bigcode

/

starcoder2-15b

like 558

I have finetuned bigcode/starcoder2-15b and generated adapter weights. Now while trying to load adapter model with base model I'm getting the following error.

RuntimeError: Error(s) in loading state_dict for PeftModelForCausalLM:
size mismatch for base_model.model.model.embed_tokens.modules_to_save.adapter_model.weight: copying a param with shape torch.Size([49156, 6144]) from checkpoint, the shape in current model is torch.Size([49152, 6144]).
size mismatch for base_model.model.lm_head.modules_to_save.adapter_model.weight: copying a param with shape torch.Size([49156, 6144]) from checkpoint, the shape in current model is torch.Size([49152, 6144]).

I have used https://github.com/OpenAccess-AI-Collective/axolotl/tree/main for training and i have put following params in .yml

lora_modules_to_save:

• embed_tokens
• lm_head

using the following code to load the model:

def load_model_and_tokenizer(adapter_path, bnb_config=None):
    base_model = AutoModelForCausalLM.from_pretrained(
                                        "bigcode/starcoder2-15b", 
                                        # load_in_4bit=True,
                                        torch_dtype=torch.bfloat16,
                                        # quantization_config=bnb_config,
                                        device_map='auto'
                                        )
    tokenizer = AutoTokenizer.from_pretrained("bigcode/starcoder2-15b")
    model = PeftModel.from_pretrained(
                        model=base_model, 
                        model_id=adapter_path, 
                        adapter_name="adapter_model",
                        torch_dtype=torch.bfloat16,
                        is_trainable= False)

     tokenizer.pad_token = tokenizer.eos_token
     model.eval()
     return model, tokenizer