ValueError: Please specify `target_modules` in `peft_config`
While training with
Set supervised fine-tuning parameters
trainer = SFTTrainer(
model=model,
train_dataset=dataset,
peft_config=peft_config,
dataset_text_field="text",
max_seq_length=max_seq_length,
tokenizer=tokenizer,
args=training_arguments,
packing=packing,
)
Train model
trainer.train()
ValueError: Please specify target_modules in peft_config
I'm attempting to use deepspeed for model training, but the script crashes when trying to load the model. Additionally, the trainer is not assigning the model to the GPU.
You want to specify the target_modules which are the layers to apply LoRA to. I did the linear layers:
target_modules=[
"q_proj",
"k_proj",
"v_proj",
"o_proj",
"gate_proj",
"up_proj",
"down_proj",
"lm_head",
],
According to the LoRA paper, the best performance you can get is if you apply it to all the linear layers of the model. (Diminishing returns if you were to choose just the Q and V but they propose all if I'm not mistaken)
Which are the ones
@harpercarroll
has mentioned. But if you want to go deeper and apply this to other models as well what you would do is instantiate your model, then go and print(model) then you will see all the layers with their names and you would want to include in the target_modules all layers that have linear or 4bitlinear (something like that if its 4bit).