Edit model card

Phi-3.5-MultiCap-mt

This model is a fine-tuned version of microsoft/Phi-3.5-mini-instruct on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7569

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 128
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 2

Training results

Training Loss Epoch Step Validation Loss
1.4288 0.1533 15 1.3449
1.0894 0.3065 30 1.1240
0.9541 0.4598 45 0.9830
0.9216 0.6130 60 0.8949
0.8675 0.7663 75 0.8414
0.8007 0.9195 90 0.8108
0.8205 1.0728 105 0.7919
0.7864 1.2261 120 0.7794
0.7983 1.3793 135 0.7705
0.7784 1.5326 150 0.7641
0.744 1.6858 165 0.7595
0.7765 1.8391 180 0.7569

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.2
  • Pytorch 2.4.0+cu124
  • Datasets 2.21.0
  • Tokenizers 0.19.1
Downloads last month
2
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for sofyc/Phi-3.5-MultiCap-mt

Adapter
(102)
this model