Hugging Face's logo

codewizardUV
/
NXAIR_M_mistral-7B

PEFT
Safetensors
trl
sft
Generated from Trainer
Model card Files Community
NXAIR_M_mistral-7B / README.md
codewizardUV's picture
codewizardUV
End of training
b55593d verified
preview code
|
raw
history blame
2.95 kB
metadata
license: apache-2.0
library_name: peft
tags:
  - trl
  - sft
  - generated_from_trainer
base_model: mistralai/Mistral-7B-v0.1
model-index:
  - name: NXAIR_M_mistral-7B
    results: []

NXAIR_M_mistral-7B

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6929

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.00025
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss
1.1435 0.0702 100 1.1971
1.0993 0.1404 200 1.0390
1.0643 0.2107 300 0.9309
0.956 0.2809 400 0.9125
0.9906 0.3511 500 0.8591
0.9083 0.4213 600 0.8703
0.8951 0.4916 700 0.8179
0.8352 0.5618 800 0.7852
0.8472 0.6320 900 0.7772
0.8733 0.7022 1000 0.7447
0.7958 0.7725 1100 0.7082
0.8726 0.8427 1200 0.7125
0.804 0.9129 1300 0.6909
0.8467 0.9831 1400 0.7287
0.4705 1.0534 1500 0.6921
0.4864 1.1236 1600 0.6648
0.4535 1.1938 1700 0.6765
0.4542 1.2640 1800 0.6620
0.4789 1.3343 1900 0.6584
0.5154 1.4045 2000 0.6492
0.459 1.4747 2100 0.6647
0.5168 1.5449 2200 0.6484
0.483 1.6152 2300 0.6795
0.4768 1.6854 2400 0.6730
0.4821 1.7556 2500 0.6404
0.4929 1.8258 2600 0.6409
0.5438 1.8961 2700 0.6551
0.4598 1.9663 2800 0.6740
0.4902 2.0365 2900 0.7287
0.5058 2.1067 3000 0.7142
0.4615 2.1770 3100 0.6929

Framework versions

  • PEFT 0.10.0
  • Transformers 4.40.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1