---
library_name: peft
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
tags:
- generated_from_trainer
model-index:
- name: qwen_checkpoints
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# qwen_checkpoints

This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0618
- Mse: 0.0618
- Mae: 0.1983
- R Squared: 0.3107

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.01
- num_epochs: 3
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Mae    | Mse    | R Squared |
|:-------------:|:------:|:----:|:---------------:|:------:|:------:|:---------:|
| 0.0856        | 0.1558 | 100  | 0.0878          | 0.2351 | 0.0878 | 0.0207    |
| 0.0843        | 0.3115 | 200  | 0.0803          | 0.2314 | 0.0803 | 0.1045    |
| 0.0851        | 0.4673 | 300  | 0.0882          | 0.2278 | 0.0882 | 0.0168    |
| 0.0676        | 0.6231 | 400  | 0.0716          | 0.2183 | 0.0716 | 0.2014    |
| 0.0737        | 0.7788 | 500  | 0.0691          | 0.2164 | 0.0691 | 0.2291    |
| 0.0694        | 0.9346 | 600  | 0.0696          | 0.2157 | 0.0696 | 0.2242    |
| 0.0569        | 1.0903 | 700  | 0.0661          | 0.2049 | 0.0661 | 0.2627    |
| 0.0589        | 1.2461 | 800  | 0.0663          | 0.2045 | 0.0663 | 0.2606    |
| 0.0648        | 1.4019 | 900  | 0.0649          | 0.2039 | 0.0649 | 0.2764    |
| 0.0652        | 1.5576 | 1000 | 0.0644          | 0.2027 | 0.0644 | 0.2813    |
| 0.0657        | 1.7134 | 1100 | 0.0649          | 0.0649 | 0.2082 | 0.2763    |
| 0.0577        | 1.8692 | 1200 | 0.0639          | 0.0639 | 0.2022 | 0.2869    |
| 0.0564        | 2.0249 | 1300 | 0.0636          | 0.0636 | 0.2006 | 0.2902    |
| 0.0613        | 2.1807 | 1400 | 0.0633          | 0.0633 | 0.1989 | 0.2939    |
| 0.0596        | 2.3364 | 1500 | 0.0624          | 0.0624 | 0.1999 | 0.3036    |
| 0.0547        | 2.4922 | 1600 | 0.0621          | 0.0621 | 0.1985 | 0.3076    |
| 0.0554        | 2.6480 | 1700 | 0.0620          | 0.0620 | 0.1974 | 0.3087    |
| 0.0581        | 2.8037 | 1800 | 0.0618          | 0.0618 | 0.1983 | 0.3107    |
| 0.0653        | 2.9595 | 1900 | 0.0618          | 0.0618 | 0.1983 | 0.3107    |


### Framework versions

- PEFT 0.13.2
- Transformers 4.45.2
- Pytorch 2.5.1+cu121
- Datasets 3.1.0
- Tokenizers 0.20.3