Hugging Face's logo

parth0908
/
peft-starcoder-finetuned

PEFT
TensorBoard
Safetensors
gpt_bigcode
Generated from Trainer
Model card Files Metrics Community
parth0908 commited on
Commit
860f546
•
1 Parent(s): b24ecd2

End of training

Browse files
Files changed (1) hide show
  1. README.md +12 -22
README.md CHANGED
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [bigcode/starcoderbase-1b](https://huggingface.co/bigcode/starcoderbase-1b) on the None dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 1.2297
20
 
21
  ## Model description
22
 
@@ -44,32 +44,22 @@ The following hyperparameters were used during training:
44
  - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
  - lr_scheduler_type: cosine
46
  - lr_scheduler_warmup_steps: 50
47
- - training_steps: 1000
48
 
49
  ### Training results
50
 
51
  | Training Loss | Epoch | Step | Validation Loss |
52
  |:-------------:|:------:|:----:|:---------------:|
53
- | 1.234 | 0.0996 | 50 | 1.2381 |
54
- | 1.0882 | 0.1992 | 100 | 1.2581 |
55
- | 0.9431 | 0.2988 | 150 | 1.2722 |
56
- | 0.8286 | 0.3984 | 200 | 1.2415 |
57
- | 0.7855 | 0.4980 | 250 | 1.2368 |
58
- | 0.7417 | 0.5976 | 300 | 1.2313 |
59
- | 0.7189 | 0.6972 | 350 | 1.2391 |
60
- | 0.6781 | 0.7968 | 400 | 1.2231 |
61
- | 0.6902 | 0.8964 | 450 | 1.2234 |
62
- | 0.653 | 0.9960 | 500 | 1.2330 |
63
- | 0.6556 | 1.0956 | 550 | 1.2255 |
64
- | 0.6343 | 1.1952 | 600 | 1.2197 |
65
- | 0.606 | 1.2948 | 650 | 1.2366 |
66
- | 0.6223 | 1.3944 | 700 | 1.2300 |
67
- | 0.6313 | 1.4940 | 750 | 1.2340 |
68
- | 0.6068 | 1.5936 | 800 | 1.2327 |
69
- | 0.6012 | 1.6932 | 850 | 1.2280 |
70
- | 0.6299 | 1.7928 | 900 | 1.2296 |
71
- | 0.5962 | 1.8924 | 950 | 1.2293 |
72
- | 0.6041 | 1.9920 | 1000 | 1.2297 |
73
 
74
 
75
  ### Framework versions
 
16
 
17
  This model is a fine-tuned version of [bigcode/starcoderbase-1b](https://huggingface.co/bigcode/starcoderbase-1b) on the None dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 1.2169
20
 
21
  ## Model description
22
 
 
44
  - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
  - lr_scheduler_type: cosine
46
  - lr_scheduler_warmup_steps: 50
47
+ - training_steps: 5000
48
 
49
  ### Training results
50
 
51
  | Training Loss | Epoch | Step | Validation Loss |
52
  |:-------------:|:------:|:----:|:---------------:|
53
+ | 0.8313 | 0.9960 | 500 | 1.2189 |
54
+ | 0.5881 | 1.9920 | 1000 | 1.2062 |
55
+ | 0.5283 | 2.9880 | 1500 | 1.2020 |
56
+ | 0.4928 | 3.9841 | 2000 | 1.1926 |
57
+ | 0.4682 | 4.9801 | 2500 | 1.2326 |
58
+ | 0.4481 | 5.9761 | 3000 | 1.2139 |
59
+ | 0.4383 | 6.9721 | 3500 | 1.2099 |
60
+ | 0.4305 | 7.9681 | 4000 | 1.2161 |
61
+ | 0.4251 | 8.9641 | 4500 | 1.2138 |
62
+ | 0.423 | 9.9602 | 5000 | 1.2169 |
 
 
 
 
 
 
 
 
 
 
63
 
64
 
65
  ### Framework versions