SQL_Final_RunPod_Last

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0215
• Bleu: 44.256
• Gen Len: 18.9114

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.2055	0.12	1000	0.0917	42.8594	18.8938
0.1187	0.23	2000	0.0709	43.1637	18.8915
0.1007	0.35	3000	0.0602	43.4304	18.9088
0.0869	0.46	4000	0.0559	43.4636	18.8961
0.0792	0.58	5000	0.0497	43.5366	18.9063
0.0736	0.69	6000	0.0464	43.5769	18.9016
0.0672	0.81	7000	0.0435	43.7471	18.9068
0.0635	0.93	8000	0.0403	43.781	18.9073
0.0564	1.04	9000	0.0389	43.7054	18.9029
0.0493	1.16	10000	0.0376	43.8362	18.9063
0.0479	1.27	11000	0.0367	43.8514	18.9126
0.0465	1.39	12000	0.0350	43.8365	18.9078
0.0449	1.5	13000	0.0335	43.8878	18.9042
0.0419	1.62	14000	0.0324	43.9035	18.9075
0.0426	1.74	15000	0.0314	43.9272	18.906
0.0405	1.85	16000	0.0302	44.0143	18.9087
0.039	1.97	17000	0.0291	43.9392	18.9089
0.0327	2.08	18000	0.0286	44.0248	18.9087
0.0311	2.2	19000	0.0288	44.0732	18.9119
0.0302	2.31	20000	0.0282	44.061	18.9055
0.029	2.43	21000	0.0279	44.0681	18.9121
0.0297	2.55	22000	0.0267	44.0958	18.91
0.0284	2.66	23000	0.0259	44.1215	18.9121
0.0272	2.78	24000	0.0259	44.0752	18.9113
0.0273	2.89	25000	0.0253	44.1104	18.909
0.0265	3.01	26000	0.0253	44.1262	18.9095
0.0215	3.12	27000	0.0251	44.137	18.9119
0.0215	3.24	28000	0.0246	44.1382	18.9096
0.0215	3.36	29000	0.0244	44.1806	18.9088
0.0206	3.47	30000	0.0237	44.169	18.911
0.0202	3.59	31000	0.0243	44.1469	18.9096
0.0204	3.7	32000	0.0231	44.1405	18.9116
0.0193	3.82	33000	0.0230	44.1613	18.9116
0.0196	3.94	34000	0.0226	44.197	18.9117
0.0177	4.05	35000	0.0228	44.1942	18.9102
0.0155	4.17	36000	0.0230	44.2241	18.9118
0.0159	4.28	37000	0.0226	44.2219	18.9107
0.0151	4.4	38000	0.0221	44.212	18.912
0.0149	4.51	39000	0.0222	44.2743	18.9115
0.0154	4.63	40000	0.0216	44.2636	18.9121
0.0149	4.75	41000	0.0215	44.2805	18.913
0.0146	4.86	42000	0.0216	44.2681	18.9125
0.0145	4.98	43000	0.0215	44.256	18.9114

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3