flan-t5-base-sql

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2463

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
36.07	0.032	100	28.3961
17.6704	0.064	200	7.5753
5.1571	0.096	300	4.2162
3.7573	0.128	400	2.9984
2.7831	0.16	500	1.7788
2.0308	0.192	600	1.2800
1.5731	0.224	700	0.9554
1.2735	0.256	800	0.7648
1.0347	0.288	900	0.6471
0.886	0.32	1000	0.5424
0.757	0.352	1100	0.4848
0.6731	0.384	1200	0.4293
0.6222	0.416	1300	0.3904
0.55	0.448	1400	0.3609
0.5226	0.48	1500	0.3372
0.4802	0.512	1600	0.3218
0.4653	0.544	1700	0.3076
0.4419	0.576	1800	0.2948
0.4252	0.608	1900	0.2865
0.4009	0.64	2000	0.2780
0.3977	0.672	2100	0.2716
0.3919	0.704	2200	0.2670
0.3788	0.736	2300	0.2618
0.3674	0.768	2400	0.2574
0.359	0.8	2500	0.2543
0.3477	0.832	2600	0.2524
0.3555	0.864	2700	0.2503
0.3461	0.896	2800	0.2482
0.3454	0.928	2900	0.2470
0.3526	0.96	3000	0.2464
0.3464	0.992	3100	0.2463

Framework versions

• Transformers 4.44.0
• Pytorch 2.3.1.post300
• Datasets 2.21.0
• Tokenizers 0.19.1