facebook_nllb-200-distilled-600M_text2sparql

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2092

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	12	1.7982
No log	2.0	24	0.5266
No log	3.0	36	0.2833
No log	4.0	48	0.2299
No log	5.0	60	0.2088
No log	6.0	72	0.2297
No log	7.0	84	0.2089
No log	8.0	96	0.2013
No log	9.0	108	0.1952
No log	10.0	120	0.2037
No log	11.0	132	0.2145
No log	12.0	144	0.2064
No log	13.0	156	0.2117
No log	14.0	168	0.2147
No log	15.0	180	0.2153
No log	16.0	192	0.2146
No log	17.0	204	0.2152
No log	18.0	216	0.2123
No log	19.0	228	0.2145
No log	20.0	240	0.2121
No log	21.0	252	0.2044
No log	22.0	264	0.2132
No log	23.0	276	0.2161
No log	24.0	288	0.2141
No log	25.0	300	0.2109
No log	26.0	312	0.2034
No log	27.0	324	0.2068
No log	28.0	336	0.2101
No log	29.0	348	0.2110
No log	30.0	360	0.2141
No log	31.0	372	0.2141
No log	32.0	384	0.2124
No log	33.0	396	0.2115
No log	34.0	408	0.2095
No log	35.0	420	0.2068
No log	36.0	432	0.2027
No log	37.0	444	0.2030
No log	38.0	456	0.2043
No log	39.0	468	0.2067
No log	40.0	480	0.2076
No log	41.0	492	0.2067
0.1497	42.0	504	0.2076
0.1497	43.0	516	0.2089
0.1497	44.0	528	0.2098
0.1497	45.0	540	0.2096
0.1497	46.0	552	0.2090
0.1497	47.0	564	0.2089
0.1497	48.0	576	0.2090
0.1497	49.0	588	0.2095
0.1497	50.0	600	0.2092

Framework versions

• Transformers 4.36.0
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0