ymoslem/whisper-small-ga2en-v3.4-r

Whisper Small GA-EN Speech Translation

This model is a fine-tuned version of openai/whisper-small on the IWSLT-2023, FLEURS, BiteSize, SpokenWords, Tatoeba, and Wikimedia dataset. It achieves the following results on the evaluation set:

• Loss: 1.4631
• Bleu: 32.04
• Chrf: 48.69
• Wer: 63.3949

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.03
• training_steps: 3000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Chrf	Wer
2.3783	0.1312	100	1.8852	7.56	22.6	113.2823
1.92	0.2625	200	1.5276	16.93	32.19	81.4498
1.6661	0.3937	300	1.3907	16.26	35.75	99.1896
1.4712	0.5249	400	1.3126	24.55	42.56	77.8478
1.3404	0.6562	500	1.2960	23.94	42.25	77.3976
1.2106	0.7874	600	1.2556	23.82	43.46	73.5705
1.0312	0.9186	700	1.3002	23.73	43.09	74.6060
0.5265	1.0499	800	1.2993	28.09	45.57	69.1580
0.4802	1.1811	900	1.3466	25.21	43.38	75.7767
0.4415	1.3123	1000	1.3456	29.77	47.56	66.9968
0.4164	1.4436	1100	1.3373	27.92	45.54	70.9140
0.3937	1.5748	1200	1.3162	30.09	46.51	64.2053
0.3391	1.7060	1300	1.3424	24.82	45.35	72.9401
0.2969	1.8373	1400	1.3271	31.78	48.51	62.5394
0.2755	1.9685	1500	1.3523	31.6	48.33	61.3237
0.1059	2.0997	1600	1.3910	30.26	45.88	65.3309
0.0975	2.2310	1700	1.4255	30.28	46.1	64.1603
0.1047	2.3622	1800	1.3923	29.99	46.44	64.9257
0.0874	2.4934	1900	1.4111	30.14	47.09	65.1058
0.0838	2.6247	2000	1.4378	25.63	45.79	77.4426
0.0757	2.7559	2100	1.4356	29.28	47.5	65.0608
0.0749	2.8871	2200	1.4532	30.56	46.58	64.3854
0.0463	3.0184	2300	1.4324	32.69	49.04	62.6294
0.0265	3.1496	2400	1.4311	31.24	48.58	62.9896
0.0266	3.2808	2500	1.4409	31.97	47.99	62.4944
0.0237	3.4121	2600	1.4310	32.44	48.86	62.2692
0.0208	3.5433	2700	1.4483	31.3	47.49	63.5299
0.0185	3.6745	2800	1.4513	32.86	48.98	62.6294
0.0178	3.8058	2900	1.4583	31.77	48.91	63.0797
0.0194	3.9370	3000	1.4631	32.04	48.69	63.3949

Framework versions

• Transformers 4.41.2
• Pytorch 2.2.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1