A model for translating cuneiform to english using google t5-small.

Akkadian: 𒄿 𒈾 𒌗 𒃶 𒌓 𒐉 𒆚 𒀀 𒈾 𒆳 𒆸 𒄭 𒇻 𒁺 𒅅 𒆳 𒁀 𒀀 𒍝 𒆳 𒊓 𒅈 𒁀 𒇷 𒀀 𒆳 𒁲 𒁺 𒀀 𒆷 𒀀 𒁲 𒌷 𒈨 𒌍 𒉌 𒃻 𒅆 𒁲 𒀀 𒇉 𒊒 𒌑 𒊒 𒊭 𒆳 𒈨 𒄴 𒊑 𒀝 𒋤 𒊩 𒆷 𒋢 𒉡 𒃻 𒋗 𒈨 𒌍 𒋗 𒉡 𒌑 𒊺 𒍝 𒀀 𒀀 𒈾 𒌷 𒅀 𒀸 𒋩 𒌒 𒆷' English: 'in the month kislimu the fourth day i marched to the land habhu i conquered the lands bazu sarbaliu and didualu together with the cities on the banks of the river ruru of the land mehru i brought forth their booty and possessions and brought them to my city assur' Prediction: 'in the mo nth tammuz iv i conquered the land s que and que i conquered the land s que and bi t yakin i conquered the cities f ro m the river i conquered and plundered the cities on the bo rd er of the land elam'

t5-small-p-l-akk-en-20240727-131059

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0397
• Bleu (max_length: 500): 23.8943
• Bleu (max_length: 100): 34.2329

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.0533	0.1657	2500	0.0495
0.0469	0.3314	5000	0.0493
0.0516	0.4971	7500	0.0491
0.0495	0.6629	10000	0.0490
0.0531	0.8286	12500	0.0488
0.056	0.9943	15000	0.0487
0.0528	1.1600	17500	0.0486
0.0515	1.3257	20000	0.0484
0.0569	1.4914	22500	0.0483
0.0487	1.6572	25000	0.0481
0.0492	1.8229	27500	0.0481
0.0477	1.9886	30000	0.0478
0.0514	2.1543	32500	0.0478
0.0499	2.3200	35000	0.0477
0.0524	2.4857	37500	0.0477
0.0493	2.6515	40000	0.0475
0.0534	2.8172	42500	0.0474
0.053	2.9829	45000	0.0471
0.0497	3.1486	47500	0.0472
0.0451	3.3143	50000	0.0471
0.0533	3.4800	52500	0.0469
0.0476	3.6458	55000	0.0469
0.0499	3.8115	57500	0.0467
0.0524	3.9772	60000	0.0467
0.0471	4.1429	62500	0.0466
0.0495	4.3086	65000	0.0465
0.0489	4.4743	67500	0.0464
0.0462	4.6401	70000	0.0462
0.0487	4.8058	72500	0.0463
0.0506	4.9715	75000	0.0461
0.0467	5.1372	77500	0.0459
0.0448	5.3029	80000	0.0462
0.0472	5.4686	82500	0.0461
0.0522	5.6344	85000	0.0459
0.0498	5.8001	87500	0.0457
0.0512	5.9658	90000	0.0456
0.0474	6.1315	92500	0.0455
0.0517	6.2972	95000	0.0454
0.0465	6.4629	97500	0.0455
0.0468	6.6287	100000	0.0453
0.0486	6.7944	102500	0.0453
0.0469	6.9601	105000	0.0451
0.0468	7.1258	107500	0.0451
0.0486	7.2915	110000	0.0449
0.0473	7.4572	112500	0.0449
0.0487	7.6230	115000	0.0449
0.0487	7.7887	117500	0.0448
0.05	7.9544	120000	0.0447
0.049	8.1201	122500	0.0447
0.0428	8.2858	125000	0.0446
0.047	8.4515	127500	0.0445
0.0447	8.6173	130000	0.0443
0.0455	8.7830	132500	0.0443
0.0464	8.9487	135000	0.0443
0.0432	9.1144	137500	0.0444
0.0463	9.2801	140000	0.0442
0.0431	9.4458	142500	0.0442
0.0448	9.6116	145000	0.0441
0.0457	9.7773	147500	0.0440
0.0478	9.9430	150000	0.0439
0.0484	10.1087	152500	0.0441
0.0418	10.2744	155000	0.0439
0.0404	10.4401	157500	0.0437
0.0449	10.6059	160000	0.0436
0.0453	10.7716	162500	0.0438
0.0441	10.9373	165000	0.0435
0.0447	11.1030	167500	0.0437
0.0455	11.2687	170000	0.0436
0.043	11.4344	172500	0.0436
0.0429	11.6002	175000	0.0435
0.0434	11.7659	177500	0.0434
0.0452	11.9316	180000	0.0433
0.0423	12.0973	182500	0.0435
0.0482	12.2630	185000	0.0433
0.0439	12.4287	187500	0.0433
0.0408	12.5945	190000	0.0432
0.0458	12.7602	192500	0.0430
0.0429	12.9259	195000	0.0431
0.0403	13.0916	197500	0.0430
0.0451	13.2573	200000	0.0429
0.0453	13.4230	202500	0.0429
0.0437	13.5888	205000	0.0427
0.0433	13.7545	207500	0.0428
0.0462	13.9202	210000	0.0426
0.0447	14.0859	212500	0.0427
0.0454	14.2516	215000	0.0427
0.0413	14.4173	217500	0.0426
0.0416	14.5831	220000	0.0426
0.0444	14.7488	222500	0.0425
0.0465	14.9145	225000	0.0424
0.0423	15.0802	227500	0.0425
0.0473	15.2459	230000	0.0424
0.0408	15.4116	232500	0.0424
0.0442	15.5774	235000	0.0424
0.0415	15.7431	237500	0.0422
0.042	15.9088	240000	0.0421
0.0414	16.0745	242500	0.0422
0.0444	16.2402	245000	0.0421
0.0435	16.4059	247500	0.0421
0.0393	16.5717	250000	0.0420
0.0436	16.7374	252500	0.0421
0.0442	16.9031	255000	0.0419
0.0461	17.0688	257500	0.0420
0.0403	17.2345	260000	0.0420
0.044	17.4002	262500	0.0420
0.0406	17.5660	265000	0.0419
0.0422	17.7317	267500	0.0418
0.0424	17.8974	270000	0.0416
0.0415	18.0631	272500	0.0418
0.0407	18.2288	275000	0.0416
0.0401	18.3945	277500	0.0418
0.0449	18.5603	280000	0.0418
0.0405	18.7260	282500	0.0415
0.0385	18.8917	285000	0.0418
0.0405	19.0574	287500	0.0415
0.0445	19.2231	290000	0.0417
0.0429	19.3888	292500	0.0414
0.0429	19.5546	295000	0.0413
0.0464	19.7203	297500	0.0414
0.0424	19.8860	300000	0.0413
0.0417	20.0517	302500	0.0414
0.0418	20.2174	305000	0.0413
0.0406	20.3831	307500	0.0414
0.0419	20.5489	310000	0.0413
0.039	20.7146	312500	0.0413
0.0392	20.8803	315000	0.0411
0.0418	21.0460	317500	0.0411
0.0363	21.2117	320000	0.0411
0.0424	21.3774	322500	0.0412
0.0402	21.5432	325000	0.0413
0.0418	21.7089	327500	0.0413
0.0412	21.8746	330000	0.0410
0.0413	22.0403	332500	0.0410
0.0413	22.2060	335000	0.0411
0.0418	22.3717	337500	0.0411
0.0424	22.5375	340000	0.0411
0.0386	22.7032	342500	0.0410
0.0399	22.8689	345000	0.0408
0.0429	23.0346	347500	0.0409
0.0384	23.2003	350000	0.0409
0.0408	23.3660	352500	0.0408
0.0405	23.5318	355000	0.0407
0.042	23.6975	357500	0.0408
0.0404	23.8632	360000	0.0407
0.0382	24.0289	362500	0.0406
0.0393	24.1946	365000	0.0408
0.0359	24.3603	367500	0.0407
0.0412	24.5261	370000	0.0408
0.0446	24.6918	372500	0.0406
0.0377	24.8575	375000	0.0406
0.0379	25.0232	377500	0.0407
0.0389	25.1889	380000	0.0407
0.0365	25.3546	382500	0.0405
0.0441	25.5203	385000	0.0405
0.0427	25.6861	387500	0.0405
0.0393	25.8518	390000	0.0405
0.0392	26.0175	392500	0.0406
0.039	26.1832	395000	0.0405
0.0401	26.3489	397500	0.0405
0.0385	26.5146	400000	0.0405
0.0413	26.6804	402500	0.0406
0.0384	26.8461	405000	0.0405
0.0388	27.0118	407500	0.0405
0.039	27.1775	410000	0.0405
0.0385	27.3432	412500	0.0404
0.0387	27.5089	415000	0.0404
0.0426	27.6747	417500	0.0403
0.0381	27.8404	420000	0.0403
0.0423	28.0061	422500	0.0405
0.0368	28.1718	425000	0.0403
0.0405	28.3375	427500	0.0403
0.0371	28.5032	430000	0.0405
0.0393	28.6690	432500	0.0403
0.0385	28.8347	435000	0.0403
0.0399	29.0004	437500	0.0402
0.0398	29.1661	440000	0.0403
0.0364	29.3318	442500	0.0402
0.0374	29.4975	445000	0.0402
0.0401	29.6633	447500	0.0401
0.0404	29.8290	450000	0.0402
0.0391	29.9947	452500	0.0401
0.0398	30.1604	455000	0.0401
0.0387	30.3261	457500	0.0403
0.0388	30.4918	460000	0.0402
0.0392	30.6576	462500	0.0400
0.037	30.8233	465000	0.0400
0.0415	30.9890	467500	0.0401
0.0407	31.1547	470000	0.0401
0.0414	31.3204	472500	0.0401
0.0401	31.4861	475000	0.0400
0.0382	31.6519	477500	0.0400
0.0412	31.8176	480000	0.0399
0.0368	31.9833	482500	0.0400
0.0384	32.1490	485000	0.0401
0.036	32.3147	487500	0.0399
0.0387	32.4804	490000	0.0400
0.0407	32.6462	492500	0.0399
0.0377	32.8119	495000	0.0400
0.0353	32.9776	497500	0.0399
0.0401	33.1433	500000	0.0400
0.0367	33.3090	502500	0.0399
0.0376	33.4747	505000	0.0399
0.0422	33.6405	507500	0.0398
0.0405	33.8062	510000	0.0398
0.0376	33.9719	512500	0.0398
0.041	34.1376	515000	0.0399
0.0372	34.3033	517500	0.0400
0.0391	34.4690	520000	0.0399
0.0399	34.6348	522500	0.0399
0.0369	34.8005	525000	0.0399
0.0409	34.9662	527500	0.0398
0.0398	35.1319	530000	0.0398
0.0379	35.2976	532500	0.0399
0.037	35.4633	535000	0.0399
0.0391	35.6291	537500	0.0398
0.0437	35.7948	540000	0.0399
0.0393	35.9605	542500	0.0397
0.0395	36.1262	545000	0.0398
0.0414	36.2919	547500	0.0399
0.0379	36.4576	550000	0.0398
0.0366	36.6234	552500	0.0398
0.0394	36.7891	555000	0.0398
0.0413	36.9548	557500	0.0398
0.0397	37.1205	560000	0.0398
0.0367	37.2862	562500	0.0397
0.0413	37.4519	565000	0.0397
0.0427	37.6177	567500	0.0398
0.0383	37.7834	570000	0.0397
0.036	37.9491	572500	0.0397
0.0389	38.1148	575000	0.0397
0.0395	38.2805	577500	0.0397
0.0391	38.4462	580000	0.0398
0.0388	38.6120	582500	0.0398
0.0412	38.7777	585000	0.0397
0.036	38.9434	587500	0.0398
0.0398	39.1091	590000	0.0398
0.0373	39.2748	592500	0.0397
0.0391	39.4405	595000	0.0397
0.0375	39.6063	597500	0.0398
0.0404	39.7720	600000	0.0398
0.0369	39.9377	602500	0.0397

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1