--- tags: - generated_from_trainer model-index: - name: AraT5v2-base-1024-p-l-akk-en-20240801-201225 results: [] --- # AraT5v2-base-1024-p-l-akk-en-20240801-201225 This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.0402 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 4e-05 - train_batch_size: 1 - eval_batch_size: 1 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 10 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:------:|:---------------:| | 0.0561 | 0.0552 | 2500 | 0.0493 | | 0.0511 | 0.1105 | 5000 | 0.0501 | | 0.0475 | 0.1657 | 7500 | 0.0499 | | 0.0465 | 0.2210 | 10000 | 0.0499 | | 0.0417 | 0.2762 | 12500 | 0.0495 | | 0.0528 | 0.3314 | 15000 | 0.0497 | | 0.0523 | 0.3867 | 17500 | 0.0492 | | 0.0492 | 0.4419 | 20000 | 0.0497 | | 0.0468 | 0.4972 | 22500 | 0.0488 | | 0.0515 | 0.5524 | 25000 | 0.0489 | | 0.0522 | 0.6076 | 27500 | 0.0487 | | 0.0522 | 0.6629 | 30000 | 0.0489 | | 0.0544 | 0.7181 | 32500 | 0.0486 | | 0.0466 | 0.7734 | 35000 | 0.0488 | | 0.0546 | 0.8286 | 37500 | 0.0491 | | 0.0558 | 0.8838 | 40000 | 0.0486 | | 0.0532 | 0.9391 | 42500 | 0.0484 | | 0.049 | 0.9943 | 45000 | 0.0484 | | 0.049 | 1.0496 | 47500 | 0.0487 | | 0.0429 | 1.1048 | 50000 | 0.0483 | | 0.0549 | 1.1600 | 52500 | 0.0482 | | 0.047 | 1.2153 | 55000 | 0.0480 | | 0.0446 | 1.2705 | 57500 | 0.0477 | | 0.0583 | 1.3258 | 60000 | 0.0478 | | 0.0471 | 1.3810 | 62500 | 0.0477 | | 0.0549 | 1.4362 | 65000 | 0.0475 | | 0.0441 | 1.4915 | 67500 | 0.0476 | | 0.0467 | 1.5467 | 70000 | 0.0471 | | 0.0434 | 1.6020 | 72500 | 0.0467 | | 0.0522 | 1.6572 | 75000 | 0.0471 | | 0.0513 | 1.7124 | 77500 | 0.0469 | | 0.0492 | 1.7677 | 80000 | 0.0465 | | 0.0428 | 1.8229 | 82500 | 0.0466 | | 0.0479 | 1.8782 | 85000 | 0.0461 | | 0.0522 | 1.9334 | 87500 | 0.0463 | | 0.0503 | 1.9886 | 90000 | 0.0462 | | 0.0378 | 2.0439 | 92500 | 0.0463 | | 0.0442 | 2.0991 | 95000 | 0.0462 | | 0.0412 | 2.1544 | 97500 | 0.0461 | | 0.0378 | 2.2096 | 100000 | 0.0460 | | 0.0437 | 2.2648 | 102500 | 0.0460 | | 0.0467 | 2.3201 | 105000 | 0.0456 | | 0.0446 | 2.3753 | 107500 | 0.0455 | | 0.0395 | 2.4306 | 110000 | 0.0456 | | 0.0425 | 2.4858 | 112500 | 0.0453 | | 0.0435 | 2.5410 | 115000 | 0.0454 | | 0.0457 | 2.5963 | 117500 | 0.0454 | | 0.048 | 2.6515 | 120000 | 0.0452 | | 0.0474 | 2.7068 | 122500 | 0.0450 | | 0.0512 | 2.7620 | 125000 | 0.0451 | | 0.0472 | 2.8172 | 127500 | 0.0445 | | 0.0513 | 2.8725 | 130000 | 0.0448 | | 0.0451 | 2.9277 | 132500 | 0.0446 | | 0.0445 | 2.9830 | 135000 | 0.0445 | | 0.0445 | 3.0382 | 137500 | 0.0447 | | 0.0396 | 3.0934 | 140000 | 0.0444 | | 0.047 | 3.1487 | 142500 | 0.0443 | | 0.0422 | 3.2039 | 145000 | 0.0444 | | 0.0443 | 3.2592 | 147500 | 0.0444 | | 0.0483 | 3.3144 | 150000 | 0.0442 | | 0.0434 | 3.3696 | 152500 | 0.0442 | | 0.05 | 3.4249 | 155000 | 0.0441 | | 0.0403 | 3.4801 | 157500 | 0.0438 | | 0.0461 | 3.5354 | 160000 | 0.0437 | | 0.0433 | 3.5906 | 162500 | 0.0437 | | 0.0418 | 3.6458 | 165000 | 0.0437 | | 0.0407 | 3.7011 | 167500 | 0.0436 | | 0.0408 | 3.7563 | 170000 | 0.0435 | | 0.0432 | 3.8116 | 172500 | 0.0434 | | 0.0411 | 3.8668 | 175000 | 0.0434 | | 0.045 | 3.9220 | 177500 | 0.0431 | | 0.0408 | 3.9773 | 180000 | 0.0430 | | 0.0413 | 4.0325 | 182500 | 0.0435 | | 0.0443 | 4.0878 | 185000 | 0.0430 | | 0.0375 | 4.1430 | 187500 | 0.0434 | | 0.0431 | 4.1982 | 190000 | 0.0428 | | 0.0381 | 4.2535 | 192500 | 0.0428 | | 0.0456 | 4.3087 | 195000 | 0.0429 | | 0.0468 | 4.3640 | 197500 | 0.0432 | | 0.0404 | 4.4192 | 200000 | 0.0426 | | 0.0417 | 4.4744 | 202500 | 0.0424 | | 0.0402 | 4.5297 | 205000 | 0.0429 | | 0.0427 | 4.5849 | 207500 | 0.0427 | | 0.0413 | 4.6402 | 210000 | 0.0427 | | 0.0417 | 4.6954 | 212500 | 0.0425 | | 0.0353 | 4.7506 | 215000 | 0.0425 | | 0.039 | 4.8059 | 217500 | 0.0423 | | 0.0427 | 4.8611 | 220000 | 0.0420 | | 0.0359 | 4.9164 | 222500 | 0.0422 | | 0.0392 | 4.9716 | 225000 | 0.0421 | | 0.0424 | 5.0268 | 227500 | 0.0426 | | 0.0398 | 5.0821 | 230000 | 0.0421 | | 0.0398 | 5.1373 | 232500 | 0.0421 | | 0.0349 | 5.1926 | 235000 | 0.0422 | | 0.0415 | 5.2478 | 237500 | 0.0422 | | 0.0413 | 5.3030 | 240000 | 0.0421 | | 0.0422 | 5.3583 | 242500 | 0.0419 | | 0.0365 | 5.4135 | 245000 | 0.0418 | | 0.0403 | 5.4688 | 247500 | 0.0416 | | 0.0398 | 5.5240 | 250000 | 0.0417 | | 0.0393 | 5.5792 | 252500 | 0.0417 | | 0.0396 | 5.6345 | 255000 | 0.0417 | | 0.0383 | 5.6897 | 257500 | 0.0417 | | 0.041 | 5.7450 | 260000 | 0.0417 | | 0.0425 | 5.8002 | 262500 | 0.0414 | | 0.0417 | 5.8554 | 265000 | 0.0412 | | 0.0352 | 5.9107 | 267500 | 0.0413 | | 0.0349 | 5.9659 | 270000 | 0.0413 | | 0.0397 | 6.0212 | 272500 | 0.0413 | | 0.037 | 6.0764 | 275000 | 0.0414 | | 0.0358 | 6.1316 | 277500 | 0.0416 | | 0.0402 | 6.1869 | 280000 | 0.0415 | | 0.0332 | 6.2421 | 282500 | 0.0417 | | 0.035 | 6.2974 | 285000 | 0.0415 | | 0.0364 | 6.3526 | 287500 | 0.0413 | | 0.0427 | 6.4078 | 290000 | 0.0412 | | 0.0387 | 6.4631 | 292500 | 0.0409 | | 0.0288 | 6.5183 | 295000 | 0.0410 | | 0.0417 | 6.5736 | 297500 | 0.0410 | | 0.0372 | 6.6288 | 300000 | 0.0410 | | 0.042 | 6.6840 | 302500 | 0.0411 | | 0.0347 | 6.7393 | 305000 | 0.0409 | | 0.0363 | 6.7945 | 307500 | 0.0408 | | 0.0413 | 6.8498 | 310000 | 0.0410 | | 0.0386 | 6.9050 | 312500 | 0.0407 | | 0.0362 | 6.9602 | 315000 | 0.0407 | | 0.0385 | 7.0155 | 317500 | 0.0410 | | 0.0412 | 7.0707 | 320000 | 0.0410 | | 0.0349 | 7.1260 | 322500 | 0.0407 | | 0.0383 | 7.1812 | 325000 | 0.0408 | | 0.0316 | 7.2364 | 327500 | 0.0410 | | 0.0387 | 7.2917 | 330000 | 0.0409 | | 0.0321 | 7.3469 | 332500 | 0.0407 | | 0.0323 | 7.4022 | 335000 | 0.0406 | | 0.0352 | 7.4574 | 337500 | 0.0406 | | 0.0342 | 7.5126 | 340000 | 0.0404 | | 0.0356 | 7.5679 | 342500 | 0.0406 | | 0.0379 | 7.6231 | 345000 | 0.0405 | | 0.0381 | 7.6784 | 347500 | 0.0406 | | 0.0359 | 7.7336 | 350000 | 0.0403 | | 0.0371 | 7.7889 | 352500 | 0.0403 | | 0.0385 | 7.8441 | 355000 | 0.0404 | | 0.0321 | 7.8993 | 357500 | 0.0405 | | 0.0366 | 7.9546 | 360000 | 0.0406 | | 0.0316 | 8.0098 | 362500 | 0.0407 | | 0.0372 | 8.0651 | 365000 | 0.0406 | | 0.0327 | 8.1203 | 367500 | 0.0406 | | 0.0334 | 8.1755 | 370000 | 0.0405 | | 0.0366 | 8.2308 | 372500 | 0.0405 | | 0.0346 | 8.2860 | 375000 | 0.0405 | | 0.0322 | 8.3413 | 377500 | 0.0404 | | 0.0429 | 8.3965 | 380000 | 0.0403 | | 0.0341 | 8.4517 | 382500 | 0.0404 | | 0.0345 | 8.5070 | 385000 | 0.0403 | | 0.0356 | 8.5622 | 387500 | 0.0403 | | 0.0352 | 8.6175 | 390000 | 0.0404 | | 0.0361 | 8.6727 | 392500 | 0.0402 | | 0.0335 | 8.7279 | 395000 | 0.0404 | | 0.0361 | 8.7832 | 397500 | 0.0403 | | 0.035 | 8.8384 | 400000 | 0.0403 | | 0.0327 | 8.8937 | 402500 | 0.0402 | | 0.0336 | 8.9489 | 405000 | 0.0401 | | 0.035 | 9.0041 | 407500 | 0.0404 | | 0.0363 | 9.0594 | 410000 | 0.0403 | | 0.0317 | 9.1146 | 412500 | 0.0403 | | 0.033 | 9.1699 | 415000 | 0.0404 | | 0.0368 | 9.2251 | 417500 | 0.0403 | | 0.0333 | 9.2803 | 420000 | 0.0403 | | 0.0308 | 9.3356 | 422500 | 0.0402 | | 0.0329 | 9.3908 | 425000 | 0.0403 | | 0.0371 | 9.4461 | 427500 | 0.0402 | | 0.0331 | 9.5013 | 430000 | 0.0403 | | 0.0328 | 9.5565 | 432500 | 0.0402 | | 0.0323 | 9.6118 | 435000 | 0.0402 | | 0.0283 | 9.6670 | 437500 | 0.0402 | | 0.0338 | 9.7223 | 440000 | 0.0401 | | 0.0323 | 9.7775 | 442500 | 0.0402 | | 0.0374 | 9.8327 | 445000 | 0.0402 | | 0.0357 | 9.8880 | 447500 | 0.0402 | | 0.037 | 9.9432 | 450000 | 0.0402 | | 0.0348 | 9.9985 | 452500 | 0.0402 | ### Framework versions - Transformers 4.44.0.dev0 - Pytorch 2.5.0.dev20240625 - Datasets 2.20.0 - Tokenizers 0.19.1