PhoBert_Dataset59KBoDuoi

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3898
• Accuracy: 0.8943
• F1: 0.8949

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 256
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.0230	200	0.3239	0.8676	0.8657
No log	2.0460	400	0.2895	0.8761	0.8750
No log	3.0691	600	0.2810	0.8862	0.8865
0.2918	4.0921	800	0.2887	0.8842	0.8856
0.2918	5.1151	1000	0.2770	0.8938	0.8945
0.2918	6.1381	1200	0.3323	0.8837	0.8856
0.2918	7.1611	1400	0.3013	0.8935	0.8942
0.1744	8.1841	1600	0.3146	0.8919	0.8935
0.1744	9.2072	1800	0.3165	0.8977	0.8978
0.1744	10.2302	2000	0.3452	0.8889	0.8903
0.1744	11.2532	2200	0.3487	0.8956	0.8964
0.1208	12.2762	2400	0.3420	0.8956	0.8963
0.1208	13.2992	2600	0.3441	0.8983	0.8984
0.1208	14.3223	2800	0.3713	0.8962	0.8966
0.1208	15.3453	3000	0.3696	0.8962	0.8968
0.0881	16.3683	3200	0.3812	0.8957	0.8964
0.0881	17.3913	3400	0.3824	0.8952	0.8958
0.0881	18.4143	3600	0.3838	0.8975	0.8978
0.0881	19.4373	3800	0.3898	0.8943	0.8949

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1