cls-comment-phobert-base-v2-v2.3

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6228
• Accuracy: 0.9272
• F1 Score: 0.8944
• Recall: 0.8843
• Precision: 0.9063

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 4000
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Score	Recall	Precision
1.1884	1.04	100	1.0905	0.7277	0.4371	0.4229	0.5017
1.0502	2.08	200	0.9278	0.7956	0.5245	0.5274	0.5244
0.9031	3.12	300	0.7793	0.8446	0.5653	0.5897	0.7067
0.7772	4.17	400	0.7090	0.8697	0.6917	0.6804	0.7269
0.6741	5.21	500	0.6483	0.8959	0.8078	0.7818	0.8798
0.6156	6.25	600	0.6268	0.9079	0.8601	0.8345	0.8933
0.5745	7.29	700	0.6176	0.9161	0.8739	0.8610	0.8881
0.5466	8.33	800	0.6134	0.9181	0.8730	0.8555	0.8951
0.5254	9.38	900	0.6196	0.9184	0.8794	0.8634	0.8980
0.5059	10.42	1000	0.6175	0.9220	0.8799	0.8702	0.8925
0.4971	11.46	1100	0.6110	0.9226	0.8771	0.8639	0.8929
0.4872	12.5	1200	0.6191	0.9230	0.8834	0.8774	0.8914
0.4756	13.54	1300	0.6240	0.9243	0.8929	0.8931	0.8938
0.4737	14.58	1400	0.6246	0.9203	0.8794	0.8582	0.9038
0.4626	15.62	1500	0.6267	0.9249	0.8890	0.8842	0.8955
0.4641	16.67	1600	0.6228	0.9272	0.8944	0.8843	0.9063
0.4562	17.71	1700	0.6256	0.9278	0.8923	0.8800	0.9065
0.4522	18.75	1800	0.6203	0.9285	0.8913	0.8845	0.8993
0.4476	19.79	1900	0.6258	0.9262	0.8849	0.8692	0.9034
0.4474	20.83	2000	0.6333	0.9272	0.8932	0.8751	0.9134
0.4421	21.88	2100	0.6340	0.9292	0.8929	0.8896	0.8975

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2