cls-comment-phobert-base-v2-v2.4.1

This model is a fine-tuned version of vinai/phobert-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3274
• Accuracy: 0.9268
• F1 Score: 0.8919
• Recall: 0.8944
• Precision: 0.8897

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 4000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Score	Recall	Precision
1.7049	0.96	100	1.4950	0.4614	0.1080	0.1681	0.2101
1.3066	1.91	200	1.0451	0.6598	0.2493	0.2970	0.2150
0.9457	2.87	300	0.7491	0.7972	0.5219	0.5230	0.5238
0.6975	3.83	400	0.5574	0.8497	0.5700	0.5935	0.7143
0.5187	4.78	500	0.4681	0.8665	0.6685	0.6592	0.7077
0.4183	5.74	600	0.4121	0.8821	0.7747	0.7478	0.8761
0.3323	6.7	700	0.3488	0.9040	0.8505	0.8391	0.8647
0.2705	7.66	800	0.3179	0.9124	0.8680	0.8694	0.8683
0.229	8.61	900	0.3109	0.9160	0.8739	0.8778	0.8704
0.1964	9.57	1000	0.3028	0.9175	0.8776	0.8813	0.8741
0.1771	10.53	1100	0.3032	0.9181	0.8807	0.8877	0.8743
0.1518	11.48	1200	0.3151	0.9166	0.8762	0.8702	0.8828
0.1368	12.44	1300	0.2938	0.9214	0.8794	0.8800	0.8789
0.1116	13.4	1400	0.2971	0.9205	0.8795	0.8815	0.8776
0.1136	14.35	1500	0.3011	0.9235	0.8858	0.8825	0.8894
0.094	15.31	1600	0.2937	0.9268	0.8891	0.8933	0.8855
0.0905	16.27	1700	0.3049	0.9265	0.8850	0.8819	0.8886
0.0838	17.22	1800	0.3061	0.9244	0.8823	0.8869	0.8784
0.0749	18.18	1900	0.3275	0.9205	0.8771	0.8839	0.8717
0.0686	19.14	2000	0.3092	0.9295	0.8915	0.8990	0.8846
0.0669	20.1	2100	0.3168	0.9250	0.8836	0.8849	0.8825
0.0582	21.05	2200	0.3339	0.9235	0.8763	0.8926	0.8631
0.0516	22.01	2300	0.3274	0.9268	0.8919	0.8944	0.8897
0.0543	22.97	2400	0.3230	0.9295	0.8913	0.8882	0.8946
0.0435	23.92	2500	0.3364	0.9253	0.8806	0.8705	0.8918
0.0405	24.88	2600	0.3492	0.9241	0.8816	0.8821	0.8819
0.0398	25.84	2700	0.3558	0.9238	0.8799	0.8796	0.8807
0.0363	26.79	2800	0.3605	0.9223	0.8742	0.8795	0.8698

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2