tiny-rubert

This model is a fine-tuned version of cointegrated/rubert-tiny2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5730
• Accuracy: 0.4956
• F1: 0.6380
• Precision: 0.7116
• Recall: 0.5873

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
No log	0.2569	500	4.0706	0.0551	0.0257	0.0397	0.0556
4.3917	0.5139	1000	3.3738	0.2871	0.2256	0.4551	0.2810
2.4111	0.7708	1500	3.0120	0.4041	0.4675	0.6818	0.4345
1.6023	1.0277	2000	2.8194	0.4454	0.5570	0.7232	0.4930
1.2666	1.2847	2500	2.7362	0.4553	0.5615	0.7195	0.5
1.0944	1.5416	3000	2.6636	0.4513	0.5783	0.7227	0.5106
1.0944	1.7986	3500	2.5940	0.4543	0.5842	0.7290	0.5134
1.0351	2.0555	4000	2.5506	0.4690	0.5953	0.7435	0.5254
0.9259	2.3124	4500	2.5396	0.4474	0.5780	0.7272	0.5127
0.802	2.5694	5000	2.4499	0.4680	0.6044	0.7420	0.5310
0.7777	2.8263	5500	2.4295	0.4661	0.5902	0.7239	0.5232
0.7247	3.0832	6000	2.4434	0.4631	0.5880	0.7245	0.5197
0.7247	3.3402	6500	2.4479	0.4769	0.6023	0.7401	0.5352
0.6062	3.5971	7000	2.4713	0.4720	0.6076	0.7465	0.5359
0.6207	3.8541	7500	2.4590	0.4779	0.6020	0.7284	0.5359
0.6021	4.1110	8000	2.4468	0.4926	0.6333	0.7359	0.5676
0.4891	4.3679	8500	2.4930	0.4848	0.6232	0.7313	0.5599
0.4983	4.6249	9000	2.4374	0.4936	0.6249	0.7239	0.5676
0.4983	4.8818	9500	2.4792	0.4956	0.6246	0.7208	0.5648
0.4789	5.1387	10000	2.5257	0.4897	0.6355	0.7117	0.5845
0.4353	5.3957	10500	2.5430	0.4946	0.6358	0.7276	0.5761
0.3995	5.6526	11000	2.5579	0.4887	0.6340	0.7188	0.5782
0.4005	5.9096	11500	2.5249	0.4828	0.6305	0.7014	0.5824
0.3774	6.1665	12000	2.6100	0.4838	0.6295	0.7194	0.5725
0.3774	6.4234	12500	2.5730	0.4956	0.6380	0.7116	0.5873
0.3502	6.6804	13000	2.6117	0.4916	0.6358	0.7066	0.5880
0.3562	6.9373	13500	2.6457	0.4956	0.6373	0.7185	0.5838
0.3453	7.1942	14000	2.6547	0.4848	0.6316	0.7062	0.5810
0.3213	7.4512	14500	2.6828	0.4877	0.6258	0.7035	0.5746

Framework versions

• Transformers 4.40.1
• Pytorch 2.2.1+cu121
• Tokenizers 0.19.1