plus2_model

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7698
• Eader: {'precision': 0.3232876712328767, 'recall': 0.2532188841201717, 'f1': 0.28399518652226236, 'number': 466}
• Nswer: {'precision': 0.43698252069917204, 'recall': 0.38183279742765275, 'f1': 0.4075504075504076, 'number': 1244}
• Uestion: {'precision': 0.392904953145917, 'recall': 0.425670775924583, 'f1': 0.40863209189001043, 'number': 1379}
• Overall Precision: 0.4005
• Overall Recall: 0.3820
• Overall F1: 0.3911
• Overall Accuracy: 0.5776

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500

Training results

Training Loss	Epoch	Step	Validation Loss	Eader	Nswer	Uestion	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.1902	0.7	200	0.2840	{'precision': 0.20205479452054795, 'recall': 0.12660944206008584, 'f1': 0.15567282321899734, 'number': 466}	{'precision': 0.2560738581146744, 'recall': 0.42363344051446944, 'f1': 0.3192004845548152, 'number': 1244}	{'precision': 0.3292753623188406, 'recall': 0.41189267585206674, 'f1': 0.36597938144329895, 'number': 1379}	0.2832	0.3736	0.3222	0.6408
0.1315	1.4	400	0.3785	{'precision': 0.2557544757033248, 'recall': 0.2145922746781116, 'f1': 0.23337222870478413, 'number': 466}	{'precision': 0.3143491124260355, 'recall': 0.34163987138263663, 'f1': 0.327426810477658, 'number': 1244}	{'precision': 0.31990924560408396, 'recall': 0.40899202320522116, 'f1': 0.35900700190961177, 'number': 1379}	0.3106	0.3525	0.3303	0.5741
0.1117	2.11	600	0.5260	{'precision': 0.33003300330033003, 'recall': 0.2145922746781116, 'f1': 0.2600780234070221, 'number': 466}	{'precision': 0.37556973564266183, 'recall': 0.3311897106109325, 'f1': 0.35198633062793677, 'number': 1244}	{'precision': 0.37993311036789296, 'recall': 0.41189267585206674, 'f1': 0.39526791927627, 'number': 1379}	0.3731	0.3496	0.3610	0.5467
0.0824	2.81	800	0.4716	{'precision': 0.29289940828402367, 'recall': 0.21244635193133046, 'f1': 0.2462686567164179, 'number': 466}	{'precision': 0.38620689655172413, 'recall': 0.36012861736334406, 'f1': 0.3727121464226289, 'number': 1244}	{'precision': 0.36801541425818884, 'recall': 0.41551849166062366, 'f1': 0.3903269754768393, 'number': 1379}	0.3666	0.3626	0.3646	0.5599
0.0654	3.51	1000	0.3955	{'precision': 0.3408360128617363, 'recall': 0.22746781115879827, 'f1': 0.2728442728442728, 'number': 466}	{'precision': 0.3290094339622642, 'recall': 0.44855305466237944, 'f1': 0.37959183673469393, 'number': 1244}	{'precision': 0.3595634692705342, 'recall': 0.45395213923132705, 'f1': 0.4012820512820513, 'number': 1379}	0.3442	0.4176	0.3774	0.6413
0.0701	4.21	1200	0.5372	{'precision': 0.3501577287066246, 'recall': 0.23819742489270387, 'f1': 0.2835249042145594, 'number': 466}	{'precision': 0.39559543230016314, 'recall': 0.38987138263665594, 'f1': 0.3927125506072874, 'number': 1244}	{'precision': 0.3765156349712827, 'recall': 0.4278462654097172, 'f1': 0.40054310930074677, 'number': 1379}	0.3814	0.3839	0.3826	0.5792
0.048	4.91	1400	0.7393	{'precision': 0.37168141592920356, 'recall': 0.18025751072961374, 'f1': 0.24277456647398846, 'number': 466}	{'precision': 0.4155972359328727, 'recall': 0.3384244372990354, 'f1': 0.3730615861763403, 'number': 1244}	{'precision': 0.39614855570839064, 'recall': 0.4176939811457578, 'f1': 0.4066360748323332, 'number': 1379}	0.4014	0.3500	0.3739	0.5258
0.0372	5.61	1600	0.6617	{'precision': 0.34726688102893893, 'recall': 0.2317596566523605, 'f1': 0.277992277992278, 'number': 466}	{'precision': 0.390325271059216, 'recall': 0.3762057877813505, 'f1': 0.38313548915268114, 'number': 1244}	{'precision': 0.39173228346456695, 'recall': 0.43292240754169686, 'f1': 0.4112986565621771, 'number': 1379}	0.3866	0.3797	0.3831	0.5713
0.0347	6.32	1800	0.7866	{'precision': 0.3680297397769517, 'recall': 0.21244635193133046, 'f1': 0.2693877551020408, 'number': 466}	{'precision': 0.42815814850530376, 'recall': 0.35691318327974275, 'f1': 0.38930293730819815, 'number': 1244}	{'precision': 0.39893617021276595, 'recall': 0.43509789702683105, 'f1': 0.4162330905306972, 'number': 1379}	0.4068	0.3700	0.3875	0.5454
0.0278	7.02	2000	0.6686	{'precision': 0.34110787172011664, 'recall': 0.2510729613733906, 'f1': 0.2892459826946848, 'number': 466}	{'precision': 0.4066115702479339, 'recall': 0.3954983922829582, 'f1': 0.40097799511002447, 'number': 1244}	{'precision': 0.39158163265306123, 'recall': 0.4452501812907904, 'f1': 0.4166949440108585, 'number': 1379}	0.3919	0.3959	0.3939	0.6009
0.0233	7.72	2200	0.7673	{'precision': 0.2876712328767123, 'recall': 0.2703862660944206, 'f1': 0.2787610619469027, 'number': 466}	{'precision': 0.4287020109689214, 'recall': 0.3770096463022508, 'f1': 0.4011976047904192, 'number': 1244}	{'precision': 0.38976109215017063, 'recall': 0.41406816533720087, 'f1': 0.4015471167369901, 'number': 1379}	0.3891	0.3775	0.3832	0.5676
0.0205	8.42	2400	0.7698	{'precision': 0.3232876712328767, 'recall': 0.2532188841201717, 'f1': 0.28399518652226236, 'number': 466}	{'precision': 0.43698252069917204, 'recall': 0.38183279742765275, 'f1': 0.4075504075504076, 'number': 1244}	{'precision': 0.392904953145917, 'recall': 0.425670775924583, 'f1': 0.40863209189001043, 'number': 1379}	0.4005	0.3820	0.3911	0.5776

Framework versions

• Transformers 4.34.0
• Pytorch 2.1.0.dev20230810
• Datasets 2.14.4
• Tokenizers 0.14.1