distilbert-mouse-enhancers

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6932
• Accuracy: 0.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	242	0.6932	0.5
No log	2.0	484	0.6949	0.5
0.693	3.0	726	0.6931	0.5
0.693	4.0	968	0.6931	0.5
0.694	5.0	1210	0.6932	0.5
0.694	6.0	1452	0.6935	0.5
0.6954	7.0	1694	0.6933	0.5
0.6954	8.0	1936	0.6932	0.5
0.6937	9.0	2178	0.6932	0.5
0.6937	10.0	2420	0.6932	0.5
0.6935	11.0	2662	0.6932	0.5
0.6935	12.0	2904	0.6934	0.5
0.6955	13.0	3146	0.6932	0.5
0.6955	14.0	3388	0.6931	0.5
0.6941	15.0	3630	0.6931	0.5
0.6941	16.0	3872	0.6932	0.5
0.6953	17.0	4114	0.6932	0.5
0.6953	18.0	4356	0.6931	0.5
0.6932	19.0	4598	0.6932	0.5
0.6932	20.0	4840	0.6931	0.5
0.6945	21.0	5082	0.6933	0.5
0.6945	22.0	5324	0.6932	0.5
0.6939	23.0	5566	0.6931	0.5
0.6939	24.0	5808	0.6931	0.5
0.6951	25.0	6050	0.6932	0.5
0.6951	26.0	6292	0.6931	0.5
0.6943	27.0	6534	0.6932	0.5
0.6943	28.0	6776	0.6931	0.5
0.6944	29.0	7018	0.6931	0.5
0.6944	30.0	7260	0.6932	0.5
0.6955	31.0	7502	0.6931	0.5
0.6955	32.0	7744	0.6933	0.5
0.6955	33.0	7986	0.6932	0.5
0.694	34.0	8228	0.6931	0.5
0.694	35.0	8470	0.6932	0.5
0.6937	36.0	8712	0.6932	0.5
0.6937	37.0	8954	0.6931	0.5
0.6923	38.0	9196	0.6932	0.5
0.6923	39.0	9438	0.6932	0.5
0.6931	40.0	9680	0.6931	0.5
0.6931	41.0	9922	0.6932	0.5
0.6937	42.0	10164	0.6932	0.5
0.6937	43.0	10406	0.6932	0.5
0.6936	44.0	10648	0.6932	0.5
0.6936	45.0	10890	0.6932	0.5
0.6933	46.0	11132	0.6932	0.5
0.6933	47.0	11374	0.6932	0.5
0.6924	48.0	11616	0.6932	0.5
0.6924	49.0	11858	0.6932	0.5
0.6929	50.0	12100	0.6932	0.5

Framework versions

• Transformers 4.26.1
• Pytorch 2.0.0+cu117
• Datasets 2.19.0
• Tokenizers 0.13.3