metadata

license: apache-2.0
base_model: facebook/hubert-base-ls960
tags:
  - generated_from_trainer
metrics:
  - accuracy
  - precision
  - recall
  - f1
model-index:
  - name: hubert-classifier-aug-fold-0
    results: []

hubert-classifier-aug-fold-0

This model is a fine-tuned version of facebook/hubert-base-ls960 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.5649
Accuracy: 0.8679
Precision: 0.8791
Recall: 0.8679
F1: 0.8667
Binary: 0.9082

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 30
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1	Binary
No log	0.13	50	3.8775	0.0512	0.0179	0.0512	0.0103	0.3205
No log	0.27	100	3.4967	0.0526	0.0224	0.0526	0.0156	0.3290
No log	0.4	150	3.2301	0.0970	0.0222	0.0970	0.0291	0.3659
No log	0.54	200	3.0363	0.1361	0.0437	0.1361	0.0582	0.3920
No log	0.67	250	2.7181	0.2385	0.1581	0.2385	0.1495	0.4625
No log	0.81	300	2.3875	0.3019	0.1996	0.3019	0.2063	0.5101
No log	0.94	350	2.1936	0.3895	0.3562	0.3895	0.3154	0.5706
3.2299	1.08	400	1.9583	0.4555	0.3983	0.4555	0.3887	0.6168
3.2299	1.21	450	1.7394	0.5	0.4895	0.5	0.4446	0.6497
3.2299	1.35	500	1.5140	0.6065	0.6096	0.6065	0.5686	0.7244
3.2299	1.48	550	1.3457	0.6361	0.6197	0.6361	0.5975	0.7460
3.2299	1.62	600	1.2439	0.6644	0.6718	0.6644	0.6320	0.7644
3.2299	1.75	650	1.0984	0.7035	0.6871	0.7035	0.6718	0.7927
3.2299	1.89	700	1.0212	0.7480	0.7557	0.7480	0.7313	0.8225
1.7016	2.02	750	0.9402	0.7412	0.7504	0.7412	0.7217	0.8187
1.7016	2.16	800	0.9117	0.7412	0.7498	0.7412	0.7285	0.8178
1.7016	2.29	850	0.8195	0.7763	0.8018	0.7763	0.7704	0.8439
1.7016	2.43	900	0.7736	0.7898	0.8195	0.7898	0.7817	0.8531
1.7016	2.56	950	0.7723	0.7965	0.8227	0.7965	0.7855	0.8597
1.7016	2.7	1000	0.7737	0.7857	0.8131	0.7857	0.7780	0.8512
1.7016	2.83	1050	0.6884	0.8113	0.8281	0.8113	0.8057	0.8679
1.7016	2.97	1100	0.7205	0.8032	0.8297	0.8032	0.7987	0.8646
1.0665	3.1	1150	0.6086	0.8342	0.8483	0.8342	0.8297	0.8864
1.0665	3.24	1200	0.5848	0.8531	0.8690	0.8531	0.8503	0.8991
1.0665	3.37	1250	0.6821	0.8369	0.8551	0.8369	0.8315	0.8861
1.0665	3.51	1300	0.6050	0.8315	0.8491	0.8315	0.8287	0.8833
1.0665	3.64	1350	0.5871	0.8504	0.8708	0.8504	0.8477	0.8960
1.0665	3.78	1400	0.6485	0.8329	0.8560	0.8329	0.8298	0.8837
1.0665	3.91	1450	0.5727	0.8518	0.8673	0.8518	0.8484	0.8970
0.7598	4.05	1500	0.5555	0.8652	0.8752	0.8652	0.8623	0.9069
0.7598	4.18	1550	0.5407	0.8585	0.8702	0.8585	0.8558	0.9031
0.7598	4.32	1600	0.5282	0.8679	0.8813	0.8679	0.8658	0.9082
0.7598	4.45	1650	0.5557	0.8558	0.8693	0.8558	0.8540	0.9008
0.7598	4.59	1700	0.5832	0.8693	0.8866	0.8693	0.8686	0.9097
0.7598	4.72	1750	0.5623	0.8720	0.8818	0.8720	0.8707	0.9097
0.7598	4.86	1800	0.5598	0.8612	0.8811	0.8612	0.8614	0.9040
0.7598	4.99	1850	0.6415	0.8504	0.8627	0.8504	0.8479	0.8970
0.5874	5.12	1900	0.5866	0.8652	0.8788	0.8652	0.8620	0.9074
0.5874	5.26	1950	0.6092	0.8612	0.8725	0.8612	0.8603	0.9036
0.5874	5.39	2000	0.5544	0.8733	0.8870	0.8733	0.8730	0.9125
0.5874	5.53	2050	0.5478	0.8598	0.8726	0.8598	0.8591	0.9026
0.5874	5.66	2100	0.5683	0.8733	0.8908	0.8733	0.8713	0.9116
0.5874	5.8	2150	0.5644	0.8679	0.8782	0.8679	0.8643	0.9102
0.5874	5.93	2200	0.5471	0.8666	0.8759	0.8666	0.8639	0.9093
0.4963	6.07	2250	0.6050	0.8612	0.8757	0.8612	0.8603	0.9044
0.4963	6.2	2300	0.5908	0.8625	0.8770	0.8625	0.8613	0.9054
0.4963	6.34	2350	0.5994	0.8598	0.8773	0.8598	0.8589	0.9036

Framework versions

Transformers 4.38.2
Pytorch 2.3.0
Datasets 2.19.1
Tokenizers 0.15.1