--- license: apache-2.0 base_model: facebook/hubert-base-ls960 tags: - generated_from_trainer metrics: - accuracy - precision - recall - f1 model-index: - name: hubert-classifier-aug-fold-3 results: [] --- # hubert-classifier-aug-fold-3 This model is a fine-tuned version of [facebook/hubert-base-ls960](https://huggingface.co/facebook/hubert-base-ls960) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.6214 - Accuracy: 0.8544 - Precision: 0.8720 - Recall: 0.8544 - F1: 0.8540 - Binary: 0.8989 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 32 - eval_batch_size: 32 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 128 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 500 - num_epochs: 100 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1 | Binary | |:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|:------:| | No log | 0.24 | 50 | 4.4176 | 0.0172 | 0.0140 | 0.0172 | 0.0083 | 0.1549 | | No log | 0.48 | 100 | 4.3072 | 0.0434 | 0.0497 | 0.0434 | 0.0197 | 0.2778 | | No log | 0.72 | 150 | 3.9604 | 0.0906 | 0.0606 | 0.0906 | 0.0446 | 0.3581 | | No log | 0.96 | 200 | 3.6355 | 0.1191 | 0.0507 | 0.1191 | 0.0558 | 0.3783 | | 4.235 | 1.2 | 250 | 3.3489 | 0.1700 | 0.0800 | 0.1700 | 0.0858 | 0.4157 | | 4.235 | 1.44 | 300 | 3.1008 | 0.2015 | 0.1270 | 0.2015 | 0.1158 | 0.4382 | | 4.235 | 1.68 | 350 | 2.8220 | 0.2906 | 0.2075 | 0.2906 | 0.2012 | 0.5014 | | 4.235 | 1.92 | 400 | 2.5557 | 0.3843 | 0.3160 | 0.3843 | 0.3099 | 0.5671 | | 3.2055 | 2.16 | 450 | 2.1790 | 0.4801 | 0.4047 | 0.4801 | 0.4036 | 0.6344 | | 3.2055 | 2.4 | 500 | 1.9034 | 0.5790 | 0.5557 | 0.5790 | 0.5261 | 0.7028 | | 3.2055 | 2.63 | 550 | 1.6707 | 0.6135 | 0.6116 | 0.6135 | 0.5701 | 0.7273 | | 3.2055 | 2.87 | 600 | 1.4658 | 0.6285 | 0.6047 | 0.6285 | 0.5817 | 0.7381 | | 2.1878 | 3.11 | 650 | 1.3665 | 0.6457 | 0.6522 | 0.6457 | 0.6153 | 0.7534 | | 2.1878 | 3.35 | 700 | 1.2309 | 0.6757 | 0.6806 | 0.6757 | 0.6446 | 0.7730 | | 2.1878 | 3.59 | 750 | 1.1077 | 0.7169 | 0.7307 | 0.7169 | 0.6966 | 0.7999 | | 2.1878 | 3.83 | 800 | 1.0393 | 0.7341 | 0.7548 | 0.7341 | 0.7226 | 0.8130 | | 1.534 | 4.07 | 850 | 0.9478 | 0.7678 | 0.7794 | 0.7678 | 0.7572 | 0.8384 | | 1.534 | 4.31 | 900 | 0.8755 | 0.7715 | 0.7789 | 0.7715 | 0.7627 | 0.8395 | | 1.534 | 4.55 | 950 | 0.8563 | 0.7618 | 0.7737 | 0.7618 | 0.7491 | 0.8330 | | 1.534 | 4.79 | 1000 | 0.7866 | 0.8007 | 0.8046 | 0.8007 | 0.7921 | 0.8616 | | 1.2035 | 5.03 | 1050 | 0.7462 | 0.8007 | 0.8212 | 0.8007 | 0.7945 | 0.8591 | | 1.2035 | 5.27 | 1100 | 0.7003 | 0.8157 | 0.8272 | 0.8157 | 0.8126 | 0.8717 | | 1.2035 | 5.51 | 1150 | 0.7421 | 0.8105 | 0.8262 | 0.8105 | 0.8074 | 0.8672 | | 1.2035 | 5.75 | 1200 | 0.7638 | 0.7993 | 0.8294 | 0.7993 | 0.7979 | 0.8595 | | 1.2035 | 5.99 | 1250 | 0.6872 | 0.8187 | 0.8330 | 0.8187 | 0.8171 | 0.8742 | | 0.9909 | 6.23 | 1300 | 0.6378 | 0.8345 | 0.8462 | 0.8345 | 0.8338 | 0.8840 | | 0.9909 | 6.47 | 1350 | 0.6835 | 0.8075 | 0.8266 | 0.8075 | 0.8063 | 0.8669 | | 0.9909 | 6.71 | 1400 | 0.6367 | 0.8345 | 0.8480 | 0.8345 | 0.8337 | 0.8874 | | 0.9909 | 6.95 | 1450 | 0.5793 | 0.8434 | 0.8521 | 0.8434 | 0.8425 | 0.8931 | | 0.8826 | 7.19 | 1500 | 0.6528 | 0.8307 | 0.8458 | 0.8307 | 0.8293 | 0.8824 | | 0.8826 | 7.43 | 1550 | 0.6361 | 0.8225 | 0.8382 | 0.8225 | 0.8218 | 0.8761 | | 0.8826 | 7.66 | 1600 | 0.6189 | 0.8360 | 0.8478 | 0.8360 | 0.8334 | 0.8855 | | 0.8826 | 7.9 | 1650 | 0.6078 | 0.8337 | 0.8433 | 0.8337 | 0.8321 | 0.8831 | | 0.7752 | 8.14 | 1700 | 0.6868 | 0.8315 | 0.8436 | 0.8315 | 0.8289 | 0.8835 | | 0.7752 | 8.38 | 1750 | 0.6118 | 0.8419 | 0.8549 | 0.8419 | 0.8411 | 0.8897 | | 0.7752 | 8.62 | 1800 | 0.5837 | 0.8532 | 0.8660 | 0.8532 | 0.8531 | 0.8974 | | 0.7752 | 8.86 | 1850 | 0.5758 | 0.8487 | 0.8613 | 0.8487 | 0.8494 | 0.8956 | | 0.7067 | 9.1 | 1900 | 0.6950 | 0.8307 | 0.8490 | 0.8307 | 0.8279 | 0.8827 | | 0.7067 | 9.34 | 1950 | 0.5968 | 0.8479 | 0.8595 | 0.8479 | 0.8470 | 0.8942 | | 0.7067 | 9.58 | 2000 | 0.5714 | 0.8614 | 0.8696 | 0.8614 | 0.8613 | 0.9035 | | 0.7067 | 9.82 | 2050 | 0.6389 | 0.8427 | 0.8538 | 0.8427 | 0.8415 | 0.8903 | | 0.6457 | 10.06 | 2100 | 0.6504 | 0.8502 | 0.8639 | 0.8502 | 0.8504 | 0.8948 | | 0.6457 | 10.3 | 2150 | 0.5776 | 0.8547 | 0.8659 | 0.8547 | 0.8534 | 0.8988 | | 0.6457 | 10.54 | 2200 | 0.6775 | 0.8434 | 0.8570 | 0.8434 | 0.8438 | 0.8912 | | 0.6457 | 10.78 | 2250 | 0.5849 | 0.8569 | 0.8686 | 0.8569 | 0.8579 | 0.9013 | | 0.6098 | 11.02 | 2300 | 0.5767 | 0.8622 | 0.8706 | 0.8622 | 0.8632 | 0.9037 | | 0.6098 | 11.26 | 2350 | 0.6875 | 0.8404 | 0.8588 | 0.8404 | 0.8404 | 0.8898 | | 0.6098 | 11.5 | 2400 | 0.7397 | 0.8352 | 0.8483 | 0.8352 | 0.8340 | 0.8865 | | 0.6098 | 11.74 | 2450 | 0.5998 | 0.8629 | 0.8716 | 0.8629 | 0.8618 | 0.9053 | | 0.6098 | 11.98 | 2500 | 0.6435 | 0.8449 | 0.8549 | 0.8449 | 0.8441 | 0.8918 | | 0.5538 | 12.22 | 2550 | 0.6969 | 0.8502 | 0.8640 | 0.8502 | 0.8508 | 0.8965 | | 0.5538 | 12.46 | 2600 | 0.6323 | 0.8577 | 0.8710 | 0.8577 | 0.8566 | 0.9006 | | 0.5538 | 12.69 | 2650 | 0.6989 | 0.8532 | 0.8660 | 0.8532 | 0.8525 | 0.8981 | | 0.5538 | 12.93 | 2700 | 0.6736 | 0.8554 | 0.8666 | 0.8554 | 0.8552 | 0.8994 | | 0.5356 | 13.17 | 2750 | 0.6737 | 0.8487 | 0.8584 | 0.8487 | 0.8469 | 0.8960 | | 0.5356 | 13.41 | 2800 | 0.6893 | 0.8457 | 0.8565 | 0.8457 | 0.8452 | 0.8921 | ### Framework versions - Transformers 4.38.2 - Pytorch 2.3.0 - Datasets 2.19.1 - Tokenizers 0.15.1