scenario-kd-pre-ner-full-xlmr_data-univner_full55
This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.4608
- Precision: 0.8156
- Recall: 0.8175
- F1: 0.8165
- Accuracy: 0.9809
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 8
- eval_batch_size: 32
- seed: 55
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
Training results
Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
---|---|---|---|---|---|---|---|
1.4819 | 0.2911 | 500 | 0.9261 | 0.6821 | 0.6631 | 0.6725 | 0.9684 |
0.7602 | 0.5822 | 1000 | 0.7517 | 0.7360 | 0.7280 | 0.7320 | 0.9744 |
0.6658 | 0.8732 | 1500 | 0.6692 | 0.7309 | 0.7741 | 0.7519 | 0.9759 |
0.5797 | 1.1643 | 2000 | 0.6374 | 0.7530 | 0.7754 | 0.7640 | 0.9771 |
0.5073 | 1.4554 | 2500 | 0.6143 | 0.7486 | 0.7814 | 0.7646 | 0.9768 |
0.4981 | 1.7465 | 3000 | 0.5840 | 0.7764 | 0.7886 | 0.7825 | 0.9782 |
0.4623 | 2.0375 | 3500 | 0.5865 | 0.7928 | 0.7762 | 0.7844 | 0.9785 |
0.4163 | 2.3286 | 4000 | 0.5635 | 0.7767 | 0.7969 | 0.7866 | 0.9786 |
0.3968 | 2.6197 | 4500 | 0.5500 | 0.7826 | 0.7844 | 0.7835 | 0.9782 |
0.3882 | 2.9108 | 5000 | 0.5628 | 0.7999 | 0.7878 | 0.7938 | 0.9792 |
0.3671 | 3.2019 | 5500 | 0.5415 | 0.7851 | 0.8002 | 0.7926 | 0.9790 |
0.3478 | 3.4929 | 6000 | 0.5258 | 0.7988 | 0.8015 | 0.8001 | 0.9800 |
0.3402 | 3.7840 | 6500 | 0.5184 | 0.7948 | 0.8113 | 0.8029 | 0.9801 |
0.3272 | 4.0751 | 7000 | 0.5122 | 0.7873 | 0.8145 | 0.8007 | 0.9796 |
0.3138 | 4.3662 | 7500 | 0.5116 | 0.7927 | 0.7995 | 0.7961 | 0.9796 |
0.3078 | 4.6573 | 8000 | 0.5158 | 0.8017 | 0.7970 | 0.7994 | 0.9793 |
0.3041 | 4.9483 | 8500 | 0.4921 | 0.7932 | 0.8155 | 0.8042 | 0.9799 |
0.2874 | 5.2394 | 9000 | 0.5006 | 0.7984 | 0.8055 | 0.8019 | 0.9799 |
0.2805 | 5.5305 | 9500 | 0.4859 | 0.8075 | 0.8091 | 0.8083 | 0.9802 |
0.2803 | 5.8216 | 10000 | 0.4845 | 0.8046 | 0.8123 | 0.8084 | 0.9809 |
0.2738 | 6.1126 | 10500 | 0.4837 | 0.8033 | 0.8117 | 0.8075 | 0.9805 |
0.2618 | 6.4037 | 11000 | 0.4872 | 0.8089 | 0.8108 | 0.8099 | 0.9809 |
0.2625 | 6.6948 | 11500 | 0.4765 | 0.8049 | 0.8075 | 0.8062 | 0.9803 |
0.2588 | 6.9859 | 12000 | 0.4760 | 0.8104 | 0.8155 | 0.8129 | 0.9810 |
0.2471 | 7.2770 | 12500 | 0.4727 | 0.8030 | 0.8146 | 0.8088 | 0.9807 |
0.2448 | 7.5680 | 13000 | 0.4599 | 0.7999 | 0.8207 | 0.8101 | 0.9808 |
0.2448 | 7.8591 | 13500 | 0.4828 | 0.8114 | 0.8059 | 0.8087 | 0.9803 |
0.2422 | 8.1502 | 14000 | 0.4690 | 0.8170 | 0.8090 | 0.8130 | 0.9809 |
0.2368 | 8.4413 | 14500 | 0.4695 | 0.8105 | 0.8145 | 0.8125 | 0.9810 |
0.2356 | 8.7324 | 15000 | 0.4659 | 0.8076 | 0.8121 | 0.8099 | 0.9806 |
0.2312 | 9.0234 | 15500 | 0.4677 | 0.8194 | 0.8124 | 0.8159 | 0.9810 |
0.2333 | 9.3145 | 16000 | 0.4627 | 0.8115 | 0.8153 | 0.8134 | 0.9810 |
0.2279 | 9.6056 | 16500 | 0.4672 | 0.8111 | 0.8146 | 0.8128 | 0.9807 |
0.2299 | 9.8967 | 17000 | 0.4608 | 0.8156 | 0.8175 | 0.8165 | 0.9809 |
Framework versions
- Transformers 4.44.2
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.19.1
- Downloads last month
- 6
Model tree for haryoaw/scenario-kd-pre-ner-full-xlmr_data-univner_full55
Base model
FacebookAI/xlm-roberta-base