|
--- |
|
license: mit |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- precision |
|
- recall |
|
- f1 |
|
- accuracy |
|
model-index: |
|
- name: fedcsis-slot_baseline-xlm_r-es |
|
results: [] |
|
datasets: |
|
- cartesinus/leyzer-fedcsis |
|
language: |
|
- es |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# fedcsis-slot_baseline-xlm_r-es |
|
|
|
This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the |
|
[leyzer-fedcsis](https://huggingface.co/cartesinus/leyzer-fedcsis) dataset. |
|
|
|
Result on test set: |
|
- Precision: 0.9696 |
|
- Recall: 0.9686 |
|
- F1: 0.9691 |
|
- Accuracy: 0.9904 |
|
|
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.0521 |
|
- Precision: 0.9728 |
|
- Recall: 0.9711 |
|
- F1: 0.9720 |
|
- Accuracy: 0.9914 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 10 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:| |
|
| 0.7183 | 1.0 | 941 | 0.1287 | 0.9389 | 0.9429 | 0.9409 | 0.9802 | |
|
| 0.0792 | 2.0 | 1882 | 0.0698 | 0.9551 | 0.9609 | 0.9580 | 0.9876 | |
|
| 0.0502 | 3.0 | 2823 | 0.0586 | 0.9623 | 0.9624 | 0.9624 | 0.9886 | |
|
| 0.0312 | 4.0 | 3764 | 0.0511 | 0.9697 | 0.9668 | 0.9682 | 0.9904 | |
|
| 0.0229 | 5.0 | 4705 | 0.0494 | 0.9715 | 0.9687 | 0.9701 | 0.9913 | |
|
| 0.021 | 6.0 | 5646 | 0.0447 | 0.9697 | 0.9680 | 0.9689 | 0.9911 | |
|
| 0.0139 | 7.0 | 6587 | 0.0512 | 0.9715 | 0.9691 | 0.9703 | 0.9915 | |
|
| 0.0126 | 8.0 | 7528 | 0.0507 | 0.9713 | 0.9699 | 0.9706 | 0.9913 | |
|
| 0.01 | 9.0 | 8469 | 0.0500 | 0.9720 | 0.9702 | 0.9711 | 0.9913 | |
|
| 0.0072 | 10.0 | 9410 | 0.0521 | 0.9728 | 0.9711 | 0.9720 | 0.9914 | |
|
|
|
### Per slot evaluation |
|
|
|
| slot_name | precision | recall | f1 | tc_size | |
|
|-----------|-----------|--------|----|---------| |
|
| album | 0.9500 | 0.9135 | 0.9314 | 104 | |
|
| all_lang | 0.7500 | 1.0000 | 0.8571 | 3 | |
|
| artist | 0.9556 | 0.9685 | 0.9620 | 222 | |
|
| av_alias | 1.0000 | 1.0000 | 1.0000 | 18 | |
|
| caption | 0.9565 | 0.9362 | 0.9462 | 47 | |
|
| category | 0.9091 | 1.0000 | 0.9524 | 10 | |
|
| channel | 0.7857 | 0.7857 | 0.7857 | 14 | |
|
| channel_id | 0.9500 | 1.0000 | 0.9744 | 19 | |
|
| count | 1.0000 | 1.0000 | 1.0000 | 8 | |
|
| date | 0.9762 | 0.9762 | 0.9762 | 42 | |
|
| date_day | 1.0000 | 1.0000 | 1.0000 | 6 | |
|
| date_month | 1.0000 | 1.0000 | 1.0000 | 7 | |
|
| device_name | 0.9770 | 1.0000 | 0.9884 | 85 | |
|
| email | 1.0000 | 0.9740 | 0.9868 | 192 | |
|
| event_name | 1.0000 | 1.0000 | 1.0000 | 35 | |
|
| file_name | 1.0000 | 1.0000 | 1.0000 | 10 | |
|
| file_size | 1.0000 | 1.0000 | 1.0000 | 2 | |
|
| filter | 1.0000 | 1.0000 | 1.0000 | 15 | |
|
| hashtag | 1.0000 | 0.9565 | 0.9778 | 46 | |
|
| img_query | 0.9843 | 0.9843 | 0.9843 | 764 | |
|
| label | 1.0000 | 1.0000 | 1.0000 | 7 | |
|
| location | 0.9753 | 0.9875 | 0.9814 | 80 | |
|
| mail | 1.0000 | 1.0000 | 1.0000 | 5 | |
|
| message | 0.9577 | 0.9607 | 0.9592 | 636 | |
|
| mime_type | 1.0000 | 1.0000 | 1.0000 | 1 | |
|
| name | 0.9677 | 0.9677 | 0.9677 | 31 | |
|
| percent | 0.8571 | 1.0000 | 0.9231 | 6 | |
|
| phone_number | 0.9429 | 0.9763 | 0.9593 | 169 | |
|
| phone_type | 1.0000 | 0.6667 | 0.8000 | 3 | |
|
| picture_url | 1.0000 | 0.9286 | 0.9630 | 42 | |
|
| playlist | 0.9701 | 0.9630 | 0.9665 | 135 | |
|
| portal | 1.0000 | 0.9940 | 0.9970 | 168 | |
|
| priority | 1.0000 | 1.0000 | 1.0000 | 3 | |
|
| purpose | 0.0000 | 0.0000 | 0.0000 | 1 | |
|
| query | 0.9259 | 0.8929 | 0.9091 | 28 | |
|
| rating | 1.0000 | 1.0000 | 1.0000 | 3 | |
|
| review_count | 0.7500 | 0.7500 | 0.7500 | 4 | |
|
| section | 1.0000 | 1.0000 | 1.0000 | 134 | |
|
| seek_time | 1.0000 | 1.0000 | 1.0000 | 2 | |
|
| sender | 0.0000 | 0.0000 | 0.0000 | 1 | |
|
| sender_address | 1.0000 | 1.0000 | 1.0000 | 6 | |
|
| song | 0.9314 | 0.9628 | 0.9468 | 296 | |
|
| src_lang | 0.9872 | 1.0000 | 0.9935 | 77 | |
|
| status | 0.8462 | 0.9565 | 0.8980 | 23 | |
|
| subject | 0.9555 | 0.9567 | 0.9561 | 785 | |
|
| text | 0.9798 | 0.9798 | 0.9798 | 99 | |
|
| time | 1.0000 | 1.0000 | 1.0000 | 32 | |
|
| to | 0.9760 | 0.9651 | 0.9705 | 802 | |
|
| topic | 1.0000 | 1.0000 | 1.0000 | 1 | |
|
| translator | 1.0000 | 1.0000 | 1.0000 | 52 | |
|
| trg_lang | 0.9886 | 1.0000 | 0.9943 | 87 | |
|
| txt_query | 1.0000 | 0.8947 | 0.9444 | 19 | |
|
| username | 1.0000 | 1.0000 | 1.0000 | 6 | |
|
| value | 0.9318 | 0.9535 | 0.9425 | 43 | |
|
| weight | 1.0000 | 1.0000 | 1.0000 | 1 | |
|
|
|
### Framework versions |
|
|
|
- Transformers 4.27.4 |
|
- Pytorch 1.13.1+cu116 |
|
- Datasets 2.11.0 |
|
- Tokenizers 0.13.2 |
