cartesinus
/

fedcsis-slot_baseline-xlm_r-pl

@@ -10,6 +10,10 @@ metrics:
 model-index:
 - name: fedcsis-slot_baseline-xlm_r-pl
   results: []
 ---
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -17,7 +21,15 @@ should probably proofread and complete it, then remove this comment. -->
 # fedcsis-slot_baseline-xlm_r-pl
-This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the None dataset.
 It achieves the following results on the evaluation set:
 - Loss: 0.1009
 - Precision: 0.9579
@@ -65,10 +77,92 @@ The following hyperparameters were used during training:
 | 0.0168        | 9.0   | 7182 | 0.1009          | 0.9577    | 0.9516 | 0.9546 | 0.9861   |
 | 0.016         | 10.0  | 7980 | 0.1009          | 0.9579    | 0.9512 | 0.9546 | 0.9860   |
 ### Framework versions
 - Transformers 4.27.4
 - Pytorch 1.13.1+cu116
 - Datasets 2.11.0
-- Tokenizers 0.13.2

 model-index:
 - name: fedcsis-slot_baseline-xlm_r-pl
   results: []
+datasets:
+- cartesinus/leyzer-fedcsis
+language:
+- pl
 ---
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 # fedcsis-slot_baseline-xlm_r-pl
+This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the
+[leyzer-fedcsis](https://huggingface.co/cartesinus/leyzer-fedcsis) dataset.
+Results on test set:
+- Precision: 0.9621
+- Recall: 0.9583
+- F1: 0.9602
+- Accuracy: 0.9857
 It achieves the following results on the evaluation set:
 - Loss: 0.1009
 - Precision: 0.9579
 | 0.0168        | 9.0   | 7182 | 0.1009          | 0.9577    | 0.9516 | 0.9546 | 0.9861   |
 | 0.016         | 10.0  | 7980 | 0.1009          | 0.9579    | 0.9512 | 0.9546 | 0.9860   |
+### Per slot evaluation
+| slot_name | precision | recall | f1 | tc_size |
+|-----------|-----------|--------|----|---------|
+| album | 0.2000 | 0.3333 | 0.2500 | 9 |
+| all_lang | 1.0000 | 1.0000 | 1.0000 | 5 |
+| artist | 0.9341 | 0.9444 | 0.9392 | 90 |
+| av_alias | 0.6667 | 0.8000 | 0.7273 | 5 |
+| caption | 0.9651 | 0.9432 | 0.9540 | 88 |
+| category | 0.0000 | 0.0000 | 0.0000 | 1 |
+| category_a | 1.0000 | 0.9167 | 0.9565 | 12 |
+| category_b | 1.0000 | 1.0000 | 1.0000 | 25 |
+| channel | 0.9492 | 0.9333 | 0.9412 | 60 |
+| channel_id | 0.9701 | 0.9644 | 0.9673 | 337 |
+| count | 1.0000 | 0.9167 | 0.9565 | 12 |
+| date | 0.9764 | 0.9841 | 0.9802 | 126 |
+| date_day | 1.0000 | 0.9500 | 0.9744 | 20 |
+| date_month | 0.9677 | 1.0000 | 0.9836 | 30 |
+| device_name | 0.9091 | 1.0000 | 0.9524 | 10 |
+| email | 1.0000 | 0.9913 | 0.9956 | 115 |
+| event_name | 0.8788 | 0.9355 | 0.9063 | 31 |
+| file_name | 0.9778 | 0.9778 | 0.9778 | 45 |
+| file_size | 1.0000 | 1.0000 | 1.0000 | 12 |
+| filename | 0.9722 | 0.9589 | 0.9655 | 73 |
+| filter | 1.0000 | 1.0000 | 1.0000 | 35 |
+| from | 0.9811 | 0.9123 | 0.9455 | 57 |
+| hashtag | 1.0000 | 1.0000 | 1.0000 | 28 |
+| img_query | 0.9707 | 0.9678 | 0.9693 | 342 |
+| label | 1.0000 | 1.0000 | 1.0000 | 5 |
+| location | 0.9766 | 0.9728 | 0.9747 | 257 |
+| mail | 1.0000 | 1.0000 | 1.0000 | 3 |
+| message | 0.9250 | 0.9487 | 0.9367 | 117 |
+| mime_type | 0.9375 | 1.0000 | 0.9677 | 15 |
+| name | 0.9412 | 0.9796 | 0.9600 | 49 |
+| pathname | 0.8889 | 0.8889 | 0.8889 | 18 |
+| percent | 1.0000 | 1.0000 | 1.0000 | 3 |
+| phone_number | 0.9774 | 0.9774 | 0.9774 | 177 |
+| phone_type | 1.0000 | 1.0000 | 1.0000 | 21 |
+| picture_url | 0.9846 | 0.9412 | 0.9624 | 68 |
+| playlist | 0.9516 | 0.9672 | 0.9593 | 122 |
+| portal | 0.9869 | 0.9869 | 0.9869 | 153 |
+| priority | 0.7500 | 1.0000 | 0.8571 | 6 |
+| purpose | 0.0000 | 0.0000 | 0.0000 | 5 |
+| query | 0.9663 | 0.9690 | 0.9677 | 355 |
+| rating | 0.9630 | 0.9286 | 0.9455 | 28 |
+| review_count | 1.0000 | 1.0000 | 1.0000 | 20 |
+| section | 0.9730 | 0.9730 | 0.9730 | 74 |
+| seek_time | 1.0000 | 1.0000 | 1.0000 | 3 |
+| sender | 1.0000 | 1.0000 | 1.0000 | 6 |
+| sender_address | 1.0000 | 0.9444 | 0.9714 | 18 |
+| song | 0.8824 | 0.8898 | 0.8861 | 118 |
+| src_lang_de | 0.9880 | 0.9762 | 0.9820 | 84 |
+| src_lang_en | 0.9455 | 0.9630 | 0.9541 | 54 |
+| src_lang_es | 0.9853 | 0.9306 | 0.9571 | 72 |
+| src_lang_fr | 0.9733 | 0.9733 | 0.9733 | 75 |
+| src_lang_it | 0.9872 | 0.9506 | 0.9686 | 81 |
+| src_lang_pl | 0.9818 | 1.0000 | 0.9908 | 54 |
+| status | 0.8810 | 0.9487 | 0.9136 | 39 |
+| subject | 0.9636 | 0.9725 | 0.9680 | 109 |
+| text_de | 0.9762 | 0.9762 | 0.9762 | 84 |
+| text_en | 0.9796 | 0.9697 | 0.9746 | 99 |
+| text_es | 0.8734 | 0.9583 | 0.9139 | 72 |
+| text_fr | 0.9733 | 0.9733 | 0.9733 | 75 |
+| text_it | 0.9872 | 0.9506 | 0.9686 | 81 |
+| text_multi | 0.0000 | 0.0000 | 0.0000 | 4 |
+| text_pl | 0.9310 | 1.0000 | 0.9643 | 54 |
+| time | 0.9063 | 0.8788 | 0.8923 | 33 |
+| to | 0.9648 | 0.9648 | 0.9648 | 199 |
+| topic | 0.0000 | 0.0000 | 0.0000 | 3 |
+| translator | 0.9838 | 0.9838 | 0.9838 | 185 |
+| trg_lang_de | 0.9474 | 0.9730 | 0.9600 | 37 |
+| trg_lang_en | 1.0000 | 0.9565 | 0.9778 | 46 |
+| trg_lang_es | 0.9792 | 0.9792 | 0.9792 | 48 |
+| trg_lang_fr | 0.9808 | 1.0000 | 0.9903 | 51 |
+| trg_lang_general | 0.9500 | 0.9500 | 0.9500 | 20 |
+| trg_lang_it | 0.9825 | 0.9492 | 0.9655 | 59 |
+| trg_lang_pl | 0.9302 | 0.9756 | 0.9524 | 41 |
+| txt_query | 0.9375 | 0.9146 | 0.9259 | 82 |
+| username | 0.9615 | 0.8929 | 0.9259 | 28 |
+| value | 0.8750 | 0.8750 | 0.8750 | 8 |
+| weight | 1.0000 | 1.0000 | 1.0000 | 3 |
 ### Framework versions
 - Transformers 4.27.4
 - Pytorch 1.13.1+cu116
 - Datasets 2.11.0
+- Tokenizers 0.13.2