---
library_name: transformers
license: mit
base_model: FacebookAI/xlm-roberta-base
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: scenario-non-kd-pre-ner-full-xlmr_data-univner_half44
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-non-kd-pre-ner-full-xlmr_data-univner_half44

This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1692
- Precision: 0.7997
- Recall: 0.8083
- F1: 0.8040
- Accuracy: 0.9794

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 44
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-------:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.1456        | 0.5828  | 500   | 0.0795          | 0.7321    | 0.7445 | 0.7383 | 0.9744   |
| 0.0686        | 1.1655  | 1000  | 0.0786          | 0.7478    | 0.7850 | 0.7660 | 0.9763   |
| 0.0505        | 1.7483  | 1500  | 0.0765          | 0.7499    | 0.8058 | 0.7768 | 0.9775   |
| 0.039         | 2.3310  | 2000  | 0.0850          | 0.7407    | 0.8025 | 0.7704 | 0.9759   |
| 0.0308        | 2.9138  | 2500  | 0.0935          | 0.7340    | 0.8178 | 0.7736 | 0.9755   |
| 0.0222        | 3.4965  | 3000  | 0.0941          | 0.7587    | 0.8097 | 0.7834 | 0.9776   |
| 0.0202        | 4.0793  | 3500  | 0.0978          | 0.7615    | 0.8139 | 0.7868 | 0.9781   |
| 0.0149        | 4.6620  | 4000  | 0.1010          | 0.7738    | 0.8039 | 0.7886 | 0.9780   |
| 0.0134        | 5.2448  | 4500  | 0.1072          | 0.7917    | 0.7973 | 0.7945 | 0.9791   |
| 0.0112        | 5.8275  | 5000  | 0.1023          | 0.7866    | 0.7948 | 0.7907 | 0.9789   |
| 0.0091        | 6.4103  | 5500  | 0.1151          | 0.7765    | 0.8083 | 0.7921 | 0.9784   |
| 0.0088        | 6.9930  | 6000  | 0.1144          | 0.7838    | 0.7980 | 0.7908 | 0.9785   |
| 0.0077        | 7.5758  | 6500  | 0.1150          | 0.7748    | 0.7979 | 0.7862 | 0.9780   |
| 0.0067        | 8.1585  | 7000  | 0.1191          | 0.7889    | 0.7960 | 0.7924 | 0.9787   |
| 0.0059        | 8.7413  | 7500  | 0.1269          | 0.7703    | 0.8140 | 0.7916 | 0.9780   |
| 0.0053        | 9.3240  | 8000  | 0.1267          | 0.7857    | 0.7944 | 0.7900 | 0.9784   |
| 0.0053        | 9.9068  | 8500  | 0.1273          | 0.7957    | 0.7928 | 0.7942 | 0.9788   |
| 0.0043        | 10.4895 | 9000  | 0.1321          | 0.7772    | 0.7990 | 0.7879 | 0.9782   |
| 0.0039        | 11.0723 | 9500  | 0.1313          | 0.7940    | 0.7945 | 0.7943 | 0.9789   |
| 0.0033        | 11.6550 | 10000 | 0.1361          | 0.7964    | 0.8031 | 0.7997 | 0.9788   |
| 0.0033        | 12.2378 | 10500 | 0.1394          | 0.7828    | 0.8057 | 0.7941 | 0.9785   |
| 0.0032        | 12.8205 | 11000 | 0.1445          | 0.7760    | 0.7928 | 0.7843 | 0.9781   |
| 0.0029        | 13.4033 | 11500 | 0.1358          | 0.7833    | 0.8083 | 0.7956 | 0.9789   |
| 0.003         | 13.9860 | 12000 | 0.1383          | 0.7830    | 0.8031 | 0.7929 | 0.9787   |
| 0.0024        | 14.5688 | 12500 | 0.1403          | 0.7731    | 0.8130 | 0.7925 | 0.9779   |
| 0.0028        | 15.1515 | 13000 | 0.1423          | 0.7998    | 0.7956 | 0.7977 | 0.9789   |
| 0.0021        | 15.7343 | 13500 | 0.1443          | 0.7831    | 0.8062 | 0.7945 | 0.9785   |
| 0.002         | 16.3170 | 14000 | 0.1396          | 0.7815    | 0.8088 | 0.7950 | 0.9785   |
| 0.0018        | 16.8998 | 14500 | 0.1469          | 0.7808    | 0.8072 | 0.7938 | 0.9783   |
| 0.0019        | 17.4825 | 15000 | 0.1450          | 0.7969    | 0.8029 | 0.7999 | 0.9790   |
| 0.0018        | 18.0653 | 15500 | 0.1516          | 0.7906    | 0.8098 | 0.8001 | 0.9788   |
| 0.0011        | 18.6480 | 16000 | 0.1591          | 0.7810    | 0.8093 | 0.7949 | 0.9787   |
| 0.0014        | 19.2308 | 16500 | 0.1561          | 0.8002    | 0.8051 | 0.8026 | 0.9791   |
| 0.0011        | 19.8135 | 17000 | 0.1568          | 0.7901    | 0.8093 | 0.7996 | 0.9788   |
| 0.0013        | 20.3963 | 17500 | 0.1573          | 0.7980    | 0.7950 | 0.7965 | 0.9790   |
| 0.001         | 20.9790 | 18000 | 0.1566          | 0.7987    | 0.7915 | 0.7951 | 0.9787   |
| 0.001         | 21.5618 | 18500 | 0.1594          | 0.7926    | 0.8041 | 0.7983 | 0.9787   |
| 0.001         | 22.1445 | 19000 | 0.1616          | 0.8068    | 0.7899 | 0.7983 | 0.9790   |
| 0.0011        | 22.7273 | 19500 | 0.1644          | 0.7908    | 0.8098 | 0.8002 | 0.9788   |
| 0.0007        | 23.3100 | 20000 | 0.1628          | 0.7870    | 0.8018 | 0.7943 | 0.9788   |
| 0.0008        | 23.8928 | 20500 | 0.1615          | 0.7960    | 0.7995 | 0.7977 | 0.9787   |
| 0.0006        | 24.4755 | 21000 | 0.1614          | 0.8002    | 0.7993 | 0.7998 | 0.9790   |
| 0.0007        | 25.0583 | 21500 | 0.1612          | 0.8001    | 0.7979 | 0.7990 | 0.9791   |
| 0.0005        | 25.6410 | 22000 | 0.1627          | 0.7940    | 0.8104 | 0.8021 | 0.9791   |
| 0.0005        | 26.2238 | 22500 | 0.1664          | 0.7945    | 0.8062 | 0.8003 | 0.9793   |
| 0.0005        | 26.8065 | 23000 | 0.1663          | 0.7871    | 0.8175 | 0.8020 | 0.9792   |
| 0.0004        | 27.3893 | 23500 | 0.1691          | 0.8037    | 0.8091 | 0.8064 | 0.9796   |
| 0.0005        | 27.9720 | 24000 | 0.1684          | 0.7979    | 0.8087 | 0.8032 | 0.9794   |
| 0.0003        | 28.5548 | 24500 | 0.1686          | 0.7999    | 0.8064 | 0.8031 | 0.9793   |
| 0.0004        | 29.1375 | 25000 | 0.1692          | 0.8005    | 0.8081 | 0.8043 | 0.9794   |
| 0.0003        | 29.7203 | 25500 | 0.1692          | 0.7997    | 0.8083 | 0.8040 | 0.9794   |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.19.1