---
library_name: transformers
license: mit
base_model: FacebookAI/xlm-roberta-base
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: scenario-kd-pre-ner-full_data-univner_full66
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-kd-pre-ner-full_data-univner_full66

This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5549
- Precision: 0.7660
- Recall: 0.7319
- F1: 0.7485
- Accuracy: 0.9802

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 66
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-------:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 1.2484        | 1.2755  | 500   | 0.8737          | 0.6792    | 0.5631 | 0.6157 | 0.9709   |
| 0.6459        | 2.5510  | 1000  | 0.7190          | 0.6926    | 0.6739 | 0.6831 | 0.9771   |
| 0.5071        | 3.8265  | 1500  | 0.6650          | 0.7076    | 0.6863 | 0.6968 | 0.9773   |
| 0.4233        | 5.1020  | 2000  | 0.6513          | 0.6933    | 0.7019 | 0.6975 | 0.9775   |
| 0.3655        | 6.3776  | 2500  | 0.6252          | 0.7421    | 0.6822 | 0.7109 | 0.9778   |
| 0.3251        | 7.6531  | 3000  | 0.6172          | 0.7412    | 0.7174 | 0.7291 | 0.9791   |
| 0.2963        | 8.9286  | 3500  | 0.6204          | 0.7143    | 0.6677 | 0.6902 | 0.9773   |
| 0.2699        | 10.2041 | 4000  | 0.5919          | 0.7310    | 0.7288 | 0.7299 | 0.9792   |
| 0.2469        | 11.4796 | 4500  | 0.6168          | 0.7560    | 0.6863 | 0.7195 | 0.9788   |
| 0.2313        | 12.7551 | 5000  | 0.5871          | 0.7353    | 0.7133 | 0.7241 | 0.9792   |
| 0.2148        | 14.0306 | 5500  | 0.5947          | 0.7358    | 0.7122 | 0.7238 | 0.9794   |
| 0.2022        | 15.3061 | 6000  | 0.5830          | 0.7298    | 0.7019 | 0.7156 | 0.9790   |
| 0.1933        | 16.5816 | 6500  | 0.5734          | 0.7427    | 0.7143 | 0.7282 | 0.9794   |
| 0.185         | 17.8571 | 7000  | 0.5814          | 0.7352    | 0.6957 | 0.7149 | 0.9792   |
| 0.1767        | 19.1327 | 7500  | 0.5670          | 0.7516    | 0.7236 | 0.7373 | 0.9797   |
| 0.1688        | 20.4082 | 8000  | 0.5770          | 0.7551    | 0.6957 | 0.7241 | 0.9791   |
| 0.1634        | 21.6837 | 8500  | 0.5621          | 0.7443    | 0.7143 | 0.7290 | 0.9792   |
| 0.1592        | 22.9592 | 9000  | 0.5691          | 0.7495    | 0.7091 | 0.7287 | 0.9790   |
| 0.1538        | 24.2347 | 9500  | 0.5557          | 0.7481    | 0.7195 | 0.7335 | 0.9802   |
| 0.1513        | 25.5102 | 10000 | 0.5687          | 0.7446    | 0.7091 | 0.7264 | 0.9791   |
| 0.1489        | 26.7857 | 10500 | 0.5554          | 0.7623    | 0.7236 | 0.7424 | 0.9801   |
| 0.145         | 28.0612 | 11000 | 0.5488          | 0.7564    | 0.7329 | 0.7445 | 0.9804   |
| 0.144         | 29.3367 | 11500 | 0.5549          | 0.7660    | 0.7319 | 0.7485 | 0.9802   |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.1.1+cu121
- Datasets 2.14.5
- Tokenizers 0.19.1