File size: 3,171 Bytes
fc1a052 8aef9de fc1a052 8aef9de fc1a052 8aef9de fc1a052 8aef9de fc1a052 8aef9de fc1a052 8aef9de fc1a052 8aef9de |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
---
license: mit
base_model: microsoft/deberta-base
tags:
- generated_from_trainer
datasets:
- squad_v2
model-index:
- name: deberta-base-finetuned-squad2
results: []
language:
- en
metrics:
- exact_match
- f1
pipeline_tag: question-answering
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
## Model description
DeBERTabase fine-tuned on SQuAD 2.0 : Encoder-based Transformer Language model.
DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder.
It outperforms BERT and RoBERTa on majority of NLU tasks with 80GB training data.<br>
Suitable for Question-Answering tasks, predicts answer spans within the context provided.<br>
**Language model:** microsoft/deberta-base
**Language:** English
**Downstream-task:** Question-Answering
**Training data:** Train-set SQuAD 2.0
**Evaluation data:** Evaluation-set SQuAD 2.0
**Hardware Accelerator used**: GPU Tesla T4
## Intended uses & limitations
For Question-Answering -
```python
!pip install transformers
from transformers import pipeline
model_checkpoint = "IProject-10/deberta-base-finetuned-squad2"
question_answerer = pipeline("question-answering", model=model_checkpoint)
context = """
🤗 Transformers is backed by the three most popular deep learning libraries — Jax, PyTorch and TensorFlow — with a seamless integration
between them. It's straightforward to train your models with one before loading them for inference with the other.
"""
question = "Which deep learning libraries back 🤗 Transformers?"
question_answerer(question=question, context=context)
```
## Results
Evaluation on SQuAD 2.0 validation dataset:
```
exact: 81.03259496336226,
f1: 84.42279239924598,
total: 11873,
HasAns_exact: 79.30161943319838,
HasAns_f1: 86.09173653108105,
HasAns_total: 5928,
NoAns_exact: 82.75862068965517,
NoAns_f1: 82.75862068965517,
NoAns_total: 5945,
best_exact: 81.03259496336226,
best_exact_thresh: 0.9992604851722717,
best_f1: 84.42279239924635,
best_f1_thresh: 0.9992604851722717,
total_time_in_seconds: 326.41847440000004,
samples_per_second: 36.37355398411236,
latency_in_seconds: 0.027492501844521185
```
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
### Training results
| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 0.8054 | 1.0 | 8238 | 0.7902 |
| 0.5368 | 2.0 | 16476 | 0.7901 |
| 0.3845 | 3.0 | 24714 | 0.9334 |
This model is a fine-tuned version of [microsoft/deberta-base](https://huggingface.co/microsoft/deberta-base) on the squad_v2 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9334
### Framework versions
- Transformers 4.31.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.3
- Tokenizers 0.13.3 |