---
license: mit
base_model: microsoft/deberta-base
tags:
- generated_from_trainer
datasets:
- squad_v2
model-index:
- name: deberta-base-finetuned-squad2
  results: []
language:
- en
metrics:
- exact_match
- f1
pipeline_tag: question-answering
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

## Model description

DeBERTabase fine-tuned on SQuAD 2.0 : Encoder-based Transformer Language model.
DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder.
It outperforms BERT and RoBERTa on majority of NLU tasks with 80GB training data.<br>
Suitable for Question-Answering tasks, predicts answer spans within the context provided.<br>

**Language model:** microsoft/deberta-base  
**Language:** English  
**Downstream-task:** Question-Answering  
**Training data:** Train-set SQuAD 2.0  
**Evaluation data:** Evaluation-set SQuAD 2.0   
**Hardware Accelerator used**: GPU Tesla T4

## Intended uses & limitations

For Question-Answering - 

```python
!pip install transformers
from transformers import pipeline
model_checkpoint = "IProject-10/deberta-base-finetuned-squad2"
question_answerer = pipeline("question-answering", model=model_checkpoint)

context = """
🤗 Transformers is backed by the three most popular deep learning libraries — Jax, PyTorch and TensorFlow — with a seamless integration
between them. It's straightforward to train your models with one before loading them for inference with the other.
"""

question = "Which deep learning libraries back 🤗 Transformers?"
question_answerer(question=question, context=context)
```

## Results

Evaluation on SQuAD 2.0 validation dataset:

```
 exact: 81.03259496336226,
 f1: 84.42279239924598,
 total: 11873,
 HasAns_exact: 79.30161943319838,
 HasAns_f1: 86.09173653108105,
 HasAns_total: 5928,
 NoAns_exact: 82.75862068965517,
 NoAns_f1: 82.75862068965517,
 NoAns_total: 5945,
 best_exact: 81.03259496336226,
 best_exact_thresh: 0.9992604851722717,
 best_f1: 84.42279239924635,
 best_f1_thresh: 0.9992604851722717,
 total_time_in_seconds: 326.41847440000004,
 samples_per_second: 36.37355398411236,
 latency_in_seconds: 0.027492501844521185
```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 0.8054        | 1.0   | 8238  | 0.7902          |
| 0.5368        | 2.0   | 16476 | 0.7901          |
| 0.3845        | 3.0   | 24714 | 0.9334          |

This model is a fine-tuned version of [microsoft/deberta-base](https://huggingface.co/microsoft/deberta-base) on the squad_v2 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9334
  
### Framework versions

- Transformers 4.31.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.3
- Tokenizers 0.13.3