---
license: apache-2.0
base_model: google/electra-base-generator
tags:
- generated_from_trainer
datasets:
- datasets/all_binary_and_xe_ey_fae_counterfactual
metrics:
- accuracy
model-index:
- name: electra-base-finetuned-xe_ey_fae
  results:
  - task:
      name: Masked Language Modeling
      type: fill-mask
    dataset:
      name: datasets/all_binary_and_xe_ey_fae_counterfactual
      type: datasets/all_binary_and_xe_ey_fae_counterfactual
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.667333329363415
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# electra-base-finetuned-xe_ey_fae

This model is a fine-tuned version of [google/electra-base-generator](https://huggingface.co/google/electra-base-generator) on the datasets/all_binary_and_xe_ey_fae_counterfactual dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7211
- Accuracy: 0.6673

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 100
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 2.5359        | 0.06  | 500   | 2.0696          | 0.6228   |
| 2.1807        | 0.13  | 1000  | 1.9677          | 0.6352   |
| 2.1028        | 0.19  | 1500  | 1.9192          | 0.6415   |
| 2.0658        | 0.26  | 2000  | 1.8923          | 0.6451   |
| 2.0426        | 0.32  | 2500  | 1.8699          | 0.6478   |
| 2.0133        | 0.39  | 3000  | 1.8580          | 0.6490   |
| 1.9978        | 0.45  | 3500  | 1.8411          | 0.6507   |
| 1.9862        | 0.52  | 4000  | 1.8297          | 0.6524   |
| 1.9745        | 0.58  | 4500  | 1.8154          | 0.6545   |
| 1.9606        | 0.64  | 5000  | 1.8056          | 0.6557   |
| 1.9486        | 0.71  | 5500  | 1.8033          | 0.6560   |
| 1.9416        | 0.77  | 6000  | 1.7894          | 0.6581   |
| 1.9279        | 0.84  | 6500  | 1.7848          | 0.6582   |
| 1.9196        | 0.9   | 7000  | 1.7786          | 0.6593   |
| 1.9168        | 0.97  | 7500  | 1.7762          | 0.6592   |
| 1.9123        | 1.03  | 8000  | 1.7744          | 0.6597   |
| 1.8942        | 1.1   | 8500  | 1.7625          | 0.6611   |
| 1.9053        | 1.16  | 9000  | 1.7576          | 0.6623   |
| 1.898         | 1.22  | 9500  | 1.7588          | 0.6620   |
| 1.8896        | 1.29  | 10000 | 1.7518          | 0.6625   |
| 1.8796        | 1.35  | 10500 | 1.7557          | 0.6619   |
| 1.8838        | 1.42  | 11000 | 1.7511          | 0.6628   |
| 1.8869        | 1.48  | 11500 | 1.7437          | 0.6640   |
| 1.8756        | 1.55  | 12000 | 1.7425          | 0.6641   |
| 1.8775        | 1.61  | 12500 | 1.7409          | 0.6641   |
| 1.8757        | 1.68  | 13000 | 1.7372          | 0.6649   |
| 1.8616        | 1.74  | 13500 | 1.7387          | 0.6646   |
| 1.8675        | 1.8   | 14000 | 1.7335          | 0.6648   |
| 1.8725        | 1.87  | 14500 | 1.7288          | 0.6660   |
| 1.8678        | 1.93  | 15000 | 1.7305          | 0.6659   |
| 1.8611        | 2.0   | 15500 | 1.7256          | 0.6666   |
| 1.853         | 2.06  | 16000 | 1.7286          | 0.6661   |
| 1.8487        | 2.13  | 16500 | 1.7285          | 0.6659   |
| 1.8543        | 2.19  | 17000 | 1.7229          | 0.6668   |
| 1.8519        | 2.26  | 17500 | 1.7240          | 0.6670   |
| 1.851         | 2.32  | 18000 | 1.7275          | 0.6662   |
| 1.8547        | 2.38  | 18500 | 1.7197          | 0.6673   |
| 1.8476        | 2.45  | 19000 | 1.7164          | 0.6675   |
| 1.8444        | 2.51  | 19500 | 1.7214          | 0.6676   |
| 1.8544        | 2.58  | 20000 | 1.7217          | 0.6668   |
| 1.8491        | 2.64  | 20500 | 1.7175          | 0.6678   |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2