---
license: apache-2.0
base_model: allenai/longformer-base-4096
tags:
- generated_from_trainer
datasets:
- essays_su_g
metrics:
- accuracy
model-index:
- name: longformer-simple
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: essays_su_g
      type: essays_su_g
      config: simple
      split: train[20%:40%]
      args: simple
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8525299930594573
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# longformer-simple

This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6282
- Claim: {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0}
- Majorclaim: {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0}
- O: {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0}
- Premise: {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0}
- Accuracy: 0.8525
- Macro avg: {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0}
- Weighted avg: {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0}

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 16

### Training results

| Training Loss | Epoch | Step | Validation Loss | Claim                                                                                                                | Majorclaim                                                                                                         | O                                                                                                                   | Premise                                                                                                             | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
| No log        | 1.0   | 41   | 0.6017          | {'precision': 0.44954648526077096, 'recall': 0.36426274689940286, 'f1-score': 0.4024359299670135, 'support': 4354.0} | {'precision': 0.452642328312484, 'recall': 0.7547892720306514, 'f1-score': 0.5659112671560804, 'support': 2349.0}  | {'precision': 0.8588395638629284, 'recall': 0.8666011787819253, 'f1-score': 0.8627029141404263, 'support': 10180.0} | {'precision': 0.8723285486443381, 'recall': 0.8179303125467324, 'f1-score': 0.8442540711584471, 'support': 13374.0} | 0.7641   | {'precision': 0.6583392315201303, 'recall': 0.7008958775646781, 'f1-score': 0.6688260456054919, 'support': 30257.0} | {'precision': 0.774369269779734, 'recall': 0.764120699342301, 'f1-score': 0.765274191732446, 'support': 30257.0}    |
| No log        | 2.0   | 82   | 0.4534          | {'precision': 0.5700215450907972, 'recall': 0.4253559944878273, 'f1-score': 0.48717611469156913, 'support': 4354.0}  | {'precision': 0.6134453781512605, 'recall': 0.8701575138356747, 'f1-score': 0.719591621193452, 'support': 2349.0}  | {'precision': 0.884556428434566, 'recall': 0.9069744597249508, 'f1-score': 0.8956251818799107, 'support': 10180.0}  | {'precision': 0.8789847408974165, 'recall': 0.8700463586062509, 'f1-score': 0.8744927100556139, 'support': 13374.0} | 0.8185   | {'precision': 0.7367520231435101, 'recall': 0.768133581663676, 'f1-score': 0.7442214069551364, 'support': 30257.0}  | {'precision': 0.8157842273466825, 'recall': 0.8184882837029448, 'f1-score': 0.8138419333500274, 'support': 30257.0} |
| No log        | 3.0   | 123  | 0.4068          | {'precision': 0.5364131531168045, 'recall': 0.61070280202113, 'f1-score': 0.5711524003866395, 'support': 4354.0}     | {'precision': 0.7406428885953324, 'recall': 0.7160493827160493, 'f1-score': 0.7281385281385281, 'support': 2349.0} | {'precision': 0.9368757079600453, 'recall': 0.8937131630648331, 'f1-score': 0.9147855814187321, 'support': 10180.0} | {'precision': 0.8885718576362818, 'recall': 0.8848512038283236, 'f1-score': 0.8867076277536341, 'support': 13374.0} | 0.8353   | {'precision': 0.775625901827116, 'recall': 0.7763291379075841, 'f1-score': 0.7751960344243833, 'support': 30257.0}  | {'precision': 0.842663441353799, 'recall': 0.8352777869583898, 'f1-score': 0.8384354029249637, 'support': 30257.0}  |
| No log        | 4.0   | 164  | 0.4188          | {'precision': 0.5978233358643381, 'recall': 0.5424896646761599, 'f1-score': 0.5688139674894642, 'support': 4354.0}   | {'precision': 0.7550200803212851, 'recall': 0.800340570455513, 'f1-score': 0.7770200454639389, 'support': 2349.0}  | {'precision': 0.9040139616055847, 'recall': 0.9159135559921414, 'f1-score': 0.9099248560554309, 'support': 10180.0} | {'precision': 0.8849800029625241, 'recall': 0.8934499775684164, 'f1-score': 0.8891948206578361, 'support': 13374.0} | 0.8433   | {'precision': 0.7854593451884331, 'recall': 0.7880484421730577, 'f1-score': 0.7862384224166675, 'support': 30257.0} | {'precision': 0.8399725571535075, 'recall': 0.8432759361470074, 'f1-score': 0.8413577905068613, 'support': 30257.0} |
| No log        | 5.0   | 205  | 0.4474          | {'precision': 0.5675675675675675, 'recall': 0.5884244372990354, 'f1-score': 0.5778078484438429, 'support': 4354.0}   | {'precision': 0.728060263653484, 'recall': 0.8229033631332482, 'f1-score': 0.7725819344524381, 'support': 2349.0}  | {'precision': 0.9372398001665279, 'recall': 0.8845776031434185, 'f1-score': 0.9101475641803114, 'support': 10180.0} | {'precision': 0.887240356083086, 'recall': 0.8942724689696426, 'f1-score': 0.8907425337007523, 'support': 13374.0}  | 0.8415   | {'precision': 0.7800269968676663, 'recall': 0.7975444681363362, 'f1-score': 0.7878199701943362, 'support': 30257.0} | {'precision': 0.845703686302729, 'recall': 0.8414581749677761, 'f1-score': 0.8430665031306045, 'support': 30257.0}  |
| No log        | 6.0   | 246  | 0.4609          | {'precision': 0.6318574213311056, 'recall': 0.5211299954065227, 'f1-score': 0.5711768407803649, 'support': 4354.0}   | {'precision': 0.7856852379015861, 'recall': 0.822477650063857, 'f1-score': 0.8036605657237937, 'support': 2349.0}  | {'precision': 0.9054745582697692, 'recall': 0.9212180746561887, 'f1-score': 0.9132784729999514, 'support': 10180.0} | {'precision': 0.8784115523465704, 'recall': 0.9096754897562435, 'f1-score': 0.8937702027622686, 'support': 13374.0} | 0.8509   | {'precision': 0.8003571924622578, 'recall': 0.793625302470703, 'f1-score': 0.7954715205665946, 'support': 30257.0}  | {'precision': 0.8448388452449266, 'recall': 0.8508774828965198, 'f1-score': 0.8469167525043788, 'support': 30257.0} |
| No log        | 7.0   | 287  | 0.4865          | {'precision': 0.630808729139923, 'recall': 0.5643086816720257, 'f1-score': 0.5957085707358469, 'support': 4354.0}    | {'precision': 0.7307692307692307, 'recall': 0.8573861217539378, 'f1-score': 0.7890303623898138, 'support': 2349.0} | {'precision': 0.9172843166320783, 'recall': 0.9117878192534381, 'f1-score': 0.9145278092516873, 'support': 10180.0} | {'precision': 0.8916734633350634, 'recall': 0.8992074173770002, 'f1-score': 0.8954245932764975, 'support': 13374.0} | 0.8520   | {'precision': 0.7926339349690739, 'recall': 0.8081725100141004, 'f1-score': 0.7986728339134614, 'support': 30257.0} | {'precision': 0.8502598860333094, 'recall': 0.8520011898073173, 'f1-score': 0.8504626713454606, 'support': 30257.0} |
| No log        | 8.0   | 328  | 0.5096          | {'precision': 0.5821842854016196, 'recall': 0.6109324758842444, 'f1-score': 0.5962120363106579, 'support': 4354.0}   | {'precision': 0.774493927125506, 'recall': 0.8143891017454236, 'f1-score': 0.7939406515874662, 'support': 2349.0}  | {'precision': 0.9332583810302535, 'recall': 0.8969548133595285, 'f1-score': 0.9147465437788018, 'support': 10180.0} | {'precision': 0.8931070418341521, 'recall': 0.8971138029011515, 'f1-score': 0.8951059385258131, 'support': 13374.0} | 0.8495   | {'precision': 0.7957609088478829, 'recall': 0.804847548472587, 'f1-score': 0.8000012925506848, 'support': 30257.0}  | {'precision': 0.8526655157429486, 'recall': 0.8494563241563936, 'f1-score': 0.8508490740717186, 'support': 30257.0} |
| No log        | 9.0   | 369  | 0.5327          | {'precision': 0.6261045190608432, 'recall': 0.5695911805236564, 'f1-score': 0.5965123271196633, 'support': 4354.0}   | {'precision': 0.7516019600452318, 'recall': 0.8488718603661133, 'f1-score': 0.7972810875649741, 'support': 2349.0} | {'precision': 0.9026343722860176, 'recall': 0.918860510805501, 'f1-score': 0.9106751691573773, 'support': 10180.0}  | {'precision': 0.8932981927710844, 'recall': 0.8870195902497383, 'f1-score': 0.8901478202146019, 'support': 13374.0} | 0.8491   | {'precision': 0.7934097610407942, 'recall': 0.8060857854862523, 'f1-score': 0.7986541010141541, 'support': 30257.0} | {'precision': 0.846989457650438, 'recall': 0.8490927719205473, 'f1-score': 0.8475902474317125, 'support': 30257.0}  |
| No log        | 10.0  | 410  | 0.5611          | {'precision': 0.6031589338598223, 'recall': 0.5613229214515388, 'f1-score': 0.5814894123245301, 'support': 4354.0}   | {'precision': 0.7980400511291009, 'recall': 0.7973605789697744, 'f1-score': 0.7977001703577512, 'support': 2349.0} | {'precision': 0.9056966897613549, 'recall': 0.9245579567779961, 'f1-score': 0.9150301380517207, 'support': 10180.0} | {'precision': 0.8834100698054359, 'recall': 0.8894870644534171, 'f1-score': 0.8864381520119226, 'support': 13374.0} | 0.8469   | {'precision': 0.7975764361389286, 'recall': 0.7931821304131816, 'f1-score': 0.7951644681864812, 'support': 30257.0} | {'precision': 0.8439524293048358, 'recall': 0.8469114585054698, 'f1-score': 0.8452864874840641, 'support': 30257.0} |
| No log        | 11.0  | 451  | 0.5648          | {'precision': 0.5827433628318585, 'recall': 0.6049609554432706, 'f1-score': 0.5936443542934415, 'support': 4354.0}   | {'precision': 0.7425330812854443, 'recall': 0.8361004682843763, 'f1-score': 0.7865438526231477, 'support': 2349.0} | {'precision': 0.938811369509044, 'recall': 0.8922396856581533, 'f1-score': 0.9149332661798036, 'support': 10180.0}  | {'precision': 0.890064843109488, 'recall': 0.8929265739494542, 'f1-score': 0.8914934119667053, 'support': 13374.0}  | 0.8468   | {'precision': 0.7885381641839586, 'recall': 0.8065569208338136, 'f1-score': 0.7966537212657745, 'support': 30257.0} | {'precision': 0.8507883056171393, 'recall': 0.8468453580989523, 'f1-score': 0.8483713709144507, 'support': 30257.0} |
| No log        | 12.0  | 492  | 0.6091          | {'precision': 0.6024649589173514, 'recall': 0.5725769407441433, 'f1-score': 0.5871408384361753, 'support': 4354.0}   | {'precision': 0.749317738791423, 'recall': 0.8182205193699447, 'f1-score': 0.7822547822547823, 'support': 2349.0}  | {'precision': 0.9307965499746321, 'recall': 0.9010805500982318, 'f1-score': 0.9156975293236835, 'support': 10180.0} | {'precision': 0.8842981239506533, 'recall': 0.9057873485868102, 'f1-score': 0.8949137517083441, 'support': 13374.0} | 0.8495   | {'precision': 0.791719342908515, 'recall': 0.7994163396997825, 'f1-score': 0.7950017254307464, 'support': 30257.0}  | {'precision': 0.8489074193741941, 'recall': 0.8494563241563936, 'f1-score': 0.848871502724331, 'support': 30257.0}  |
| 0.2687        | 13.0  | 533  | 0.6049          | {'precision': 0.6140061306295685, 'recall': 0.5980707395498392, 'f1-score': 0.605933682373473, 'support': 4354.0}    | {'precision': 0.7614920874152223, 'recall': 0.8603661132396765, 'f1-score': 0.8079152508494902, 'support': 2349.0} | {'precision': 0.921222343486457, 'recall': 0.9120825147347741, 'f1-score': 0.9166296460832224, 'support': 10180.0}  | {'precision': 0.8964089437627042, 'recall': 0.8903095558546433, 'f1-score': 0.8933488389541209, 'support': 13374.0} | 0.8533   | {'precision': 0.798282376323488, 'recall': 0.8152072308447333, 'f1-score': 0.8059568545650766, 'support': 30257.0}  | {'precision': 0.8536452482623537, 'recall': 0.8532570975311499, 'f1-score': 0.8531898518226914, 'support': 30257.0} |
| 0.2687        | 14.0  | 574  | 0.6213          | {'precision': 0.6094716801523085, 'recall': 0.588194763435921, 'f1-score': 0.5986442262739597, 'support': 4354.0}    | {'precision': 0.759737755495565, 'recall': 0.8386547467007237, 'f1-score': 0.7972480777013357, 'support': 2349.0}  | {'precision': 0.9096567149664495, 'recall': 0.918860510805501, 'f1-score': 0.914235449347603, 'support': 10180.0}   | {'precision': 0.8965778890659383, 'recall': 0.8835053088081352, 'f1-score': 0.88999359771024, 'support': 13374.0}   | 0.8494   | {'precision': 0.7938610099200654, 'recall': 0.8073038324375702, 'f1-score': 0.8000303377582847, 'support': 30257.0} | {'precision': 0.8490399487645354, 'recall': 0.8494232739531348, 'f1-score': 0.8490241579089998, 'support': 30257.0} |
| 0.2687        | 15.0  | 615  | 0.6240          | {'precision': 0.6123274631128034, 'recall': 0.5909508497932935, 'f1-score': 0.6014492753623188, 'support': 4354.0}   | {'precision': 0.7543659832953683, 'recall': 0.8458918688803746, 'f1-score': 0.7975115392333936, 'support': 2349.0} | {'precision': 0.9263157894736842, 'recall': 0.9076620825147348, 'f1-score': 0.9168940709501364, 'support': 10180.0} | {'precision': 0.8923843522237096, 'recall': 0.8971885748467175, 'f1-score': 0.894780014914243, 'support': 13374.0}  | 0.8527   | {'precision': 0.7963483970263914, 'recall': 0.8104233440087801, 'f1-score': 0.802658725115023, 'support': 30257.0}  | {'precision': 0.8527852243327482, 'recall': 0.8526621938724923, 'f1-score': 0.8524584166415128, 'support': 30257.0} |
| 0.2687        | 16.0  | 656  | 0.6282          | {'precision': 0.6189892051030422, 'recall': 0.5794671566375746, 'f1-score': 0.5985765124555161, 'support': 4354.0}   | {'precision': 0.7602001539645882, 'recall': 0.8407833120476799, 'f1-score': 0.7984637153830604, 'support': 2349.0} | {'precision': 0.9195254572417202, 'recall': 0.9136542239685658, 'f1-score': 0.916580438531658, 'support': 10180.0}  | {'precision': 0.8907038907038907, 'recall': 0.8969642590100194, 'f1-score': 0.8938231130318157, 'support': 13374.0} | 0.8525   | {'precision': 0.7973546767533104, 'recall': 0.8077172379159598, 'f1-score': 0.8018609448505126, 'support': 30257.0} | {'precision': 0.8511693872385236, 'recall': 0.8525299930594573, 'f1-score': 0.8515904610703608, 'support': 30257.0} |


### Framework versions

- Transformers 4.37.2
- Pytorch 2.2.0+cu121
- Datasets 2.17.0
- Tokenizers 0.15.2