Hugging Face's logo

Theoreticallyhugo
/
longformer-simple

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Theoreticallyhugo commited on
Commit
d4cdc9a
•
1 Parent(s): 7bfcc3d

trainer: training complete at 2024-10-26 21:03:14.375078.

Browse files
Files changed (4) hide show
  1. README.md +26 -40
  2. meta_data/README_s42_e5.md +27 -25
  3. meta_data/meta_s42_e5_cvi0.json +1 -1
  4. model.safetensors +1 -1
README.md CHANGED
@@ -1,10 +1,11 @@
1
  ---
 
2
  license: apache-2.0
3
  base_model: allenai/longformer-base-4096
4
  tags:
5
  - generated_from_trainer
6
  datasets:
7
- - essays_su_g
8
  metrics:
9
  - accuracy
10
  model-index:
@@ -14,15 +15,15 @@ model-index:
14
  name: Token Classification
15
  type: token-classification
16
  dataset:
17
- name: essays_su_g
18
- type: essays_su_g
19
  config: simple
20
  split: train[0%:20%]
21
  args: simple
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.8399468193904684
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -30,16 +31,16 @@ should probably proofread and complete it, then remove this comment. -->
30
 
31
  # longformer-simple
32
 
33
- This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 1.0670
36
- - Claim: {'precision': 0.5894120517199317, 'recall': 0.5668700140778977, 'f1-score': 0.5779213012797513, 'support': 4262.0}
37
- - Majorclaim: {'precision': 0.7822390174775626, 'recall': 0.7648960739030023, 'f1-score': 0.7734703409621673, 'support': 2165.0}
38
- - O: {'precision': 0.9166666666666666, 'recall': 0.882853668423186, 'f1-score': 0.8994424943217014, 'support': 9868.0}
39
- - Premise: {'precision': 0.8707947700896136, 'recall': 0.9091954904517218, 'f1-score': 0.889580910216486, 'support': 13039.0}
40
- - Accuracy: 0.8399
41
- - Macro avg: {'precision': 0.7897781264884436, 'recall': 0.780953811713952, 'f1-score': 0.7851037616950265, 'support': 29334.0}
42
- - Weighted avg: {'precision': 0.8388075717984049, 'recall': 0.8399468193904684, 'f1-score': 0.8390471090378641, 'support': 29334.0}
43
 
44
  ## Model description
45
 
@@ -64,37 +65,22 @@ The following hyperparameters were used during training:
64
  - seed: 42
65
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
66
  - lr_scheduler_type: linear
67
- - num_epochs: 20
68
 
69
  ### Training results
70
 
71
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
72
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
73
- | No log | 1.0 | 81 | 0.5489 | {'precision': 0.43257049448304047, 'recall': 0.496715157203191, 'f1-score': 0.46242900830056793, 'support': 4262.0} | {'precision': 0.6522533495736906, 'recall': 0.49468822170900695, 'f1-score': 0.5626477541371159, 'support': 2165.0} | {'precision': 0.911293908403735, 'recall': 0.8307661126874747, 'f1-score': 0.8691687871077185, 'support': 9868.0} | {'precision': 0.8377046804810897, 'recall': 0.88672444205844, 'f1-score': 0.8615178272046495, 'support': 13039.0} | 0.7823 | {'precision': 0.708455608235389, 'recall': 0.6772234834145282, 'f1-score': 0.6889408441875129, 'support': 29334.0} | {'precision': 0.7899101236188295, 'recall': 0.7823004022635849, 'f1-score': 0.7840489998358311, 'support': 29334.0} |
74
- | No log | 2.0 | 162 | 0.5075 | {'precision': 0.5107383923092657, 'recall': 0.5858751759737213, 'f1-score': 0.5457327068079991, 'support': 4262.0} | {'precision': 0.5687340153452686, 'recall': 0.8217090069284064, 'f1-score': 0.6722085773663329, 'support': 2165.0} | {'precision': 0.9344863131370977, 'recall': 0.8268139440616133, 'f1-score': 0.8773589977955805, 'support': 9868.0} | {'precision': 0.8834419195931988, 'recall': 0.852749443975765, 'f1-score': 0.8678243902439025, 'support': 13039.0} | 0.8030 | {'precision': 0.7243501600962077, 'recall': 0.7717868927348766, 'f1-score': 0.7407811680534537, 'support': 29334.0} | {'precision': 0.8232353684753937, 'recall': 0.8029590236585532, 'f1-score': 0.8097969994222006, 'support': 29334.0} |
75
- | No log | 3.0 | 243 | 0.5219 | {'precision': 0.5516552511415526, 'recall': 0.45354293758798686, 'f1-score': 0.4978109708987896, 'support': 4262.0} | {'precision': 0.8036105032822757, 'recall': 0.6785219399538106, 'f1-score': 0.7357876283496118, 'support': 2165.0} | {'precision': 0.9331699710403207, 'recall': 0.8490068909606809, 'f1-score': 0.8891011355194736, 'support': 9868.0} | {'precision': 0.8191560170394037, 'recall': 0.9438607255157604, 'f1-score': 0.8770979581655561, 'support': 13039.0} | 0.8211 | {'precision': 0.7768979356258882, 'recall': 0.7312331235045597, 'f1-score': 0.7499494232333578, 'support': 29334.0} | {'precision': 0.8174973750724106, 'recall': 0.821129065248517, 'f1-score': 0.815598992812927, 'support': 29334.0} |
76
- | No log | 4.0 | 324 | 0.4725 | {'precision': 0.5851569933396765, 'recall': 0.5771938057250118, 'f1-score': 0.5811481218993623, 'support': 4262.0} | {'precision': 0.7335329341317365, 'recall': 0.792147806004619, 'f1-score': 0.761714412613813, 'support': 2165.0} | {'precision': 0.9085720215857203, 'recall': 0.8872111876773409, 'f1-score': 0.8977645611156685, 'support': 9868.0} | {'precision': 0.8851474612344178, 'recall': 0.8930899608865711, 'f1-score': 0.8891009734682191, 'support': 13039.0} | 0.8378 | {'precision': 0.7781023525728877, 'recall': 0.7874106900733857, 'f1-score': 0.7824320172742658, 'support': 29334.0} | {'precision': 0.8382513248807655, 'recall': 0.8377650507943001, 'f1-score': 0.8378705011585708, 'support': 29334.0} |
77
- | No log | 5.0 | 405 | 0.5539 | {'precision': 0.5784176029962547, 'recall': 0.5797747536367902, 'f1-score': 0.5790953831731895, 'support': 4262.0} | {'precision': 0.8008497079129049, 'recall': 0.6965357967667436, 'f1-score': 0.7450592885375493, 'support': 2165.0} | {'precision': 0.9040794979079498, 'recall': 0.8758613700851237, 'f1-score': 0.8897467572575665, 'support': 9868.0} | {'precision': 0.8672442910639547, 'recall': 0.905820998542833, 'f1-score': 0.8861129867206842, 'support': 13039.0} | 0.8329 | {'precision': 0.787647774970266, 'recall': 0.7644982297578727, 'f1-score': 0.7750036039222473, 'support': 29334.0} | {'precision': 0.8327711951367025, 'recall': 0.8329242517215518, 'f1-score': 0.8323176558681596, 'support': 29334.0} |
78
- | No log | 6.0 | 486 | 0.5790 | {'precision': 0.5325670498084292, 'recall': 0.6848897231346786, 'f1-score': 0.5991994252283692, 'support': 4262.0} | {'precision': 0.7400087834870444, 'recall': 0.7782909930715936, 'f1-score': 0.7586672669968483, 'support': 2165.0} | {'precision': 0.9212211784799317, 'recall': 0.8745439805431698, 'f1-score': 0.8972759409440632, 'support': 9868.0} | {'precision': 0.9062909567496723, 'recall': 0.8485313290896541, 'f1-score': 0.8764605695726225, 'support': 13039.0} | 0.8283 | {'precision': 0.7750219921312693, 'recall': 0.796564006459774, 'f1-score': 0.7829008006854759, 'support': 29334.0} | {'precision': 0.8447418748493872, 'recall': 0.8283220835890094, 'f1-score': 0.8344852708551485, 'support': 29334.0} |
79
- | 0.3561 | 7.0 | 567 | 0.7058 | {'precision': 0.5955997904662127, 'recall': 0.5335523228531206, 'f1-score': 0.5628712871287128, 'support': 4262.0} | {'precision': 0.8422198041349293, 'recall': 0.7150115473441109, 'f1-score': 0.7734199350487135, 'support': 2165.0} | {'precision': 0.9193378321383383, 'recall': 0.8835630320226996, 'f1-score': 0.9010954940057875, 'support': 9868.0} | {'precision': 0.848316189939411, 'recall': 0.9234603880665695, 'f1-score': 0.8842947894099071, 'support': 13039.0} | 0.8380 | {'precision': 0.8013684041697228, 'recall': 0.7638968225716252, 'f1-score': 0.7804203763982802, 'support': 29334.0} | {'precision': 0.8350403187795808, 'recall': 0.838003681734506, 'f1-score': 0.835063123988816, 'support': 29334.0} |
80
- | 0.3561 | 8.0 | 648 | 0.6876 | {'precision': 0.5858841386288894, 'recall': 0.5434068512435476, 'f1-score': 0.5638466220328667, 'support': 4262.0} | {'precision': 0.8025641025641026, 'recall': 0.7228637413394919, 'f1-score': 0.7606318347509112, 'support': 2165.0} | {'precision': 0.8864763037874281, 'recall': 0.9060599918929875, 'f1-score': 0.896161170692593, 'support': 9868.0} | {'precision': 0.8807043836642937, 'recall': 0.9013728046629342, 'f1-score': 0.8909187386294724, 'support': 13039.0} | 0.8378 | {'precision': 0.7889072321611785, 'recall': 0.7684258472847404, 'f1-score': 0.7778895915264609, 'support': 29334.0} | {'precision': 0.8340438435010799, 'recall': 0.8377650507943001, 'f1-score': 0.8355454452418354, 'support': 29334.0} |
81
- | 0.3561 | 9.0 | 729 | 0.6963 | {'precision': 0.5602836879432624, 'recall': 0.6302205537306429, 'f1-score': 0.5931978798586572, 'support': 4262.0} | {'precision': 0.8248823836905385, 'recall': 0.7288683602771363, 'f1-score': 0.7739087788131438, 'support': 2165.0} | {'precision': 0.927653083460449, 'recall': 0.8627888123226591, 'f1-score': 0.8940459939094823, 'support': 9868.0} | {'precision': 0.8708454160160607, 'recall': 0.8982283917478334, 'f1-score': 0.8843249773482331, 'support': 13039.0} | 0.8349 | {'precision': 0.7959161427775777, 'recall': 0.7800265295195679, 'f1-score': 0.786369407482379, 'support': 29334.0} | {'precision': 0.8414411074427396, 'recall': 0.8348673893775141, 'f1-score': 0.8371473756606818, 'support': 29334.0} |
82
- | 0.3561 | 10.0 | 810 | 0.7715 | {'precision': 0.5701775147928994, 'recall': 0.5652275926794932, 'f1-score': 0.5676917638741604, 'support': 4262.0} | {'precision': 0.7745940783190067, 'recall': 0.7491916859122402, 'f1-score': 0.7616811458088754, 'support': 2165.0} | {'precision': 0.9296124365756234, 'recall': 0.8726185650587759, 'f1-score': 0.9002143118498772, 'support': 9868.0} | {'precision': 0.8669284467713787, 'recall': 0.9143339213129841, 'f1-score': 0.8900003732596766, 'support': 13039.0} | 0.8374 | {'precision': 0.785328119114727, 'recall': 0.7753429412408733, 'f1-score': 0.7798968986981474, 'support': 29334.0} | {'precision': 0.8380850988337167, 'recall': 0.8373900593168337, 'f1-score': 0.8371368267053725, 'support': 29334.0} |
83
- | 0.3561 | 11.0 | 891 | 0.7798 | {'precision': 0.5522299306243805, 'recall': 0.6536837165649929, 'f1-score': 0.5986891586977544, 'support': 4262.0} | {'precision': 0.7361830742659758, 'recall': 0.7875288683602771, 'f1-score': 0.760990850256639, 'support': 2165.0} | {'precision': 0.9179415855354659, 'recall': 0.8694770976895014, 'f1-score': 0.8930523028883683, 'support': 9868.0} | {'precision': 0.8987802946301283, 'recall': 0.8703121405015722, 'f1-score': 0.8843171634521723, 'support': 13039.0} | 0.8324 | {'precision': 0.7762837212639876, 'recall': 0.795250455779086, 'f1-score': 0.7842623688237336, 'support': 29334.0} | {'precision': 0.8428746215263233, 'recall': 0.83244698984114, 'f1-score': 0.8366540534646059, 'support': 29334.0} |
84
- | 0.3561 | 12.0 | 972 | 0.8434 | {'precision': 0.5933806146572104, 'recall': 0.5889253871421868, 'f1-score': 0.5911446066886481, 'support': 4262.0} | {'precision': 0.7994011976047904, 'recall': 0.7399538106235566, 'f1-score': 0.7685296234108899, 'support': 2165.0} | {'precision': 0.9005203550658096, 'recall': 0.8944061613295501, 'f1-score': 0.8974528445777621, 'support': 9868.0} | {'precision': 0.8794646214001053, 'recall': 0.8970013037809648, 'f1-score': 0.8881464044346572, 'support': 13039.0} | 0.8398 | {'precision': 0.7931916971819789, 'recall': 0.7800716657190646, 'f1-score': 0.7863183697779894, 'support': 29334.0} | {'precision': 0.8390729472526346, 'recall': 0.8397763687188927, 'f1-score': 0.8392967405095946, 'support': 29334.0} |
85
- | 0.0633 | 13.0 | 1053 | 0.9362 | {'precision': 0.5771842462652784, 'recall': 0.5983106522759268, 'f1-score': 0.5875576036866359, 'support': 4262.0} | {'precision': 0.7786116322701688, 'recall': 0.766743648960739, 'f1-score': 0.7726320688852689, 'support': 2165.0} | {'precision': 0.9130388953304522, 'recall': 0.8777867855695176, 'f1-score': 0.8950658744510462, 'support': 9868.0} | {'precision': 0.8741821463488004, 'recall': 0.891479407930056, 'f1-score': 0.8827460510328068, 'support': 13039.0} | 0.8351 | {'precision': 0.785754230053675, 'recall': 0.7835801236840598, 'f1-score': 0.7845003995139396, 'support': 29334.0} | {'precision': 0.8370485534468686, 'recall': 0.835071930183405, 'f1-score': 0.83587491458883, 'support': 29334.0} |
86
- | 0.0633 | 14.0 | 1134 | 1.0311 | {'precision': 0.6124338624338624, 'recall': 0.5431722196152041, 'f1-score': 0.5757274309873166, 'support': 4262.0} | {'precision': 0.7854137447405329, 'recall': 0.7759815242494227, 'f1-score': 0.7806691449814126, 'support': 2165.0} | {'precision': 0.916153682869879, 'recall': 0.8747466558573166, 'f1-score': 0.894971487817522, 'support': 9868.0} | {'precision': 0.8590723933395269, 'recall': 0.9219265281079837, 'f1-score': 0.8893903521751998, 'support': 13039.0} | 0.8403 | {'precision': 0.7932684208459503, 'recall': 0.7789567319574817, 'f1-score': 0.7851896039903627, 'support': 29334.0} | {'precision': 0.8370035916809992, 'recall': 0.8402536305993046, 'f1-score': 0.8376709093048489, 'support': 29334.0} |
87
- | 0.0633 | 15.0 | 1215 | 1.0063 | {'precision': 0.5736224028906955, 'recall': 0.5959643359924918, 'f1-score': 0.5845799769850402, 'support': 4262.0} | {'precision': 0.8367459878251245, 'recall': 0.6983833718244804, 'f1-score': 0.7613293051359518, 'support': 2165.0} | {'precision': 0.9253507550605119, 'recall': 0.8755573571139035, 'f1-score': 0.8997656860192659, 'support': 9868.0} | {'precision': 0.8641912512716174, 'recall': 0.9121098243730348, 'f1-score': 0.8875041976045669, 'support': 13039.0} | 0.8381 | {'precision': 0.7999775992619873, 'recall': 0.7705037223259776, 'f1-score': 0.7832947914362062, 'support': 29334.0} | {'precision': 0.8405224217982303, 'recall': 0.8381059521374514, 'f1-score': 0.8383041122838222, 'support': 29334.0} |
88
- | 0.0633 | 16.0 | 1296 | 0.9864 | {'precision': 0.6114068441064638, 'recall': 0.5659314875645237, 'f1-score': 0.587790910198611, 'support': 4262.0} | {'precision': 0.8076540755467196, 'recall': 0.7505773672055427, 'f1-score': 0.7780703854440987, 'support': 2165.0} | {'precision': 0.9003660024400163, 'recall': 0.8974462910417511, 'f1-score': 0.8989037758830695, 'support': 9868.0} | {'precision': 0.8745292075917583, 'recall': 0.908198481478641, 'f1-score': 0.89104589917231, 'support': 13039.0} | 0.8432 | {'precision': 0.7984890324212395, 'recall': 0.7805384068226147, 'f1-score': 0.7889527426745223, 'support': 29334.0} | {'precision': 0.8400553996388973, 'recall': 0.8432194722847208, 'f1-score': 0.8412905564694496, 'support': 29334.0} |
89
- | 0.0633 | 17.0 | 1377 | 1.0474 | {'precision': 0.5777574788764558, 'recall': 0.5936180197090568, 'f1-score': 0.5855803726420553, 'support': 4262.0} | {'precision': 0.7891123099558607, 'recall': 0.7431870669745958, 'f1-score': 0.7654614652711702, 'support': 2165.0} | {'precision': 0.9293800539083558, 'recall': 0.8735306039724362, 'f1-score': 0.9005902941022829, 'support': 9868.0} | {'precision': 0.869437724507001, 'recall': 0.9095789554413682, 'f1-score': 0.8890554722638682, 'support': 13039.0} | 0.8393 | {'precision': 0.7914218918119184, 'recall': 0.7799786615243642, 'f1-score': 0.7851719010698441, 'support': 29334.0} | {'precision': 0.8412951315142951, 'recall': 0.8392650167041659, 'f1-score': 0.8397213794764584, 'support': 29334.0} |
90
- | 0.0633 | 18.0 | 1458 | 1.0609 | {'precision': 0.5889078083191438, 'recall': 0.5680431722196152, 'f1-score': 0.5782873522035114, 'support': 4262.0} | {'precision': 0.7991159135559921, 'recall': 0.751501154734411, 'f1-score': 0.7745774815520113, 'support': 2165.0} | {'precision': 0.9093172857439302, 'recall': 0.8881232265910012, 'f1-score': 0.8985953040090229, 'support': 9868.0} | {'precision': 0.8732747804265998, 'recall': 0.9074315514993481, 'f1-score': 0.8900255754475703, 'support': 13039.0} | 0.8401 | {'precision': 0.7926539470114164, 'recall': 0.7787747762610939, 'f1-score': 0.7853714283030289, 'support': 29334.0} | {'precision': 0.8386099362381009, 'recall': 0.8401172700620441, 'f1-score': 0.8390946642419506, 'support': 29334.0} |
91
- | 0.0173 | 19.0 | 1539 | 1.0791 | {'precision': 0.586526726873322, 'recall': 0.5638198029094322, 'f1-score': 0.5749491565976791, 'support': 4262.0} | {'precision': 0.7679227941176471, 'recall': 0.771824480369515, 'f1-score': 0.7698686938493434, 'support': 2165.0} | {'precision': 0.9245000534702171, 'recall': 0.8760640453992704, 'f1-score': 0.8996305739112337, 'support': 9868.0} | {'precision': 0.8675419401896426, 'recall': 0.912186517370964, 'f1-score': 0.8893042730569367, 'support': 13039.0} | 0.8391 | {'precision': 0.7866228786627072, 'recall': 0.7809737115122954, 'f1-score': 0.7834381743537983, 'support': 29334.0} | {'precision': 0.8385210215100449, 'recall': 0.839060475898275, 'f1-score': 0.8382897643467849, 'support': 29334.0} |
92
- | 0.0173 | 20.0 | 1620 | 1.0670 | {'precision': 0.5894120517199317, 'recall': 0.5668700140778977, 'f1-score': 0.5779213012797513, 'support': 4262.0} | {'precision': 0.7822390174775626, 'recall': 0.7648960739030023, 'f1-score': 0.7734703409621673, 'support': 2165.0} | {'precision': 0.9166666666666666, 'recall': 0.882853668423186, 'f1-score': 0.8994424943217014, 'support': 9868.0} | {'precision': 0.8707947700896136, 'recall': 0.9091954904517218, 'f1-score': 0.889580910216486, 'support': 13039.0} | 0.8399 | {'precision': 0.7897781264884436, 'recall': 0.780953811713952, 'f1-score': 0.7851037616950265, 'support': 29334.0} | {'precision': 0.8388075717984049, 'recall': 0.8399468193904684, 'f1-score': 0.8390471090378641, 'support': 29334.0} |
93
 
94
 
95
  ### Framework versions
96
 
97
- - Transformers 4.38.2
98
- - Pytorch 2.2.1+cu121
99
- - Datasets 2.18.0
100
- - Tokenizers 0.15.2
 
1
  ---
2
+ library_name: transformers
3
  license: apache-2.0
4
  base_model: allenai/longformer-base-4096
5
  tags:
6
  - generated_from_trainer
7
  datasets:
8
+ - stab-gurevych-essays
9
  metrics:
10
  - accuracy
11
  model-index:
 
15
  name: Token Classification
16
  type: token-classification
17
  dataset:
18
+ name: stab-gurevych-essays
19
+ type: stab-gurevych-essays
20
  config: simple
21
  split: train[0%:20%]
22
  args: simple
23
  metrics:
24
  - name: Accuracy
25
  type: accuracy
26
+ value: 0.8751580602166706
27
  ---
28
 
29
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
31
 
32
  # longformer-simple
33
 
34
+ This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the stab-gurevych-essays dataset.
35
  It achieves the following results on the evaluation set:
36
+ - Loss: 0.3326
37
+ - Claim: {'precision': 0.6375421311900441, 'recall': 0.6000488042947779, 'f1-score': 0.6182275298554368, 'support': 4098.0}
38
+ - Majorclaim: {'precision': 0.8534005037783375, 'recall': 0.7853500231803431, 'f1-score': 0.8179623370352487, 'support': 2157.0}
39
+ - O: {'precision': 0.9584632404706829, 'recall': 0.9674144756877474, 'f1-score': 0.9629180559765586, 'support': 9851.0}
40
+ - Premise: {'precision': 0.884906500445236, 'recall': 0.9064994298745724, 'f1-score': 0.8955728286583305, 'support': 13155.0}
41
+ - Accuracy: 0.8752
42
+ - Macro avg: {'precision': 0.8335780939710751, 'recall': 0.8148281832593602, 'f1-score': 0.8236701878813937, 'support': 29261.0}
43
+ - Weighted avg: {'precision': 0.8727042457708365, 'recall': 0.8751580602166706, 'f1-score': 0.8736819489681839, 'support': 29261.0}
44
 
45
  ## Model description
46
 
 
65
  - seed: 42
66
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
67
  - lr_scheduler_type: linear
68
+ - num_epochs: 5
69
 
70
  ### Training results
71
 
72
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
73
+ |:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
74
+ | No log | 1.0 | 41 | 0.5843 | {'precision': 0.48984771573604063, 'recall': 0.14128843338213762, 'f1-score': 0.21931818181818183, 'support': 4098.0} | {'precision': 0.5396243701328447, 'recall': 0.5461288827074641, 'f1-score': 0.5428571428571428, 'support': 2157.0} | {'precision': 0.8916069169126951, 'recall': 0.858390011166379, 'f1-score': 0.8746832169640548, 'support': 9851.0} | {'precision': 0.7743724104313917, 'recall': 0.9660965412390726, 'f1-score': 0.8596746372645179, 'support': 13155.0} | 0.7834 | {'precision': 0.673862853303243, 'recall': 0.6279759671237634, 'f1-score': 0.6241332947259743, 'support': 29261.0} | {'precision': 0.7566882370115429, 'recall': 0.7833635214107515, 'f1-score': 0.7516910901801511, 'support': 29261.0} |
75
+ | No log | 2.0 | 82 | 0.4171 | {'precision': 0.5352343493936415, 'recall': 0.39848706686188384, 'f1-score': 0.45684711148412366, 'support': 4098.0} | {'precision': 0.8575712143928036, 'recall': 0.5303662494204914, 'f1-score': 0.6553995989687769, 'support': 2157.0} | {'precision': 0.9516030844155844, 'recall': 0.952086082631205, 'f1-score': 0.9518445222509768, 'support': 9851.0} | {'precision': 0.8249001331557922, 'recall': 0.9418472063854048, 'f1-score': 0.8795031055900621, 'support': 13155.0} | 0.8389 | {'precision': 0.7923271953394554, 'recall': 0.7056966513247462, 'f1-score': 0.7358985845734849, 'support': 29261.0} | {'precision': 0.8293966272342979, 'recall': 0.8388640169508903, 'f1-score': 0.8281446341741303, 'support': 29261.0} |
76
+ | No log | 3.0 | 123 | 0.3525 | {'precision': 0.6357409713574097, 'recall': 0.49829184968277207, 'f1-score': 0.5586867305061559, 'support': 4098.0} | {'precision': 0.7525641025641026, 'recall': 0.8164116828929068, 'f1-score': 0.7831887925283523, 'support': 2157.0} | {'precision': 0.9471273523847455, 'recall': 0.9655872500253782, 'f1-score': 0.956268221574344, 'support': 9851.0} | {'precision': 0.8749451192741109, 'recall': 0.9089319650323071, 'f1-score': 0.8916147794638529, 'support': 13155.0} | 0.8637 | {'precision': 0.8025943863950922, 'recall': 0.797305686908341, 'f1-score': 0.7974396310181762, 'support': 29261.0} | {'precision': 0.8567240306977373, 'recall': 0.863675199070435, 'f1-score': 0.8587617347894375, 'support': 29261.0} |
77
+ | No log | 4.0 | 164 | 0.3385 | {'precision': 0.6185015290519877, 'recall': 0.5922401171303074, 'f1-score': 0.6050860134629769, 'support': 4098.0} | {'precision': 0.7913082842915347, 'recall': 0.8103847936949466, 'f1-score': 0.8007329363261567, 'support': 2157.0} | {'precision': 0.9529177057356608, 'recall': 0.9697492640341082, 'f1-score': 0.9612598108271282, 'support': 9851.0} | {'precision': 0.8938411050904373, 'recall': 0.8903078677309008, 'f1-score': 0.8920709878894051, 'support': 13155.0} | 0.8694 | {'precision': 0.8141421560424051, 'recall': 0.8156705106475658, 'f1-score': 0.8147874371264168, 'support': 29261.0} | {'precision': 0.8676102420265399, 'recall': 0.8694166296435528, 'f1-score': 0.8684387980236479, 'support': 29261.0} |
78
+ | No log | 5.0 | 205 | 0.3326 | {'precision': 0.6375421311900441, 'recall': 0.6000488042947779, 'f1-score': 0.6182275298554368, 'support': 4098.0} | {'precision': 0.8534005037783375, 'recall': 0.7853500231803431, 'f1-score': 0.8179623370352487, 'support': 2157.0} | {'precision': 0.9584632404706829, 'recall': 0.9674144756877474, 'f1-score': 0.9629180559765586, 'support': 9851.0} | {'precision': 0.884906500445236, 'recall': 0.9064994298745724, 'f1-score': 0.8955728286583305, 'support': 13155.0} | 0.8752 | {'precision': 0.8335780939710751, 'recall': 0.8148281832593602, 'f1-score': 0.8236701878813937, 'support': 29261.0} | {'precision': 0.8727042457708365, 'recall': 0.8751580602166706, 'f1-score': 0.8736819489681839, 'support': 29261.0} |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
 
80
 
81
  ### Framework versions
82
 
83
+ - Transformers 4.45.2
84
+ - Pytorch 2.5.0+cu124
85
+ - Datasets 2.19.1
86
+ - Tokenizers 0.20.1
meta_data/README_s42_e5.md CHANGED
@@ -1,9 +1,11 @@
1
  ---
 
 
2
  base_model: allenai/longformer-base-4096
3
  tags:
4
  - generated_from_trainer
5
  datasets:
6
- - essays_su_g
7
  metrics:
8
  - accuracy
9
  model-index:
@@ -13,15 +15,15 @@ model-index:
13
  name: Token Classification
14
  type: token-classification
15
  dataset:
16
- name: essays_su_g
17
- type: essays_su_g
18
  config: simple
19
- split: train[80%:100%]
20
  args: simple
21
  metrics:
22
  - name: Accuracy
23
  type: accuracy
24
- value: 0.8379014446576633
25
  ---
26
 
27
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -29,16 +31,16 @@ should probably proofread and complete it, then remove this comment. -->
29
 
30
  # longformer-simple
31
 
32
- This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
33
  It achieves the following results on the evaluation set:
34
- - Loss: 0.4267
35
- - Claim: {'precision': 0.6011011011011012, 'recall': 0.5762955854126679, 'f1-score': 0.58843704066634, 'support': 4168.0}
36
- - Majorclaim: {'precision': 0.7353560893383903, 'recall': 0.8108736059479554, 'f1-score': 0.7712707182320443, 'support': 2152.0}
37
- - O: {'precision': 0.9331677579589072, 'recall': 0.8959462388900932, 'f1-score': 0.9141782791417828, 'support': 9226.0}
38
- - Premise: {'precision': 0.8658005164622337, 'recall': 0.8886772136171622, 'f1-score': 0.8770897200081749, 'support': 12073.0}
39
- - Accuracy: 0.8379
40
- - Macro avg: {'precision': 0.7838563662151581, 'recall': 0.7929481609669696, 'f1-score': 0.7877439395120855, 'support': 27619.0}
41
- - Weighted avg: {'precision': 0.838194397473588, 'recall': 0.8379014446576633, 'f1-score': 0.8376730933108891, 'support': 27619.0}
42
 
43
  ## Model description
44
 
@@ -67,18 +69,18 @@ The following hyperparameters were used during training:
67
 
68
  ### Training results
69
 
70
- | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
71
- |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
72
- | No log | 1.0 | 41 | 0.6166 | {'precision': 0.4200196270853778, 'recall': 0.2053742802303263, 'f1-score': 0.27586206896551724, 'support': 4168.0} | {'precision': 0.6073394495412844, 'recall': 0.46143122676579923, 'f1-score': 0.524425666754687, 'support': 2152.0} | {'precision': 0.897315672254132, 'recall': 0.8297203555170172, 'f1-score': 0.8621951906290477, 'support': 9226.0} | {'precision': 0.7481024975673046, 'recall': 0.9551892653027416, 'f1-score': 0.8390570430733411, 'support': 12073.0} | 0.7616 | {'precision': 0.6681943116120247, 'recall': 0.612928781953971, 'f1-score': 0.6253849923556483, 'support': 27619.0} | {'precision': 0.7374674009360002, 'recall': 0.7616495890510157, 'f1-score': 0.7372788894627758, 'support': 27619.0} |
73
- | No log | 2.0 | 82 | 0.4575 | {'precision': 0.5743048897411314, 'recall': 0.43114203454894434, 'f1-score': 0.49253117719610806, 'support': 4168.0} | {'precision': 0.7058560572194904, 'recall': 0.7337360594795539, 'f1-score': 0.7195260879471406, 'support': 2152.0} | {'precision': 0.9206993795826283, 'recall': 0.8846737481031867, 'f1-score': 0.9023271239843015, 'support': 9226.0} | {'precision': 0.8243949805796236, 'recall': 0.9141886854965626, 'f1-score': 0.8669730175562625, 'support': 12073.0} | 0.8174 | {'precision': 0.7563138267807185, 'recall': 0.7409351319070618, 'f1-score': 0.7453393516709531, 'support': 27619.0} | {'precision': 0.8095875336596003, 'recall': 0.8173720989174119, 'f1-score': 0.8107869718183696, 'support': 27619.0} |
74
- | No log | 3.0 | 123 | 0.4417 | {'precision': 0.6082102988836874, 'recall': 0.4052303262955854, 'f1-score': 0.4863930885529157, 'support': 4168.0} | {'precision': 0.7309513560051657, 'recall': 0.7890334572490706, 'f1-score': 0.7588826815642457, 'support': 2152.0} | {'precision': 0.9306548632391329, 'recall': 0.8887925428137872, 'f1-score': 0.9092421134334979, 'support': 9226.0} | {'precision': 0.8175517945725124, 'recall': 0.9282696927027251, 'f1-score': 0.8693999456964432, 'support': 12073.0} | 0.8253 | {'precision': 0.7718420781751247, 'recall': 0.7528315047652921, 'f1-score': 0.7559794573117755, 'support': 27619.0} | {'precision': 0.8169938241061772, 'recall': 0.8253014229334878, 'f1-score': 0.8162980269649668, 'support': 27619.0} |
75
- | No log | 4.0 | 164 | 0.4247 | {'precision': 0.5918674698795181, 'recall': 0.5657389635316699, 'f1-score': 0.5785083415112856, 'support': 4168.0} | {'precision': 0.7616387337057728, 'recall': 0.7602230483271375, 'f1-score': 0.7609302325581395, 'support': 2152.0} | {'precision': 0.918848167539267, 'recall': 0.9130717537394321, 'f1-score': 0.9159508535391975, 'support': 9226.0} | {'precision': 0.8669534864842926, 'recall': 0.8846185703636213, 'f1-score': 0.8756969498196131, 'support': 12073.0} | 0.8363 | {'precision': 0.7848269644022126, 'recall': 0.7809130839904652, 'f1-score': 0.782771594357059, 'support': 27619.0} | {'precision': 0.8345694197992249, 'recall': 0.8363083384626525, 'f1-score': 0.8353523472178203, 'support': 27619.0} |
76
- | No log | 5.0 | 205 | 0.4267 | {'precision': 0.6011011011011012, 'recall': 0.5762955854126679, 'f1-score': 0.58843704066634, 'support': 4168.0} | {'precision': 0.7353560893383903, 'recall': 0.8108736059479554, 'f1-score': 0.7712707182320443, 'support': 2152.0} | {'precision': 0.9331677579589072, 'recall': 0.8959462388900932, 'f1-score': 0.9141782791417828, 'support': 9226.0} | {'precision': 0.8658005164622337, 'recall': 0.8886772136171622, 'f1-score': 0.8770897200081749, 'support': 12073.0} | 0.8379 | {'precision': 0.7838563662151581, 'recall': 0.7929481609669696, 'f1-score': 0.7877439395120855, 'support': 27619.0} | {'precision': 0.838194397473588, 'recall': 0.8379014446576633, 'f1-score': 0.8376730933108891, 'support': 27619.0} |
77
 
78
 
79
  ### Framework versions
80
 
81
- - Transformers 4.37.2
82
- - Pytorch 2.2.0+cu121
83
- - Datasets 2.17.0
84
- - Tokenizers 0.15.2
 
1
  ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
  base_model: allenai/longformer-base-4096
5
  tags:
6
  - generated_from_trainer
7
  datasets:
8
+ - stab-gurevych-essays
9
  metrics:
10
  - accuracy
11
  model-index:
 
15
  name: Token Classification
16
  type: token-classification
17
  dataset:
18
+ name: stab-gurevych-essays
19
+ type: stab-gurevych-essays
20
  config: simple
21
+ split: train[0%:20%]
22
  args: simple
23
  metrics:
24
  - name: Accuracy
25
  type: accuracy
26
+ value: 0.8751580602166706
27
  ---
28
 
29
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
31
 
32
  # longformer-simple
33
 
34
+ This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the stab-gurevych-essays dataset.
35
  It achieves the following results on the evaluation set:
36
+ - Loss: 0.3326
37
+ - Claim: {'precision': 0.6375421311900441, 'recall': 0.6000488042947779, 'f1-score': 0.6182275298554368, 'support': 4098.0}
38
+ - Majorclaim: {'precision': 0.8534005037783375, 'recall': 0.7853500231803431, 'f1-score': 0.8179623370352487, 'support': 2157.0}
39
+ - O: {'precision': 0.9584632404706829, 'recall': 0.9674144756877474, 'f1-score': 0.9629180559765586, 'support': 9851.0}
40
+ - Premise: {'precision': 0.884906500445236, 'recall': 0.9064994298745724, 'f1-score': 0.8955728286583305, 'support': 13155.0}
41
+ - Accuracy: 0.8752
42
+ - Macro avg: {'precision': 0.8335780939710751, 'recall': 0.8148281832593602, 'f1-score': 0.8236701878813937, 'support': 29261.0}
43
+ - Weighted avg: {'precision': 0.8727042457708365, 'recall': 0.8751580602166706, 'f1-score': 0.8736819489681839, 'support': 29261.0}
44
 
45
  ## Model description
46
 
 
69
 
70
  ### Training results
71
 
72
+ | Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
73
+ |:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
74
+ | No log | 1.0 | 41 | 0.5843 | {'precision': 0.48984771573604063, 'recall': 0.14128843338213762, 'f1-score': 0.21931818181818183, 'support': 4098.0} | {'precision': 0.5396243701328447, 'recall': 0.5461288827074641, 'f1-score': 0.5428571428571428, 'support': 2157.0} | {'precision': 0.8916069169126951, 'recall': 0.858390011166379, 'f1-score': 0.8746832169640548, 'support': 9851.0} | {'precision': 0.7743724104313917, 'recall': 0.9660965412390726, 'f1-score': 0.8596746372645179, 'support': 13155.0} | 0.7834 | {'precision': 0.673862853303243, 'recall': 0.6279759671237634, 'f1-score': 0.6241332947259743, 'support': 29261.0} | {'precision': 0.7566882370115429, 'recall': 0.7833635214107515, 'f1-score': 0.7516910901801511, 'support': 29261.0} |
75
+ | No log | 2.0 | 82 | 0.4171 | {'precision': 0.5352343493936415, 'recall': 0.39848706686188384, 'f1-score': 0.45684711148412366, 'support': 4098.0} | {'precision': 0.8575712143928036, 'recall': 0.5303662494204914, 'f1-score': 0.6553995989687769, 'support': 2157.0} | {'precision': 0.9516030844155844, 'recall': 0.952086082631205, 'f1-score': 0.9518445222509768, 'support': 9851.0} | {'precision': 0.8249001331557922, 'recall': 0.9418472063854048, 'f1-score': 0.8795031055900621, 'support': 13155.0} | 0.8389 | {'precision': 0.7923271953394554, 'recall': 0.7056966513247462, 'f1-score': 0.7358985845734849, 'support': 29261.0} | {'precision': 0.8293966272342979, 'recall': 0.8388640169508903, 'f1-score': 0.8281446341741303, 'support': 29261.0} |
76
+ | No log | 3.0 | 123 | 0.3525 | {'precision': 0.6357409713574097, 'recall': 0.49829184968277207, 'f1-score': 0.5586867305061559, 'support': 4098.0} | {'precision': 0.7525641025641026, 'recall': 0.8164116828929068, 'f1-score': 0.7831887925283523, 'support': 2157.0} | {'precision': 0.9471273523847455, 'recall': 0.9655872500253782, 'f1-score': 0.956268221574344, 'support': 9851.0} | {'precision': 0.8749451192741109, 'recall': 0.9089319650323071, 'f1-score': 0.8916147794638529, 'support': 13155.0} | 0.8637 | {'precision': 0.8025943863950922, 'recall': 0.797305686908341, 'f1-score': 0.7974396310181762, 'support': 29261.0} | {'precision': 0.8567240306977373, 'recall': 0.863675199070435, 'f1-score': 0.8587617347894375, 'support': 29261.0} |
77
+ | No log | 4.0 | 164 | 0.3385 | {'precision': 0.6185015290519877, 'recall': 0.5922401171303074, 'f1-score': 0.6050860134629769, 'support': 4098.0} | {'precision': 0.7913082842915347, 'recall': 0.8103847936949466, 'f1-score': 0.8007329363261567, 'support': 2157.0} | {'precision': 0.9529177057356608, 'recall': 0.9697492640341082, 'f1-score': 0.9612598108271282, 'support': 9851.0} | {'precision': 0.8938411050904373, 'recall': 0.8903078677309008, 'f1-score': 0.8920709878894051, 'support': 13155.0} | 0.8694 | {'precision': 0.8141421560424051, 'recall': 0.8156705106475658, 'f1-score': 0.8147874371264168, 'support': 29261.0} | {'precision': 0.8676102420265399, 'recall': 0.8694166296435528, 'f1-score': 0.8684387980236479, 'support': 29261.0} |
78
+ | No log | 5.0 | 205 | 0.3326 | {'precision': 0.6375421311900441, 'recall': 0.6000488042947779, 'f1-score': 0.6182275298554368, 'support': 4098.0} | {'precision': 0.8534005037783375, 'recall': 0.7853500231803431, 'f1-score': 0.8179623370352487, 'support': 2157.0} | {'precision': 0.9584632404706829, 'recall': 0.9674144756877474, 'f1-score': 0.9629180559765586, 'support': 9851.0} | {'precision': 0.884906500445236, 'recall': 0.9064994298745724, 'f1-score': 0.8955728286583305, 'support': 13155.0} | 0.8752 | {'precision': 0.8335780939710751, 'recall': 0.8148281832593602, 'f1-score': 0.8236701878813937, 'support': 29261.0} | {'precision': 0.8727042457708365, 'recall': 0.8751580602166706, 'f1-score': 0.8736819489681839, 'support': 29261.0} |
79
 
80
 
81
  ### Framework versions
82
 
83
+ - Transformers 4.45.2
84
+ - Pytorch 2.5.0+cu124
85
+ - Datasets 2.19.1
86
+ - Tokenizers 0.20.1
meta_data/meta_s42_e5_cvi0.json CHANGED
@@ -1 +1 @@
1
- {"Claim": {"precision": 0.6357409713574097, "recall": 0.49829184968277207, "f1-score": 0.5586867305061559, "support": 4098.0}, "MajorClaim": {"precision": 0.7525641025641026, "recall": 0.8164116828929068, "f1-score": 0.7831887925283523, "support": 2157.0}, "O": {"precision": 0.9471273523847455, "recall": 0.9655872500253782, "f1-score": 0.956268221574344, "support": 9851.0}, "Premise": {"precision": 0.8749451192741109, "recall": 0.9089319650323071, "f1-score": 0.8916147794638529, "support": 13155.0}, "accuracy": 0.863675199070435, "macro avg": {"precision": 0.8025943863950922, "recall": 0.797305686908341, "f1-score": 0.7974396310181762, "support": 29261.0}, "weighted avg": {"precision": 0.8567240306977373, "recall": 0.863675199070435, "f1-score": 0.8587617347894375, "support": 29261.0}}
 
1
+ {"Claim": {"precision": 0.6375421311900441, "recall": 0.6000488042947779, "f1-score": 0.6182275298554368, "support": 4098.0}, "MajorClaim": {"precision": 0.8534005037783375, "recall": 0.7853500231803431, "f1-score": 0.8179623370352487, "support": 2157.0}, "O": {"precision": 0.9584632404706829, "recall": 0.9674144756877474, "f1-score": 0.9629180559765586, "support": 9851.0}, "Premise": {"precision": 0.884906500445236, "recall": 0.9064994298745724, "f1-score": 0.8955728286583305, "support": 13155.0}, "accuracy": 0.8751580602166706, "macro avg": {"precision": 0.8335780939710751, "recall": 0.8148281832593602, "f1-score": 0.8236701878813937, "support": 29261.0}, "weighted avg": {"precision": 0.8727042457708365, "recall": 0.8751580602166706, "f1-score": 0.8736819489681839, "support": 29261.0}}
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f8ce6bdec635b30b63b8e52599e9146f7866666b010d86b0c5ccd1807c2b1980
3
  size 592324828
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c89f233a7a2cb18fa41b152316ed74186d34fc99cb342d2f74d6710b03c2dd9
3
  size 592324828