Theoreticallyhugo
commited on
trainer: training complete at 2024-02-19 21:00:31.064789.
Browse files- README.md +19 -18
- meta_data/README_s42_e7.md +87 -0
- model.safetensors +1 -1
README.md
CHANGED
@@ -22,7 +22,7 @@ model-index:
|
|
22 |
metrics:
|
23 |
- name: Accuracy
|
24 |
type: accuracy
|
25 |
-
value: 0.
|
26 |
---
|
27 |
|
28 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
@@ -32,14 +32,14 @@ should probably proofread and complete it, then remove this comment. -->
|
|
32 |
|
33 |
This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
|
34 |
It achieves the following results on the evaluation set:
|
35 |
-
- Loss: 0.
|
36 |
-
- Claim: {'precision': 0.
|
37 |
-
- Majorclaim: {'precision': 0.
|
38 |
-
- O: {'precision': 0.
|
39 |
-
- Premise: {'precision': 0.
|
40 |
-
- Accuracy: 0.
|
41 |
-
- Macro avg: {'precision': 0.
|
42 |
-
- Weighted avg: {'precision': 0.
|
43 |
|
44 |
## Model description
|
45 |
|
@@ -64,18 +64,19 @@ The following hyperparameters were used during training:
|
|
64 |
- seed: 42
|
65 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
66 |
- lr_scheduler_type: linear
|
67 |
-
- num_epochs:
|
68 |
|
69 |
### Training results
|
70 |
|
71 |
-
| Training Loss | Epoch | Step | Validation Loss | Claim
|
72 |
-
|
73 |
-
| No log | 1.0 | 41 | 0.
|
74 |
-
| No log | 2.0 | 82 | 0.
|
75 |
-
| No log | 3.0 | 123 | 0.
|
76 |
-
| No log | 4.0 | 164 | 0.
|
77 |
-
| No log | 5.0 | 205 | 0.
|
78 |
-
| No log | 6.0 | 246 | 0.
|
|
|
79 |
|
80 |
|
81 |
### Framework versions
|
|
|
22 |
metrics:
|
23 |
- name: Accuracy
|
24 |
type: accuracy
|
25 |
+
value: 0.8374001218245011
|
26 |
---
|
27 |
|
28 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
|
|
32 |
|
33 |
This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
|
34 |
It achieves the following results on the evaluation set:
|
35 |
+
- Loss: 0.4624
|
36 |
+
- Claim: {'precision': 0.5906025179856115, 'recall': 0.617826904985889, 'f1-score': 0.6039080459770114, 'support': 4252.0}
|
37 |
+
- Majorclaim: {'precision': 0.7631810193321616, 'recall': 0.7960586617781852, 'f1-score': 0.7792732166890982, 'support': 2182.0}
|
38 |
+
- O: {'precision': 0.9296403841858387, 'recall': 0.897466307277628, 'f1-score': 0.913270064183444, 'support': 9275.0}
|
39 |
+
- Premise: {'precision': 0.8734363502575423, 'recall': 0.875655737704918, 'f1-score': 0.8745446359133887, 'support': 12200.0}
|
40 |
+
- Accuracy: 0.8374
|
41 |
+
- Macro avg: {'precision': 0.7892150679402886, 'recall': 0.7967519029366551, 'f1-score': 0.7927489906907357, 'support': 27909.0}
|
42 |
+
- Weighted avg: {'precision': 0.8404042039171331, 'recall': 0.8374001218245011, 'f1-score': 0.8387335832080923, 'support': 27909.0}
|
43 |
|
44 |
## Model description
|
45 |
|
|
|
64 |
- seed: 42
|
65 |
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
66 |
- lr_scheduler_type: linear
|
67 |
+
- num_epochs: 7
|
68 |
|
69 |
### Training results
|
70 |
|
71 |
+
| Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
|
72 |
+
|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
|
73 |
+
| No log | 1.0 | 41 | 0.5869 | {'precision': 0.50130958617077, 'recall': 0.2250705550329257, 'f1-score': 0.310663853270573, 'support': 4252.0} | {'precision': 0.6411985018726591, 'recall': 0.3923006416131989, 'f1-score': 0.48677850440716514, 'support': 2182.0} | {'precision': 0.8172767203513909, 'recall': 0.9027493261455526, 'f1-score': 0.8578893442622951, 'support': 9275.0} | {'precision': 0.7879334257975035, 'recall': 0.931311475409836, 'f1-score': 0.8536438767843726, 'support': 12200.0} | 0.7721 | {'precision': 0.6869295585480809, 'recall': 0.6128579995503783, 'f1-score': 0.6272438946811014, 'support': 27909.0} | {'precision': 0.7425451598936884, 'recall': 0.7720806908165825, 'f1-score': 0.7436480119504476, 'support': 27909.0} |
|
74 |
+
| No log | 2.0 | 82 | 0.4605 | {'precision': 0.5800683670786222, 'recall': 0.5188146754468486, 'f1-score': 0.5477343265052762, 'support': 4252.0} | {'precision': 0.679080824088748, 'recall': 0.7855178735105408, 'f1-score': 0.7284317892052699, 'support': 2182.0} | {'precision': 0.9250369696280286, 'recall': 0.8767654986522911, 'f1-score': 0.900254621941769, 'support': 9275.0} | {'precision': 0.8497380970995231, 'recall': 0.8909016393442623, 'f1-score': 0.8698331399303749, 'support': 12200.0} | 0.8213 | {'precision': 0.7584810644737304, 'recall': 0.7679999217384857, 'f1-score': 0.7615634693956725, 'support': 27909.0} | {'precision': 0.8203349361458346, 'recall': 0.8212762908022502, 'f1-score': 0.8198154876923865, 'support': 27909.0} |
|
75 |
+
| No log | 3.0 | 123 | 0.4587 | {'precision': 0.6081277213352685, 'recall': 0.39416745061147695, 'f1-score': 0.478310502283105, 'support': 4252.0} | {'precision': 0.7005473025801408, 'recall': 0.8212648945921174, 'f1-score': 0.7561181434599156, 'support': 2182.0} | {'precision': 0.9445551517993201, 'recall': 0.8687870619946092, 'f1-score': 0.905088172526115, 'support': 9275.0} | {'precision': 0.8125, 'recall': 0.9366393442622951, 'f1-score': 0.8701644837039293, 'support': 12200.0} | 0.8224 | {'precision': 0.7664325439286823, 'recall': 0.7552146878651247, 'f1-score': 0.7524203254932662, 'support': 27909.0} | {'precision': 0.8164965537384401, 'recall': 0.8224228743416102, 'f1-score': 0.8131543783763286, 'support': 27909.0} |
|
76 |
+
| No log | 4.0 | 164 | 0.4491 | {'precision': 0.5829145728643216, 'recall': 0.6274694261523989, 'f1-score': 0.6043719560539133, 'support': 4252.0} | {'precision': 0.7112758486149044, 'recall': 0.8354720439963337, 'f1-score': 0.7683877766069548, 'support': 2182.0} | {'precision': 0.9357652656621729, 'recall': 0.8905660377358491, 'f1-score': 0.9126063418406806, 'support': 9275.0} | {'precision': 0.881426896667225, 'recall': 0.8627868852459016, 'f1-score': 0.8720072901996521, 'support': 12200.0} | 0.8340 | {'precision': 0.7778456459521561, 'recall': 0.8040735982826209, 'f1-score': 0.7893433411753001, 'support': 27909.0} | {'precision': 0.8407032729174679, 'recall': 0.8340320326776308, 'f1-score': 0.8366234708053201, 'support': 27909.0} |
|
77 |
+
| No log | 5.0 | 205 | 0.4611 | {'precision': 0.5805860805860806, 'recall': 0.5964252116650988, 'f1-score': 0.588399071925754, 'support': 4252.0} | {'precision': 0.7489102005231038, 'recall': 0.7873510540788268, 'f1-score': 0.7676496872207329, 'support': 2182.0} | {'precision': 0.9323308270676691, 'recall': 0.8957412398921832, 'f1-score': 0.9136698559331353, 'support': 9275.0} | {'precision': 0.8673800259403373, 'recall': 0.8770491803278688, 'f1-score': 0.8721878056732963, 'support': 12200.0} | 0.8335 | {'precision': 0.7823017835292977, 'recall': 0.7891416714909945, 'f1-score': 0.7854766051882296, 'support': 27909.0} | {'precision': 0.8360091300196414, 'recall': 0.8334945716435559, 'f1-score': 0.8345646069131102, 'support': 27909.0} |
|
78 |
+
| No log | 6.0 | 246 | 0.4642 | {'precision': 0.5962333486449242, 'recall': 0.6105362182502352, 'f1-score': 0.6033000232396003, 'support': 4252.0} | {'precision': 0.7385488447507094, 'recall': 0.8350137488542622, 'f1-score': 0.783824478382448, 'support': 2182.0} | {'precision': 0.9409678526484384, 'recall': 0.8867924528301887, 'f1-score': 0.9130772646536413, 'support': 9275.0} | {'precision': 0.8715477443913501, 'recall': 0.8820491803278688, 'f1-score': 0.8767670183729173, 'support': 12200.0} | 0.8386 | {'precision': 0.7868244476088555, 'recall': 0.8035979000656387, 'f1-score': 0.7942421961621517, 'support': 27909.0} | {'precision': 0.8422751475356695, 'recall': 0.8385825360994661, 'f1-score': 0.8399041873394746, 'support': 27909.0} |
|
79 |
+
| No log | 7.0 | 287 | 0.4624 | {'precision': 0.5906025179856115, 'recall': 0.617826904985889, 'f1-score': 0.6039080459770114, 'support': 4252.0} | {'precision': 0.7631810193321616, 'recall': 0.7960586617781852, 'f1-score': 0.7792732166890982, 'support': 2182.0} | {'precision': 0.9296403841858387, 'recall': 0.897466307277628, 'f1-score': 0.913270064183444, 'support': 9275.0} | {'precision': 0.8734363502575423, 'recall': 0.875655737704918, 'f1-score': 0.8745446359133887, 'support': 12200.0} | 0.8374 | {'precision': 0.7892150679402886, 'recall': 0.7967519029366551, 'f1-score': 0.7927489906907357, 'support': 27909.0} | {'precision': 0.8404042039171331, 'recall': 0.8374001218245011, 'f1-score': 0.8387335832080923, 'support': 27909.0} |
|
80 |
|
81 |
|
82 |
### Framework versions
|
meta_data/README_s42_e7.md
ADDED
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
base_model: allenai/longformer-base-4096
|
4 |
+
tags:
|
5 |
+
- generated_from_trainer
|
6 |
+
datasets:
|
7 |
+
- essays_su_g
|
8 |
+
metrics:
|
9 |
+
- accuracy
|
10 |
+
model-index:
|
11 |
+
- name: longformer-simple
|
12 |
+
results:
|
13 |
+
- task:
|
14 |
+
name: Token Classification
|
15 |
+
type: token-classification
|
16 |
+
dataset:
|
17 |
+
name: essays_su_g
|
18 |
+
type: essays_su_g
|
19 |
+
config: simple
|
20 |
+
split: test
|
21 |
+
args: simple
|
22 |
+
metrics:
|
23 |
+
- name: Accuracy
|
24 |
+
type: accuracy
|
25 |
+
value: 0.8374001218245011
|
26 |
+
---
|
27 |
+
|
28 |
+
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
29 |
+
should probably proofread and complete it, then remove this comment. -->
|
30 |
+
|
31 |
+
# longformer-simple
|
32 |
+
|
33 |
+
This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
|
34 |
+
It achieves the following results on the evaluation set:
|
35 |
+
- Loss: 0.4624
|
36 |
+
- Claim: {'precision': 0.5906025179856115, 'recall': 0.617826904985889, 'f1-score': 0.6039080459770114, 'support': 4252.0}
|
37 |
+
- Majorclaim: {'precision': 0.7631810193321616, 'recall': 0.7960586617781852, 'f1-score': 0.7792732166890982, 'support': 2182.0}
|
38 |
+
- O: {'precision': 0.9296403841858387, 'recall': 0.897466307277628, 'f1-score': 0.913270064183444, 'support': 9275.0}
|
39 |
+
- Premise: {'precision': 0.8734363502575423, 'recall': 0.875655737704918, 'f1-score': 0.8745446359133887, 'support': 12200.0}
|
40 |
+
- Accuracy: 0.8374
|
41 |
+
- Macro avg: {'precision': 0.7892150679402886, 'recall': 0.7967519029366551, 'f1-score': 0.7927489906907357, 'support': 27909.0}
|
42 |
+
- Weighted avg: {'precision': 0.8404042039171331, 'recall': 0.8374001218245011, 'f1-score': 0.8387335832080923, 'support': 27909.0}
|
43 |
+
|
44 |
+
## Model description
|
45 |
+
|
46 |
+
More information needed
|
47 |
+
|
48 |
+
## Intended uses & limitations
|
49 |
+
|
50 |
+
More information needed
|
51 |
+
|
52 |
+
## Training and evaluation data
|
53 |
+
|
54 |
+
More information needed
|
55 |
+
|
56 |
+
## Training procedure
|
57 |
+
|
58 |
+
### Training hyperparameters
|
59 |
+
|
60 |
+
The following hyperparameters were used during training:
|
61 |
+
- learning_rate: 2e-05
|
62 |
+
- train_batch_size: 8
|
63 |
+
- eval_batch_size: 8
|
64 |
+
- seed: 42
|
65 |
+
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
66 |
+
- lr_scheduler_type: linear
|
67 |
+
- num_epochs: 7
|
68 |
+
|
69 |
+
### Training results
|
70 |
+
|
71 |
+
| Training Loss | Epoch | Step | Validation Loss | Claim | Majorclaim | O | Premise | Accuracy | Macro avg | Weighted avg |
|
72 |
+
|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
|
73 |
+
| No log | 1.0 | 41 | 0.5869 | {'precision': 0.50130958617077, 'recall': 0.2250705550329257, 'f1-score': 0.310663853270573, 'support': 4252.0} | {'precision': 0.6411985018726591, 'recall': 0.3923006416131989, 'f1-score': 0.48677850440716514, 'support': 2182.0} | {'precision': 0.8172767203513909, 'recall': 0.9027493261455526, 'f1-score': 0.8578893442622951, 'support': 9275.0} | {'precision': 0.7879334257975035, 'recall': 0.931311475409836, 'f1-score': 0.8536438767843726, 'support': 12200.0} | 0.7721 | {'precision': 0.6869295585480809, 'recall': 0.6128579995503783, 'f1-score': 0.6272438946811014, 'support': 27909.0} | {'precision': 0.7425451598936884, 'recall': 0.7720806908165825, 'f1-score': 0.7436480119504476, 'support': 27909.0} |
|
74 |
+
| No log | 2.0 | 82 | 0.4605 | {'precision': 0.5800683670786222, 'recall': 0.5188146754468486, 'f1-score': 0.5477343265052762, 'support': 4252.0} | {'precision': 0.679080824088748, 'recall': 0.7855178735105408, 'f1-score': 0.7284317892052699, 'support': 2182.0} | {'precision': 0.9250369696280286, 'recall': 0.8767654986522911, 'f1-score': 0.900254621941769, 'support': 9275.0} | {'precision': 0.8497380970995231, 'recall': 0.8909016393442623, 'f1-score': 0.8698331399303749, 'support': 12200.0} | 0.8213 | {'precision': 0.7584810644737304, 'recall': 0.7679999217384857, 'f1-score': 0.7615634693956725, 'support': 27909.0} | {'precision': 0.8203349361458346, 'recall': 0.8212762908022502, 'f1-score': 0.8198154876923865, 'support': 27909.0} |
|
75 |
+
| No log | 3.0 | 123 | 0.4587 | {'precision': 0.6081277213352685, 'recall': 0.39416745061147695, 'f1-score': 0.478310502283105, 'support': 4252.0} | {'precision': 0.7005473025801408, 'recall': 0.8212648945921174, 'f1-score': 0.7561181434599156, 'support': 2182.0} | {'precision': 0.9445551517993201, 'recall': 0.8687870619946092, 'f1-score': 0.905088172526115, 'support': 9275.0} | {'precision': 0.8125, 'recall': 0.9366393442622951, 'f1-score': 0.8701644837039293, 'support': 12200.0} | 0.8224 | {'precision': 0.7664325439286823, 'recall': 0.7552146878651247, 'f1-score': 0.7524203254932662, 'support': 27909.0} | {'precision': 0.8164965537384401, 'recall': 0.8224228743416102, 'f1-score': 0.8131543783763286, 'support': 27909.0} |
|
76 |
+
| No log | 4.0 | 164 | 0.4491 | {'precision': 0.5829145728643216, 'recall': 0.6274694261523989, 'f1-score': 0.6043719560539133, 'support': 4252.0} | {'precision': 0.7112758486149044, 'recall': 0.8354720439963337, 'f1-score': 0.7683877766069548, 'support': 2182.0} | {'precision': 0.9357652656621729, 'recall': 0.8905660377358491, 'f1-score': 0.9126063418406806, 'support': 9275.0} | {'precision': 0.881426896667225, 'recall': 0.8627868852459016, 'f1-score': 0.8720072901996521, 'support': 12200.0} | 0.8340 | {'precision': 0.7778456459521561, 'recall': 0.8040735982826209, 'f1-score': 0.7893433411753001, 'support': 27909.0} | {'precision': 0.8407032729174679, 'recall': 0.8340320326776308, 'f1-score': 0.8366234708053201, 'support': 27909.0} |
|
77 |
+
| No log | 5.0 | 205 | 0.4611 | {'precision': 0.5805860805860806, 'recall': 0.5964252116650988, 'f1-score': 0.588399071925754, 'support': 4252.0} | {'precision': 0.7489102005231038, 'recall': 0.7873510540788268, 'f1-score': 0.7676496872207329, 'support': 2182.0} | {'precision': 0.9323308270676691, 'recall': 0.8957412398921832, 'f1-score': 0.9136698559331353, 'support': 9275.0} | {'precision': 0.8673800259403373, 'recall': 0.8770491803278688, 'f1-score': 0.8721878056732963, 'support': 12200.0} | 0.8335 | {'precision': 0.7823017835292977, 'recall': 0.7891416714909945, 'f1-score': 0.7854766051882296, 'support': 27909.0} | {'precision': 0.8360091300196414, 'recall': 0.8334945716435559, 'f1-score': 0.8345646069131102, 'support': 27909.0} |
|
78 |
+
| No log | 6.0 | 246 | 0.4642 | {'precision': 0.5962333486449242, 'recall': 0.6105362182502352, 'f1-score': 0.6033000232396003, 'support': 4252.0} | {'precision': 0.7385488447507094, 'recall': 0.8350137488542622, 'f1-score': 0.783824478382448, 'support': 2182.0} | {'precision': 0.9409678526484384, 'recall': 0.8867924528301887, 'f1-score': 0.9130772646536413, 'support': 9275.0} | {'precision': 0.8715477443913501, 'recall': 0.8820491803278688, 'f1-score': 0.8767670183729173, 'support': 12200.0} | 0.8386 | {'precision': 0.7868244476088555, 'recall': 0.8035979000656387, 'f1-score': 0.7942421961621517, 'support': 27909.0} | {'precision': 0.8422751475356695, 'recall': 0.8385825360994661, 'f1-score': 0.8399041873394746, 'support': 27909.0} |
|
79 |
+
| No log | 7.0 | 287 | 0.4624 | {'precision': 0.5906025179856115, 'recall': 0.617826904985889, 'f1-score': 0.6039080459770114, 'support': 4252.0} | {'precision': 0.7631810193321616, 'recall': 0.7960586617781852, 'f1-score': 0.7792732166890982, 'support': 2182.0} | {'precision': 0.9296403841858387, 'recall': 0.897466307277628, 'f1-score': 0.913270064183444, 'support': 9275.0} | {'precision': 0.8734363502575423, 'recall': 0.875655737704918, 'f1-score': 0.8745446359133887, 'support': 12200.0} | 0.8374 | {'precision': 0.7892150679402886, 'recall': 0.7967519029366551, 'f1-score': 0.7927489906907357, 'support': 27909.0} | {'precision': 0.8404042039171331, 'recall': 0.8374001218245011, 'f1-score': 0.8387335832080923, 'support': 27909.0} |
|
80 |
+
|
81 |
+
|
82 |
+
### Framework versions
|
83 |
+
|
84 |
+
- Transformers 4.37.2
|
85 |
+
- Pytorch 2.2.0+cu121
|
86 |
+
- Datasets 2.17.0
|
87 |
+
- Tokenizers 0.15.2
|
model.safetensors
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 592324828
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:30f305587ff6545597b5296b28378ab26a4416dc09e0081f2ee57c1edba11f55
|
3 |
size 592324828
|