Update README.md
Browse files
README.md
CHANGED
|
@@ -26,6 +26,26 @@ It achieves the following results on the evaluation set:
|
|
| 26 |
- Accuracy: 0.1881
|
| 27 |
- Num Input Tokens Seen: 766509056
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
## Training procedure
|
| 30 |
|
| 31 |
### Training hyperparameters
|
|
|
|
| 26 |
- Accuracy: 0.1881
|
| 27 |
- Num Input Tokens Seen: 766509056
|
| 28 |
|
| 29 |
+
## evals
|
| 30 |
+
|
| 31 |
+
tl;dr its bad/would need more training:
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
hf (pretrained=pszemraj/griffin-llama3t-8L-v0.02-fineweb,trust_remote_code=True,dtype=float), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 4
|
| 35 |
+
|
| 36 |
+
| Tasks |Version|Filter|n-shot| Metric | Value | | Stderr |
|
| 37 |
+
|--------------|------:|------|-----:|----------|----------:|---|---------:|
|
| 38 |
+
|winogrande | 1|none | 0|acc | 0.4964|± | 0.0141|
|
| 39 |
+
|piqa | 1|none | 0|acc | 0.5332|± | 0.0116|
|
| 40 |
+
| | |none | 0|acc_norm | 0.5299|± | 0.0116|
|
| 41 |
+
|openbookqa | 1|none | 0|acc | 0.1280|± | 0.0150|
|
| 42 |
+
| | |none | 0|acc_norm | 0.2320|± | 0.0189|
|
| 43 |
+
|lambada_openai| 1|none | 0|perplexity|638060.0702|± |43608.0044|
|
| 44 |
+
| | |none | 0|acc | 0.0000|± | 0.0000|
|
| 45 |
+
|boolq | 2|none | 0|acc | 0.3783|± | 0.0085|
|
| 46 |
+
|arc_easy | 1|none | 0|acc | 0.2614|± | 0.0090|
|
| 47 |
+
| | |none | 0|acc_norm | 0.2744|± | 0.0092|
|
| 48 |
+
|
| 49 |
## Training procedure
|
| 50 |
|
| 51 |
### Training hyperparameters
|