BlinkDL
/

rwkv-4-pile-14b

Text Generation

Model card Files Files and versions Community

BlinkDL commited on Jan 1, 2023

Commit

82ffb69

•

1 Parent(s): 6d0e61b

Update README.md

Files changed (1) hide show

README.md +7 -7

README.md CHANGED Viewed

@@ -26,10 +26,10 @@ ctx_len = 1024
 n_layer = 40
 n_embd = 5120
-Preview checkpoint: RWKV-4-Pile-14B-20221217-3794.pth : Trained on the Pile for 156B tokens.
-* Pile loss 1.8189
-* LAMBADA ppl 4.04, acc 69.47%
-* PIQA acc 77.20%
-* SC2016 acc 74.35%
-* Hellaswag acc_norm 67.76%
-* WinoGrande acc 63.54%

 n_layer = 40
 n_embd = 5120
+Preview checkpoint: RWKV-4-Pile-14B-20221231-4514.pth : Trained on the Pile for 186B tokens.
+* Pile loss 1.8014
+* LAMBADA ppl 3.92, acc 69.96%
+* PIQA acc 77.42%
+* SC2016 acc 74.18%
+* Hellaswag acc_norm 68.47%
+* WinoGrande acc 62.35%