BlinkDL
/

rwkv-4-pile-14b

Text Generation

Model card Files Files and versions Community

BlinkDL commited on Dec 18, 2022

Commit

f3de8f7

•

1 Parent(s): 474f10a

Update README.md

Files changed (1) hide show

README.md +7 -7

README.md CHANGED Viewed

@@ -26,10 +26,10 @@ ctx_len = 1024
 n_layer = 40
 n_embd = 5120
-Preview checkpoint: RWKV-4-Pile-14B-20221128-3061.pth : Trained on the Pile for 126B tokens.
-* Pile loss 1.8412
-* LAMBADA ppl 4.14, acc 68.78%
-* PIQA acc 76.66%
-* SC2016 acc 73.86%
-* Hellaswag acc_norm 67.06%
-* WinoGrande acc 64.01%

 n_layer = 40
 n_embd = 5120
+Preview checkpoint: RWKV-4-Pile-14B-20221217-3794.pth : Trained on the Pile for 156B tokens.
+* Pile loss 1.8189
+* LAMBADA ppl 4.04, acc 69.47%
+* PIQA acc 77.20%
+* SC2016 acc 74.35%
+* Hellaswag acc_norm 67.76%
+* WinoGrande acc 63.54%