BlinkDL commited on
Commit
a3ce566
1 Parent(s): a4ba8d0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -24,10 +24,10 @@ ctx_len = 1024
24
  n_layer = 40
25
  n_embd = 5120
26
 
27
- Preview checkpoint: RWKV-4-Pile-14B-20230121-6210.pth : Trained on the Pile for 256B tokens.
28
- * Pile loss 1.7746
29
- * LAMBADA ppl 3.83, acc 70.79%
30
- * PIQA acc 77.53%
31
- * SC2016 acc 74.83%
32
- * Hellaswag acc_norm 69.78%
33
- * WinoGrande acc 63.54%
 
24
  n_layer = 40
25
  n_embd = 5120
26
 
27
+ Final checkpoint: RWKV-4-Pile-14B-20230213-8019.pth : Trained on the Pile for 331B tokens.
28
+ * Pile loss 1.7579
29
+ * LAMBADA ppl 3.81, acc 71.05%
30
+ * PIQA acc 77.42%
31
+ * SC2016 acc 75.57%
32
+ * Hellaswag acc_norm 70.24%
33
+ * WinoGrande acc 62.98%