Saving weights and logs at step 320000

README.md

@@ -14,6 +14,8 @@ datasets:
 ---
 # GPT2-Medium pre-trained on cleaned Dutch mC4 🇳🇱
 
+Training is not finished!
+
 Dataset:
 
 * [mC4 NL Cleaned](https://huggingface.co/datasets/yhavinga/mc4_nl_cleaned)
@@ -26,7 +28,7 @@ Tokenizer:
 
 Training details:
 
-* Trained for 280k steps (30 dec 2021)
+* Trained for 320K of 520K steps (31 dec 2021)
 * Block size: 512
 * Optimizer: adam, lr 8e-4, beta1 0.9, beta2 0.98
 * Warmup steps: 5000

flax_model.msgpack

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d2bc942466bedf81fea88c9bbeaaafa7dfb2fec485a78c89c52705b841a2bf0a
+oid sha256:a5b7d6e0a2d7b5a04dbf348d2196ad0fc3797612d0a846685cea7685fb916543
 size 1419302302

runs/events.out.tfevents.1640332964.t1v-n-f9cfcc28-w-0.384322.0.v2

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c60bb6a82a55ceae1859f8fc81e83b0c19ee72e64de5ecdc95e012746328f4c6
-size 43681985
+oid sha256:78a17fcd88365f6bcb23b316b06024faf91960c419d1cab9d39b055a0ae3fcb8
+size 49426035