versae's picture
Saving weights and logs of step 1000
5f161cc
raw
history blame
5.3 kB
[13:56:50] - WARNING - __main__ - Unable to display metrics through TensorBoard because some package are not installed: No module named 'tensorflow'
Epoch ... (1/3): 0%| | 0/3 [00:00<?, ?it/s]
Training...: 2%|████▏ | 1000/54217 [18:07<14:47:08, 1.00s/it]
Training...: 2%|████▏ | 1000/54217 [18:09<14:47:08, 1.00s/it]
Evaluating ...: 0%| | 0/1167 [00:00<?, ?it/s]
Training...: 4%|████████▎ | 2000/54217 [46:46<14:17:16, 1.02it/s]
Training...: 4%|████████▎ | 2000/54217 [46:47<14:17:16, 1.02it/s]
Step... (2000 | Loss: 1.330615520477295, Acc: 0.7090045213699341): 0%| | 0/3 [1:15:55<?, ?it/s]
Traceback (most recent call last):
File "run_mlm_flax.py", line 815, in <module>
main()
File "run_mlm_flax.py", line 718, in main
samples = [tokenized_datasets["train"][int(idx)] for idx in batch_idx]
File "run_mlm_flax.py", line 718, in <listcomp>
samples = [tokenized_datasets["train"][int(idx)] for idx in batch_idx]
File "/data/flax/lib/python3.8/site-packages/jax/_src/device_array.py", line 247, in <genexpr>
return (sl for chunk in self._chunk_iter(100) for sl in chunk._unstack())
File "/data/flax/lib/python3.8/site-packages/jax/_src/numpy/lax_numpy.py", line 6792, in _chunk_iter
for i in range(num_chunks):
File "/usr/lib/python3.8/functools.py", line 399, in _method
return self.func(cls_or_self, *self.args, *args, **keywords)
File "/data/flax/lib/python3.8/site-packages/jax/_src/device_array.py", line 41, in _forward_method
return fun(getattr(self, attrname), *args)
KeyboardInterrupt