marinone94 commited on
Commit
7c3cc17
1 Parent(s): e45caa6

Training in progress, step 1500

Browse files
.gitattributes CHANGED
@@ -25,3 +25,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
28
+ wandb/run-20220127_103723-scy0vyln/run-scy0vyln.wandb filter=lfs diff=lfs merge=lfs -text
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:34a701553b13c726a8968f4902973c74969369bed0d173e46ea4c9f4633f9fa7
3
  size 1262075377
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4b43f6d3376e4159d59d4515731a13e694d35e3d25d2d8c5518d3de2edea5ee
3
  size 1262075377
wandb/run-20220127_103723-scy0vyln/files/output.log CHANGED
@@ -775,3 +775,390 @@ Configuration saved in ./checkpoint-1000/config.json
775
  {'eval_loss': 2.963273763656616, 'eval_wer': 1.0, 'eval_runtime': 130.7049, 'eval_samples_per_second': 35.347, 'eval_steps_per_second': 4.422, 'epoch': 2.91}
776
  Model weights saved in ./checkpoint-1000/pytorch_model.bin
777
  Configuration saved in ./checkpoint-1000/preprocessor_config.json
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
775
  {'eval_loss': 2.963273763656616, 'eval_wer': 1.0, 'eval_runtime': 130.7049, 'eval_samples_per_second': 35.347, 'eval_steps_per_second': 4.422, 'epoch': 2.91}
776
  Model weights saved in ./checkpoint-1000/pytorch_model.bin
777
  Configuration saved in ./checkpoint-1000/preprocessor_config.json
778
+ Configuration saved in ./preprocessor_config.json
779
+
780
+
781
+
782
+
783
+
784
+
785
+
786
+
787
+
788
+
789
+
790
+
791
+
792
+
793
+
794
+
795
+
796
+
797
+
798
+
799
+
800
+
801
+
802
+
803
+
804
+
805
+
806
+
807
+
808
+
809
+
810
+
811
+
812
+
813
+
814
+
815
+
816
+
817
+
818
+
819
+
820
+
821
+
822
+
823
+
824
+
825
+
826
+
827
+
828
+
829
+
830
+
831
+
832
+
833
+
834
+
835
+
836
+
837
+
838
+ 6%|█████████▊ | 1098/17200 [30:25<6:37:11, 1.48s/it]
839
+
840
+
841
+
842
+
843
+
844
+
845
+
846
+
847
+
848
+
849
+
850
+
851
+
852
+
853
+
854
+
855
+
856
+
857
+
858
+
859
+
860
+
861
+
862
+
863
+
864
+
865
+
866
+
867
+
868
+
869
+
870
+
871
+
872
+
873
+
874
+
875
+
876
+
877
+
878
+
879
+
880
+
881
+
882
+
883
+
884
+
885
+
886
+
887
+
888
+
889
+
890
+
891
+
892
+
893
+
894
+
895
+
896
+
897
+
898
+
899
+
900
+
901
+
902
+ 7%|██████████▋ | 1200/17200 [32:33<6:11:06, 1.39s/it]
903
+
904
+
905
+
906
+
907
+
908
+
909
+
910
+
911
+
912
+
913
+
914
+
915
+
916
+
917
+
918
+
919
+
920
+
921
+
922
+
923
+
924
+
925
+
926
+
927
+
928
+
929
+
930
+
931
+
932
+
933
+
934
+
935
+
936
+
937
+
938
+
939
+
940
+
941
+
942
+
943
+
944
+
945
+
946
+
947
+
948
+
949
+
950
+
951
+
952
+
953
+
954
+
955
+
956
+
957
+
958
+
959
+
960
+
961
+
962
+
963
+
964
+
965
+ 8%|███████████▋ | 1300/17200 [34:38<6:10:35, 1.40s/it]
966
+
967
+
968
+
969
+
970
+
971
+
972
+
973
+
974
+
975
+
976
+
977
+
978
+
979
+
980
+
981
+
982
+
983
+
984
+
985
+
986
+
987
+
988
+
989
+
990
+
991
+
992
+
993
+
994
+
995
+
996
+
997
+
998
+
999
+
1000
+
1001
+
1002
+
1003
+
1004
+
1005
+
1006
+
1007
+
1008
+
1009
+
1010
+
1011
+
1012
+
1013
+
1014
+
1015
+
1016
+
1017
+
1018
+
1019
+
1020
+
1021
+
1022
+
1023
+
1024
+
1025
+
1026
+
1027
+
1028
+ 8%|████████████▌ | 1398/17200 [36:42<5:48:31, 1.32s/it]
1029
+
1030
+
1031
+
1032
+
1033
+
1034
+
1035
+
1036
+
1037
+
1038
+
1039
+
1040
+
1041
+
1042
+
1043
+
1044
+
1045
+
1046
+
1047
+
1048
+
1049
+
1050
+
1051
+
1052
+
1053
+
1054
+
1055
+
1056
+
1057
+
1058
+
1059
+
1060
+
1061
+
1062
+
1063
+
1064
+
1065
+
1066
+
1067
+
1068
+
1069
+
1070
+
1071
+
1072
+
1073
+
1074
+
1075
+
1076
+
1077
+
1078
+
1079
+
1080
+
1081
+
1082
+
1083
+
1084
+
1085
+
1086
+
1087
+
1088
+
1089
+
1090
+
1091
+ 9%|█████████████▍ | 1498/17200 [38:48<5:49:49, 1.34s/it]
1092
+ 9%|█████████████▍ | 1500/17200 [38:50<5:34:26, 1.28s/it]The following columns in the evaluation set don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
1093
+ ***** Running Evaluation *****
1094
+ Num examples = 4620
1095
+ Batch size = 8
1096
+
1097
+
1098
+
1099
+
1100
+
1101
+
1102
+
1103
+
1104
+
1105
+
1106
+
1107
+
1108
+
1109
+
1110
+
1111
+
1112
+
1113
+
1114
+
1115
+
1116
+
1117
+
1118
+
1119
+
1120
+
1121
+
1122
+
1123
+
1124
+
1125
+
1126
+
1127
+
1128
+
1129
+
1130
+
1131
+
1132
+
1133
+
1134
+
1135
+
1136
+
1137
+
1138
+
1139
+
1140
+
1141
+
1142
+
1143
+
1144
+
1145
+
1146
+
1147
+
1148
+
1149
+
1150
+
1151
+
1152
+
1153
+
1154
+
1155
+
1156
+
1157
+
1158
+ 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 578/578 [02:05<00:00, 4.82it/s]
1159
+
1160
+ Configuration saved in ./checkpoint-1500/config.json
1161
+ Model weights saved in ./checkpoint-1500/pytorch_model.bin
1162
+ Configuration saved in ./checkpoint-1500/preprocessor_config.json
1163
+ Configuration saved in ./preprocessor_config.json
1164
+ 01/27/2022 11:19:35 - WARNING - huggingface_hub.repository - Adding files tracked by Git LFS: ['wandb/run-20220127_103723-scy0vyln/run-scy0vyln.wandb']. This may take a bit of time if the files are large.
wandb/run-20220127_103723-scy0vyln/files/wandb-summary.json CHANGED
The diff for this file is too large to render. See raw diff
 
wandb/run-20220127_103723-scy0vyln/logs/debug-internal.log CHANGED
@@ -2324,3 +2324,546 @@
2324
  2022-01-27 11:05:34,273 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2325
  2022-01-27 11:05:34,274 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2326
  2022-01-27 11:05:35,226 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2324
  2022-01-27 11:05:34,273 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2325
  2022-01-27 11:05:34,274 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2326
  2022-01-27 11:05:35,226 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2327
+ 2022-01-27 11:05:49,396 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2328
+ 2022-01-27 11:05:49,461 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2329
+ 2022-01-27 11:05:49,461 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2330
+ 2022-01-27 11:05:51,396 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2331
+ 2022-01-27 11:05:53,398 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2332
+ 2022-01-27 11:05:55,399 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2333
+ 2022-01-27 11:05:57,400 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2334
+ 2022-01-27 11:05:59,402 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2335
+ 2022-01-27 11:06:01,404 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2336
+ 2022-01-27 11:06:03,405 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2337
+ 2022-01-27 11:06:04,733 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2338
+ 2022-01-27 11:06:04,733 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2339
+ 2022-01-27 11:06:05,462 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2340
+ 2022-01-27 11:06:06,406 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2341
+ 2022-01-27 11:06:08,408 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2342
+ 2022-01-27 11:06:10,409 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2343
+ 2022-01-27 11:06:12,410 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2344
+ 2022-01-27 11:06:14,411 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2345
+ 2022-01-27 11:06:16,412 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2346
+ 2022-01-27 11:06:18,413 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2347
+ 2022-01-27 11:06:19,931 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2348
+ 2022-01-27 11:06:19,931 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2349
+ 2022-01-27 11:06:20,415 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2350
+ 2022-01-27 11:06:22,416 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2351
+ 2022-01-27 11:06:24,417 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2352
+ 2022-01-27 11:06:28,420 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2353
+ 2022-01-27 11:06:30,421 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2354
+ 2022-01-27 11:06:32,422 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2355
+ 2022-01-27 11:06:34,424 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2356
+ 2022-01-27 11:06:35,143 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2357
+ 2022-01-27 11:06:35,144 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2358
+ 2022-01-27 11:06:35,690 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2359
+ 2022-01-27 11:06:36,425 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2360
+ 2022-01-27 11:06:38,427 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2361
+ 2022-01-27 11:06:40,429 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2362
+ 2022-01-27 11:06:42,430 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2363
+ 2022-01-27 11:06:44,432 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2364
+ 2022-01-27 11:06:46,433 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2365
+ 2022-01-27 11:06:48,435 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2366
+ 2022-01-27 11:06:50,383 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2367
+ 2022-01-27 11:06:50,383 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2368
+ 2022-01-27 11:06:50,436 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2369
+ 2022-01-27 11:06:52,437 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2370
+ 2022-01-27 11:06:54,439 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2371
+ 2022-01-27 11:06:56,441 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2372
+ 2022-01-27 11:06:58,443 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2373
+ 2022-01-27 11:07:00,445 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2374
+ 2022-01-27 11:07:02,447 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2375
+ 2022-01-27 11:07:04,449 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2376
+ 2022-01-27 11:07:05,587 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2377
+ 2022-01-27 11:07:05,587 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2378
+ 2022-01-27 11:07:05,915 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2379
+ 2022-01-27 11:07:06,451 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2380
+ 2022-01-27 11:07:08,453 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2381
+ 2022-01-27 11:07:10,455 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2382
+ 2022-01-27 11:07:12,457 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2383
+ 2022-01-27 11:07:14,459 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2384
+ 2022-01-27 11:07:16,461 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2385
+ 2022-01-27 11:07:18,462 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2386
+ 2022-01-27 11:07:20,464 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2387
+ 2022-01-27 11:07:20,754 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2388
+ 2022-01-27 11:07:20,754 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2389
+ 2022-01-27 11:07:22,466 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2390
+ 2022-01-27 11:07:24,468 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2391
+ 2022-01-27 11:07:26,470 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2392
+ 2022-01-27 11:07:30,473 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2393
+ 2022-01-27 11:07:32,475 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2394
+ 2022-01-27 11:07:34,477 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2395
+ 2022-01-27 11:07:36,016 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2396
+ 2022-01-27 11:07:36,016 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2397
+ 2022-01-27 11:07:36,189 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2398
+ 2022-01-27 11:07:36,479 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2399
+ 2022-01-27 11:07:38,481 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2400
+ 2022-01-27 11:07:40,483 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2401
+ 2022-01-27 11:07:42,485 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2402
+ 2022-01-27 11:07:44,486 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2403
+ 2022-01-27 11:07:46,488 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2404
+ 2022-01-27 11:07:48,490 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2405
+ 2022-01-27 11:07:51,199 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2406
+ 2022-01-27 11:07:51,200 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2407
+ 2022-01-27 11:07:51,492 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2408
+ 2022-01-27 11:07:53,494 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2409
+ 2022-01-27 11:07:54,225 DEBUG SenderThread:1977561 [sender.py:send():234] send: history
2410
+ 2022-01-27 11:07:54,365 DEBUG SenderThread:1977561 [sender.py:send():234] send: summary
2411
+ 2022-01-27 11:07:54,428 INFO SenderThread:1977561 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2412
+ 2022-01-27 11:07:54,496 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/wandb-summary.json
2413
+ 2022-01-27 11:07:55,497 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2414
+ 2022-01-27 11:07:57,499 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2415
+ 2022-01-27 11:07:59,501 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2416
+ 2022-01-27 11:08:01,503 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2417
+ 2022-01-27 11:08:03,505 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2418
+ 2022-01-27 11:08:05,506 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2419
+ 2022-01-27 11:08:06,371 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2420
+ 2022-01-27 11:08:06,372 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2421
+ 2022-01-27 11:08:06,538 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2422
+ 2022-01-27 11:08:07,508 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2423
+ 2022-01-27 11:08:09,509 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2424
+ 2022-01-27 11:08:11,511 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2425
+ 2022-01-27 11:08:13,512 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2426
+ 2022-01-27 11:08:15,514 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2427
+ 2022-01-27 11:08:17,516 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2428
+ 2022-01-27 11:08:19,518 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2429
+ 2022-01-27 11:08:21,520 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2430
+ 2022-01-27 11:08:21,556 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2431
+ 2022-01-27 11:08:21,556 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2432
+ 2022-01-27 11:08:23,522 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2433
+ 2022-01-27 11:08:25,523 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2434
+ 2022-01-27 11:08:27,525 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2435
+ 2022-01-27 11:08:29,527 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2436
+ 2022-01-27 11:08:31,529 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2437
+ 2022-01-27 11:08:33,531 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2438
+ 2022-01-27 11:08:35,532 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2439
+ 2022-01-27 11:08:36,710 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2440
+ 2022-01-27 11:08:36,710 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2441
+ 2022-01-27 11:08:36,877 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2442
+ 2022-01-27 11:08:37,534 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2443
+ 2022-01-27 11:08:39,536 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2444
+ 2022-01-27 11:08:41,538 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2445
+ 2022-01-27 11:08:43,540 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2446
+ 2022-01-27 11:08:45,541 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2447
+ 2022-01-27 11:08:47,543 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2448
+ 2022-01-27 11:08:49,545 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2449
+ 2022-01-27 11:08:51,547 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2450
+ 2022-01-27 11:08:51,919 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2451
+ 2022-01-27 11:08:51,919 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2452
+ 2022-01-27 11:08:53,549 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2453
+ 2022-01-27 11:08:55,551 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2454
+ 2022-01-27 11:08:57,553 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2455
+ 2022-01-27 11:08:59,555 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2456
+ 2022-01-27 11:09:01,556 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2457
+ 2022-01-27 11:09:03,558 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2458
+ 2022-01-27 11:09:05,559 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2459
+ 2022-01-27 11:09:06,960 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2460
+ 2022-01-27 11:09:07,155 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2461
+ 2022-01-27 11:09:07,155 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2462
+ 2022-01-27 11:09:07,561 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2463
+ 2022-01-27 11:09:09,563 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2464
+ 2022-01-27 11:09:11,565 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2465
+ 2022-01-27 11:09:13,567 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2466
+ 2022-01-27 11:09:15,569 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2467
+ 2022-01-27 11:09:17,571 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2468
+ 2022-01-27 11:09:19,573 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2469
+ 2022-01-27 11:09:21,575 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2470
+ 2022-01-27 11:09:22,322 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2471
+ 2022-01-27 11:09:22,323 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2472
+ 2022-01-27 11:09:23,577 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2473
+ 2022-01-27 11:09:25,579 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2474
+ 2022-01-27 11:09:27,580 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2475
+ 2022-01-27 11:09:29,582 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2476
+ 2022-01-27 11:09:31,583 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2477
+ 2022-01-27 11:09:33,585 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2478
+ 2022-01-27 11:09:35,587 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2479
+ 2022-01-27 11:09:37,181 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2480
+ 2022-01-27 11:09:37,568 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2481
+ 2022-01-27 11:09:37,568 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2482
+ 2022-01-27 11:09:37,589 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2483
+ 2022-01-27 11:09:40,591 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2484
+ 2022-01-27 11:09:42,593 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2485
+ 2022-01-27 11:09:44,595 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2486
+ 2022-01-27 11:09:46,597 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2487
+ 2022-01-27 11:09:48,599 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2488
+ 2022-01-27 11:09:50,601 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2489
+ 2022-01-27 11:09:52,602 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2490
+ 2022-01-27 11:09:52,802 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2491
+ 2022-01-27 11:09:52,803 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2492
+ 2022-01-27 11:09:54,603 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2493
+ 2022-01-27 11:09:56,605 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2494
+ 2022-01-27 11:09:58,607 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2495
+ 2022-01-27 11:09:59,386 DEBUG SenderThread:1977561 [sender.py:send():234] send: history
2496
+ 2022-01-27 11:09:59,472 DEBUG SenderThread:1977561 [sender.py:send():234] send: summary
2497
+ 2022-01-27 11:09:59,503 INFO SenderThread:1977561 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2498
+ 2022-01-27 11:09:59,608 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/wandb-summary.json
2499
+ 2022-01-27 11:10:00,609 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2500
+ 2022-01-27 11:10:01,610 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2501
+ 2022-01-27 11:10:02,612 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2502
+ 2022-01-27 11:10:04,614 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2503
+ 2022-01-27 11:10:06,616 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2504
+ 2022-01-27 11:10:07,411 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2505
+ 2022-01-27 11:10:07,960 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2506
+ 2022-01-27 11:10:07,960 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2507
+ 2022-01-27 11:10:08,618 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2508
+ 2022-01-27 11:10:10,620 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2509
+ 2022-01-27 11:10:12,621 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2510
+ 2022-01-27 11:10:14,623 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2511
+ 2022-01-27 11:10:16,625 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2512
+ 2022-01-27 11:10:18,627 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2513
+ 2022-01-27 11:10:20,629 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2514
+ 2022-01-27 11:10:22,631 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2515
+ 2022-01-27 11:10:23,176 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2516
+ 2022-01-27 11:10:23,177 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2517
+ 2022-01-27 11:10:24,633 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2518
+ 2022-01-27 11:10:26,635 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2519
+ 2022-01-27 11:10:28,637 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2520
+ 2022-01-27 11:10:30,639 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2521
+ 2022-01-27 11:10:32,641 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2522
+ 2022-01-27 11:10:34,643 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2523
+ 2022-01-27 11:10:36,644 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2524
+ 2022-01-27 11:10:37,619 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2525
+ 2022-01-27 11:10:38,345 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2526
+ 2022-01-27 11:10:38,345 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2527
+ 2022-01-27 11:10:38,646 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2528
+ 2022-01-27 11:10:40,648 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2529
+ 2022-01-27 11:10:42,651 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2530
+ 2022-01-27 11:10:44,653 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2531
+ 2022-01-27 11:10:46,655 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2532
+ 2022-01-27 11:10:48,657 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2533
+ 2022-01-27 11:10:50,659 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2534
+ 2022-01-27 11:10:52,661 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2535
+ 2022-01-27 11:10:53,628 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2536
+ 2022-01-27 11:10:53,629 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2537
+ 2022-01-27 11:10:54,663 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2538
+ 2022-01-27 11:10:56,664 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2539
+ 2022-01-27 11:10:58,666 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2540
+ 2022-01-27 11:11:00,668 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2541
+ 2022-01-27 11:11:02,670 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2542
+ 2022-01-27 11:11:04,672 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2543
+ 2022-01-27 11:11:06,674 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2544
+ 2022-01-27 11:11:07,833 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2545
+ 2022-01-27 11:11:08,676 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2546
+ 2022-01-27 11:11:08,796 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2547
+ 2022-01-27 11:11:08,797 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2548
+ 2022-01-27 11:11:10,678 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2549
+ 2022-01-27 11:11:12,680 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2550
+ 2022-01-27 11:11:14,681 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2551
+ 2022-01-27 11:11:16,683 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2552
+ 2022-01-27 11:11:18,685 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2553
+ 2022-01-27 11:11:20,687 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2554
+ 2022-01-27 11:11:22,689 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2555
+ 2022-01-27 11:11:24,004 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2556
+ 2022-01-27 11:11:24,004 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2557
+ 2022-01-27 11:11:24,690 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2558
+ 2022-01-27 11:11:26,692 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2559
+ 2022-01-27 11:11:28,694 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2560
+ 2022-01-27 11:11:30,695 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2561
+ 2022-01-27 11:11:32,697 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2562
+ 2022-01-27 11:11:34,698 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2563
+ 2022-01-27 11:11:36,700 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2564
+ 2022-01-27 11:11:38,052 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2565
+ 2022-01-27 11:11:38,702 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2566
+ 2022-01-27 11:11:39,241 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2567
+ 2022-01-27 11:11:39,242 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2568
+ 2022-01-27 11:11:40,704 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2569
+ 2022-01-27 11:11:42,706 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2570
+ 2022-01-27 11:11:44,708 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2571
+ 2022-01-27 11:11:46,710 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2572
+ 2022-01-27 11:11:48,712 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2573
+ 2022-01-27 11:11:50,714 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2574
+ 2022-01-27 11:11:53,716 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2575
+ 2022-01-27 11:11:54,418 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2576
+ 2022-01-27 11:11:54,418 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2577
+ 2022-01-27 11:11:55,718 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2578
+ 2022-01-27 11:11:57,719 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2579
+ 2022-01-27 11:11:59,721 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2580
+ 2022-01-27 11:12:01,722 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2581
+ 2022-01-27 11:12:03,723 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2582
+ 2022-01-27 11:12:04,402 DEBUG SenderThread:1977561 [sender.py:send():234] send: history
2583
+ 2022-01-27 11:12:04,450 DEBUG SenderThread:1977561 [sender.py:send():234] send: summary
2584
+ 2022-01-27 11:12:04,481 INFO SenderThread:1977561 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2585
+ 2022-01-27 11:12:04,724 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/wandb-summary.json
2586
+ 2022-01-27 11:12:05,725 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2587
+ 2022-01-27 11:12:07,726 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2588
+ 2022-01-27 11:12:08,292 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2589
+ 2022-01-27 11:12:09,679 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2590
+ 2022-01-27 11:12:09,680 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2591
+ 2022-01-27 11:12:09,727 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2592
+ 2022-01-27 11:12:11,728 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2593
+ 2022-01-27 11:12:13,730 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2594
+ 2022-01-27 11:12:15,732 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2595
+ 2022-01-27 11:12:17,734 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2596
+ 2022-01-27 11:12:19,735 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2597
+ 2022-01-27 11:12:21,737 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2598
+ 2022-01-27 11:12:23,739 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2599
+ 2022-01-27 11:12:24,844 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2600
+ 2022-01-27 11:12:24,845 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2601
+ 2022-01-27 11:12:25,741 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2602
+ 2022-01-27 11:12:27,742 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2603
+ 2022-01-27 11:12:29,744 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2604
+ 2022-01-27 11:12:31,746 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2605
+ 2022-01-27 11:12:33,748 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2606
+ 2022-01-27 11:12:35,750 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2607
+ 2022-01-27 11:12:37,754 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2608
+ 2022-01-27 11:12:38,514 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2609
+ 2022-01-27 11:12:39,756 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2610
+ 2022-01-27 11:12:40,104 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2611
+ 2022-01-27 11:12:40,105 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2612
+ 2022-01-27 11:12:41,757 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2613
+ 2022-01-27 11:12:43,759 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2614
+ 2022-01-27 11:12:45,761 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2615
+ 2022-01-27 11:12:47,763 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2616
+ 2022-01-27 11:12:49,765 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2617
+ 2022-01-27 11:12:51,766 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2618
+ 2022-01-27 11:12:53,768 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2619
+ 2022-01-27 11:12:55,265 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2620
+ 2022-01-27 11:12:55,266 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2621
+ 2022-01-27 11:12:55,770 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2622
+ 2022-01-27 11:12:57,772 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2623
+ 2022-01-27 11:12:59,774 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2624
+ 2022-01-27 11:13:01,776 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2625
+ 2022-01-27 11:13:03,778 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2626
+ 2022-01-27 11:13:05,780 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2627
+ 2022-01-27 11:13:07,782 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2628
+ 2022-01-27 11:13:08,755 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2629
+ 2022-01-27 11:13:09,784 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2630
+ 2022-01-27 11:13:10,552 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2631
+ 2022-01-27 11:13:10,553 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2632
+ 2022-01-27 11:13:11,786 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2633
+ 2022-01-27 11:13:13,788 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2634
+ 2022-01-27 11:13:15,791 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2635
+ 2022-01-27 11:13:17,793 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2636
+ 2022-01-27 11:13:19,794 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2637
+ 2022-01-27 11:13:21,796 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2638
+ 2022-01-27 11:13:23,798 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2639
+ 2022-01-27 11:13:25,718 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2640
+ 2022-01-27 11:13:25,719 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2641
+ 2022-01-27 11:13:25,800 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2642
+ 2022-01-27 11:13:27,801 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2643
+ 2022-01-27 11:13:29,803 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2644
+ 2022-01-27 11:13:31,805 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2645
+ 2022-01-27 11:13:33,807 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2646
+ 2022-01-27 11:13:35,809 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2647
+ 2022-01-27 11:13:37,811 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2648
+ 2022-01-27 11:13:38,972 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2649
+ 2022-01-27 11:13:39,813 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2650
+ 2022-01-27 11:13:40,887 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2651
+ 2022-01-27 11:13:40,887 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2652
+ 2022-01-27 11:13:41,815 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2653
+ 2022-01-27 11:13:43,816 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2654
+ 2022-01-27 11:13:45,818 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2655
+ 2022-01-27 11:13:47,819 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2656
+ 2022-01-27 11:13:49,821 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2657
+ 2022-01-27 11:13:51,823 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2658
+ 2022-01-27 11:13:53,824 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2659
+ 2022-01-27 11:13:55,826 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2660
+ 2022-01-27 11:13:56,115 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2661
+ 2022-01-27 11:13:56,115 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2662
+ 2022-01-27 11:13:58,829 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2663
+ 2022-01-27 11:14:00,831 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2664
+ 2022-01-27 11:14:02,832 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2665
+ 2022-01-27 11:14:04,833 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2666
+ 2022-01-27 11:14:06,835 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2667
+ 2022-01-27 11:14:08,837 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2668
+ 2022-01-27 11:14:09,186 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2669
+ 2022-01-27 11:14:10,839 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2670
+ 2022-01-27 11:14:10,951 DEBUG SenderThread:1977561 [sender.py:send():234] send: history
2671
+ 2022-01-27 11:14:11,046 DEBUG SenderThread:1977561 [sender.py:send():234] send: summary
2672
+ 2022-01-27 11:14:11,098 INFO SenderThread:1977561 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2673
+ 2022-01-27 11:14:11,290 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2674
+ 2022-01-27 11:14:11,291 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2675
+ 2022-01-27 11:14:11,840 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/wandb-summary.json
2676
+ 2022-01-27 11:14:11,840 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2677
+ 2022-01-27 11:14:12,841 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2678
+ 2022-01-27 11:14:14,843 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2679
+ 2022-01-27 11:14:16,844 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2680
+ 2022-01-27 11:14:18,846 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2681
+ 2022-01-27 11:14:20,847 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2682
+ 2022-01-27 11:14:22,849 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2683
+ 2022-01-27 11:14:24,850 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2684
+ 2022-01-27 11:14:26,558 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2685
+ 2022-01-27 11:14:26,559 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2686
+ 2022-01-27 11:14:26,852 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2687
+ 2022-01-27 11:14:28,853 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2688
+ 2022-01-27 11:14:30,854 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2689
+ 2022-01-27 11:14:32,856 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2690
+ 2022-01-27 11:14:34,857 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2691
+ 2022-01-27 11:14:36,859 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2692
+ 2022-01-27 11:14:38,861 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2693
+ 2022-01-27 11:14:39,408 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2694
+ 2022-01-27 11:14:40,863 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2695
+ 2022-01-27 11:14:41,758 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2696
+ 2022-01-27 11:14:41,759 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2697
+ 2022-01-27 11:14:42,865 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2698
+ 2022-01-27 11:14:44,866 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2699
+ 2022-01-27 11:14:46,869 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2700
+ 2022-01-27 11:14:48,871 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2701
+ 2022-01-27 11:14:50,875 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2702
+ 2022-01-27 11:14:52,877 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2703
+ 2022-01-27 11:14:54,879 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2704
+ 2022-01-27 11:14:56,881 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2705
+ 2022-01-27 11:14:56,944 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2706
+ 2022-01-27 11:14:56,945 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2707
+ 2022-01-27 11:14:58,883 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2708
+ 2022-01-27 11:15:00,885 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2709
+ 2022-01-27 11:15:02,886 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2710
+ 2022-01-27 11:15:04,887 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2711
+ 2022-01-27 11:15:06,888 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2712
+ 2022-01-27 11:15:08,890 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2713
+ 2022-01-27 11:15:09,634 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2714
+ 2022-01-27 11:15:10,891 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2715
+ 2022-01-27 11:15:12,162 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2716
+ 2022-01-27 11:15:12,162 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2717
+ 2022-01-27 11:15:12,892 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2718
+ 2022-01-27 11:15:14,893 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2719
+ 2022-01-27 11:15:16,894 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2720
+ 2022-01-27 11:15:18,895 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2721
+ 2022-01-27 11:15:20,896 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2722
+ 2022-01-27 11:15:22,898 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2723
+ 2022-01-27 11:15:24,900 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2724
+ 2022-01-27 11:15:26,902 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2725
+ 2022-01-27 11:15:27,320 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2726
+ 2022-01-27 11:15:27,321 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2727
+ 2022-01-27 11:15:28,904 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2728
+ 2022-01-27 11:15:30,906 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2729
+ 2022-01-27 11:15:32,908 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2730
+ 2022-01-27 11:15:34,910 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2731
+ 2022-01-27 11:15:36,912 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2732
+ 2022-01-27 11:15:38,914 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2733
+ 2022-01-27 11:15:39,845 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2734
+ 2022-01-27 11:15:40,916 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2735
+ 2022-01-27 11:15:42,488 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2736
+ 2022-01-27 11:15:42,488 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2737
+ 2022-01-27 11:15:42,918 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2738
+ 2022-01-27 11:15:44,920 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2739
+ 2022-01-27 11:15:47,922 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2740
+ 2022-01-27 11:15:49,924 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2741
+ 2022-01-27 11:15:51,926 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2742
+ 2022-01-27 11:15:53,928 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2743
+ 2022-01-27 11:15:55,930 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2744
+ 2022-01-27 11:15:57,656 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2745
+ 2022-01-27 11:15:57,657 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2746
+ 2022-01-27 11:15:57,932 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2747
+ 2022-01-27 11:15:59,934 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2748
+ 2022-01-27 11:16:01,936 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2749
+ 2022-01-27 11:16:03,938 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2750
+ 2022-01-27 11:16:05,940 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2751
+ 2022-01-27 11:16:07,942 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2752
+ 2022-01-27 11:16:09,943 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2753
+ 2022-01-27 11:16:10,054 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2754
+ 2022-01-27 11:16:11,945 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2755
+ 2022-01-27 11:16:12,850 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2756
+ 2022-01-27 11:16:12,851 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2757
+ 2022-01-27 11:16:13,947 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2758
+ 2022-01-27 11:16:15,949 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2759
+ 2022-01-27 11:16:16,173 DEBUG SenderThread:1977561 [sender.py:send():234] send: history
2760
+ 2022-01-27 11:16:16,278 DEBUG SenderThread:1977561 [sender.py:send():234] send: summary
2761
+ 2022-01-27 11:16:16,322 INFO SenderThread:1977561 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2762
+ 2022-01-27 11:16:16,951 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/wandb-summary.json
2763
+ 2022-01-27 11:16:16,951 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2764
+ 2022-01-27 11:16:17,952 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2765
+ 2022-01-27 11:16:19,955 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2766
+ 2022-01-27 11:16:21,957 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2767
+ 2022-01-27 11:16:23,958 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2768
+ 2022-01-27 11:16:25,960 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2769
+ 2022-01-27 11:16:27,961 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2770
+ 2022-01-27 11:16:28,042 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2771
+ 2022-01-27 11:16:28,043 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2772
+ 2022-01-27 11:16:29,963 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2773
+ 2022-01-27 11:16:31,964 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2774
+ 2022-01-27 11:16:33,965 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2775
+ 2022-01-27 11:16:35,967 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2776
+ 2022-01-27 11:16:37,968 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2777
+ 2022-01-27 11:16:39,970 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2778
+ 2022-01-27 11:16:40,306 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2779
+ 2022-01-27 11:16:41,972 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2780
+ 2022-01-27 11:16:43,246 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2781
+ 2022-01-27 11:16:43,246 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2782
+ 2022-01-27 11:16:43,973 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2783
+ 2022-01-27 11:16:45,975 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2784
+ 2022-01-27 11:16:47,976 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2785
+ 2022-01-27 11:16:49,978 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2786
+ 2022-01-27 11:16:51,979 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2787
+ 2022-01-27 11:16:53,981 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2788
+ 2022-01-27 11:16:55,983 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2789
+ 2022-01-27 11:16:57,985 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2790
+ 2022-01-27 11:16:58,442 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2791
+ 2022-01-27 11:16:58,443 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2792
+ 2022-01-27 11:16:59,986 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2793
+ 2022-01-27 11:17:01,989 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2794
+ 2022-01-27 11:17:04,991 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2795
+ 2022-01-27 11:17:06,993 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2796
+ 2022-01-27 11:17:08,995 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2797
+ 2022-01-27 11:17:10,564 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2798
+ 2022-01-27 11:17:10,997 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2799
+ 2022-01-27 11:17:12,999 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2800
+ 2022-01-27 11:17:13,635 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2801
+ 2022-01-27 11:17:13,635 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2802
+ 2022-01-27 11:17:15,001 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2803
+ 2022-01-27 11:17:17,003 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2804
+ 2022-01-27 11:17:19,005 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2805
+ 2022-01-27 11:17:21,007 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2806
+ 2022-01-27 11:17:23,009 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2807
+ 2022-01-27 11:17:25,011 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2808
+ 2022-01-27 11:17:27,013 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2809
+ 2022-01-27 11:17:28,817 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2810
+ 2022-01-27 11:17:28,817 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2811
+ 2022-01-27 11:17:29,014 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2812
+ 2022-01-27 11:17:31,016 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2813
+ 2022-01-27 11:17:33,017 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2814
+ 2022-01-27 11:17:35,019 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2815
+ 2022-01-27 11:17:37,021 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2816
+ 2022-01-27 11:17:39,023 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2817
+ 2022-01-27 11:17:40,817 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2818
+ 2022-01-27 11:17:41,025 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2819
+ 2022-01-27 11:17:43,027 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2820
+ 2022-01-27 11:17:44,073 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2821
+ 2022-01-27 11:17:44,073 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2822
+ 2022-01-27 11:17:45,029 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2823
+ 2022-01-27 11:17:47,030 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2824
+ 2022-01-27 11:17:49,032 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2825
+ 2022-01-27 11:17:51,033 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2826
+ 2022-01-27 11:17:53,035 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2827
+ 2022-01-27 11:17:55,037 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2828
+ 2022-01-27 11:17:57,039 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2829
+ 2022-01-27 11:17:59,041 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2830
+ 2022-01-27 11:17:59,332 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2831
+ 2022-01-27 11:17:59,332 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2832
+ 2022-01-27 11:18:01,043 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2833
+ 2022-01-27 11:18:03,045 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2834
+ 2022-01-27 11:18:05,046 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2835
+ 2022-01-27 11:18:07,048 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2836
+ 2022-01-27 11:18:09,050 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2837
+ 2022-01-27 11:18:11,014 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2838
+ 2022-01-27 11:18:11,052 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2839
+ 2022-01-27 11:18:13,054 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2840
+ 2022-01-27 11:18:14,573 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2841
+ 2022-01-27 11:18:14,574 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2842
+ 2022-01-27 11:18:15,056 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2843
+ 2022-01-27 11:18:17,058 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2844
+ 2022-01-27 11:18:19,059 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2845
+ 2022-01-27 11:18:21,061 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2846
+ 2022-01-27 11:18:23,062 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2847
+ 2022-01-27 11:18:25,116 DEBUG SenderThread:1977561 [sender.py:send():234] send: history
2848
+ 2022-01-27 11:18:25,147 DEBUG SenderThread:1977561 [sender.py:send():234] send: summary
2849
+ 2022-01-27 11:18:25,195 INFO SenderThread:1977561 [sender.py:_save_file():939] saving file wandb-summary.json with policy end
2850
+ 2022-01-27 11:18:26,065 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/wandb-summary.json
2851
+ 2022-01-27 11:18:27,066 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2852
+ 2022-01-27 11:18:29,068 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2853
+ 2022-01-27 11:18:29,745 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2854
+ 2022-01-27 11:18:29,749 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2855
+ 2022-01-27 11:18:39,075 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2856
+ 2022-01-27 11:18:41,328 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2857
+ 2022-01-27 11:18:44,945 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2858
+ 2022-01-27 11:18:44,946 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2859
+ 2022-01-27 11:19:00,114 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2860
+ 2022-01-27 11:19:00,115 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2861
+ 2022-01-27 11:19:11,558 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2862
+ 2022-01-27 11:19:16,328 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2863
+ 2022-01-27 11:19:16,329 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2864
+ 2022-01-27 11:19:31,493 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2865
+ 2022-01-27 11:19:31,494 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
2866
+ 2022-01-27 11:19:37,112 INFO Thread-8 :1977561 [dir_watcher.py:_on_file_modified():230] file/dir modified: /workspace/xls-r-300m-sv-robust/wandb/run-20220127_103723-scy0vyln/files/output.log
2867
+ 2022-01-27 11:19:41,795 DEBUG SenderThread:1977561 [sender.py:send():234] send: stats
2868
+ 2022-01-27 11:19:46,663 DEBUG HandlerThread:1977561 [handler.py:handle_request():130] handle_request: stop_status
2869
+ 2022-01-27 11:19:46,664 DEBUG SenderThread:1977561 [sender.py:send_request():248] send_request: stop_status
wandb/run-20220127_103723-scy0vyln/run-scy0vyln.wandb CHANGED
Binary files a/wandb/run-20220127_103723-scy0vyln/run-scy0vyln.wandb and b/wandb/run-20220127_103723-scy0vyln/run-scy0vyln.wandb differ