| 2024-07-04 08:29:16 | INFO | model_worker | args: Namespace(awq_ckpt=None, awq_groupsize=-1, awq_wbits=16, controller_address='http://127.0.0.1:21002', conv_template=None, cpu_offloading=False, debug=False, device='cuda', dtype=None, embed_in_truncate=False, enable_exllama=False, enable_xft=False, exllama_cache_8bit=False, exllama_gpu_split=None, exllama_max_seq_len=4096, gptq_act_order=False, gptq_ckpt=None, gptq_groupsize=-1, gptq_wbits=16, gpus=None, host='127.0.0.1', limit_worker_concurrency=5, load_8bit=False, max_gpu_memory=None, model_names=None, model_path='lmsys/vicuna-7b-v1.5', no_register=False, num_gpus=1, port=21003, revision='main', seed=None, ssl=False, stream_interval=2, worker_address='http://127.0.0.1:21003', xft_dtype=None, xft_max_seq_len=4096) | |
| 2024-07-04 08:29:16 | INFO | model_worker | Loading the model ['vicuna-7b-v1.5'] on worker daef9b78 ... | |
| 2024-07-04 08:29:16 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/torch/storage.py:315: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. | |
| 2024-07-04 08:29:16 | ERROR | stderr | warnings.warn(message, UserWarning) | |
| 2024-07-04 08:29:16 | ERROR | stderr | Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] | |
| 2024-07-04 08:29:21 | ERROR | stderr | Loading checkpoint shards: 50%|▌| 1/2 [00:04<00:04, 4.98 | |
| 2024-07-04 08:29:23 | ERROR | stderr | Loading checkpoint shards: 100%|█| 2/2 [00:06<00:00, 3.06 | |
| 2024-07-04 08:29:23 | ERROR | stderr | Loading checkpoint shards: 100%|█| 2/2 [00:06<00:00, 3.34 | |
| 2024-07-04 08:29:23 | ERROR | stderr | | |
| 2024-07-04 08:29:23 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. | |
| 2024-07-04 08:29:23 | ERROR | stderr | warnings.warn( | |
| 2024-07-04 08:29:23 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. | |
| 2024-07-04 08:29:23 | ERROR | stderr | warnings.warn( | |
| 2024-07-04 08:29:23 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. | |
| 2024-07-04 08:29:23 | ERROR | stderr | warnings.warn( | |
| 2024-07-04 08:29:23 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. | |
| 2024-07-04 08:29:23 | ERROR | stderr | warnings.warn( | |
| 2024-07-04 08:29:47 | INFO | model_worker | Register to controller | |
| 2024-07-04 08:29:48 | ERROR | stderr | [32mINFO[0m: Started server process [[36m18410[0m] | |
| 2024-07-04 08:29:48 | ERROR | stderr | [32mINFO[0m: Waiting for application startup. | |
| 2024-07-04 08:29:48 | ERROR | stderr | [32mINFO[0m: Application startup complete. | |
| 2024-07-04 08:29:48 | ERROR | stderr | [32mINFO[0m: Uvicorn running on [1mhttp://127.0.0.1:21003[0m (Press CTRL+C to quit) | |
| 2024-07-04 08:30:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: daef9b78. | |
| 2024-07-04 08:31:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: None. call_ct: 0. worker_id: daef9b78. | |
| 2024-07-04 08:31:53 | INFO | stdout | [32mINFO[0m: 127.0.0.1:55282 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m | |
| 2024-07-04 08:32:01 | INFO | stdout | [32mINFO[0m: 127.0.0.1:37832 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m | |
| 2024-07-04 08:32:02 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=4, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:32:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:33:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:34:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:35:02 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:35:17 | INFO | stdout | [32mINFO[0m: 127.0.0.1:32778 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m | |
| 2024-07-04 08:35:47 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:36:32 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:36:59 | INFO | stdout | [32mINFO[0m: 127.0.0.1:52544 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m | |
| 2024-07-04 08:37:17 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:38:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:38:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:39:15 | INFO | stdout | [32mINFO[0m: 127.0.0.1:58500 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m | |
| 2024-07-04 08:39:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:40:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:41:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:41:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:42:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:43:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:44:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:44:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:45:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:46:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:47:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:47:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:48:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:49:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:50:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:50:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:51:33 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:52:18 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:53:03 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:53:48 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:54:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:55:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:56:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:56:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:57:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:58:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:59:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 08:59:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:00:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:01:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:02:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:02:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:03:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:04:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:05:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:05:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:06:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:07:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:08:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:08:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:09:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:10:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:11:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:11:49 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:12:34 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:13:19 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:14:04 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:14:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:15:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:16:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:17:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:17:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:18:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:19:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:20:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:20:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:21:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:22:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:23:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:23:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:24:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:25:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:26:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:26:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:27:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:28:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:29:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:29:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:30:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:31:20 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:32:05 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:32:50 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:33:35 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:34:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:35:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:35:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:36:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:37:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:38:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:38:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:39:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:40:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:41:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:41:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:42:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:43:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:44:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:44:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:45:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:46:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:47:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:47:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:48:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:49:21 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:50:06 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:50:51 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:51:36 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:52:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:53:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:53:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:54:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:55:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:56:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:56:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:57:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:58:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:59:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 09:59:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:00:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:01:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:02:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:02:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:03:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:04:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:05:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:05:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:06:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:07:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:08:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:08:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:09:37 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:10:22 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:11:07 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:11:52 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:12:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:13:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:14:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:14:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:15:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:16:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:17:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:17:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:18:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:19:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:20:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:20:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:21:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:22:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:23:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:23:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:24:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:25:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:26:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:26:53 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:27:38 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:28:23 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:29:08 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:29:54 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:30:39 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:31:24 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:32:09 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:32:54 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:33:39 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:34:24 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:35:09 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:35:54 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:36:39 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:37:24 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:38:09 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:38:54 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:39:39 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:40:24 | INFO | model_worker | Send heart beat. Models: ['vicuna-7b-v1.5']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: daef9b78. | |
| 2024-07-04 10:40:39 | ERROR | stderr | [32mINFO[0m: Shutting down | |
| 2024-07-04 10:40:39 | ERROR | stderr | [32mINFO[0m: Waiting for application shutdown. | |
| 2024-07-04 10:40:39 | ERROR | stderr | [32mINFO[0m: Application shutdown complete. | |
| 2024-07-04 10:40:39 | ERROR | stderr | [32mINFO[0m: Finished server process [[36m18410[0m] | |
| 2024-07-04 10:40:39 | ERROR | stderr | Traceback (most recent call last): | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main | |
| 2024-07-04 10:40:39 | ERROR | stderr | return _run_code(code, main_globals, None, | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/lib/python3.8/runpy.py", line 87, in _run_code | |
| 2024-07-04 10:40:39 | ERROR | stderr | exec(code, run_globals) | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/LLM_32T/evelyn/FastChat/fastchat/serve/model_worker.py", line 425, in <module> | |
| 2024-07-04 10:40:39 | ERROR | stderr | uvicorn.run(app, host=args.host, port=args.port, log_level="info") | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 577, in run | |
| 2024-07-04 10:40:39 | ERROR | stderr | server.run() | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 65, in run | |
| 2024-07-04 10:40:39 | ERROR | stderr | return asyncio.run(self.serve(sockets=sockets)) | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run | |
| 2024-07-04 10:40:39 | ERROR | stderr | return loop.run_until_complete(main) | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "uvloop/loop.pyx", line 555, in uvloop.loop.Loop._run | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "uvloop/loop.pyx", line 474, in uvloop.loop.Loop._on_idle | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "uvloop/cbhandles.pyx", line 83, in uvloop.loop.Handle._run | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 69, in serve | |
| 2024-07-04 10:40:39 | ERROR | stderr | await self._serve(sockets) | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ | |
| 2024-07-04 10:40:39 | ERROR | stderr | next(self.gen) | |
| 2024-07-04 10:40:39 | ERROR | stderr | File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 328, in capture_signals | |
| 2024-07-04 10:40:39 | ERROR | stderr | signal.raise_signal(captured_signal) | |
| 2024-07-04 10:40:39 | ERROR | stderr | KeyboardInterrupt | |