|
2024-02-26 21:45:17 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=40001, worker_address='http://localhost:40001', controller_address='http://localhost:10000', model_path='MBZUAI/MobiLlama-1B-Chat', revision='main', device='cuda', gpus=None, num_gpus=1, max_gpu_memory=None, dtype=None, load_8bit=False, cpu_offloading=False, gptq_ckpt=None, gptq_wbits=16, gptq_groupsize=-1, gptq_act_order=False, awq_ckpt=None, awq_wbits=16, awq_groupsize=-1, enable_exllama=False, exllama_max_seq_len=4096, exllama_gpu_split=None, exllama_cache_8bit=False, enable_xft=False, xft_max_seq_len=4096, xft_dtype=None, model_names=None, conv_template=None, embed_in_truncate=False, limit_worker_concurrency=5, stream_interval=2, no_register=False, seed=None, debug=False, ssl=False) |
|
2024-02-26 21:45:17 | INFO | model_worker | Loading the model ['MobiLlama-1B-Chat'] on worker 1639f093 ... |
|
2024-02-26 21:45:23 | INFO | model_worker | Register to controller |
|
2024-02-26 21:45:23 | ERROR | stderr | [32mINFO[0m: Started server process [[36m455699[0m] |
|
2024-02-26 21:45:23 | ERROR | stderr | [32mINFO[0m: Waiting for application startup. |
|
2024-02-26 21:45:23 | ERROR | stderr | [32mINFO[0m: Application startup complete. |
|
2024-02-26 21:45:23 | ERROR | stderr | [32mINFO[0m: Uvicorn running on [1mhttp://0.0.0.0:40001[0m (Press CTRL+C to quit) |
|
2024-02-26 21:46:01 | INFO | stdout | [32mINFO[0m: 127.0.0.1:60500 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:46:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: None. call_ct: 0. worker_id: 1639f093. |
|
2024-02-26 21:46:20 | INFO | stdout | [32mINFO[0m: 127.0.0.1:48788 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:46:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: None. call_ct: 0. worker_id: 1639f093. |
|
2024-02-26 21:47:06 | INFO | stdout | [32mINFO[0m: 127.0.0.1:55144 - "[1mPOST /worker_generate_stream HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:47:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:48:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:49:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:49:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:50:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:51:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:52:01 | INFO | stdout | [32mINFO[0m: 127.0.0.1:59092 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:52:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:52:20 | INFO | stdout | [32mINFO[0m: 127.0.0.1:37148 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:52:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:53:23 | INFO | stdout | [32mINFO[0m: 127.0.0.1:58944 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:53:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:53:54 | INFO | stdout | [32mINFO[0m: 127.0.0.1:41798 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:54:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:54:56 | INFO | stdout | [32mINFO[0m: 127.0.0.1:53180 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:55:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:55:14 | INFO | stdout | [32mINFO[0m: 127.0.0.1:36798 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:55:53 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:56:16 | INFO | stdout | [32mINFO[0m: 127.0.0.1:34376 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:56:38 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:56:49 | INFO | stdout | [32mINFO[0m: 127.0.0.1:38786 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:57:19 | INFO | stdout | [32mINFO[0m: 127.0.0.1:50820 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:57:23 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:57:36 | INFO | stdout | [32mINFO[0m: 127.0.0.1:45772 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 21:58:08 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:58:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 21:59:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:00:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:00:37 | INFO | stdout | [32mINFO[0m: 127.0.0.1:43378 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:01:01 | INFO | stdout | [32mINFO[0m: 127.0.0.1:43798 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:01:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:01:24 | INFO | stdout | [32mINFO[0m: 127.0.0.1:54958 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:01:43 | INFO | stdout | [32mINFO[0m: 127.0.0.1:57684 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:01:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:02:37 | INFO | stdout | [32mINFO[0m: 127.0.0.1:34256 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:02:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:02:55 | INFO | stdout | [32mINFO[0m: 127.0.0.1:55556 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:03:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:03:35 | INFO | stdout | [32mINFO[0m: 127.0.0.1:50278 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:03:56 | INFO | stdout | [32mINFO[0m: 127.0.0.1:40924 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:04:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:04:24 | INFO | stdout | [32mINFO[0m: 127.0.0.1:52980 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:04:49 | INFO | stdout | [32mINFO[0m: 127.0.0.1:37202 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:04:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:05:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:06:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:07:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:07:46 | INFO | stdout | [32mINFO[0m: 127.0.0.1:59246 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:07:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:08:12 | INFO | stdout | [32mINFO[0m: 127.0.0.1:53410 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:08:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:09:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:10:01 | INFO | stdout | [32mINFO[0m: 127.0.0.1:44898 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:10:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:10:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:11:09 | INFO | stdout | [32mINFO[0m: 127.0.0.1:58684 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:11:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:12:24 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:13:09 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:13:54 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:14:39 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:14:41 | INFO | stdout | [32mINFO[0m: 127.0.0.1:41514 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:14:59 | INFO | stdout | [32mINFO[0m: 127.0.0.1:48850 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:15:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:16:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:16:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:17:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:17:57 | INFO | stdout | [32mINFO[0m: 127.0.0.1:55990 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:18:14 | INFO | stdout | [32mINFO[0m: 127.0.0.1:36960 - "[1mPOST /worker_get_status HTTP/1.1[0m" [32m200 OK[0m |
|
2024-02-26 22:18:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:19:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:19:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:20:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:21:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:22:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:22:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:23:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:24:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:25:10 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:25:55 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:26:40 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:27:25 | INFO | model_worker | Send heart beat. Models: ['MobiLlama-1B-Chat']. Semaphore: Semaphore(value=5, locked=False). call_ct: 1. worker_id: 1639f093. |
|
2024-02-26 22:27:37 | ERROR | stderr | [32mINFO[0m: Shutting down |
|
2024-02-26 22:27:38 | ERROR | stderr | [32mINFO[0m: Waiting for application shutdown. |
|
2024-02-26 22:27:38 | ERROR | stderr | [32mINFO[0m: Application shutdown complete. |
|
2024-02-26 22:27:38 | ERROR | stderr | [32mINFO[0m: Finished server process [[36m455699[0m] |
|
|
