Spaces:
Running
on
Zero
Running
on
Zero
Frequently GPU Task Aborted
#1
by
wondervictor
- opened
Hello @nielsr @hysts , we recently built a control-to-image demo, but it often randomly encounters "GPU Task Aborted" issues during runtime. Is there any way to resolve this? I’d like to understand the possible causes of this problem.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/gradio/queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1935, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1520, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1005, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 826, in wrapper
response = f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/spaces/zero/wrappers.py", line 211, in gradio_handler
raise gr.Error("GPU task aborted")
gradio.exceptions.Error: 'GPU task aborted'
Hi
@wondervictor
, the error is raised when the function decorated with @spaces.GPU
takes longer than the specified duration
. The default value of it is 60 seconds. You might want to check out the org card of the ZeroGPU explorers org: https://huggingface.co/zero-gpu-explorers