{"message":"unknown or invalid runtime name: nvidia"}
#26
by
iholy19
- opened
Hello,
I am using GPU - 2 x NVIDIA Tesla V100 for inference in AzureML. I built the docker image successfully. But I get this error and it does not mention the code line number. I don't understand what I am missing. Please help me fix this.
ERROR:
Failed to execute command group with error API queried with a bad parameter: {"message":"unknown or invalid runtime name: nvidia"}
Okat, Thank you.
I cloned the repo and I am trying to load the model. The repo is around 330GB. Could you please tell me how much of vRAM is needed?
managed to run in 4 bit quantization using 47gb vRAM, and 8bit quantization using 95gb vRAM