Amazon sagemaker deploy
I clicked on the Amazon Sagemaker deploy and followed the steps given. However, it throws an error. Should I change the "instance type" parameter?
The inference widget doesn't run, Sagemaker also doesn't work. So I do not know how to use this model. Help is massively appreciated.
I realised that there is no way to actually run these models on system without having access to A100 GPU's. In Amazon to request that instance, it costs approx. USD 32. Therefore, unless there are folks from big companies or academia, this model cannot be used.
pinging @philschmid just for visibility
Hello @ghpkishore ,
It should be possible to run the model with Large Model Loading on Amazon SageMaker. But there is not yet a container with the supported transformers version available meaning you would need to create a custom inference.py + requirements.txt to deploy to sagemaker. example
For instance type i am not sure which one is enough I would either try g5.12xlarge or p3.8xlarge.