StarCoder in AWS Sagemaker? Error "gpt_bigcode" for not supporting Transformers last version
#41
by
apol
- opened
Hi, trying to deploy StarChat in AWS Sagemaker.
Error "gpt_bigcode" for lacking the last transformers version, that is not compatible with Sagemaker.
Any ideas to hack this?
Thanks
Hey, same here)
there are some references that transformers version 4.28.1 is required to pass "gpt_bigcode" error.
As an idea:
- there is a dlc with transformers 4.28.1 just released here, and the model could be created with help of sagemaker.huggingface (and boto3) python package(s) with any given image uri.
I've just tried, it gave me another error:ValueError: Could not load model /.sagemaker/mms/models/bigcode__starcoder with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.gpt_bigcode.modeling_gpt_bigcode.GPTBigCodeForCausalLM'>).
It worked by putting it on the AWS instance type: ml.g4dn.12xlarge and setting SM_NUM_GPUS: "4"