mosaicml

/

mpt-7b-instruct

like 467

Follow

Mosaic ML, Inc. 351

I've used this code:

model_name = "mosaicml/mpt-7b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name,
                                             low_cpu_mem_usage=True,
                                             trust_remote_code=True,
                                             load_in_8bit=True,
                                             torch_dtype=torch.float16,
                                             device_map="auto")

But gives this error:

ValueError: MPTForCausalLM does not support `device_map='auto'` yet.