How do I run this model?
I am getting this error.
KeyError Traceback (most recent call last)
in <cell line: 5>()
3 device = "cuda" # the device to load the model onto
4
----> 5 model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
6 tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
7
2 frames
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
525 _ = kwargs.pop("quantization_config")
526
--> 527 config, kwargs = AutoConfig.from_pretrained(
528 pretrained_model_name_or_path,
529 return_unused_kwargs=True,
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
1037 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
1038 elif "model_type" in config_dict:
-> 1039 config_class = CONFIG_MAPPING[config_dict["model_type"]]
1040 return config_class.from_dict(config_dict, **unused_kwargs)
1041 else:
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in getitem(self, key)
732 return self._extra_content[key]
733 if key not in self._mapping:
--> 734 raise KeyError(key)
735 value = self._mapping[key]
736 module_name = model_type_to_module_name(key)
KeyError: 'mistral'
My code
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
text = "<s>[INST] What is your favourite condiment? [/INST]"
"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
"[INST] Do you have mayonnaise recipes? [/INST]"
encodeds = tokenizer(text, return_tensors="pt", add_special_tokens=False)
model_inputs = encodeds.to(device)
model.to(device)
generated_ids = model.generate(**model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])
I am also getting this error as well using the sample code for this model.
Here is an abbreviated environment:
Package Version
absl-py 1.0.0
accelerate 0.21.0
aiohttp 3.8.4
aiosignal 1.3.1
alembic 1.12.0
appdirs 1.4.4
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
astor 0.8.1
asttokens 2.2.1
astunparse 1.6.3
async-timeout 4.0.2
attrs 21.4.0
audioread 3.0.0
azure-core 1.26.4
azure-cosmos 4.3.1b1
azure-storage-blob 12.16.0
azure-storage-file-datalake 12.11.0
backcall 0.2.0
bcrypt 3.2.0
beautifulsoup4 4.11.1
bitsandbytes 0.41.0
black 22.6.0
bleach 4.1.0
blinker 1.4
blis 0.7.9
boto3 1.24.28
botocore 1.27.28
bravado 11.0.3
bravado-core 6.1.0
cachetools 4.2.4
catalogue 2.0.8
category-encoders 2.6.0
certifi 2022.9.14
cffi 1.15.1
chardet 4.0.0
charset-normalizer 2.0.4
click 8.0.4
cloudpickle 2.0.0
cmaes 0.10.0
cmdstanpy 1.1.0
colorlog 6.7.0
confection 0.0.4
configparser 5.2.0
convertdate 2.4.0
cryptography 37.0.1
cycler 0.11.0
cymem 2.0.7
Cython 0.29.32
databricks-automl-runtime 0.2.16
databricks-cli 0.17.6
databricks-feature-store 0.12.1
dataclasses-json 0.5.7
datasets 2.12.0
dbl-tempo 0.1.23
dbus-python 1.2.18
debugpy 1.5.1
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.4
diskcache 5.6.1
distlib 0.3.6
distro 1.7.0
distro-info 1.1+ubuntu0.1
doc2text 0.2.4
docstring-to-markdown 0.12
einops 0.6.1
entrypoints 0.4
ephem 4.1.4
et-xmlfile 1.1.0
evaluate 0.4.0
executing 1.2.0
facets-overview 1.0.3
faiss-gpu 1.7.2
fastjsonschema 2.16.3
fasttext 0.9.2
filelock 3.6.0
filetype 1.2.0
Flask 1.1.2+db1
flatbuffers 23.5.9
fonttools 4.25.0
fqdn 1.5.1
frozenlist 1.3.3
fsspec 2022.7.1
future 0.18.2
gast 0.4.0
gitdb 4.0.10
GitPython 3.1.27
google-api-core 2.8.2
google-auth 1.33.0
google-auth-oauthlib 0.4.6
google-cloud-core 2.3.2
google-cloud-storage 2.9.0
google-crc32c 1.5.0
google-pasta 0.2.0
google-resumable-media 2.5.0
googleapis-common-protos 1.56.4
greenlet 1.1.1
grpcio 1.48.1
grpcio-status 1.48.1
gunicorn 20.1.0
gviz-api 1.10.0
h5py 3.7.0
hijri-converter 2.3.1
holidays 0.22
horovod 0.27.0
htmlmin 0.1.12
httplib2 0.20.2
huggingface-hub 0.17.3
idna 3.3
ImageHash 4.3.1
imbalanced-learn 0.8.1
importlib-metadata 4.11.3
ipykernel 6.17.1
ipython 8.10.0
ipython-genutils 0.2.0
ipywidgets 7.7.2
isodate 0.6.1
isoduration 20.11.0
itsdangerous 2.0.1
jedi 0.18.1
jeepney 0.7.1
Jinja2 2.11.3
jmespath 0.10.0
joblib 1.2.0
joblibspark 0.5.1
jsonpointer 2.4
jsonref 1.1.0
jsonschema 4.16.0
jupyter-client 7.3.4
jupyter_core 4.11.2
jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0
keras 2.11.0
keyring 23.5.0
kiwisolver 1.4.2
korean-lunar-calendar 0.3.1
langchain 0.0.285
langcodes 3.3.0
langsmith 0.0.41
launchpadlib 1.10.16
lazr.restfulclient 0.14.4
lazr.uri 1.0.6
lazy_loader 0.2
libclang 15.0.6.1
librosa 0.10.0
lightgbm 3.3.5
llvmlite 0.38.0
LunarCalendar 0.0.9
lxml 4.9.3
Mako 1.2.0
Markdown 3.3.4
MarkupSafe 2.0.1
marshmallow 3.19.0
marshmallow-enum 1.5.1
matplotlib 3.5.2
matplotlib-inline 0.1.6
mccabe 0.7.0
mime 0.1.0
mistune 0.8.4
mleap 0.20.0
mlflow-skinny 2.3.1
monotonic 1.6
more-itertools 8.10.0
msgpack 1.0.5
multidict 6.0.4
multimethod 1.9.1
multiprocess 0.70.12.2
murmurhash 1.0.9
mypy-extensions 0.4.3
nbclient 0.5.13
nbconvert 6.4.4
nbformat 5.5.0
neptune-client 0.16.18
neptune-optuna 1.1.0
nest-asyncio 1.5.5
networkx 2.8.4
nltk 3.7
nodeenv 1.8.0
notebook 6.4.12
numba 0.55.1
numexpr 2.8.4
numpy 1.21.5
oauthlib 3.2.0
openai 0.27.4
openapi-schema-pydantic 1.2.4
openpyxl 3.1.2
opt-einsum 3.3.0
optuna 3.3.0
packaging 21.3
pandas 1.4.4
pandocfilters 1.5.0
paramiko 2.9.2
parso 0.8.3
pathspec 0.9.0
pathy 0.10.1
patsy 0.5.2
petastorm 0.12.1
pexpect 4.8.0
phik 0.12.3
pickleshare 0.7.5
Pillow 9.2.0
pip 23.2.1
platformdirs 2.5.2
plotly 5.9.0
pluggy 1.0.0
pmdarima 2.0.3
pooch 1.7.0
preshed 3.0.8
prompt-toolkit 3.0.36
prophet 1.1.2
protobuf 3.19.4
psutil 5.9.0
psycopg2 2.9.3
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 8.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pybind11 2.10.4
pycparser 2.21
pydantic 1.10.6
pyflakes 3.0.1
Pygments 2.11.2
PyGObject 3.42.1
PyJWT 2.3.0
PyMeeus 0.5.12
PyNaCl 1.5.0
pyodbc 4.0.32
pyparsing 3.0.9
PyPDF2 3.0.1
pyright 1.1.294
pyrsistent 0.18.0
pytesseract 0.3.10
python-apt 2.4.0+ubuntu2
python-dateutil 2.8.2
python-docx 0.8.11
python-editor 1.0.4
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.7.1
python-magic 0.4.27
pytoolconfig 1.2.2
pytz 2022.1
PyWavelets 1.3.0
PyYAML 6.0
pyzmq 23.2.0
ragas 0.0.12
regex 2022.7.9
requests 2.28.1
requests-oauthlib 1.3.1
responses 0.18.0
rfc3339-validator 0.1.4
rfc3987 1.3.8
rope 1.7.0
rsa 4.9
s3transfer 0.6.0
safetensors 0.3.3
scikit-learn 1.1.1
scipy 1.9.1
seaborn 0.11.2
SecretStorage 3.3.1
Send2Trash 1.8.0
sentence-transformers 2.2.2
sentencepiece 0.1.97
setuptools 63.4.1
shap 0.41.0
simplejson 3.17.6
six 1.16.0
skops 0.8.0
slicer 0.0.7
smart-open 5.2.1
smmap 5.0.0
soundfile 0.12.1
soupsieve 2.3.1
soxr 0.3.5
spacy 3.5.1
spacy-cleaner 3.1.3
spacy-legacy 3.0.12
spacy-loggers 1.0.4
spacy-lookups-data 1.0.5
spark-tensorflow-distributor 1.0.0
SQLAlchemy 1.4.39
sqlparse 0.4.2
srsly 2.4.6
ssh-import-id 5.11
stack-data 0.6.2
statsmodels 0.13.2
swagger-spec-validator 3.0.3
tabulate 0.8.10
tangled-up-in-unicode 0.2.0
tenacity 8.1.0
tensorboard 2.11.0
tensorboard-data-server 0.6.1
tensorboard-plugin-profile 2.11.2
tensorboard-plugin-wit 1.8.1
tensorflow 2.11.1
tensorflow-estimator 2.11.0
tensorflow-io-gcs-filesystem 0.32.0
termcolor 2.3.0
terminado 0.13.1
testpath 0.6.0
thinc 8.1.10
threadpoolctl 2.2.0
tiktoken 0.3.3
tokenize-rt 4.2.1
tokenizers 0.13.3
tomli 2.0.1
torch 1.13.1+cu117
torchvision 0.14.1+cu117
tornado 6.1
tqdm 4.64.1
traitlets 5.1.1
transformers 4.33.3
typeguard 2.13.3
typer 0.7.0
typing_extensions 4.3.0
typing-inspect 0.8.0
ujson 5.4.0
unattended-upgrades 0.1
unstructured 0.9.0
uri-template 1.3.0
urllib3 1.26.11
virtualenv 20.16.3
visions 0.7.5
wadllib 1.3.6
wasabi 0.10.1
watermark 2.4.3
wcwidth 0.2.5
webcolors 1.13
webencodings 0.5.1
websocket-client 0.58.0
Werkzeug 2.0.3
whatthepatch 1.0.2
wheel 0.37.1
widgetsnbextension 3.6.1
wrapt 1.14.1
xgboost 1.7.5
xxhash 3.2.0
yapf 0.31.0
yarl 1.9.2
ydata-profiling 4.1.2
zipp 3.8.0
Here are two lines of the sample code pulled from the model card:
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
Here is the error I am receiving:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File <command-4309329534262302>:1
----> 1 model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
2 tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:441, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
438 if kwargs_copy.get("torch_dtype", None) == "auto":
439 _ = kwargs_copy.pop("torch_dtype")
--> 441 config, kwargs = AutoConfig.from_pretrained(
442 pretrained_model_name_or_path,
443 return_unused_kwargs=True,
444 trust_remote_code=trust_remote_code,
445 **hub_kwargs,
446 **kwargs_copy,
447 )
448 if hasattr(config, "auto_map") and cls.__name__ in config.auto_map:
449 if not trust_remote_code:
File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:937, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
935 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
936 elif "model_type" in config_dict:
--> 937 config_class = CONFIG_MAPPING[config_dict["model_type"]]
938 return config_class.from_dict(config_dict, **unused_kwargs)
939 else:
940 # Fallback: use pattern matching on the string.
941 # We go from longer names to shorter names to catch roberta before bert (for instance)
File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:643, in _LazyConfigMapping.__getitem__(self, key)
641 return self._extra_content[key]
642 if key not in self._mapping:
--> 643 raise KeyError(key)
644 value = self._mapping[key]
645 module_name = model_type_to_module_name(key)
KeyError: 'mistral'
you need to update the transformers package to the latest.
I am also getting this error as well using the sample code for this model.
Here is an abbreviated environment:
Package Version
absl-py 1.0.0
accelerate 0.21.0
aiohttp 3.8.4
aiosignal 1.3.1
alembic 1.12.0
appdirs 1.4.4
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
astor 0.8.1
asttokens 2.2.1
astunparse 1.6.3
async-timeout 4.0.2
attrs 21.4.0
audioread 3.0.0
azure-core 1.26.4
azure-cosmos 4.3.1b1
azure-storage-blob 12.16.0
azure-storage-file-datalake 12.11.0
backcall 0.2.0
bcrypt 3.2.0
beautifulsoup4 4.11.1
bitsandbytes 0.41.0
black 22.6.0
bleach 4.1.0
blinker 1.4
blis 0.7.9
boto3 1.24.28
botocore 1.27.28
bravado 11.0.3
bravado-core 6.1.0
cachetools 4.2.4
catalogue 2.0.8
category-encoders 2.6.0
certifi 2022.9.14
cffi 1.15.1
chardet 4.0.0
charset-normalizer 2.0.4
click 8.0.4
cloudpickle 2.0.0
cmaes 0.10.0
cmdstanpy 1.1.0
colorlog 6.7.0
confection 0.0.4
configparser 5.2.0
convertdate 2.4.0
cryptography 37.0.1
cycler 0.11.0
cymem 2.0.7
Cython 0.29.32
databricks-automl-runtime 0.2.16
databricks-cli 0.17.6
databricks-feature-store 0.12.1
dataclasses-json 0.5.7
datasets 2.12.0
dbl-tempo 0.1.23
dbus-python 1.2.18
debugpy 1.5.1
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.4
diskcache 5.6.1
distlib 0.3.6
distro 1.7.0
distro-info 1.1+ubuntu0.1
doc2text 0.2.4
docstring-to-markdown 0.12
einops 0.6.1
entrypoints 0.4
ephem 4.1.4
et-xmlfile 1.1.0
evaluate 0.4.0
executing 1.2.0
facets-overview 1.0.3
faiss-gpu 1.7.2
fastjsonschema 2.16.3
fasttext 0.9.2
filelock 3.6.0
filetype 1.2.0
Flask 1.1.2+db1
flatbuffers 23.5.9
fonttools 4.25.0
fqdn 1.5.1
frozenlist 1.3.3
fsspec 2022.7.1
future 0.18.2
gast 0.4.0
gitdb 4.0.10
GitPython 3.1.27
google-api-core 2.8.2
google-auth 1.33.0
google-auth-oauthlib 0.4.6
google-cloud-core 2.3.2
google-cloud-storage 2.9.0
google-crc32c 1.5.0
google-pasta 0.2.0
google-resumable-media 2.5.0
googleapis-common-protos 1.56.4
greenlet 1.1.1
grpcio 1.48.1
grpcio-status 1.48.1
gunicorn 20.1.0
gviz-api 1.10.0
h5py 3.7.0
hijri-converter 2.3.1
holidays 0.22
horovod 0.27.0
htmlmin 0.1.12
httplib2 0.20.2
huggingface-hub 0.17.3
idna 3.3
ImageHash 4.3.1
imbalanced-learn 0.8.1
importlib-metadata 4.11.3
ipykernel 6.17.1
ipython 8.10.0
ipython-genutils 0.2.0
ipywidgets 7.7.2
isodate 0.6.1
isoduration 20.11.0
itsdangerous 2.0.1
jedi 0.18.1
jeepney 0.7.1
Jinja2 2.11.3
jmespath 0.10.0
joblib 1.2.0
joblibspark 0.5.1
jsonpointer 2.4
jsonref 1.1.0
jsonschema 4.16.0
jupyter-client 7.3.4
jupyter_core 4.11.2
jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0
keras 2.11.0
keyring 23.5.0
kiwisolver 1.4.2
korean-lunar-calendar 0.3.1
langchain 0.0.285
langcodes 3.3.0
langsmith 0.0.41
launchpadlib 1.10.16
lazr.restfulclient 0.14.4
lazr.uri 1.0.6
lazy_loader 0.2
libclang 15.0.6.1
librosa 0.10.0
lightgbm 3.3.5
llvmlite 0.38.0
LunarCalendar 0.0.9
lxml 4.9.3
Mako 1.2.0
Markdown 3.3.4
MarkupSafe 2.0.1
marshmallow 3.19.0
marshmallow-enum 1.5.1
matplotlib 3.5.2
matplotlib-inline 0.1.6
mccabe 0.7.0
mime 0.1.0
mistune 0.8.4
mleap 0.20.0
mlflow-skinny 2.3.1
monotonic 1.6
more-itertools 8.10.0
msgpack 1.0.5
multidict 6.0.4
multimethod 1.9.1
multiprocess 0.70.12.2
murmurhash 1.0.9
mypy-extensions 0.4.3
nbclient 0.5.13
nbconvert 6.4.4
nbformat 5.5.0
neptune-client 0.16.18
neptune-optuna 1.1.0
nest-asyncio 1.5.5
networkx 2.8.4
nltk 3.7
nodeenv 1.8.0
notebook 6.4.12
numba 0.55.1
numexpr 2.8.4
numpy 1.21.5
oauthlib 3.2.0
openai 0.27.4
openapi-schema-pydantic 1.2.4
openpyxl 3.1.2
opt-einsum 3.3.0
optuna 3.3.0
packaging 21.3
pandas 1.4.4
pandocfilters 1.5.0
paramiko 2.9.2
parso 0.8.3
pathspec 0.9.0
pathy 0.10.1
patsy 0.5.2
petastorm 0.12.1
pexpect 4.8.0
phik 0.12.3
pickleshare 0.7.5
Pillow 9.2.0
pip 23.2.1
platformdirs 2.5.2
plotly 5.9.0
pluggy 1.0.0
pmdarima 2.0.3
pooch 1.7.0
preshed 3.0.8
prompt-toolkit 3.0.36
prophet 1.1.2
protobuf 3.19.4
psutil 5.9.0
psycopg2 2.9.3
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 8.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pybind11 2.10.4
pycparser 2.21
pydantic 1.10.6
pyflakes 3.0.1
Pygments 2.11.2
PyGObject 3.42.1
PyJWT 2.3.0
PyMeeus 0.5.12
PyNaCl 1.5.0
pyodbc 4.0.32
pyparsing 3.0.9
PyPDF2 3.0.1
pyright 1.1.294
pyrsistent 0.18.0
pytesseract 0.3.10
python-apt 2.4.0+ubuntu2
python-dateutil 2.8.2
python-docx 0.8.11
python-editor 1.0.4
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.7.1
python-magic 0.4.27
pytoolconfig 1.2.2
pytz 2022.1
PyWavelets 1.3.0
PyYAML 6.0
pyzmq 23.2.0
ragas 0.0.12
regex 2022.7.9
requests 2.28.1
requests-oauthlib 1.3.1
responses 0.18.0
rfc3339-validator 0.1.4
rfc3987 1.3.8
rope 1.7.0
rsa 4.9
s3transfer 0.6.0
safetensors 0.3.3
scikit-learn 1.1.1
scipy 1.9.1
seaborn 0.11.2
SecretStorage 3.3.1
Send2Trash 1.8.0
sentence-transformers 2.2.2
sentencepiece 0.1.97
setuptools 63.4.1
shap 0.41.0
simplejson 3.17.6
six 1.16.0
skops 0.8.0
slicer 0.0.7
smart-open 5.2.1
smmap 5.0.0
soundfile 0.12.1
soupsieve 2.3.1
soxr 0.3.5
spacy 3.5.1
spacy-cleaner 3.1.3
spacy-legacy 3.0.12
spacy-loggers 1.0.4
spacy-lookups-data 1.0.5
spark-tensorflow-distributor 1.0.0
SQLAlchemy 1.4.39
sqlparse 0.4.2
srsly 2.4.6
ssh-import-id 5.11
stack-data 0.6.2
statsmodels 0.13.2
swagger-spec-validator 3.0.3
tabulate 0.8.10
tangled-up-in-unicode 0.2.0
tenacity 8.1.0
tensorboard 2.11.0
tensorboard-data-server 0.6.1
tensorboard-plugin-profile 2.11.2
tensorboard-plugin-wit 1.8.1
tensorflow 2.11.1
tensorflow-estimator 2.11.0
tensorflow-io-gcs-filesystem 0.32.0
termcolor 2.3.0
terminado 0.13.1
testpath 0.6.0
thinc 8.1.10
threadpoolctl 2.2.0
tiktoken 0.3.3
tokenize-rt 4.2.1
tokenizers 0.13.3
tomli 2.0.1
torch 1.13.1+cu117
torchvision 0.14.1+cu117
tornado 6.1
tqdm 4.64.1
traitlets 5.1.1
transformers 4.33.3
typeguard 2.13.3
typer 0.7.0
typing_extensions 4.3.0
typing-inspect 0.8.0
ujson 5.4.0
unattended-upgrades 0.1
unstructured 0.9.0
uri-template 1.3.0
urllib3 1.26.11
virtualenv 20.16.3
visions 0.7.5
wadllib 1.3.6
wasabi 0.10.1
watermark 2.4.3
wcwidth 0.2.5
webcolors 1.13
webencodings 0.5.1
websocket-client 0.58.0
Werkzeug 2.0.3
whatthepatch 1.0.2
wheel 0.37.1
widgetsnbextension 3.6.1
wrapt 1.14.1
xgboost 1.7.5
xxhash 3.2.0
yapf 0.31.0
yarl 1.9.2
ydata-profiling 4.1.2
zipp 3.8.0Here are two lines of the sample code pulled from the model card:
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")Here is the error I am receiving:
--------------------------------------------------------------------------- KeyError Traceback (most recent call last) File <command-4309329534262302>:1 ----> 1 model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") 2 tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:441, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 438 if kwargs_copy.get("torch_dtype", None) == "auto": 439 _ = kwargs_copy.pop("torch_dtype") --> 441 config, kwargs = AutoConfig.from_pretrained( 442 pretrained_model_name_or_path, 443 return_unused_kwargs=True, 444 trust_remote_code=trust_remote_code, 445 **hub_kwargs, 446 **kwargs_copy, 447 ) 448 if hasattr(config, "auto_map") and cls.__name__ in config.auto_map: 449 if not trust_remote_code: File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:937, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs) 935 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs) 936 elif "model_type" in config_dict: --> 937 config_class = CONFIG_MAPPING[config_dict["model_type"]] 938 return config_class.from_dict(config_dict, **unused_kwargs) 939 else: 940 # Fallback: use pattern matching on the string. 941 # We go from longer names to shorter names to catch roberta before bert (for instance) File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:643, in _LazyConfigMapping.__getitem__(self, key) 641 return self._extra_content[key] 642 if key not in self._mapping: --> 643 raise KeyError(key) 644 value = self._mapping[key] 645 module_name = model_type_to_module_name(key) KeyError: 'mistral'
you need to update the transformers package to the latest.
transformers == 4.33.3 is not the latest?
That is what I shared above and from what I can see on GitHub, 4.33.3 appears to be the latest release (released 12 hours ago).
See comment below
I am also getting this error as well using the sample code for this model.
Here is an abbreviated environment:
Package Version
absl-py 1.0.0
accelerate 0.21.0
aiohttp 3.8.4
aiosignal 1.3.1
alembic 1.12.0
appdirs 1.4.4
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
astor 0.8.1
asttokens 2.2.1
astunparse 1.6.3
async-timeout 4.0.2
attrs 21.4.0
audioread 3.0.0
azure-core 1.26.4
azure-cosmos 4.3.1b1
azure-storage-blob 12.16.0
azure-storage-file-datalake 12.11.0
backcall 0.2.0
bcrypt 3.2.0
beautifulsoup4 4.11.1
bitsandbytes 0.41.0
black 22.6.0
bleach 4.1.0
blinker 1.4
blis 0.7.9
boto3 1.24.28
botocore 1.27.28
bravado 11.0.3
bravado-core 6.1.0
cachetools 4.2.4
catalogue 2.0.8
category-encoders 2.6.0
certifi 2022.9.14
cffi 1.15.1
chardet 4.0.0
charset-normalizer 2.0.4
click 8.0.4
cloudpickle 2.0.0
cmaes 0.10.0
cmdstanpy 1.1.0
colorlog 6.7.0
confection 0.0.4
configparser 5.2.0
convertdate 2.4.0
cryptography 37.0.1
cycler 0.11.0
cymem 2.0.7
Cython 0.29.32
databricks-automl-runtime 0.2.16
databricks-cli 0.17.6
databricks-feature-store 0.12.1
dataclasses-json 0.5.7
datasets 2.12.0
dbl-tempo 0.1.23
dbus-python 1.2.18
debugpy 1.5.1
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.4
diskcache 5.6.1
distlib 0.3.6
distro 1.7.0
distro-info 1.1+ubuntu0.1
doc2text 0.2.4
docstring-to-markdown 0.12
einops 0.6.1
entrypoints 0.4
ephem 4.1.4
et-xmlfile 1.1.0
evaluate 0.4.0
executing 1.2.0
facets-overview 1.0.3
faiss-gpu 1.7.2
fastjsonschema 2.16.3
fasttext 0.9.2
filelock 3.6.0
filetype 1.2.0
Flask 1.1.2+db1
flatbuffers 23.5.9
fonttools 4.25.0
fqdn 1.5.1
frozenlist 1.3.3
fsspec 2022.7.1
future 0.18.2
gast 0.4.0
gitdb 4.0.10
GitPython 3.1.27
google-api-core 2.8.2
google-auth 1.33.0
google-auth-oauthlib 0.4.6
google-cloud-core 2.3.2
google-cloud-storage 2.9.0
google-crc32c 1.5.0
google-pasta 0.2.0
google-resumable-media 2.5.0
googleapis-common-protos 1.56.4
greenlet 1.1.1
grpcio 1.48.1
grpcio-status 1.48.1
gunicorn 20.1.0
gviz-api 1.10.0
h5py 3.7.0
hijri-converter 2.3.1
holidays 0.22
horovod 0.27.0
htmlmin 0.1.12
httplib2 0.20.2
huggingface-hub 0.17.3
idna 3.3
ImageHash 4.3.1
imbalanced-learn 0.8.1
importlib-metadata 4.11.3
ipykernel 6.17.1
ipython 8.10.0
ipython-genutils 0.2.0
ipywidgets 7.7.2
isodate 0.6.1
isoduration 20.11.0
itsdangerous 2.0.1
jedi 0.18.1
jeepney 0.7.1
Jinja2 2.11.3
jmespath 0.10.0
joblib 1.2.0
joblibspark 0.5.1
jsonpointer 2.4
jsonref 1.1.0
jsonschema 4.16.0
jupyter-client 7.3.4
jupyter_core 4.11.2
jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0
keras 2.11.0
keyring 23.5.0
kiwisolver 1.4.2
korean-lunar-calendar 0.3.1
langchain 0.0.285
langcodes 3.3.0
langsmith 0.0.41
launchpadlib 1.10.16
lazr.restfulclient 0.14.4
lazr.uri 1.0.6
lazy_loader 0.2
libclang 15.0.6.1
librosa 0.10.0
lightgbm 3.3.5
llvmlite 0.38.0
LunarCalendar 0.0.9
lxml 4.9.3
Mako 1.2.0
Markdown 3.3.4
MarkupSafe 2.0.1
marshmallow 3.19.0
marshmallow-enum 1.5.1
matplotlib 3.5.2
matplotlib-inline 0.1.6
mccabe 0.7.0
mime 0.1.0
mistune 0.8.4
mleap 0.20.0
mlflow-skinny 2.3.1
monotonic 1.6
more-itertools 8.10.0
msgpack 1.0.5
multidict 6.0.4
multimethod 1.9.1
multiprocess 0.70.12.2
murmurhash 1.0.9
mypy-extensions 0.4.3
nbclient 0.5.13
nbconvert 6.4.4
nbformat 5.5.0
neptune-client 0.16.18
neptune-optuna 1.1.0
nest-asyncio 1.5.5
networkx 2.8.4
nltk 3.7
nodeenv 1.8.0
notebook 6.4.12
numba 0.55.1
numexpr 2.8.4
numpy 1.21.5
oauthlib 3.2.0
openai 0.27.4
openapi-schema-pydantic 1.2.4
openpyxl 3.1.2
opt-einsum 3.3.0
optuna 3.3.0
packaging 21.3
pandas 1.4.4
pandocfilters 1.5.0
paramiko 2.9.2
parso 0.8.3
pathspec 0.9.0
pathy 0.10.1
patsy 0.5.2
petastorm 0.12.1
pexpect 4.8.0
phik 0.12.3
pickleshare 0.7.5
Pillow 9.2.0
pip 23.2.1
platformdirs 2.5.2
plotly 5.9.0
pluggy 1.0.0
pmdarima 2.0.3
pooch 1.7.0
preshed 3.0.8
prompt-toolkit 3.0.36
prophet 1.1.2
protobuf 3.19.4
psutil 5.9.0
psycopg2 2.9.3
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 8.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pybind11 2.10.4
pycparser 2.21
pydantic 1.10.6
pyflakes 3.0.1
Pygments 2.11.2
PyGObject 3.42.1
PyJWT 2.3.0
PyMeeus 0.5.12
PyNaCl 1.5.0
pyodbc 4.0.32
pyparsing 3.0.9
PyPDF2 3.0.1
pyright 1.1.294
pyrsistent 0.18.0
pytesseract 0.3.10
python-apt 2.4.0+ubuntu2
python-dateutil 2.8.2
python-docx 0.8.11
python-editor 1.0.4
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.7.1
python-magic 0.4.27
pytoolconfig 1.2.2
pytz 2022.1
PyWavelets 1.3.0
PyYAML 6.0
pyzmq 23.2.0
ragas 0.0.12
regex 2022.7.9
requests 2.28.1
requests-oauthlib 1.3.1
responses 0.18.0
rfc3339-validator 0.1.4
rfc3987 1.3.8
rope 1.7.0
rsa 4.9
s3transfer 0.6.0
safetensors 0.3.3
scikit-learn 1.1.1
scipy 1.9.1
seaborn 0.11.2
SecretStorage 3.3.1
Send2Trash 1.8.0
sentence-transformers 2.2.2
sentencepiece 0.1.97
setuptools 63.4.1
shap 0.41.0
simplejson 3.17.6
six 1.16.0
skops 0.8.0
slicer 0.0.7
smart-open 5.2.1
smmap 5.0.0
soundfile 0.12.1
soupsieve 2.3.1
soxr 0.3.5
spacy 3.5.1
spacy-cleaner 3.1.3
spacy-legacy 3.0.12
spacy-loggers 1.0.4
spacy-lookups-data 1.0.5
spark-tensorflow-distributor 1.0.0
SQLAlchemy 1.4.39
sqlparse 0.4.2
srsly 2.4.6
ssh-import-id 5.11
stack-data 0.6.2
statsmodels 0.13.2
swagger-spec-validator 3.0.3
tabulate 0.8.10
tangled-up-in-unicode 0.2.0
tenacity 8.1.0
tensorboard 2.11.0
tensorboard-data-server 0.6.1
tensorboard-plugin-profile 2.11.2
tensorboard-plugin-wit 1.8.1
tensorflow 2.11.1
tensorflow-estimator 2.11.0
tensorflow-io-gcs-filesystem 0.32.0
termcolor 2.3.0
terminado 0.13.1
testpath 0.6.0
thinc 8.1.10
threadpoolctl 2.2.0
tiktoken 0.3.3
tokenize-rt 4.2.1
tokenizers 0.13.3
tomli 2.0.1
torch 1.13.1+cu117
torchvision 0.14.1+cu117
tornado 6.1
tqdm 4.64.1
traitlets 5.1.1
transformers 4.33.3
typeguard 2.13.3
typer 0.7.0
typing_extensions 4.3.0
typing-inspect 0.8.0
ujson 5.4.0
unattended-upgrades 0.1
unstructured 0.9.0
uri-template 1.3.0
urllib3 1.26.11
virtualenv 20.16.3
visions 0.7.5
wadllib 1.3.6
wasabi 0.10.1
watermark 2.4.3
wcwidth 0.2.5
webcolors 1.13
webencodings 0.5.1
websocket-client 0.58.0
Werkzeug 2.0.3
whatthepatch 1.0.2
wheel 0.37.1
widgetsnbextension 3.6.1
wrapt 1.14.1
xgboost 1.7.5
xxhash 3.2.0
yapf 0.31.0
yarl 1.9.2
ydata-profiling 4.1.2
zipp 3.8.0Here are two lines of the sample code pulled from the model card:
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")Here is the error I am receiving:
--------------------------------------------------------------------------- KeyError Traceback (most recent call last) File <command-4309329534262302>:1 ----> 1 model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") 2 tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:441, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 438 if kwargs_copy.get("torch_dtype", None) == "auto": 439 _ = kwargs_copy.pop("torch_dtype") --> 441 config, kwargs = AutoConfig.from_pretrained( 442 pretrained_model_name_or_path, 443 return_unused_kwargs=True, 444 trust_remote_code=trust_remote_code, 445 **hub_kwargs, 446 **kwargs_copy, 447 ) 448 if hasattr(config, "auto_map") and cls.__name__ in config.auto_map: 449 if not trust_remote_code: File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:937, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs) 935 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs) 936 elif "model_type" in config_dict: --> 937 config_class = CONFIG_MAPPING[config_dict["model_type"]] 938 return config_class.from_dict(config_dict, **unused_kwargs) 939 else: 940 # Fallback: use pattern matching on the string. 941 # We go from longer names to shorter names to catch roberta before bert (for instance) File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:643, in _LazyConfigMapping.__getitem__(self, key) 641 return self._extra_content[key] 642 if key not in self._mapping: --> 643 raise KeyError(key) 644 value = self._mapping[key] 645 module_name = model_type_to_module_name(key) KeyError: 'mistral'you need to update the transformers package to the latest.
transformers == 4.33.3 is not the latest?
That is what I shared above and from what I can see on GitHub, 4.33.3 appears to be the latest release (released 12 hours ago).
I am also getting this error as well using the sample code for this model.
Here is an abbreviated environment:
Package Version
absl-py 1.0.0
accelerate 0.21.0
aiohttp 3.8.4
aiosignal 1.3.1
alembic 1.12.0
appdirs 1.4.4
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
astor 0.8.1
asttokens 2.2.1
astunparse 1.6.3
async-timeout 4.0.2
attrs 21.4.0
audioread 3.0.0
azure-core 1.26.4
azure-cosmos 4.3.1b1
azure-storage-blob 12.16.0
azure-storage-file-datalake 12.11.0
backcall 0.2.0
bcrypt 3.2.0
beautifulsoup4 4.11.1
bitsandbytes 0.41.0
black 22.6.0
bleach 4.1.0
blinker 1.4
blis 0.7.9
boto3 1.24.28
botocore 1.27.28
bravado 11.0.3
bravado-core 6.1.0
cachetools 4.2.4
catalogue 2.0.8
category-encoders 2.6.0
certifi 2022.9.14
cffi 1.15.1
chardet 4.0.0
charset-normalizer 2.0.4
click 8.0.4
cloudpickle 2.0.0
cmaes 0.10.0
cmdstanpy 1.1.0
colorlog 6.7.0
confection 0.0.4
configparser 5.2.0
convertdate 2.4.0
cryptography 37.0.1
cycler 0.11.0
cymem 2.0.7
Cython 0.29.32
databricks-automl-runtime 0.2.16
databricks-cli 0.17.6
databricks-feature-store 0.12.1
dataclasses-json 0.5.7
datasets 2.12.0
dbl-tempo 0.1.23
dbus-python 1.2.18
debugpy 1.5.1
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.4
diskcache 5.6.1
distlib 0.3.6
distro 1.7.0
distro-info 1.1+ubuntu0.1
doc2text 0.2.4
docstring-to-markdown 0.12
einops 0.6.1
entrypoints 0.4
ephem 4.1.4
et-xmlfile 1.1.0
evaluate 0.4.0
executing 1.2.0
facets-overview 1.0.3
faiss-gpu 1.7.2
fastjsonschema 2.16.3
fasttext 0.9.2
filelock 3.6.0
filetype 1.2.0
Flask 1.1.2+db1
flatbuffers 23.5.9
fonttools 4.25.0
fqdn 1.5.1
frozenlist 1.3.3
fsspec 2022.7.1
future 0.18.2
gast 0.4.0
gitdb 4.0.10
GitPython 3.1.27
google-api-core 2.8.2
google-auth 1.33.0
google-auth-oauthlib 0.4.6
google-cloud-core 2.3.2
google-cloud-storage 2.9.0
google-crc32c 1.5.0
google-pasta 0.2.0
google-resumable-media 2.5.0
googleapis-common-protos 1.56.4
greenlet 1.1.1
grpcio 1.48.1
grpcio-status 1.48.1
gunicorn 20.1.0
gviz-api 1.10.0
h5py 3.7.0
hijri-converter 2.3.1
holidays 0.22
horovod 0.27.0
htmlmin 0.1.12
httplib2 0.20.2
huggingface-hub 0.17.3
idna 3.3
ImageHash 4.3.1
imbalanced-learn 0.8.1
importlib-metadata 4.11.3
ipykernel 6.17.1
ipython 8.10.0
ipython-genutils 0.2.0
ipywidgets 7.7.2
isodate 0.6.1
isoduration 20.11.0
itsdangerous 2.0.1
jedi 0.18.1
jeepney 0.7.1
Jinja2 2.11.3
jmespath 0.10.0
joblib 1.2.0
joblibspark 0.5.1
jsonpointer 2.4
jsonref 1.1.0
jsonschema 4.16.0
jupyter-client 7.3.4
jupyter_core 4.11.2
jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0
keras 2.11.0
keyring 23.5.0
kiwisolver 1.4.2
korean-lunar-calendar 0.3.1
langchain 0.0.285
langcodes 3.3.0
langsmith 0.0.41
launchpadlib 1.10.16
lazr.restfulclient 0.14.4
lazr.uri 1.0.6
lazy_loader 0.2
libclang 15.0.6.1
librosa 0.10.0
lightgbm 3.3.5
llvmlite 0.38.0
LunarCalendar 0.0.9
lxml 4.9.3
Mako 1.2.0
Markdown 3.3.4
MarkupSafe 2.0.1
marshmallow 3.19.0
marshmallow-enum 1.5.1
matplotlib 3.5.2
matplotlib-inline 0.1.6
mccabe 0.7.0
mime 0.1.0
mistune 0.8.4
mleap 0.20.0
mlflow-skinny 2.3.1
monotonic 1.6
more-itertools 8.10.0
msgpack 1.0.5
multidict 6.0.4
multimethod 1.9.1
multiprocess 0.70.12.2
murmurhash 1.0.9
mypy-extensions 0.4.3
nbclient 0.5.13
nbconvert 6.4.4
nbformat 5.5.0
neptune-client 0.16.18
neptune-optuna 1.1.0
nest-asyncio 1.5.5
networkx 2.8.4
nltk 3.7
nodeenv 1.8.0
notebook 6.4.12
numba 0.55.1
numexpr 2.8.4
numpy 1.21.5
oauthlib 3.2.0
openai 0.27.4
openapi-schema-pydantic 1.2.4
openpyxl 3.1.2
opt-einsum 3.3.0
optuna 3.3.0
packaging 21.3
pandas 1.4.4
pandocfilters 1.5.0
paramiko 2.9.2
parso 0.8.3
pathspec 0.9.0
pathy 0.10.1
patsy 0.5.2
petastorm 0.12.1
pexpect 4.8.0
phik 0.12.3
pickleshare 0.7.5
Pillow 9.2.0
pip 23.2.1
platformdirs 2.5.2
plotly 5.9.0
pluggy 1.0.0
pmdarima 2.0.3
pooch 1.7.0
preshed 3.0.8
prompt-toolkit 3.0.36
prophet 1.1.2
protobuf 3.19.4
psutil 5.9.0
psycopg2 2.9.3
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 8.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pybind11 2.10.4
pycparser 2.21
pydantic 1.10.6
pyflakes 3.0.1
Pygments 2.11.2
PyGObject 3.42.1
PyJWT 2.3.0
PyMeeus 0.5.12
PyNaCl 1.5.0
pyodbc 4.0.32
pyparsing 3.0.9
PyPDF2 3.0.1
pyright 1.1.294
pyrsistent 0.18.0
pytesseract 0.3.10
python-apt 2.4.0+ubuntu2
python-dateutil 2.8.2
python-docx 0.8.11
python-editor 1.0.4
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.7.1
python-magic 0.4.27
pytoolconfig 1.2.2
pytz 2022.1
PyWavelets 1.3.0
PyYAML 6.0
pyzmq 23.2.0
ragas 0.0.12
regex 2022.7.9
requests 2.28.1
requests-oauthlib 1.3.1
responses 0.18.0
rfc3339-validator 0.1.4
rfc3987 1.3.8
rope 1.7.0
rsa 4.9
s3transfer 0.6.0
safetensors 0.3.3
scikit-learn 1.1.1
scipy 1.9.1
seaborn 0.11.2
SecretStorage 3.3.1
Send2Trash 1.8.0
sentence-transformers 2.2.2
sentencepiece 0.1.97
setuptools 63.4.1
shap 0.41.0
simplejson 3.17.6
six 1.16.0
skops 0.8.0
slicer 0.0.7
smart-open 5.2.1
smmap 5.0.0
soundfile 0.12.1
soupsieve 2.3.1
soxr 0.3.5
spacy 3.5.1
spacy-cleaner 3.1.3
spacy-legacy 3.0.12
spacy-loggers 1.0.4
spacy-lookups-data 1.0.5
spark-tensorflow-distributor 1.0.0
SQLAlchemy 1.4.39
sqlparse 0.4.2
srsly 2.4.6
ssh-import-id 5.11
stack-data 0.6.2
statsmodels 0.13.2
swagger-spec-validator 3.0.3
tabulate 0.8.10
tangled-up-in-unicode 0.2.0
tenacity 8.1.0
tensorboard 2.11.0
tensorboard-data-server 0.6.1
tensorboard-plugin-profile 2.11.2
tensorboard-plugin-wit 1.8.1
tensorflow 2.11.1
tensorflow-estimator 2.11.0
tensorflow-io-gcs-filesystem 0.32.0
termcolor 2.3.0
terminado 0.13.1
testpath 0.6.0
thinc 8.1.10
threadpoolctl 2.2.0
tiktoken 0.3.3
tokenize-rt 4.2.1
tokenizers 0.13.3
tomli 2.0.1
torch 1.13.1+cu117
torchvision 0.14.1+cu117
tornado 6.1
tqdm 4.64.1
traitlets 5.1.1
transformers 4.33.3
typeguard 2.13.3
typer 0.7.0
typing_extensions 4.3.0
typing-inspect 0.8.0
ujson 5.4.0
unattended-upgrades 0.1
unstructured 0.9.0
uri-template 1.3.0
urllib3 1.26.11
virtualenv 20.16.3
visions 0.7.5
wadllib 1.3.6
wasabi 0.10.1
watermark 2.4.3
wcwidth 0.2.5
webcolors 1.13
webencodings 0.5.1
websocket-client 0.58.0
Werkzeug 2.0.3
whatthepatch 1.0.2
wheel 0.37.1
widgetsnbextension 3.6.1
wrapt 1.14.1
xgboost 1.7.5
xxhash 3.2.0
yapf 0.31.0
yarl 1.9.2
ydata-profiling 4.1.2
zipp 3.8.0Here are two lines of the sample code pulled from the model card:
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")Here is the error I am receiving:
--------------------------------------------------------------------------- KeyError Traceback (most recent call last) File <command-4309329534262302>:1 ----> 1 model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") 2 tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1") File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:441, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs) 438 if kwargs_copy.get("torch_dtype", None) == "auto": 439 _ = kwargs_copy.pop("torch_dtype") --> 441 config, kwargs = AutoConfig.from_pretrained( 442 pretrained_model_name_or_path, 443 return_unused_kwargs=True, 444 trust_remote_code=trust_remote_code, 445 **hub_kwargs, 446 **kwargs_copy, 447 ) 448 if hasattr(config, "auto_map") and cls.__name__ in config.auto_map: 449 if not trust_remote_code: File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:937, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs) 935 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs) 936 elif "model_type" in config_dict: --> 937 config_class = CONFIG_MAPPING[config_dict["model_type"]] 938 return config_class.from_dict(config_dict, **unused_kwargs) 939 else: 940 # Fallback: use pattern matching on the string. 941 # We go from longer names to shorter names to catch roberta before bert (for instance) File /databricks/python/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:643, in _LazyConfigMapping.__getitem__(self, key) 641 return self._extra_content[key] 642 if key not in self._mapping: --> 643 raise KeyError(key) 644 value = self._mapping[key] 645 module_name = model_type_to_module_name(key) KeyError: 'mistral'you need to update the transformers package to the latest.
transformers == 4.33.3is not the latest?That is what I shared above and from what I can see on GitHub,
4.33.3appears to be the latest release (released 12 hours ago).
pip install git+https://huggingface/transformers.git
4.34.0dev support it
Running the following command seems to be working:
pip install git+https://github.com/huggingface/transformers
Did the model load for anyone on Free Colab? The RAM keeps crashing for me
Did the model load for anyone on Free Colab? The RAM keeps crashing for me
maybe try using gguf model by TheBloke?
has anyone ran this on a MBP16 32k ?
this code: import torch
import transformers
from transformers import GenerationConfig, pipeline
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import BitsAndBytesConfig
import bitsandbytes as bnb
import torch
import transformers
from transformers import GenerationConfig, pipeline
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("ehartford/samantha-mistral-7b",use_fast=False)
model = AutoModelForCausalLM.from_pretrained("ehartford/samantha-mistral-7b",
load_in_8bit=True,
device_map='auto',
torch_dtype=torch.float16,
low_cpu_mem_usage=True,
)
gave the same mentioned error: ---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
in <cell line: 14>()
12
13 tokenizer = AutoTokenizer.from_pretrained("ehartford/samantha-mistral-7b",use_fast=False)
---> 14 model = AutoModelForCausalLM.from_pretrained("ehartford/samantha-mistral-7b",
15 load_in_8bit=True,
16 device_map='auto',
2 frames
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py in getitem(self, key)
708
709 def model_type_to_module_name(key):
--> 710 """Converts a config key to the corresponding module."""
711 # Special treatment
712 if key in SPECIAL_MODEL_TYPE_TO_MODULE_NAME:
KeyError: 'mistral'
then u tried the installation of tranformers as given above: pip install git+https://github.com/huggingface/transformers
I tried following this guide from hugging face and got the issue: https://huggingface.co/blog/Andyrasika/samantha-and-mistral-7b
I DID THIS ON GOOGLE COLLAB FREE VERSION.
Help would be appreciated, I was tryna run langchain to see its performance in presenting information from a chroma vector database (Chroma seems to be the goto), I tried for falcon and got repetition error, LLama 13b ggml quantized had a guide and it worked beautifully. So trying to run with Mistral, whichever works.
I just told the last part for context. Help will be appreciated!!
Install the last version of transformers:
%pip install git+https://github.com/huggingface/transformers
And then restart the runtime to apply the changes
@Kromtar
Ah I just ran a GGML quantised version of Mistral 7b with Llama-index and it worked perfectly well with fast inferences. The framework just made the entire process more convenient. Trying to figure out how to calculate max input/output token length for queries and cpu/gpu usage and of course time elapsed. If you have any leads please tell me.
Did the model load for anyone on Free Colab? The RAM keeps crashing for me
Try using Kaggle. Use a T4 or P100 instance
@i-darrshan yes. I ran it on Collab with llama-index and llama-cpp. But I ran a GGUF version of it.
@AbishekNairM . Hey mate, I have a doubt. Have you ever used ctransformers. I used "TheBloke's Mistral 7B gguf with Q4_K_M" in colab with ctransformers. I'm getting incomplete answers and also sometimes no answer. Being a beginner, I'm unable to figure out why. Could anyone help on this?
@i-darrshan I have only used hugging faces auto tokenizers before this but since mistral didn't have one at the time I was about to use ctransformers since it's the goto if u don't have prebuilt tokenizers. But I read llama index worked really well with these two. In my opinion you should just use llama index. If you were trying to use maybe falcon you can consider ctransformer, since your gonna have to edit preprompts in llamaindex. Well I would still just try to figure that out in llamaindex tbh. It's just so much better and convenient. I vouch for llama index for Mistral and Llama models 100 percent.
@i-darrshan I have only used hugging faces auto tokenizers before this but since mistral didn't have one at the time I was about to use ctransformers since it's the goto if u don't have prebuilt tokenizers. But I read llama index worked really well with these two. In my opinion you should just use llama index. If you were trying to use maybe falcon you can consider ctransformer, since your gonna have to edit preprompts in llamaindex. Well I would still just try to figure that out in llamaindex tbh. It's just so much better and convenient. I vouch for llama index for Mistral and Llama models 100 percent.
@AbishekNairM sure, I will try using llama-cpp-python. Thanks