ValueError: Unrecognized configuration class <class 'transformers_modules.falcon-7b.configuration_RW.RWConfig'> for this kind of AutoModel....
when I downloaded falcon 7b using git clone into the dir tiiuae/falcon_7b and run it with the following code:
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch
model = "tiiuae/falcon-7b"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
torch_dtype=torch.bfloat16,
trust_remote_code=True,
device_map="auto",
)
sequences = pipeline(
"Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
max_length=200,
do_sample=True,
top_k=10,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
print(seq['generated_text'])
But it throws and error:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28/3467703408.py", line 8, in
pipeline = transformers.pipeline(
File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/init.py", line 824, in pipeline
image_processor is None
File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/base.py", line 282, in infer_framework_load_model
ValueError: Could not load model tiiuae/falcon-7b with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.auto.modeling_tf_auto.TFAutoModelForCausalLM'>). See the original errors:
while loading with AutoModelForCausalLM, an error is thrown:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 488, in load_state_dict
loaded_keys = [k.replace(f"{module_name}.", "") for k in state_dict_keys if k.startswith(f"{module_name}.")]
File "/opt/conda/lib/python3.10/site-packages/torch/serialization.py", line 797, in load
with _open_zipfile_reader(opened_file) as opened_zipfile:
File "/opt/conda/lib/python3.10/site-packages/torch/serialization.py", line 283, in init
super().init(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 492, in load_state_dict
File "/opt/conda/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 128: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/base.py", line 269, in infer_framework_load_model
model = model_class.from_pretrained(model, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
from_pretrained.doc = from_pretrained_docstring
File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3175, in from_pretrained
# This should always be a list but, just to be sure.
File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3548, in _load_pretrained_model
return x
File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py", line 504, in load_state_dict
old_keys.append(key)
OSError: Unable to load weights from pytorch checkpoint file for 'tiiuae/falcon-7b/pytorch_model-00001-of-00002.bin' at 'tiiuae/falcon-7b/pytorch_model-00001-of-00002.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
while loading with TFAutoModelForCausalLM, an error is thrown:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/base.py", line 269, in infer_framework_load_model
model = model_class.from_pretrained(model, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
for model in model_mapping.values():
ValueError: Unrecognized configuration class <class 'transformers_modules.falcon-7b.configuration_RW.RWConfig'> for this kind of AutoModel: TFAutoModelForCausalLM.
Model type should be one of BertConfig, CamembertConfig, CTRLConfig, GPT2Config, GPT2Config, GPTJConfig, OpenAIGPTConfig, OPTConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoFormerConfig, TransfoXLConfig, XGLMConfig, XLMConfig, XLMRobertaConfig, XLNetConfig.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 2105, in showtraceback
stb = self.InteractiveTB.structured_traceback(
File "/opt/conda/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1428, in structured_traceback
return FormattedTB.structured_traceback(
File "/opt/conda/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1319, in structured_traceback
return VerboseTB.structured_traceback(
File "/opt/conda/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1172, in structured_traceback
formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
File "/opt/conda/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1087, in format_exception_as_a_whole
frames.append(self.format_record(record))
File "/opt/conda/lib/python3.10/site-packages/IPython/core/ultratb.py", line 969, in format_record
frame_info.lines, Colors, self.has_colors, lvals
File "/opt/conda/lib/python3.10/site-packages/IPython/core/ultratb.py", line 792, in lines
return self._sd.lines
File "/opt/conda/lib/python3.10/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
value = obj.dict[self.func.name] = self.func(obj)
File "/opt/conda/lib/python3.10/site-packages/stack_data/core.py", line 734, in lines
pieces = self.included_pieces
File "/opt/conda/lib/python3.10/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
value = obj.dict[self.func.name] = self.func(obj)
File "/opt/conda/lib/python3.10/site-packages/stack_data/core.py", line 681, in included_pieces
pos = scope_pieces.index(self.executing_piece)
File "/opt/conda/lib/python3.10/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
value = obj.dict[self.func.name] = self.func(obj)
File "/opt/conda/lib/python3.10/site-packages/stack_data/core.py", line 660, in executing_piece
return only(
File "/opt/conda/lib/python3.10/site-packages/executing/executing.py", line 190, in only
raise NotOneValueFound('Expected one value, found 0')
executing.executing.NotOneValueFound: Expected one value, found 0
How to fix this?
I already tried this, doesn't work:
pip install -U git+https://github.com/huggingface/transformers.git@e03a9cc &&
pip install -U git+https://github.com/huggingface/peft.git@42a184f &&
pip install -U git+https://github.com/huggingface/accelerate.git@c9fbb71 &&
pip install einops==0.6.1 &&
pip install torch==2.0.1 &&
pip install bitsandbytes==0.39.0 &&
pip install scipy &&
pip install loralib==0.1.1 && \
I'm also having similar problems. I just used the example code from: https://huggingface.co/tiiuae/falcon-7b. I get the following error:
2023-09-18 10:34:37.337197: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
File "/Users/badger/dev/transformers/falcoln7b.py", line 8, in <module>
pipeline = transformers.pipeline(
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/pipelines/__init__.py", line 824, in pipeline
framework, model = infer_framework_load_model(
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/pipelines/base.py", line 282, in infer_framework_load_model
raise ValueError(
ValueError: Could not load model tiiuae/falcon-7b with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.auto.modeling_tf_auto.TFAutoModelForCausalLM'>). See the original errors:
while loading with AutoModelForCausalLM, an error is thrown:
Traceback (most recent call last):
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/pipelines/base.py", line 269, in infer_framework_load_model
model = model_class.from_pretrained(model, **kwargs)
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
return model_class.from_pretrained(
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3175, in from_pretrained
) = cls._load_pretrained_model(
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3296, in _load_pretrained_model
raise ValueError(
ValueError: The current `device_map` had weights offloaded to the disk. Please provide an `offload_folder` for them. Alternatively, make sure you have `safetensors` installed if the model you are using offers the weights in this format.
while loading with TFAutoModelForCausalLM, an error is thrown:
Traceback (most recent call last):
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/pipelines/base.py", line 269, in infer_framework_load_model
model = model_class.from_pretrained(model, **kwargs)
File "/Users/badger/dev/transformers/.veenv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers_modules.tiiuae.falcon-7b.f7796529e36b2d49094450fb038cc7c4c86afa44.configuration_RW.RWConfig'> for this kind of AutoModel: TFAutoModelForCausalLM.
Model type should be one of BertConfig, CamembertConfig, CTRLConfig, GPT2Config, GPT2Config, GPTJConfig, OpenAIGPTConfig, OPTConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoFormerConfig, TransfoXLConfig, XGLMConfig, XLMConfig, XLMRobertaConfig, XLNetConfig.
Note that I've added offload_folder="offload" to the AutoTokenizer.from_pretrained call and that doesn't get rid of the offload_folder error.
It's hard to tell from the error which issue is causing which. I suspect one of the errors is likely spurious and is simply a knock on effect of the other error.
Anyone any ideas?
I think the fix is simple. Just put model = AutoModelForCausalLM.from_pretrained(model) after tokenizer = AutoTokenizer.from_pretrained(model, offload_folder="offload", torch_dtype=torch.bfloat16) and that should fix it.