Error while running.
Traceback (most recent call last):
File "/backup/Vidya/test1.py", line 6, in
generator = pipeline(model=model, tokenizer=tokenizer)
File "/backup/Vidya/llama/lib/python3.9/site-packages/transformers/pipelines/init.py", line 712, in pipeline
raise RuntimeError(
RuntimeError: Inferring the task automatically requires to check the hub with a model_id defined as a str.GPTNeoXForCausalLM(
(gpt_neox): GPTNeoXModel(
(embed_in): Embedding(50280, 5120)
(layers): ModuleList(
(0-35): 36 x GPTNeoXLayer(
(input_layernorm): LayerNorm((5120,), eps=1e-05, elementwise_affine=True)
(post_attention_layernorm): LayerNorm((5120,), eps=1e-05, elementwise_affine=True)
(attention): GPTNeoXAttention(
(rotary_emb): RotaryEmbedding()
(query_key_value): Linear(in_features=5120, out_features=15360, bias=True)
(dense): Linear(in_features=5120, out_features=5120, bias=True)
)
(mlp): GPTNeoXMLP(
(dense_h_to_4h): Linear(in_features=5120, out_features=20480, bias=True)
(dense_4h_to_h): Linear(in_features=20480, out_features=5120, bias=True)
(act): GELUActivation()
)
)
)
(final_layer_norm): LayerNorm((5120,), eps=1e-05, elementwise_affine=True)
)
(embed_out): Linear(in_features=5120, out_features=50280, bias=False)
) is not a valid model_id.
code:-
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
tokenizer = AutoTokenizer.from_pretrained("/backup/Vidya/dolly-v2-12b")
model = AutoModelForCausalLM.from_pretrained("/backup/Vidya/dolly-v2-12b")
generator = pipeline(model=model, tokenizer=tokenizer)
print(generator("I can't believe you did such a "))
RAM:- 120GB
What do you have in "/backup/Vidya/dolly-v2-12b" ? it is saying that is not a saved model folder. Just pass "databricks/dolly-v2-12b" instead
I have some space issues, where the model is actually stored. so I downloaded the model and give the path to it.
I have a different idea. Try adding task='text-generation' to your pipeline() call?
OMG, it worked. thanks for the help.
I did both things that you have mentioned
- direct model name.
- text="text-generation"
The code I have used:-
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-12b")
model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-12b")
generator = pipeline(task='text-generation', model=model, tokenizer=tokenizer)
print(generator("I can't believe you did such a "))
RAM:- 120GB.
while running it went up to like 55GB.
OK that makes some sense. I think Matt is fixing the custom pipeline class for this model to set that task type and that should resolve too.
Should be all set now. See the latest examples in the HF model pages.