Strange behavior
#2
by
alexcardo
- opened
Upd: I suppose, there should be a prompt template in prompt!
In [3]: response = generate(model, tokenizer, prompt="hello", verbose=True)
Prompt: hello
,
I have a problem with the "add to cart" button.
I have a product with a "add to cart" button.
When I click on the button, the product is added to the cart.
But when I click on the "cart" button, the product is not in the cart.
I have a "cart" button on the product page.
When I click on the "cart" button, the product is not in the cart.
I have
Prompt: 0.104 tokens-per-sec
Generation: 9.004 tokens-per-sec
That's because this is a chat model.
messages = [{"role": "user", "content": "Anneme onu ne kadar sevdiğimi anlatan bir mektup yaz"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="mlx")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Anneme onu ne kadar sevdiğimi anlatan bir mektup yaz<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
Read more here:
https://huggingface.co/CohereForAI/aya-23-8B
prince-canuma
changed discussion status to
closed