Text only generation for Idefics3
Hi everyone, is it possible to only pass text to Idefics3, so text to text generation? I am using approximately the same code as with Idefics2, which seems to be in line with Idefics3. In the following code I do not pass images to the prompt:
Create inputs
messages = [
{
"role": "user",
"content": [
]
}
]
format_prompt = prompt.format(truncated_text)
messages[0]['content'].append({"type": "text", "text": format_prompt})
idefics_prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(text=idefics_prompt, return_tensors="pt")
inputs = {k: v.to(DEVICE) for k, v in inputs.items()}
However, this line "inputs = processor(text=idefics_prompt, return_tensors="pt")" gives me the following error:
ValueError: The number of images in the text [0] and images [] should be the same.
Does anyone know if there is a solution? I installed the newest Transformers version also from source but it does not work. Thanks in advance!
I have the same problem
It works by just tokenizing (using Idefics3 tokenizer) the idefics_prompt as the input.
It works by just tokenizing (using Idefics3 tokenizer) the idefics_prompt as the input.
@Guanzheng
Something like ?
inputs = tokenizer.apply_chat_template(
messages,
tokenize = True,
add_generation_prompt = True, # False for fine-tuning
return_tensors = "pt",).to(DEVICE)