microsoft

/

Phi-3-vision-128k-instruct

like 935

Follow

Microsoft 5,515

I just used the example for the multi-turn input.

Just like this. This is the part of my code and it works fine :)

    chat = [
        {"role": "user", "content": '<|image_1|>' + '\n' + sys_prompt},
        {"role": "assistant", "content": few_shot_txt},
        {"role": "user", "content": '<|image_2|>\nCreate complex questions beyond describing the scene with this image too.'}
    ]

    images = [few_shot_img, image]

    prompt = processor.tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)

    if prompt.endswith("<|endoftext|>"):
        prompt = prompt.rstrip("<|endoftext|>")

    inputs = processor(prompt, images, return_tensors="pt").to("cuda")