Extract only output
Hi,
I ran the Gemma Model and I use tokenizer.decode(outputs[0]) and it give me the with the input text I have given to the tokenizer and then the output followed by an . Is there any way that I can just get the output as a string without the input text in the generated output?
Hi @tkaintura , Models will give input text in generated output. To skip it Please follow below solutions:
Use the length of the input tokens to determine where the generated tokens start.
Decode the tokens that follow the input length.
Kindly find the below screenshot for more reference. Thank you.
I tried this but the model still has some extra tokens as a part of the input text in the generated_text. Any idea why it would add extra tokens to the input text and then give the actual answer?
Hi @tkaintura ,
I tested the provided code with two different inputs and did not encounter any additional tokens. For reference, you can check this gist file. If you are still facing the issue, please share your code, and I will review it to provide an appropriate solution.
Thank you.