Model Generating Prefix
#85
by
dongyulin
- opened
I am trying get rid of "<|start_header_id|>assistant<|end_header_id|>" and "<|eot_id|>" when generating output on TextIteratorStreamer, is there anyway I can achieve this on when generating output?
streamer = TextIteratorStreamer(tokenizer, skip_prompt=True)
model.generate(model_inputs, streamer=streamer, pad_token_id=tokenizer.eos_token_id, max_new_tokens=1000,do_sample=True)