This is a wrong solution, a new PR has been submitted, sorry...

#11

by zjyhf - opened May 15

base: refs/heads/main

←

from: refs/pr/11

Discussion Files changed

+65

-6

The tokenizer adds a special token '<|im_end|>' to solve the problem of non-stop generation when encountering <|im_end|>.26140d90

zjyhf

May 15

This comment has been hidden

zjyhf changed pull request status to closed May 15

zjyhf changed pull request title from The tokenizer adds a special token '<|im_end|>' to solve the problem of non-stop generation when encountering <|im_end|>. to This is a wrong solution, a new PR has been submitted, sorry... May 15

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment