Fix eos_token in tokenizer_config.json

#4

This fix has been added by Meta 5 days ago : https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/commit/a8977699a3d0820e80129fb3c93c20fbd9972c41

Fixes a bug when Llama 3 does not stop after generating an end of sentence token.

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment