Max Context Lenght

by Desm0nt - opened Jan 15

Jan 15

•

Hello.
How long real max context for this model? Config says that it's same as mixtral/mistral - 32k. But in reality it produces just noise even on 20k (while original mixtral handle it correctly).

mlabonne

Owner Jan 15

Hi @Desm0nt this is a model made with 4 Mistral-7b models with a context of 8k and not a Mixtral model with a context of 32k. This behavior is expected.

I updated the model and the config.json to reflect that. I hope it's clearer now.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment