Is this based on instruction tuned gemma or gemma base model?

by jsgreenawalt - opened Sep 29

Sep 29

I'm getting strange results (nonsense output) when trying to add some fine-tunes based on this model to a merge with gemma-2-9b-it variants, just wanted to confirm this is based on the non-it version of the original Gemma weights?

Thanks!

intervitens

Intervitens AI Innovations Inc. org Sep 30

This is the same weights as the base model, not -it, not finetuned either, the only modification is replacing two of the reserved tokens in the tokenizer, and changing the eos token in the config.

jsgreenawalt

Oct 2

Thanks for the info

jsgreenawalt changed discussion status to closed Oct 2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment