function/tool calling capability still in place?
#10
by
aleclaza
- opened
Does the WizardLM-2 8x22B preserve the function/tool calling capabilities and tokenizer compatibility with the original Mixtral 8x22B - https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1#function-calling-and-special-tokens
For mixtral: "vocab_size": 32768.
And you can find it is 32000 in this repo, which is the original mistral/llama one.
So, I think it is lost as these are no longer special tokens. And I think they train the lm_head...