Function calling - force tool calling
#64
by
rtilman
- opened
Hi,
Is there a way to force tool usage with the Transformers library, similar to what is explained in https://docs.mistral.ai/capabilities/function_calling/#tool_choice , using the tool_choice parameter in the model request?
Many thanks
Hi
@rtilman
, this isn't supported in the chat templates, but you can force it manually. You would have to add the tokens for the start of a tool response, then generate text following that point. One easy way to get this is to add a dummy tool call to the end of your current chat, then format with apply_chat_template, then trim the end of the sequence back to [TOOL_CALLS] or whatever token indicates the start of tool calling. The model will then generate a tool call.