NousResearch

/

Hermes-3-Llama-3.1-8B

like 236

Follow

NousResearch 812

Thanks for adding the PR, while this version doesn't use <|python_tag|> and <|eom_id|> for tool-use like llama-3.1, we have an updated tool use template that implements a <scratch_pad> for intermediate reasoning steps using Goal Oriented Action Planning (GOAP) framework.

Here's the go template that I created for ollama for reference:

{{- if .Messages }}
{{- if or .System .Tools }}<|im_start|>system
{{ .System }}
{{- if .Tools }}
You are a function calling AI model. You are provided with function signatures within <tools> </tools> XML tags. You may call one or more functions to assist with the user query. If available tools are not relevant in assisting with user query, just respond in natural conversational language. Don't make assumptions about what values to plug into functions. After calling & executing the functions, you will be provided with function results within <tool_response> </tool_response> XML tags.
<tools>
{{- range .Tools }} 
{{ json . }}
{{- end }}
</tools>
For each function call return a JSON object, with the following pydantic model json schema:
{'title': 'FunctionCall', 'type': 'object', 'properties': {'name': {'title': 'Name', 'type': 'string'}, 'arguments': {'title': 'Arguments', 'type': 'object'}}, 'required': ['arguments', 'name']}
Each function call should be enclosed within <tool_call> </tool_call> XML tags. You must use <scratch_pad> </scratch_pad> XML tags to record your reasoning and planning before you call the functions as follows.
Example:
<scratch_pad>
Goal: <state task assigned by user>
Actions:
<if tool calls need to be generated:>
- {result_var_name1} = functions.{function_name1}({param1}={value1},...)
- {result_var_name2, result_var_name3} = ...
<if no tool call needs to be generated:> None
Observation: <set observation 'None' with tool calls; plan final tools results summary when provided>
Reflection: <evaluate query-tool relevance and required parameters when tools called; analyze overall task status when observations made>
</scratch_pad>
<tool_call>
{'name': <function-name>, 'arguments': <args-dict>}
</tool_call>
{{- end }}<|im_end|>
{{- end }}

{{- $hasToolResponses := false }}
{{- range .Messages }}
{{- if eq .Role "tool" }}
{{- if not $hasToolResponses }}
<|im_start|>tool
{{- $hasToolResponses = true }}
{{- end }}
<tool_response>
{{ .Content }}
</tool_response>
{{- else }}
{{- if $hasToolResponses }}<|im_end|>
{{- $hasToolResponses = false }}
{{- end }}
<|im_start|>{{ .Role }}
{{- if and (eq .Role "assistant") .ToolCalls }}
{{- range .ToolCalls }}
<tool_call>
{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
</tool_call>
{{- end }}
{{- else }}
{{ .Content }}
{{- end }}<|im_end|>
{{- end }}
{{- end }}
{{- if $hasToolResponses }}<|im_end|>
{{- end }}

{{- else }}
{{- if .System }}
<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}

{{- if .Prompt }}
<|im_start|>user
{{ .Prompt }}<|im_end|>
{{- end }}

<|im_start|>assistant
{{ .Response }}<|im_end|>
{{- end }}

Updated with tool use format from H2 Pro

This simple templpate for ollama is also right ?
TEMPLATE """
{{- if .System }}
<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}
{{- if and (eq .Role "user") $.Tools }}
Here are the available tools:

{{ json $.Tools }}

{{- end }}
{{- if and (eq .Role "assistant") .ToolCalls }}

{{- range .ToolCalls }}
{"arguments": {{ json .Function.Arguments }}, "name": "{{ .Function.Name }}"}
{{- end }}

{{- end }}
{{- if eq .Role "tool" }}

{{ .Content }}

{{- end }}
<|im_end|>
{{- end }}
<|im_start|>assistant
"""

No idea