ollama
#2
by
goodasdgood
- opened
The model is very slow in ollama
not slow with lama cpp
@goodasdgood I don't use ollama, but ensure you have the right prompt format:
FROM your_model_name.gguf
TEMPLATE """
<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|><|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>
"""
SYSTEM ""
add in rest of parameters as necessary.
Orenguteng
changed discussion status to
closed
Orenguteng
changed discussion status to
open
sloooooooow on ollama
it ius run good
thank you
Orenguteng
changed discussion status to
closed