Datasets for function calling and JSON
Hi! Are the datasets used for function calling and JSON available somewhere?
Hi! Are the datasets used for function calling and JSON available somewhere?
Not yet
Why does this model perform less than capybara hermes 2.5 on json evaluation?
I get 1.8 on capybara and 2.9 at hermes 2 pro
what evaluation framework/dataset are you using for json?
I made a combination of my auto-gpt logs and the teknium json eval.parquet which I parsed into json than txt.
I am using oobabooga text gen training-pro plugin with perplexity evaluation.
The JSON from capybara is better mostly but if often forgets delimiters or ending brackets.
Hermes 2 pro though just stops the json mid generation or makes out of context ones.
Currently I use them both for gpt-pilot and my own mini auto-gpt.
