argilla/ultrafeedback-binarized-preferences-cleaned-kto
Viewer
•
Updated
•
231k
•
148
•
9
Preference Datasets for KTO
This collection contains a list of curated preference datasets for KTO fine-tuning for intent alignment of LLMs through signals.
Note KTO transformed version of "argilla/ultrafeedback-binarized-preferences-cleaned".
argilla/distilabel-intel-orca-kto
Viewer • Updated • 23.1k • 36 • 5Note KTO transformed version of "argilla/distilabel-intel-orca-dpo-pairs"
argilla/distilabel-capybara-kto-15k-binarized
Viewer • Updated • 15.1k • 56 • 5Note KTO transformed version of "argilla/distilabel-capybara-dpo-7k-binarized".
argilla/kto-mix-15k
Viewer • Updated • 15.3k • 110 • 13Note KTO transformed version of "argilla/dpo-mix-7k".
-
KTO: Model Alignment as Prospect Theoretic Optimization
Paper • 2402.01306 • Published • 15