| license: apache-2.0 | |
| datasets: | |
| - jondurbin/truthy-dpo-v0.1 | |
| ## Solarized-18B-truthy | |
| Solarized-18B-dpo fine-tuned to improve truthfulness. | |
| It is a frankenmerge model created using mergekit, alternating layers of Nous-Hermes-2-SOLAR-10.7B and SOLAR-10.7B-Instruct. Then, we applied DPO over a high-quality preference dataset. | |
|  |