Victor Gallego

vicgalle

AI & ML interests

Preference fine-tuning, alignment & synthetic data. Building LLMs in general!

Recent Activity

liked a model 3 days ago
Qwen/QwQ-32B-Preview
liked a model 9 days ago
AIDC-AI/Marco-o1
liked a model 16 days ago
Qwen/Qwen2.5-Coder-32B-Instruct
View all activity

Organizations

Spaces-explorers's profile picture Stable Diffusion concepts library's profile picture Komorebi AI's profile picture Test ORG's profile picture Social Post Explorers's profile picture

Posts 1

view post
Post
Can you merge models of different sizes? ⚗️

Well, yes, if the models are somewhat compatible. Here is an experiment I did. I wanted to merge two of the best performing models: mlabonne/NeuralBeagle14-7B and jeonsworld/CarbonVillain-en-10.7B-v4

Here is my recipe:
1. Expand the layers of NeuralBeagle to 10.7B ala frankenmerge.
2. DPO-tune the previous model with a high-quality preference dataset, argilla/distilabel-intel-orca-dpo-pairs
3. Merge the previous model with CarbonVillain (needs —allow-crimes in mergekit! 🔪)

And here is the resulting model, CarbonBeagle-11B, which ranked top in the leaderboard for its size class:
vicgalle/CarbonBeagle-11B