293 121 792

Maxime Labonne PRO

mlabonne

https://mlabonne.github.io/blog

AI & ML interests

Post-training, model editing, quantization

Articles

Organizations

mlabonne's activity

upvoted an article 1 day ago

Article

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

2 days ago

• 96

upvoted a collection about 1 month ago

🧠 Abliteration

Collection

Uncensored models using abliteration. See this article for more information: huggingface.co/blog/mlabonne/abliteration • 7 items • Updated Aug 17 • 12

upvoted an article about 1 month ago

Article

Introduction to ggml

Aug 13

• 91

upvoted a paper about 1 month ago

The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines

Paper • 2408.01050 • Published Aug 2 • 8

upvoted an article about 2 months ago

Article

The case for specialized pre-training: ultra-fast foundation models for dedicated tasks

•

Aug 4

• 24

upvoted a paper about 2 months ago

Improving Text Embeddings for Smaller Language Models Using Contrastive Fine-tuning

Paper • 2408.00690 • Published Aug 1 • 21

upvoted a collection about 2 months ago

Probably function calling datasets

Collection

Created using the https://huggingface.co/spaces/librarian-bots/dataset-column-search-api Space. • 39 items • Updated Jul 17 • 35

upvoted 2 papers about 2 months ago

Executable Code Actions Elicit Better LLM Agents

Paper • 2402.01030 • Published Feb 1 • 27

Understanding Reference Policies in Direct Preference Optimization

Paper • 2407.13709 • Published Jul 18 • 15

upvoted 3 collections about 2 months ago

Bad Data Toolbox

Collection

PleIAs collection of models for the data processing of challenging document and data sources. • 5 items • Updated Jul 18 • 10

Common Corpus

Collection

The largest public domain dataset for training LLMs. • 27 items • Updated Jul 17 • 111

Finance Commons

Collection

A large collection of multimodal financial documents in open data. • 7 items • Updated Jul 17 • 3

upvoted an article about 2 months ago

Article

Fine-tune Llama 3.1 Ultra-Efficiently with Unsloth

•

Jul 29

• 193

upvoted a paper about 2 months ago

The Importance of Online Data: Understanding Preference Fine-tuning via Coverage

Paper • 2406.01462 • Published Jun 3 • 6

upvoted an article about 2 months ago

Article

The Rise of Agentic Data Generation

•

Jul 15

• 74

upvoted a collection 2 months ago

NuminaMath

Collection

Datasets and models for training SOTA math LLMs. See our GitHub for training & inference code: https://github.com/project-numina/aimo-progress-prize • 6 items • Updated Jul 21 • 54

upvoted 2 articles 2 months ago

Article

A brief analysis of automerger data, feat. SLERP and DARE-TIES LLM merging

•

Mar 24

• 1

Article

How we leveraged distilabel to create an Argilla 2.0 Chatbot

Jul 16

• 30

upvoted 2 papers 2 months ago

Qwen2 Technical Report

Paper • 2407.10671 • Published Jul 15 • 153

AgentInstruct: Toward Generative Teaching with Agentic Flows

Paper • 2407.03502 • Published Jul 3 • 43

upvoted an article 2 months ago

Article

Our Transformers Code Agent beats the GAIA benchmark!

Jul 1

• 44

upvoted a collection 3 months ago

📚 FineWeb-Edu

Collection

FineWeb-Edu datasets, classifier and ablation model • 5 items • Updated Jun 12 • 7

upvoted a paper 3 months ago

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing

Paper • 2406.08464 • Published Jun 12 • 61

upvoted 2 articles 4 months ago

Article

Uncensor any LLM with abliteration

•

Jun 13

• 312

Article

⚗️ 🔥 Building High-Quality Datasets with distilabel and Prometheus 2

•

Jun 3

• 26

upvoted 2 papers 4 months ago

Zamba: A Compact 7B SSM Hybrid Model

Paper • 2405.16712 • Published May 26 • 20

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Paper • 2405.15319 • Published May 24 • 24

upvoted 2 collections 4 months ago

👿 Daredevil-8B

Collection

Fine-tuned abliterated merge of the best Llama 3 8B model. Highest MMLU score in its category. • 5 items • Updated Aug 16 • 8

🚀GGUF

Collection

Llama.cpp compatible models, can be used on CPUs and GPUs! • 698 items • Updated about 23 hours ago • 30

upvoted 3 papers 5 months ago

Model Merging by Uncertainty-Based Gradient Matching

Paper • 2310.12808 • Published Oct 19, 2023 • 6

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design

Paper • 2401.14112 • Published Jan 25 • 17

WildChat: 1M ChatGPT Interaction Logs in the Wild

Paper • 2405.01470 • Published May 2 • 59

upvoted an article 5 months ago

Article

🧑‍⚖️ "Replacing Judges with Juries" using distilabel

•

May 3

• 17

upvoted 3 papers 5 months ago

Self-Play Preference Optimization for Language Model Alignment

Paper • 2405.00675 • Published May 1 • 22

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

Paper • 2405.01535 • Published May 2 • 114

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Paper • 2404.18796 • Published Apr 29 • 68

upvoted 3 articles 5 months ago

Article

StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation

Apr 29

• 71

Article

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

•

Apr 29

• 28

Article

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

•

Jun 4

• 67

upvoted a paper 5 months ago

Sailor: Open Language Models for South-East Asia

Paper • 2404.03608 • Published Apr 4 • 20

upvoted 5 articles 5 months ago

Article

Merge Large Language Models with mergekit

•

Jan 9

• 67

Article

Create Mixtures of Experts with MergeKit

•

Mar 28

• 10

Article

Fine-tune Llama 3 with ORPO

•

Apr 22

• 221

Article

Welcome Llama 3 - Meta's new open LLM

Apr 18

• 272

Article

Releasing Youtube-Commons: a massive open corpus for conversational and multimodal data

•

Apr 18

• 21

upvoted a collection 5 months ago

fuck quadratic attention

Collection

11 items • Updated Apr 24 • 20

upvoted 2 papers 5 months ago

JetMoE: Reaching Llama2 Performance with 0.1M Dollars

Paper • 2404.07413 • Published Apr 11 • 35

Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences

Paper • 2404.03715 • Published Apr 4 • 59

upvoted 7 papers 6 months ago

ReFT: Representation Finetuning for Language Models

Paper • 2404.03592 • Published Apr 4 • 86

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Paper • 2403.03507 • Published Mar 6 • 182

The Unreasonable Ineffectiveness of the Deeper Layers

Paper • 2403.17887 • Published Mar 26 • 77

ORPO: Monolithic Preference Optimization without Reference Model

Paper • 2403.07691 • Published Mar 12 • 59

Evolutionary Optimization of Model Merging Recipes

Paper • 2403.13187 • Published Mar 19 • 49

Arcee's MergeKit: A Toolkit for Merging Large Language Models

Paper • 2403.13257 • Published Mar 20 • 19

Magicoder: Source Code Is All You Need

Paper • 2312.02120 • Published Dec 4, 2023 • 79

upvoted 3 papers 7 months ago

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

Paper • 2402.19427 • Published Feb 29 • 52

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Paper • 2402.17764 • Published Feb 27 • 590

FuseChat: Knowledge Fusion of Chat Models

Paper • 2402.16107 • Published Feb 25 • 36

upvoted 2 collections 7 months ago

Mamba

Collection

Mamba checkpoints compatible with transformers • 6 items • Updated Feb 19 • 2

Gemma release

Collection

Groups the Gemma models released by the Google team. • 40 items • Updated Jul 31 • 325