RichardErkhov
/

01-ai_-_Yi-1.5-34B-Chat-16K-gguf

GGUF

Model card Files Files and versions Community

01-ai_-_Yi-1.5-34B-Chat-16K-gguf / README.md

RichardErkhov

uploaded readme

0afbfae verified 3 days ago

preview code

raw

history blame contribute delete

No virus

9.88 kB

	Quantization made by Richard Erkhov.

	[Github](https://github.com/RichardErkhov)

	[Discord](https://discord.gg/pvy7H8DZMG)

	[Request more models](https://github.com/RichardErkhov/quant_request)


	Yi-1.5-34B-Chat-16K - GGUF
	- Model creator: https://huggingface.co/01-ai/
	- Original model: https://huggingface.co/01-ai/Yi-1.5-34B-Chat-16K/


	\| Name \| Quant method \| Size \|
	\| ---- \| ---- \| ---- \|
	\| [Yi-1.5-34B-Chat-16K.Q2_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q2_K.gguf) \| Q2_K \| 11.94GB \|
	\| [Yi-1.5-34B-Chat-16K.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ3_XS.gguf) \| IQ3_XS \| 13.26GB \|
	\| [Yi-1.5-34B-Chat-16K.IQ3_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ3_S.gguf) \| IQ3_S \| 13.99GB \|
	\| [Yi-1.5-34B-Chat-16K.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K_S.gguf) \| Q3_K_S \| 13.93GB \|
	\| [Yi-1.5-34B-Chat-16K.IQ3_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ3_M.gguf) \| IQ3_M \| 14.5GB \|
	\| [Yi-1.5-34B-Chat-16K.Q3_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K.gguf) \| Q3_K \| 15.51GB \|
	\| [Yi-1.5-34B-Chat-16K.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K_M.gguf) \| Q3_K_M \| 15.51GB \|
	\| [Yi-1.5-34B-Chat-16K.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K_L.gguf) \| Q3_K_L \| 16.89GB \|
	\| [Yi-1.5-34B-Chat-16K.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ4_XS.gguf) \| IQ4_XS \| 17.36GB \|
	\| [Yi-1.5-34B-Chat-16K.Q4_0.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_0.gguf) \| Q4_0 \| 18.13GB \|
	\| [Yi-1.5-34B-Chat-16K.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ4_NL.gguf) \| IQ4_NL \| 18.3GB \|
	\| [Yi-1.5-34B-Chat-16K.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_K_S.gguf) \| Q4_K_S \| 18.25GB \|
	\| [Yi-1.5-34B-Chat-16K.Q4_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_K.gguf) \| Q4_K \| 19.24GB \|
	\| [Yi-1.5-34B-Chat-16K.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_K_M.gguf) \| Q4_K_M \| 19.24GB \|
	\| [Yi-1.5-34B-Chat-16K.Q4_1.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_1.gguf) \| Q4_1 \| 20.1GB \|
	\| [Yi-1.5-34B-Chat-16K.Q5_0.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_0.gguf) \| Q5_0 \| 22.08GB \|
	\| [Yi-1.5-34B-Chat-16K.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_K_S.gguf) \| Q5_K_S \| 22.08GB \|
	\| [Yi-1.5-34B-Chat-16K.Q5_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_K.gguf) \| Q5_K \| 22.65GB \|
	\| [Yi-1.5-34B-Chat-16K.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_K_M.gguf) \| Q5_K_M \| 22.65GB \|
	\| [Yi-1.5-34B-Chat-16K.Q5_1.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_1.gguf) \| Q5_1 \| 24.05GB \|
	\| [Yi-1.5-34B-Chat-16K.Q6_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q6_K.gguf) \| Q6_K \| 26.28GB \|
	\| [Yi-1.5-34B-Chat-16K.Q8_0.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q8_0.gguf) \| Q8_0 \| 34.03GB \|




	Original model description:
	---
	license: apache-2.0
	---
	<div align="center">

	<picture>
	<img src="https://raw.githubusercontent.com/01-ai/Yi/main/assets/img/Yi_logo_icon_light.svg" width="150px">
	</picture>

	</div>

	<p align="center">
	<a href="https://github.com/01-ai">🐙 GitHub</a> •
	<a href="https://discord.gg/hYUwWddeAu">👾 Discord</a> •
	<a href="https://twitter.com/01ai_yi">🐤 Twitter</a> •
	<a href="https://github.com/01-ai/Yi-1.5/issues/2">💬 WeChat</a>
	<br/>
	<a href="https://arxiv.org/abs/2403.04652">📝 Paper</a> •
	<a href="https://01-ai.github.io/">💪 Tech Blog</a> •
	<a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#faq">🙌 FAQ</a> •
	<a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#learning-hub">📗 Learning Hub</a>
	</p>

	# Intro

	Yi-1.5 is an upgraded version of Yi. It is continuously pre-trained on Yi with a high-quality corpus of 500B tokens and fine-tuned on 3M diverse fine-tuning samples.

	Compared with Yi, Yi-1.5 delivers stronger performance in coding, math, reasoning, and instruction-following capability, while still maintaining excellent capabilities in language understanding, commonsense reasoning, and reading comprehension.

	<div align="center">

	Model \| Context Length \| Pre-trained Tokens
	\| :------------: \| :------------: \| :------------: \|
	\| Yi-1.5 \| 4K, 16K, 32K \| 3.6T

	</div>

	# Models

	- Chat models

	<div align="center">

	\| Name \| Download \|
	\| --------------- \| ------------------------------------------------------------------------------------------------------------------------------------------------------------------- \|
	\| Yi-1.5-34B-Chat \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI)\|
	\| Yi-1.5-34B-Chat-16K \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|
	\| Yi-1.5-9B-Chat \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|
	\| Yi-1.5-9B-Chat-16K \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|
	\| Yi-1.5-6B-Chat \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|

	</div>

	- Base models

	<div align="center">

	\| Name \| Download \|
	\| ---------- \| ------------------------------------------------------------------------------------------------------------------------------------------------------------------- \|
	\| Yi-1.5-34B \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|
	\| Yi-1.5-34B-32K \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|
	\| Yi-1.5-9B \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|
	\| Yi-1.5-9B-32K \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|
	\| Yi-1.5-6B \| • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) \|

	</div>

	# Benchmarks

	- Chat models

	Yi-1.5-34B-Chat is on par with or excels beyond larger models in most benchmarks.

	![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/KcsJ9Oc1VnEmfCDEJc5cd.png)

	Yi-1.5-9B-Chat is the top performer among similarly sized open-source models.

	![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/xf6pLg5jqRCwjlh6m3t6_.png)

	- Base models

	Yi-1.5-34B is on par with or excels beyond larger models in some benchmarks.

	![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/BwU7QM-03dZvZzwdIE1xY.png)

	Yi-1.5-9B is the top performer among similarly sized open-source models.

	![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/y-EYSYPT-3aWLJ0x8R94F.png)

	# Quick Start

	For getting up and running with Yi-1.5 models quickly, see [README](https://github.com/01-ai/Yi-1.5).