Hugging Face's logo

RichardErkhov
/
01-ai_-_Yi-1.5-34B-Chat-16K-gguf

GGUF
Model card Files Community
01-ai_-_Yi-1.5-34B-Chat-16K-gguf
File size: 9,878 Bytes 
0afbfae
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


Yi-1.5-34B-Chat-16K - GGUF
- Model creator: https://huggingface.co/01-ai/
- Original model: https://huggingface.co/01-ai/Yi-1.5-34B-Chat-16K/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [Yi-1.5-34B-Chat-16K.Q2_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q2_K.gguf) | Q2_K | 11.94GB |
| [Yi-1.5-34B-Chat-16K.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ3_XS.gguf) | IQ3_XS | 13.26GB |
| [Yi-1.5-34B-Chat-16K.IQ3_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ3_S.gguf) | IQ3_S | 13.99GB |
| [Yi-1.5-34B-Chat-16K.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K_S.gguf) | Q3_K_S | 13.93GB |
| [Yi-1.5-34B-Chat-16K.IQ3_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ3_M.gguf) | IQ3_M | 14.5GB |
| [Yi-1.5-34B-Chat-16K.Q3_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K.gguf) | Q3_K | 15.51GB |
| [Yi-1.5-34B-Chat-16K.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K_M.gguf) | Q3_K_M | 15.51GB |
| [Yi-1.5-34B-Chat-16K.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q3_K_L.gguf) | Q3_K_L | 16.89GB |
| [Yi-1.5-34B-Chat-16K.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ4_XS.gguf) | IQ4_XS | 17.36GB |
| [Yi-1.5-34B-Chat-16K.Q4_0.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_0.gguf) | Q4_0 | 18.13GB |
| [Yi-1.5-34B-Chat-16K.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.IQ4_NL.gguf) | IQ4_NL | 18.3GB |
| [Yi-1.5-34B-Chat-16K.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_K_S.gguf) | Q4_K_S | 18.25GB |
| [Yi-1.5-34B-Chat-16K.Q4_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_K.gguf) | Q4_K | 19.24GB |
| [Yi-1.5-34B-Chat-16K.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_K_M.gguf) | Q4_K_M | 19.24GB |
| [Yi-1.5-34B-Chat-16K.Q4_1.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q4_1.gguf) | Q4_1 | 20.1GB |
| [Yi-1.5-34B-Chat-16K.Q5_0.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_0.gguf) | Q5_0 | 22.08GB |
| [Yi-1.5-34B-Chat-16K.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_K_S.gguf) | Q5_K_S | 22.08GB |
| [Yi-1.5-34B-Chat-16K.Q5_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_K.gguf) | Q5_K | 22.65GB |
| [Yi-1.5-34B-Chat-16K.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_K_M.gguf) | Q5_K_M | 22.65GB |
| [Yi-1.5-34B-Chat-16K.Q5_1.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q5_1.gguf) | Q5_1 | 24.05GB |
| [Yi-1.5-34B-Chat-16K.Q6_K.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q6_K.gguf) | Q6_K | 26.28GB |
| [Yi-1.5-34B-Chat-16K.Q8_0.gguf](https://huggingface.co/RichardErkhov/01-ai_-_Yi-1.5-34B-Chat-16K-gguf/blob/main/Yi-1.5-34B-Chat-16K.Q8_0.gguf) | Q8_0 | 34.03GB |




Original model description:
---
license: apache-2.0
---
<div align="center">

<picture> 
  <img src="https://raw.githubusercontent.com/01-ai/Yi/main/assets/img/Yi_logo_icon_light.svg" width="150px">
</picture>

</div>

<p align="center">
  <a href="https://github.com/01-ai">🐙 GitHub</a> •
  <a href="https://discord.gg/hYUwWddeAu">👾 Discord</a> •
  <a href="https://twitter.com/01ai_yi">🐤 Twitter</a> •
  <a href="https://github.com/01-ai/Yi-1.5/issues/2">💬 WeChat</a> 
  <br/>
  <a href="https://arxiv.org/abs/2403.04652">📝 Paper</a> •
  <a href="https://01-ai.github.io/">💪 Tech Blog</a> •
  <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#faq">🙌 FAQ</a> •
  <a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#learning-hub">📗 Learning Hub</a>
</p>

# Intro

Yi-1.5 is an upgraded version of Yi. It is continuously pre-trained on Yi with a high-quality corpus of 500B tokens and fine-tuned on 3M diverse fine-tuning samples. 

Compared with Yi, Yi-1.5 delivers stronger performance in coding, math, reasoning, and instruction-following capability, while still maintaining excellent capabilities in language understanding, commonsense reasoning, and reading comprehension.

<div align="center">
  
Model | Context Length | Pre-trained Tokens
| :------------: | :------------: | :------------: |
| Yi-1.5 | 4K, 16K, 32K | 3.6T

</div>

# Models

- Chat models

  <div align="center">
  
  | Name            | Download                                                                                                                                                            |
  | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | Yi-1.5-34B-Chat | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI)|
  | Yi-1.5-34B-Chat-16K  | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  | Yi-1.5-9B-Chat  | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  | Yi-1.5-9B-Chat-16K  | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  | Yi-1.5-6B-Chat  | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  
  </div>

- Base models

  <div align="center">
  
  | Name       | Download                                                                                                                                                            |
  | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  | Yi-1.5-34B | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  | Yi-1.5-34B-32K | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  | Yi-1.5-9B | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  | Yi-1.5-9B-32K  | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  | Yi-1.5-6B  | • [🤗 Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) • [🤖 ModelScope](https://www.modelscope.cn/organization/01ai) • [🟣 wisemodel](https://wisemodel.cn/organization/01.AI) |
  
  </div>

# Benchmarks

- Chat models

  Yi-1.5-34B-Chat is on par with or excels beyond larger models in most benchmarks.

  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/KcsJ9Oc1VnEmfCDEJc5cd.png)

  Yi-1.5-9B-Chat is the top performer among similarly sized open-source models.

  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/xf6pLg5jqRCwjlh6m3t6_.png)

- Base models

  Yi-1.5-34B is on par with or excels beyond larger models in some benchmarks.

  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/BwU7QM-03dZvZzwdIE1xY.png)

  Yi-1.5-9B is the top performer among similarly sized open-source models.

  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/656d9adce8bf55919aca7c3f/y-EYSYPT-3aWLJ0x8R94F.png)

# Quick Start

For getting up and running with Yi-1.5 models quickly, see [README](https://github.com/01-ai/Yi-1.5).