Update README.md
Browse files
README.md
CHANGED
@@ -1,10 +1,13 @@
|
|
1 |
---
|
2 |
datasets:
|
3 |
- tiiuae/falcon-refinedweb
|
|
|
4 |
language:
|
5 |
- en
|
6 |
inference: false
|
7 |
---
|
|
|
|
|
8 |
<div style="width: 100%;">
|
9 |
<img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
10 |
</div>
|
@@ -13,9 +16,10 @@ inference: false
|
|
13 |
<p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
|
14 |
</div>
|
15 |
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
16 |
-
<p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? Patreon
|
17 |
</div>
|
18 |
</div>
|
|
|
19 |
|
20 |
# Falcon-7B-Instruct GPTQ
|
21 |
|
@@ -31,29 +35,27 @@ Join me at: https://discord.gg/UBgz4VXf
|
|
31 |
|
32 |
Please note this is an experimental GPTQ model. Support for it is currently quite limited.
|
33 |
|
34 |
-
It is also expected to be **
|
35 |
|
36 |
-
|
37 |
|
38 |
-
|
39 |
-
2. `pip install einops`
|
40 |
|
41 |
-
|
42 |
|
43 |
-
|
44 |
|
45 |
-
To install AutoGPTQ please follow these instructions:
|
46 |
```
|
47 |
git clone https://github.com/PanQiWei/AutoGPTQ
|
48 |
cd AutoGPTQ
|
49 |
pip install .
|
50 |
```
|
51 |
|
52 |
-
These steps will require that you have the [Nvidia CUDA toolkit](https://developer.nvidia.com/cuda-12-0-1-download-archive) installed.
|
53 |
|
54 |
## text-generation-webui
|
55 |
|
56 |
-
There is
|
57 |
|
58 |
This requires text-generation-webui as of commit 204731952ae59d79ea3805a425c73dd171d943c3.
|
59 |
|
@@ -129,14 +131,28 @@ It was created with groupsize 64 to give higher inference quality, and without `
|
|
129 |
* Does not work with any version of GPTQ-for-LLaMa
|
130 |
* Parameters: Groupsize = 64. No act-order.
|
131 |
|
132 |
-
|
|
|
|
|
|
|
|
|
|
|
133 |
|
134 |
-
|
135 |
|
136 |
-
|
137 |
|
138 |
-
|
|
|
|
|
|
|
|
|
139 |
* Ko-Fi: https://ko-fi.com/TheBlokeAI
|
|
|
|
|
|
|
|
|
|
|
140 |
|
141 |
# ✨ Original model card: Falcon-7B-Instruct
|
142 |
|
|
|
1 |
---
|
2 |
datasets:
|
3 |
- tiiuae/falcon-refinedweb
|
4 |
+
license: apache-2.0
|
5 |
language:
|
6 |
- en
|
7 |
inference: false
|
8 |
---
|
9 |
+
|
10 |
+
<!-- header start -->
|
11 |
<div style="width: 100%;">
|
12 |
<img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
13 |
</div>
|
|
|
16 |
<p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
|
17 |
</div>
|
18 |
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
19 |
+
<p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
|
20 |
</div>
|
21 |
</div>
|
22 |
+
<!-- header end -->
|
23 |
|
24 |
# Falcon-7B-Instruct GPTQ
|
25 |
|
|
|
35 |
|
36 |
Please note this is an experimental GPTQ model. Support for it is currently quite limited.
|
37 |
|
38 |
+
It is also expected to be **SLOW**. This is currently unavoidable, but is being looked at.
|
39 |
|
40 |
+
## AutoGPTQ
|
41 |
|
42 |
+
AutoGPTQ is required: `pip install auto-gptq`
|
|
|
43 |
|
44 |
+
AutoGPTQ provides pre-compiled wheels for Windows and Linux, with CUDA toolkit 11.7 or 11.8.
|
45 |
|
46 |
+
If you are running CUDA toolkit 12.x, you will need to compile your own by following these instructions:
|
47 |
|
|
|
48 |
```
|
49 |
git clone https://github.com/PanQiWei/AutoGPTQ
|
50 |
cd AutoGPTQ
|
51 |
pip install .
|
52 |
```
|
53 |
|
54 |
+
These manual steps will require that you have the [Nvidia CUDA toolkit](https://developer.nvidia.com/cuda-12-0-1-download-archive) installed.
|
55 |
|
56 |
## text-generation-webui
|
57 |
|
58 |
+
There is provisional AutoGPTQ support in text-generation-webui.
|
59 |
|
60 |
This requires text-generation-webui as of commit 204731952ae59d79ea3805a425c73dd171d943c3.
|
61 |
|
|
|
131 |
* Does not work with any version of GPTQ-for-LLaMa
|
132 |
* Parameters: Groupsize = 64. No act-order.
|
133 |
|
134 |
+
<!-- footer start -->
|
135 |
+
## Discord
|
136 |
+
|
137 |
+
For further support, and discussions on these models and AI in general, join us at: [TheBloke AI's Discord server](https://discord.gg/UBgz4VXf)
|
138 |
+
|
139 |
+
## Thanks, and how to contribute.
|
140 |
|
141 |
+
Thanks to the [chirper.ai](https://chirper.ai) team!
|
142 |
|
143 |
+
I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
|
144 |
|
145 |
+
If you're able and willing to contribute, it'd be most gratefully received and will help me to keep providing models, and work on new AI projects.
|
146 |
+
|
147 |
+
Donaters will get priority support on any and all AI/LLM/model questions, plus other benefits.
|
148 |
+
|
149 |
+
* Patreon: https://patreon.com/TheBlokeAI
|
150 |
* Ko-Fi: https://ko-fi.com/TheBlokeAI
|
151 |
+
|
152 |
+
**Patreon special mentions**: Aemon Algiz; Johann-Peter Hartmann; Talal Aujan; Jonathan Leane; Illia Dulskyi; Khalefa Al-Ahmad; senxiiz; Sebastain Graf; Eugene Pentland; Nikolai Manek; Luke Pendergrass.
|
153 |
+
|
154 |
+
Thank you to all my generous patrons and donaters.
|
155 |
+
<!-- footer end -->
|
156 |
|
157 |
# ✨ Original model card: Falcon-7B-Instruct
|
158 |
|