TheBloke commited on
Commit
c07fcbe
1 Parent(s): 02ed56a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -14
README.md CHANGED
@@ -1,10 +1,13 @@
1
  ---
2
  datasets:
3
  - tiiuae/falcon-refinedweb
 
4
  language:
5
  - en
6
  inference: false
7
  ---
 
 
8
  <div style="width: 100%;">
9
  <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
10
  </div>
@@ -13,9 +16,10 @@ inference: false
13
  <p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
14
  </div>
15
  <div style="display: flex; flex-direction: column; align-items: flex-end;">
16
- <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? Patreon coming soon!</a></p>
17
  </div>
18
  </div>
 
19
 
20
  # Falcon-7B-Instruct GPTQ
21
 
@@ -31,29 +35,27 @@ Join me at: https://discord.gg/UBgz4VXf
31
 
32
  Please note this is an experimental GPTQ model. Support for it is currently quite limited.
33
 
34
- It is also expected to be **VERY SLOW**. This is unavoidable at the moment, but is being looked at.
35
 
36
- To use it you will require:
37
 
38
- 1. AutoGPTQ, from the latest `main` branch and compiled with `pip install .`
39
- 2. `pip install einops`
40
 
41
- You can then use it immediately from Python code - see example code below - or from text-generation-webui.
42
 
43
- ## AutoGPTQ
44
 
45
- To install AutoGPTQ please follow these instructions:
46
  ```
47
  git clone https://github.com/PanQiWei/AutoGPTQ
48
  cd AutoGPTQ
49
  pip install .
50
  ```
51
 
52
- These steps will require that you have the [Nvidia CUDA toolkit](https://developer.nvidia.com/cuda-12-0-1-download-archive) installed.
53
 
54
  ## text-generation-webui
55
 
56
- There is also provisional AutoGPTQ support in text-generation-webui.
57
 
58
  This requires text-generation-webui as of commit 204731952ae59d79ea3805a425c73dd171d943c3.
59
 
@@ -129,14 +131,28 @@ It was created with groupsize 64 to give higher inference quality, and without `
129
  * Does not work with any version of GPTQ-for-LLaMa
130
  * Parameters: Groupsize = 64. No act-order.
131
 
132
- ## Want to support my work?
 
 
 
 
 
133
 
134
- I've had a lot of people ask if they can contribute. I love providing models and helping people, but it is starting to rack up pretty big cloud computing bills.
135
 
136
- So if you're able and willing to contribute, it'd be most gratefully received and will help me to keep providing models, and work on various AI proejcts.
137
 
138
- * Patreon: coming soon! (just awaiting approval)
 
 
 
 
139
  * Ko-Fi: https://ko-fi.com/TheBlokeAI
 
 
 
 
 
140
 
141
  # ✨ Original model card: Falcon-7B-Instruct
142
 
 
1
  ---
2
  datasets:
3
  - tiiuae/falcon-refinedweb
4
+ license: apache-2.0
5
  language:
6
  - en
7
  inference: false
8
  ---
9
+
10
+ <!-- header start -->
11
  <div style="width: 100%;">
12
  <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
13
  </div>
 
16
  <p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
17
  </div>
18
  <div style="display: flex; flex-direction: column; align-items: flex-end;">
19
+ <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
20
  </div>
21
  </div>
22
+ <!-- header end -->
23
 
24
  # Falcon-7B-Instruct GPTQ
25
 
 
35
 
36
  Please note this is an experimental GPTQ model. Support for it is currently quite limited.
37
 
38
+ It is also expected to be **SLOW**. This is currently unavoidable, but is being looked at.
39
 
40
+ ## AutoGPTQ
41
 
42
+ AutoGPTQ is required: `pip install auto-gptq`
 
43
 
44
+ AutoGPTQ provides pre-compiled wheels for Windows and Linux, with CUDA toolkit 11.7 or 11.8.
45
 
46
+ If you are running CUDA toolkit 12.x, you will need to compile your own by following these instructions:
47
 
 
48
  ```
49
  git clone https://github.com/PanQiWei/AutoGPTQ
50
  cd AutoGPTQ
51
  pip install .
52
  ```
53
 
54
+ These manual steps will require that you have the [Nvidia CUDA toolkit](https://developer.nvidia.com/cuda-12-0-1-download-archive) installed.
55
 
56
  ## text-generation-webui
57
 
58
+ There is provisional AutoGPTQ support in text-generation-webui.
59
 
60
  This requires text-generation-webui as of commit 204731952ae59d79ea3805a425c73dd171d943c3.
61
 
 
131
  * Does not work with any version of GPTQ-for-LLaMa
132
  * Parameters: Groupsize = 64. No act-order.
133
 
134
+ <!-- footer start -->
135
+ ## Discord
136
+
137
+ For further support, and discussions on these models and AI in general, join us at: [TheBloke AI's Discord server](https://discord.gg/UBgz4VXf)
138
+
139
+ ## Thanks, and how to contribute.
140
 
141
+ Thanks to the [chirper.ai](https://chirper.ai) team!
142
 
143
+ I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
144
 
145
+ If you're able and willing to contribute, it'd be most gratefully received and will help me to keep providing models, and work on new AI projects.
146
+
147
+ Donaters will get priority support on any and all AI/LLM/model questions, plus other benefits.
148
+
149
+ * Patreon: https://patreon.com/TheBlokeAI
150
  * Ko-Fi: https://ko-fi.com/TheBlokeAI
151
+
152
+ **Patreon special mentions**: Aemon Algiz; Johann-Peter Hartmann; Talal Aujan; Jonathan Leane; Illia Dulskyi; Khalefa Al-Ahmad; senxiiz; Sebastain Graf; Eugene Pentland; Nikolai Manek; Luke Pendergrass.
153
+
154
+ Thank you to all my generous patrons and donaters.
155
+ <!-- footer end -->
156
 
157
  # ✨ Original model card: Falcon-7B-Instruct
158