TheBloke commited on
Commit
123edfd
1 Parent(s): 6b2ed28

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -40,8 +40,8 @@ None
40
 
41
  ## Repositories available
42
 
43
- * [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/StableBeluga2-GPTQ)
44
- * [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/StableBeluga2-GGML)
45
  * [Stability AI's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/stabilityai/StableBeluga2)
46
 
47
  ## Prompt template: Orca-Hashes
@@ -79,7 +79,7 @@ Each separate quant is in a different branch. See below for instructions on fet
79
  - In text-generation-webui, you can add `:branch` to the end of the download name, eg `TheBloke/StableBeluga2-GPTQ:gptq-4bit-32g-actorder_True`
80
  - With Git, you can clone a branch with:
81
  ```
82
- git clone --branch gptq-4bit-32g-actorder_True https://huggingface.co/TheBloke/StableBeluga2-GPTQ`
83
  ```
84
  - In Python Transformers code, the branch is the `revision` parameter; see below.
85
 
@@ -90,13 +90,13 @@ Please make sure you're using the latest version of [text-generation-webui](http
90
  It is strongly recommended to use the text-generation-webui one-click-installers unless you know how to make a manual install.
91
 
92
  1. Click the **Model tab**.
93
- 2. Under **Download custom model or LoRA**, enter `TheBloke/StableBeluga2-GPTQ`.
94
- - To download from a specific branch, enter for example `TheBloke/StableBeluga2-GPTQ:gptq-4bit-32g-actorder_True`
95
  - see Provided Files above for the list of branches for each option.
96
  3. Click **Download**.
97
  4. The model will start downloading. Once it's finished it will say "Done"
98
  5. In the top left, click the refresh icon next to **Model**.
99
- 6. In the **Model** dropdown, choose the model you just downloaded: `StableBeluga2-GPTQ`
100
  7. The model will automatically load, and is now ready for use!
101
  8. If you want any custom settings, set them and then click **Save settings for this model** followed by **Reload the Model** in the top right.
102
  * Note that you do not need to set GPTQ parameters any more. These are set automatically from the file `quantize_config.json`.
@@ -114,7 +114,7 @@ Then try the following example code:
114
  from transformers import AutoTokenizer, pipeline, logging
115
  from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
116
 
117
- model_name_or_path = "TheBloke/StableBeluga2-GPTQ"
118
  model_basename = "gptq_model-4bit--1g"
119
 
120
  use_triton = False
 
40
 
41
  ## Repositories available
42
 
43
+ * [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/StableBeluga2-70B-GPTQ)
44
+ * [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/StableBeluga2-70B-GGML)
45
  * [Stability AI's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/stabilityai/StableBeluga2)
46
 
47
  ## Prompt template: Orca-Hashes
 
79
  - In text-generation-webui, you can add `:branch` to the end of the download name, eg `TheBloke/StableBeluga2-GPTQ:gptq-4bit-32g-actorder_True`
80
  - With Git, you can clone a branch with:
81
  ```
82
+ git clone --branch gptq-4bit-32g-actorder_True --single-branch https://huggingface.co/TheBloke/StableBeluga2-GPTQ
83
  ```
84
  - In Python Transformers code, the branch is the `revision` parameter; see below.
85
 
 
90
  It is strongly recommended to use the text-generation-webui one-click-installers unless you know how to make a manual install.
91
 
92
  1. Click the **Model tab**.
93
+ 2. Under **Download custom model or LoRA**, enter `TheBloke/StableBeluga2-70B-GPTQ`.
94
+ - To download from a specific branch, enter for example `TheBloke/StableBeluga2-70B-GPTQ:gptq-4bit-32g-actorder_True`
95
  - see Provided Files above for the list of branches for each option.
96
  3. Click **Download**.
97
  4. The model will start downloading. Once it's finished it will say "Done"
98
  5. In the top left, click the refresh icon next to **Model**.
99
+ 6. In the **Model** dropdown, choose the model you just downloaded: `StableBeluga2-70B-GPTQ`
100
  7. The model will automatically load, and is now ready for use!
101
  8. If you want any custom settings, set them and then click **Save settings for this model** followed by **Reload the Model** in the top right.
102
  * Note that you do not need to set GPTQ parameters any more. These are set automatically from the file `quantize_config.json`.
 
114
  from transformers import AutoTokenizer, pipeline, logging
115
  from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
116
 
117
+ model_name_or_path = "TheBloke/StableBeluga2-70B-GPTQ"
118
  model_basename = "gptq_model-4bit--1g"
119
 
120
  use_triton = False