Hugging Face's logo

gradjitta
/
Poro-34B-AWQ

Text Generation
Transformers
Safetensors
bloom
text-generation-inference
Inference Endpoints
4-bit precision
awq
Model card Files Community
1
gradjitta commited on
Commit
077a648
•
1 Parent(s): 046a2ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -2,7 +2,7 @@
2
  license: apache-2.0
3
  ---
4
 
5
- #### Quantization config
6
  ```
7
 
8
  "zero_point": true,
@@ -12,7 +12,7 @@ license: apache-2.0
12
 
13
  ```
14
 
15
- #### Script to AWQ quantization
16
  ```
17
  from awq import AutoAWQForCausalLM
18
  from transformers import AutoTokenizer
@@ -33,7 +33,7 @@ model.save_quantized(quant_path)
33
  tokenizer.save_pretrained(quant_path)
34
  ```
35
 
36
- #### generate
37
  ```
38
  from awq import AutoAWQForCausalLM
39
  from transformers import AutoTokenizer
 
2
  license: apache-2.0
3
  ---
4
 
5
+ ### Quantization config
6
  ```
7
 
8
  "zero_point": true,
 
12
 
13
  ```
14
 
15
+ ### Script to AWQ quantization
16
  ```
17
  from awq import AutoAWQForCausalLM
18
  from transformers import AutoTokenizer
 
33
  tokenizer.save_pretrained(quant_path)
34
  ```
35
 
36
+ ### Generate
37
  ```
38
  from awq import AutoAWQForCausalLM
39
  from transformers import AutoTokenizer