linuxcl4 commited on
Commit
a1ca82e
1 Parent(s): 25f896a

Fix values in quantize_config.json

Browse files

I noticed the values in `quantize_config.json` don't match the settings listed for the main branch in the model table in the README. After pulling the latest changes from main, I get CUDA illegal memory access errors during generation. These changes allow me to run the model again.

The only other model I checked was `guanaco-65B-GPTQ`, but it seems like it might be in a similar situation.

Files changed (1) hide show
  1. quantize_config.json +2 -2
quantize_config.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "bits": 4,
3
- "group_size": 128,
4
  "damp_percent": 0.01,
5
- "desc_act": false,
6
  "sym": true,
7
  "true_sequential": true
8
  }
 
1
  {
2
  "bits": 4,
3
+ "group_size": -1,
4
  "damp_percent": 0.01,
5
+ "desc_act": true,
6
  "sym": true,
7
  "true_sequential": true
8
  }