File size: 925 Bytes
e701494
 
 
 
d3ffb36
 
 
e701494
d3ffb36
 
 
 
 
 
 
 
 
e701494
 
 
 
 
e4f6a3b
 
e701494
 
e4f6a3b
e701494
 
 
 
d3ffb36
 
e701494
575f449
d3ffb36
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
---
language:
- ar
- en
model_name: AceGPT 7B Chat
model_type: AceGPT
quantized_by: FreedomIntelligence
---

<!-- description start -->
## Description

This repo contains GPTQ model files for [AceGPT 2 7B Chat](https://huggingface.co/FreedomIntelligence/AceGPT-7B-chat).


<!-- description end -->

<!-- README_GPTQ.md-use-from-python start -->
## How to use this GPTQ model from Python code

### Install the necessary packages

Requires: Transformers 4.32.0 or later, Optimum 1.12.0 or later, and AutoGPTQ 0.4.2 or later. 


```shell
pip3 install transformers>=4.32.0 optimum>=1.12.0   #See requirements.py for verified versions.
pip3 install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/  # Use cu117 if on CUDA 11.7
```

### You can then generate a simple gradioweb with_quant.py

```bash
python web_quant.py --model-name ${model-path}

```
<!-- README_GPTQ.md-use-from-python end -->