ptrdvn commited on
Commit
aabc37c
1 Parent(s): 9ae5ab6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -22
README.md CHANGED
@@ -3,11 +3,11 @@ license: apache-2.0
3
  ---
4
  # Dataset
5
 
6
- Japanese subset of the mC4 dataset
7
 
8
  # Training
9
 
10
- Trained for 3000 steps on top of the MPT 7b checkpoint mosaicml/mpt-7b
11
 
12
  # How to load
13
 
@@ -17,7 +17,7 @@ Before running this model, please install the following pip package:
17
  pip install einops
18
  ```
19
 
20
- To run this model, you may need to load it in a lower precision in order for it to fit onto your GPU. We found for a T4 GPU, it requires loading the model in 8-bit precision. To load the model in 8-bit or 4-bit, please install the following pip packages:
21
 
22
  ```bash
23
  pip install bitsandbytes accelerate
@@ -61,25 +61,6 @@ model = AutoModelForCausalLM.from_pretrained(
61
 
62
  </details>
63
 
64
- <details>
65
- <summary><b>In 4 bit</b></summary>
66
-
67
-
68
-
69
- ```python
70
- from transformers import AutoModelForCausalLM
71
-
72
- model_name = "lightblue/japanese-mpt-7b"
73
- model = AutoModelForCausalLM.from_pretrained(
74
- model_name,
75
- torch_dtype='auto',
76
- load_in_4bit=True,
77
- trust_remote_code=True
78
- )
79
- ```
80
-
81
- </details>
82
-
83
 
84
  # How to use
85
  ```python
 
3
  ---
4
  # Dataset
5
 
6
+ Japanese subset of the [mC4](https://huggingface.co/datasets/mc4) dataset
7
 
8
  # Training
9
 
10
+ Trained for 3000 steps on top of the MPT 7b checkpoint [mosaicml/mpt-7b](https://huggingface.co/mosaicml/mpt-7b)
11
 
12
  # How to load
13
 
 
17
  pip install einops
18
  ```
19
 
20
+ To run this model, you may need to load it in a lower precision in order for it to fit onto your GPU. We found for a T4 GPU, it requires loading the model in 8-bit precision. To load the model in 8-bit, please install the following pip packages:
21
 
22
  ```bash
23
  pip install bitsandbytes accelerate
 
61
 
62
  </details>
63
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
64
 
65
  # How to use
66
  ```python