project-baize commited on
Commit
181da30
1 Parent(s): 4e1fc0c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -0
README.md CHANGED
@@ -1,3 +1,35 @@
1
  ---
2
  license: cc-by-nc-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
  ---
4
+
5
+ ---
6
+ license: cc-by-nc-4.0
7
+ ---
8
+ <p align="center">
9
+ <img width="500px" alt="Project Baize" src="https://user-images.githubusercontent.com/22514219/229195563-0cddfa74-e52f-4413-b4b4-e4ba489c4b3d.png">
10
+ </p>
11
+ <hr>
12
+
13
+ ## What's Baize?
14
+ Baize is an open-source chat model fine-tuned with [LoRA](https://github.com/microsoft/LoRA). It uses 100k dialogs generated by letting ChatGPT chat with itself. We also use Alpaca's data to improve its performance. This repo contains 13B model.
15
+
16
+ ## Why it's called Baize?
17
+ Baize (白泽) is a mythical creature in Chinese folklore, who speaks human languages and knows everything. This is exactly what we expect from a chat model.
18
+
19
+ ## Training Parameters
20
+
21
+ - Base Model: [LLaMA-13B](https://arxiv.org/pdf/2302.13971.pdf)
22
+ - Training Epoch: 1
23
+ - Batch Size: 64
24
+ - Maximum Input Length: 512
25
+ - Learning Rate: 1e-4
26
+ - LoRA Rank: 8
27
+ - Updated Modules: All Linears
28
+
29
+ ## Training Dataset
30
+
31
+ - [Standford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) (51,942)
32
+ - [Quora Dialogs](https://github.com/project-baize/baize) (54,456):
33
+ - [StackOverflow Dialogs](https://github.com/project-baize/baize) (57,046)
34
+
35
+ More details can be found in the Baize [GitHub]((https://github.com/project-baize/baize))