metadata
license: apache-2.0
Base checkpoint
augmxnt/shisa-7b-v1
- Mistral-7B base
- Pre-trained on 8B of MADLAD-Ja
- Finetuned on Japanese instructions
- Highest scoring 7B model on conversation benchmark (JA MT-Bench)
Training datasets (total ~7B)
- Aozora Bunko
- Japanese Law Precedent Dataset
- Japanese Wikipedia
- .lg.jp, .go.jp, .ac.jp domain webscrapes from CulturaX (Any documents with same first 25 characters were de-duplicated)
- English Ultrachat200K-gen (So that it doesn't forget English and chatting ability learned in the base checkpoint)