README.md

metadata

license: other
library_name: transformers
pipeline_tag: text-generation
datasets:
  - RyokoAI/ShareGPT52K
  - Hello-SimpleAI/HC3
tags:
  - koala
  - ShareGPT
  - llama
  - gptq

Koala: A Dialogue Model for Academic Research

This repo contains the weights of the Koala 13B model produced at Berkeley. It is the result of combining the diffs from https://huggingface.co/young-geng/koala with the original Llama 13B model.

This version has then been converted to HF format.

My Koala repos

I have the following Koala model repositories available:

13B models:

• Unquantized 13B model in HF format
• GPTQ quantized 4bit 13B model in pt and safetensors formats
• GPTQ quantized 4bit 13B model in GGML format for llama.cpp

7B models:

• Unquantized 7B model in HF format
• Unquantized 7B model in GGML format for llama.cpp
• GPTQ quantized 4bit 7B model in pt and safetensors formats
• GPTQ quantized 4bit 7B model in GGML format for llama.cpp

How the Koala delta weights were merged

The Koala delta weights were merged using the following commands:

git clone https://github.com/young-geng/EasyLM

git clone https://huggingface.co/TheBloke/llama-13b

mkdir koala_diffs && cd koala_diffs && wget https://huggingface.co/young-geng/koala/resolve/main/koala_13b_diff_v2

cd EasyLM

PYTHON_PATH="${PWD}:$PYTHONPATH" python \
-m EasyLM.models.llama.convert_torch_to_easylm \
--checkpoint_dir=/content/llama-13b \
--output_file=/content/llama-13b-LM \
--streaming=True

PYTHON_PATH="${PWD}:$PYTHONPATH" python \
-m EasyLM.scripts.diff_checkpoint --recover_diff=True \
--load_base_checkpoint='params::/content/llama-13b-LM' \
--load_target_checkpoint='params::/content/koala_diffs/koala_13b_diff_v2' \
--output_file=/content/koala_13b.diff.weights \
--streaming=True

PYTHON_PATH="${PWD}:$PYTHONPATH" python \
-m EasyLM.models.llama.convert_easylm_to_hf --model_size=13b \
--output_dir=/content/koala-13B-HF \
--load_checkpoint='params::/content/koala_13b.diff.weights' \
--tokenizer_path=/content/llama-13b/tokenizer.model

Check out the following links to learn more about the Berkeley Koala model.

• Blog post
• Online demo
• EasyLM: training and serving framework on GitHub
• Documentation for running Koala locally

License

The model weights are intended for academic research only, subject to the model License of LLaMA, Terms of Use of the data generated by OpenAI, and Privacy Practices of ShareGPT. Any other usage of the model weights, including but not limited to commercial usage, is strictly prohibited.