zake7749/Llama-3.2-1B-it-chinese-kyara

Kyara: Knowledge Yielding Adaptive Retrieval Augmentation for LLM Fine-tuning

🤗 Hugging Face  ｜ 🚀Github  ｜  📑 Paper  ｜  📖 English  ｜  📖 Chinese  ｜  💻 Kaggle Notebook

Kyara (Knowledge Yielding Adaptive Retrieval Augmentation) is an experimental project aimed at improving language models through knowledge retrieval processes. The project seeks to enhance the model’s ability to adapt knowledge and improve language comprehension, particularly in underrepresented languages like Traditional Chinese. Given the relatively scarce availability of Traditional Chinese data compared to the vast corpus of English data used for model training, Kyara addresses this gap by expanding the limited corpus for this language.

This is a preview model, with the stable version set to be released soon.

Benchmark

All evaluations are conducted in a zero-shot setting.

Metric	Kyara-1b-it	Llama3.2-1b-it
TMMLUPlus	31.92	30.48
- STEM	32.56	29.74
- Humanities	30.60	29.89
- Other	31.08	30.32
- Social-Science	33.42	31.98
MMLU-Redux	41.40	19.62⁺
GSM8K	31.31	31.61
MATH-L5	5.55	2.91
CRUX	14	11
AlpacaEval	10.79	7.39

⁺: Llama3.2-1b-it appears to have failed to follow the output schema of ZeroEval on MMLU, with 45.28% of examples lacking answers, which has resulted in a lower MMLU score.