QuantFactory/Llama-3-MAAL-8B-Instruct-v0.1-GGUF

This is quantized version of maum-ai/Llama-3-MAAL-8B-Instruct-v0.1 created using llama.cpp

Original Model Card

Llama-3-MAAL-8B-Instruct-v0.1

we release MAAL, Multilingual Adaptive Augmentation Language-model which comprises a groundbreaking fusion of multilingual capabilities and adaptive augmentation techniques.

• Developed by: maum.ai Brain NLP. Jaeyoon Jung, Jinjoo Lee, Yongjae Lee, Dongjun Lee, Woosung Joo
• Language(s) (NLP): Korean, English (currently, bilingual)

Model Description

Version 0.1 uses cross-lingual training to transfer instruction-following capabilities from English to Korean.

• We Trained this model on an 8 H100-80G for 1 day with cross-lingual training dataset
• we recommend using the fixed system prompt for the model unless you fine-tune it

너는 마음에이아이의 챗봇 MAAL이다. 고객의 질문에 친절하게 답하여라.

sample inference code (GPU)

import transformers
import torch

model_id = "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1"
model = transformers.AutoModelForCausalLM.from_pretrained(model_id).to("cuda")
tokenizer = transformers.AutoTokenizer.from_pretrained(model_id)
streamer = transformers.TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

# we recommend using the fixed prompt for the model unless you fine-tune it
prompt = "너는 마음에이아이의 챗봇 MAAL이다. 고객의 질문에 친절하게 답하여라."
instruction = "사과 한 박스에는 사과가 30개 들어있는데, 처음에는 사과 3박스가 있었고, 내가 사과 5개를 먹었어. 남은 사과는 총 몇개야?"

messages = [
    {"role": "system", "content": f"{prompt}"},
    {"role": "user", "content": f"{instruction}"}
    ]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    return_tensors='pt').to("cuda")
outputs = model.generate(inputs, streamer=streamer, max_new_tokens=1024, pad_token_id=tokenizer.eos_token_id)

Evaluation Results

As the main goal of version 0.1 is to transfer instruction-following capabilities from English to Korean without utilizing continuous pre-training, etc., we select LogicKor as our evaluation method to assess the Korean instruction skills.

We compare our model with a similar parameter model (less than 13B) that has been fine-tuned on the Korean dataset. * denotes our self-report result.

Model	single-turn(↑)	multi-turn(↑)	average(↑)
maum-ai/Llama-3-MAAL-8B-Instruct-v0.1*	5.80	4.66	5.23
maywell/Synatra-kiqu-10.7B	5.71	4.73	5.22
yanolja/EEVE-Korean-Instruct-10.8B-v1.0	5.78	3.92	4.85
nlpai-lab/KULLM3	4.61	4.83	4.72
MLP-KTLim/llama3-Bllossom*	2.11	1.57	1.84

Limitations

Due to this model being trained on a small dataset, it has several limitations.

• Hard to generate diverse Korean texts
• lack of Korean knowledge & Culture (localization)
• Not work with Image inputs and video inputs

Todo

we will solve these limitations one by one by upgrading this model like as...

• Enhance the Korean generation through Vocabulary Expansion & Continuous pre-training. (more Korean corpus!)
• Localize with cultural adaptation method and additional Korean knowledge data. similar idea
• Develop a Vision Language Model that can handle both video and image inputs. similar idea