Edit model card

LoRA adapter for kaitchup/Maixtchup-4x7b briefly fine-tuned on UltraChat.

To load and use this adapter:

model_name = "kaitchup/Maixtchup-4x7b"
#Tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
compute_dtype = getattr(torch, "float16")
bnb_config = BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=compute_dtype,
        bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
          model_name, quantization_config=bnb_config, device_map="auto", attn_implementation="flash_attention_2",
)

model.config.use_cache = True

model = PeftModel.from_pretrained(model, "kaitchup/Maixtchup-4x7b-QLoRA-SFT-UltraChat")

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 63.11
AI2 Reasoning Challenge (25-Shot) 60.92
HellaSwag (10-Shot) 83.23
MMLU (5-Shot) 60.78
TruthfulQA (0-shot) 53.33
Winogrande (5-shot) 77.19
GSM8k (5-shot) 43.21
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Dataset used to train kaitchup/Maixtchup-4x7b-QLoRA-SFT-UltraChat

Evaluation results