Quantization made by Richard Erkhov.

prometheus-2-llama-3-8b - GGUF

Model creator: https://huggingface.co/chargoddard/
Original model: https://huggingface.co/chargoddard/prometheus-2-llama-3-8b/

Name	Quant method	Size
prometheus-2-llama-3-8b.Q2_K.gguf	Q2_K	2.96GB
prometheus-2-llama-3-8b.IQ3_XS.gguf	IQ3_XS	3.28GB
prometheus-2-llama-3-8b.IQ3_S.gguf	IQ3_S	3.43GB
prometheus-2-llama-3-8b.Q3_K_S.gguf	Q3_K_S	3.41GB
prometheus-2-llama-3-8b.IQ3_M.gguf	IQ3_M	3.52GB
prometheus-2-llama-3-8b.Q3_K.gguf	Q3_K	3.74GB
prometheus-2-llama-3-8b.Q3_K_M.gguf	Q3_K_M	3.74GB
prometheus-2-llama-3-8b.Q3_K_L.gguf	Q3_K_L	4.03GB
prometheus-2-llama-3-8b.IQ4_XS.gguf	IQ4_XS	4.18GB
prometheus-2-llama-3-8b.Q4_0.gguf	Q4_0	4.34GB
prometheus-2-llama-3-8b.IQ4_NL.gguf	IQ4_NL	4.38GB
prometheus-2-llama-3-8b.Q4_K_S.gguf	Q4_K_S	4.37GB
prometheus-2-llama-3-8b.Q4_K.gguf	Q4_K	4.58GB
prometheus-2-llama-3-8b.Q4_K_M.gguf	Q4_K_M	4.58GB
prometheus-2-llama-3-8b.Q4_1.gguf	Q4_1	4.78GB
prometheus-2-llama-3-8b.Q5_0.gguf	Q5_0	5.21GB
prometheus-2-llama-3-8b.Q5_K_S.gguf	Q5_K_S	5.21GB
prometheus-2-llama-3-8b.Q5_K.gguf	Q5_K	5.34GB
prometheus-2-llama-3-8b.Q5_K_M.gguf	Q5_K_M	5.34GB
prometheus-2-llama-3-8b.Q5_1.gguf	Q5_1	5.65GB
prometheus-2-llama-3-8b.Q6_K.gguf	Q6_K	6.14GB
prometheus-2-llama-3-8b.Q8_0.gguf	Q8_0	7.95GB

Original model description:

base_model: - chargoddard/prometheus-llama-3-8b-preference - chargoddard/prometheus-llama-3-8b-absolute library_name: transformers tags: - mergekit - merge license: apache-2.0 datasets: - prometheus-eval/Preference-Collection - prometheus-eval/Feedback-Collection language: - en

prometheus-2-llama-3-8b

Replication of prometheus-7b-v2.0 using Llama 3 8B Instruct as a base model.

As in their paper, two different models were trained on their preference and feedback datasets then linearly merged at equal weight.

Training hyperparameters:

1 epoch
Learning rate 1e-5
Effective batch size 128
Cosine annealing
~5% warmup

Uses Llama 3 Instruct prompt format and the same prompts as prometheus-7b-v2.0. See that readme for info.

Citations

@misc{kim2023prometheus,
    title={Prometheus: Inducing Fine-grained Evaluation Capability in Language Models},
    author={Seungone Kim and Jamin Shin and Yejin Cho and Joel Jang and Shayne Longpre and Hwaran Lee and Sangdoo Yun and Seongjin Shin and Sungdong Kim and James Thorne and Minjoon Seo},
    year={2023},
    eprint={2310.08491},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

@misc{kim2024prometheus,
    title={Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models},
    author={Seungone Kim and Juyoung Suk and Shayne Longpre and Bill Yuchen Lin and Jamin Shin and Sean Welleck and Graham Neubig and Moontae Lee and Kyungjae Lee and Minjoon Seo},
    year={2024},
    eprint={2405.01535},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}