RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

LMCocktail-10.7B-v1 - GGUF

• Model creator: https://huggingface.co/Yhyu13/
• Original model: https://huggingface.co/Yhyu13/LMCocktail-10.7B-v1/

Name	Quant method	Size
LMCocktail-10.7B-v1.Q2_K.gguf	Q2_K	3.73GB
LMCocktail-10.7B-v1.IQ3_XS.gguf	IQ3_XS	4.14GB
LMCocktail-10.7B-v1.IQ3_S.gguf	IQ3_S	4.37GB
LMCocktail-10.7B-v1.Q3_K_S.gguf	Q3_K_S	4.34GB
LMCocktail-10.7B-v1.IQ3_M.gguf	IQ3_M	4.51GB
LMCocktail-10.7B-v1.Q3_K.gguf	Q3_K	4.84GB
LMCocktail-10.7B-v1.Q3_K_M.gguf	Q3_K_M	4.84GB
LMCocktail-10.7B-v1.Q3_K_L.gguf	Q3_K_L	5.26GB
LMCocktail-10.7B-v1.IQ4_XS.gguf	IQ4_XS	5.43GB
LMCocktail-10.7B-v1.Q4_0.gguf	Q4_0	5.66GB
LMCocktail-10.7B-v1.IQ4_NL.gguf	IQ4_NL	5.72GB
LMCocktail-10.7B-v1.Q4_K_S.gguf	Q4_K_S	5.7GB
LMCocktail-10.7B-v1.Q4_K.gguf	Q4_K	6.02GB
LMCocktail-10.7B-v1.Q4_K_M.gguf	Q4_K_M	6.02GB
LMCocktail-10.7B-v1.Q4_1.gguf	Q4_1	6.27GB
LMCocktail-10.7B-v1.Q5_0.gguf	Q5_0	6.89GB
LMCocktail-10.7B-v1.Q5_K_S.gguf	Q5_K_S	6.89GB
LMCocktail-10.7B-v1.Q5_K.gguf	Q5_K	7.08GB
LMCocktail-10.7B-v1.Q5_K_M.gguf	Q5_K_M	7.08GB
LMCocktail-10.7B-v1.Q5_1.gguf	Q5_1	7.51GB
LMCocktail-10.7B-v1.Q6_K.gguf	Q6_K	8.2GB
LMCocktail-10.7B-v1.Q8_0.gguf	Q8_0	10.62GB

Original model description:

license: llama2

LM-cocktail 10.7B v1

This is a 50%-50% model of the SOLAR model and meow.

https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0

https://huggingface.co/rishiraj/meow

who rank #1 and #2 among models <13B in the https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard by 2023/12/20.

Alpaca Eval

I am thrilled to announce that ChatGPT has ranked LMCocktail 10.7B as the second best model next to GPT4 on AlpcaEval in my local community run. You can also check the leaderboard at ./alpaca_eval/chatgpt_fn_--SOLAR-10-7B-LMCocktail/

                        win_rate  standard_error  n_total  avg_length
gpt4                       73.79            1.54      805        1365
SOLAR-10.7B-LMCocktail(new)73.45            1.56      804        1203
claude                     70.37            1.60      805        1082
chatgpt                    66.09            1.66      805         811
wizardlm-13b               65.16            1.67      805         985
vicuna-13b                 64.10            1.69      805        1037
guanaco-65b                62.36            1.71      805        1249
oasst-rlhf-llama-33b       62.05            1.71      805        1079
alpaca-farm-ppo-human      60.25            1.72      805         803
falcon-40b-instruct        56.52            1.74      805         662
text_davinci_003           50.00            0.00      805         307
alpaca-7b                  45.22            1.74      805         396
text_davinci_001           28.07            1.56      805         296

Code

The LM-cocktail is novel technique for merging multiple models https://arxiv.org/abs/2311.13534

Code is backed up by this repo https://github.com/FlagOpen/FlagEmbedding.git

Merging scripts available under the ./scripts folder

Result

The SOLAR model is the first model <30B that can answer this question from my test:

What will AI be like in the year 1010 A.D?

without hullicinating into 1010 A.D is a future time (like other llama2 models)

Models greater than that, like Yi-34B could answer this paradoxic question correctly as well, since it is huge enough.

SOLAR 10.7B output

LMCocktail 10.7B output1

LMCocktail 10.7B output2