Information
Uses ChatML, but Alpaca seems to work fine.
This is meant to be a smart RP model.
I did some unholy things to make the model load. (I removed all configs and replaced them with the ones from Hercules)
Important: When using this model it is necessary to include example messages, or it may sound bland. With examples it does pretty well.
Reasoning behind the selected values
To my understanding layers in the middle have a lower effect than later layers on the final output.
So keeping this in mind, I only increase the % of the smart model, at the middle. As to increase its reasoning abilities, but keep it uncensored.
Helen-v1
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the SLERP merge method.
Models Merged
The following models were included in the merge:
- l3utterfly/mistral-7b-v0.2-layla-v4
- Locutusque/Hercules-4.0-Mistral-v0.2-7B
- Weyaxi/Einstein-v5-v0.2-7B
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: l3utterfly/mistral-7b-v0.2-layla-v4
layer_range: [0, 32]
- model: Mergekit/Hercules-Einstein_low
layer_range: [0, 32]
merge_method: slerp
base_model: l3utterfly/mistral-7b-v0.2-layla-v4
parameters:
t:
- filter: self_attn
value: [0.15, 0.20, 0.35, 0.10, 0.15]
- filter: mlp
value: [0.15, 0.20, 0.35, 0.10, 0.15]
- value: 0.15 # fallback for rest of tensors
dtype: float16
- Downloads last month
- 7