license: apache-2.0
datasets:
- adamo1139/4chan_archive_ShareGPT_only5
- adamo1139/HESOYAM_v0.4
- adamo1139/uninstruct-v1-experimental-chatml
language:
- en
base_model:
- h2oai/h2o-danube3-4b-base
pipeline_tag: text-generation
Model Details
I finetuned Danube3 4B Base on adamo1139/uninstruct-v1-experimental-chatml dataset with the goal being making AI assistant slop less likely.
Then I did finetuning on adamo1139/4chan_archive_ShareGPT_only5 which is a filtered collection of 4chan threads from various boards for 1 epoch to introduce 4chan-specific slang.
Then I did finetuning on adamo1139/HESOYAM_v0.4 for 3 epochs to improve 1-on-1 chat capabilities.
This is a resulting model.
Prompt format
Use ChatML prompt format.
System message should be in the format as below:
A chat on 4chan board /3/
A chat on 4chan board /g/
A chat on 4chan board /x/
A chat on 4chan board /pol/
Evaluation
I am still vibe-checking the model but initial results are good. I might have put in a bit too much reddit style from HESOYAM, not sure.
GGUF Quants
Quants are available here: adamo1139/danube3-4b-4chan-hesoyam-2510-gguf
Training details
Training details and LoRA adapters can be made available if you request it. I am just not sure if anyone is interested in them, so I am not uploading those artifacts yet.
Future plans
A bit of experimentation on the 4B/500M Danube3 models and then I want to improve my current Yi 34B 200K HESOYAM 0208 model using 4chan archive dataset.