inflatebot commited on
Commit
478a093
1 Parent(s): 30f3b67

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -13
README.md CHANGED
@@ -1,28 +1,39 @@
1
  ---
2
  base_model:
3
  - Qwen/Qwen2.5-1.5B-Instruct
4
- library_name: transformers
5
  tags:
6
  - mergekit
7
  - merge
8
-
 
 
 
 
 
 
 
 
9
  ---
10
  # Q25-1.5-VeoLu-R2
11
 
12
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
-
14
- ## Merge Details
15
- ### Merge Method
16
 
17
- This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) as a base.
18
 
19
- ### Models Merged
20
 
21
  The following models were included in the merge:
22
- * /home/asriel/AI/text/models/scribe
23
- * /home/asriel/AI/text/models/alchemist
24
- * /home/asriel/AI/text/models/bard
25
- * /home/asriel/AI/text/models/cartographer
 
 
 
 
 
 
26
 
27
  ### Configuration
28
 
@@ -54,4 +65,4 @@ slices:
54
  weight: 1.0
55
  - layer_range: [0, 28]
56
  model: Qwen/Qwen2.5-1.5B-Instruct
57
- ```
 
1
  ---
2
  base_model:
3
  - Qwen/Qwen2.5-1.5B-Instruct
4
+ library_name: peft
5
  tags:
6
  - mergekit
7
  - merge
8
+ - llama-factory
9
+ - lora
10
+ datasets:
11
+ - allura-org/fujin-cleaned-stage-1
12
+ - Dampfinchen/Creative_Writing_Multiturn
13
+ - ToastyPigeon/SpringDragon
14
+ - allura-org/medquad_sharegpt
15
+ - allura-org/scienceqa_sharegpt
16
+ - Alignment-Lab-AI/orcamath-sharegpt
17
  ---
18
  # Q25-1.5-VeoLu-R2
19
 
20
+ Q25-1.5B-Veo Lu is a tiny General-Purpose Creative model, made up of a merge of bespoke finetunes on Qwen 2.5-1.5B-Instruct.
 
 
 
21
 
22
+ Inspired by the success of [MN-12B-Mag Mell](https://huggingface.co/inflatebot/MN-12B-Mag-Mell-R1) and [MS-Meadowlark-22B](https://huggingface.co/allura-org/MS-Meadowlark-22B), Veo Lu was trained on a healthy, balanced diet of of Internet fiction, roleplaying, adventuring, and reasoning/general knowledge.
23
 
24
+ The components of Veo Lu are:
25
 
26
  The following models were included in the merge:
27
+ * Bard (pretrain, writing): [Fujin (Cleaned/extended Rosier)](https://huggingface.co/allura-org/fujin-cleaned-stage-1)
28
+ * Scribe (pretrain, roleplay): [Creative Writing Multiturn](https://huggingface.co/Dampfinchen/Creative_Writing_Multiturn)
29
+ * Cartographer (pretrain, adventuring): [SpringDragon](https://huggingface.co/ToastyPigeon/SpringDragon)
30
+ * Alchemist (SFT, science/reasoning): [ScienceQA,](https://huggingface.co/allura-org/scienceqa_sharegpt) [MedquadQA,](https://huggingface.co/allura-org/medquad_sharegpt) [Orca Math Word Problems](https://huggingface.co/Alignment-Lab-AI/orcamath-sharegpt)
31
+
32
+ This model is capable of carrying on a scene without going completely off the rails. That being said, it only has 1.5B parameters. So please, for the love of God, *manage your expectations.*
33
+
34
+ Made by inflatebot.
35
+
36
+ Special thanks to our friends at Allura, and especially to Auri, who basically held my hand through the whole process. Her effort and enthusiasm carried this project forward.
37
 
38
  ### Configuration
39
 
 
65
  weight: 1.0
66
  - layer_range: [0, 28]
67
  model: Qwen/Qwen2.5-1.5B-Instruct
68
+ ```