Isaak Carter Augustus
commited on
Commit
•
ce3134d
1
Parent(s):
ea3de9b
Update README.md
Browse files
README.md
CHANGED
@@ -19,16 +19,26 @@ This model will be used for the base model in J.O.S.I.E.v4o and furhter trained.
|
|
19 |
|
20 |
- **Developed by:** Isaak-Carter
|
21 |
- **License:** apache-2.0
|
22 |
-
- **Finetuned from model :**
|
23 |
|
24 |
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
25 |
|
26 |
```json
|
27 |
"per_device_train_batch_size": 2,
|
28 |
"gradient_accumulation_steps": 8,
|
29 |
-
"max_steps":
|
30 |
"learning_rate": 2e-4,
|
31 |
"optim": "adamw_8bit",
|
32 |
"weight_decay": 0.01,
|
33 |
"lr_scheduler_type": "cosine"
|
34 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
|
20 |
- **Developed by:** Isaak-Carter
|
21 |
- **License:** apache-2.0
|
22 |
+
- **Finetuned from model :** Isaak-Carter/JOSIEv4o-8b-stage1-beta1
|
23 |
|
24 |
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
25 |
|
26 |
```json
|
27 |
"per_device_train_batch_size": 2,
|
28 |
"gradient_accumulation_steps": 8,
|
29 |
+
"max_steps": 300,
|
30 |
"learning_rate": 2e-4,
|
31 |
"optim": "adamw_8bit",
|
32 |
"weight_decay": 0.01,
|
33 |
"lr_scheduler_type": "cosine"
|
34 |
```
|
35 |
+
|
36 |
+
## trained on this Prompt format
|
37 |
+
|
38 |
+
```txt
|
39 |
+
"""<|begin_of_text|>system
|
40 |
+
You are J.O.S.I.E. which is an acronym for "Just an Outstandingly Smart Intelligent Entity", a private and super-intelligent AI assistant, created by Gökdeniz Gülmez.
|
41 |
+
<|begin_of_text|>main user "Gökdeniz Gülmez"
|
42 |
+
{{ .Prompt }}<|end_of_text|>
|
43 |
+
<|begin_of_text|>josie
|
44 |
+
{{ .Response }}<|end_of_text|>"""
|