Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,7 @@ tags:
|
|
| 5 |
- trl
|
| 6 |
- sft
|
| 7 |
- generated_from_trainer
|
|
|
|
| 8 |
base_model: mistralai/Mixtral-8x7B-v0.1
|
| 9 |
model-index:
|
| 10 |
- name: Mixtral_Rio_oasst2_v1
|
|
@@ -27,7 +28,16 @@ It achieves the following results on the evaluation set:
|
|
| 27 |
|
| 28 |
## Model description
|
| 29 |
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
## Intended uses & limitations
|
| 33 |
|
|
|
|
| 5 |
- trl
|
| 6 |
- sft
|
| 7 |
- generated_from_trainer
|
| 8 |
+
- text-generation
|
| 9 |
base_model: mistralai/Mixtral-8x7B-v0.1
|
| 10 |
model-index:
|
| 11 |
- name: Mixtral_Rio_oasst2_v1
|
|
|
|
| 28 |
|
| 29 |
## Model description
|
| 30 |
|
| 31 |
+
This is a LoRA trained on OpenAssistant data.
|
| 32 |
+
The settings for the base model should be:
|
| 33 |
+
Model loader: Transformers
|
| 34 |
+
Compute_dtype: bfloat16
|
| 35 |
+
quant_type: nf4
|
| 36 |
+
cpu: enabled
|
| 37 |
+
load-in-4bit: enabled
|
| 38 |
+
use_double_quant: enabled
|
| 39 |
+
set GPU memory as high as possible unless running locally to give some space for your desktop environment
|
| 40 |
+
tweak CPU usage until it loads successfully
|
| 41 |
|
| 42 |
## Intended uses & limitations
|
| 43 |
|