Creative Writing Models
Collection
Trained using the "Mutiplicative-LoRA" method on the `down_proj` matrices only.
•
5 items
•
Updated
•
1
An experimental model, fine-tuned using the "multiplicative-LoRA" method on c4ai-command-r-v01.
This model is nearly identical to creative-writer-v0.1-alfa-35b, with one key difference:
- Scaled the pre-softmax logits by
1.1during training (and then reset after training) to encourage more diverse/creative text generation (ie: increased single-token Entropy).
NOTE: For the command-r models, we can use the logit_scale parameter to do this scaling:
"logit_scale": 0.06875,
Please refer to creative-writer-v0.1-alfa-35b for full details on how to use this model.
- Downloads last month
- 38
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.