|
--- |
|
license: cc-by-nc-4.0 |
|
language: |
|
- en |
|
widget: |
|
- text: | |
|
Assistant: Howdy! Let's talk cowboy stuff. |
|
User: What color is a typical cowboy hat? |
|
Assistant: |
|
example_title: "Color of a Typical Cowboy Hat" |
|
- text: | |
|
Assistant: Yeehaw! Ready for cowboy queries. |
|
User: What's the name of a famous cowboy's horse? |
|
Assistant: |
|
example_title: "Famous Cowboy Horses" |
|
- text: | |
|
Assistant: Howdy, partner! Ask me about cowboy life. |
|
User: Do cowboys wear spurs all the time? |
|
Assistant: |
|
example_title: "Cowboys and Spurs Usage" |
|
- text: | |
|
Assistant: Howdy! Let's rustle up some cowboy knowledge! |
|
User: What's a common material for cowboy boots? |
|
Assistant: |
|
example_title: "Materials for Cowboy Boots" |
|
- text: | |
|
Assistant: Yippee Ki‐Yay! Saddle up for some cowboy facts! |
|
User: Is a lasso used for catching cattle? |
|
Assistant: |
|
example_title: "Use of a Lasso" |
|
--- |
|
 |
|
# Tiny-Cowboy-1.1b-v0.1 |
|
|
|
Tiny-Cowboy-1.1b-v0.1 is a specialized language model designed for generating cowboy-themed content. Developed by phanerozoic, this model is fine-tuned from TinyLlamaTinyLlama-1.1B-Chat-v1.0, optimized for environments with limited computing resources. |
|
|
|
### Performance |
|
The model excels in generating engaging cowboy narratives and demonstrates a strong grasp of cowboy culture and lifestyle. However, it is less effective in general language tasks, especially in scientific and technical domains. |
|
|
|
### Direct Use |
|
Ideal for thematic language generation, particularly in applications where cowboy culture and storytelling are central. Less suited for general-purpose use or scenarios requiring detailed, accurate scientific explanations. |
|
|
|
### Context Setting and Interaction Guidelines |
|
Tiny-Cowboy-1.1b-v0.1, being a narrowly focused and somewhat limited-performance model, benefits from an initial context-setting message. This setup involves a predefined assistant message that establishes its cowboy identity at the start of each interaction. This strategy is crucial for priming the model to maintain its cowboy theme throughout the conversation. It's important to note that the model has been fine-tuned for a cowboy style of speaking, so explicit instructions on how to respond in a cowboy manner are unnecessary. |
|
|
|
#### Initial Context Setting: |
|
- text: | |
|
Assistant: Howdy! I'm your cowboy assistant, ready to talk all things Wild West. What cowboy queries can I lasso for you today? |
|
example_title: "Initiating Cowboy Themed Conversation" |
|
|
|
- text: | |
|
Assistant: Yeehaw! Let's dive into the cowboy world. Ask me anything about cowboys, ranches, or the Wild West! |
|
example_title: "Engaging in Cowboy Themed Dialogue" |
|
|
|
The introduction by the assistant sets the thematic tone, guiding the user to interact within the cowboy context. |
|
|
|
### Training Data |
|
Incorporates a dataset focused on cowboy and Wild West themes, derived from the foundational TinyLlama-1.1B model. |
|
|
|
### Custom Stopping Strings |
|
Custom stopping strings were used to refine output quality: |
|
|
|
- "}," |
|
- "User:" |
|
- "You:" |
|
- "\nUser" |
|
- "\nUser:" |
|
- "me:" |
|
- "user" |
|
- "\n" |
|
|
|
### Training Hyperparameters and Fine-Tuning Details |
|
- **Base Model Name**: TinyLlamaTinyLlama-1.1B-Chat-v1.0 |
|
- **Base Model Class**: LlamaForCausalLM |
|
- **Projections**: gate, down, up, q, k, v, o |
|
- **LoRA Rank**: 16 |
|
- **LoRA Alpha**: 32 |
|
- **True Batch Size**: 4 |
|
- **Gradient Accumulation Steps**: 1 |
|
- **Epochs**: 1 |
|
- **Learning Rate**: 3e-4 |
|
- **LR Scheduler**: Linear |
|
- **LLaMA Target Projections**: All targets modified |
|
- **Loss**: 2.096 |
|
- **Stop Step**: 42 |
|
|
|
### Limitations |
|
While adept at cowboy-themed content, Tiny-Cowboy-v0.1 struggles with topics outside its specialty, particularly in scientific and technical areas. The model tends to incorporate cowboy elements into responses, regardless of the question's relevance. |
|
|
|
### Compute Infrastructure |
|
Efficiently trained, demonstrating the feasibility of specialized model training in resource-constrained environments. |
|
|
|
### Results |
|
Successfully generates cowboy-themed responses, maintaining thematic consistency. However, it shows limitations in handling more complex, non-cowboy-related queries. |
|
|
|
### Summary |
|
Tiny-Cowboy-1.1b-v0.1 is a significant development in thematic, lightweight language models, ideal for cowboy-themed storytelling and educational purposes. Its specialization, however, limits its applicability in broader contexts, particularly where accurate, technical knowledge is required. |
|
|
|
### Acknowledgments |
|
Special thanks to the TinyLlama-1.1B team, whose foundational work was instrumental in the development of Tiny-Cowboy-v0.1. |
|
|
