aznatkoiny commited on
Commit
bcb165b
1 Parent(s): d71c811

Create README.md

Browse files

# GemmaLM-for-Cannabis

This repository contains a fine-tuned version of the Gemma 2B model, specifically adapted for cannabis-related queries using Low Rank Adaptation (LoRA).

## Model Details

- **Base Model**: Gemma 2B
- **Fine-tuning Method**: Low Rank Adaptation (LoRA)
- **LoRA Rank**: 4
- **Training Data**: Custom dataset derived from cannabis strain information
- **Task**: Causal Language Modeling for cannabis-related queries

## Fine-tuning Process

The model was fine-tuned using a custom dataset created from cannabis strain information. The dataset includes details about various cannabis strains, their effects, flavors, and descriptions. The fine-tuning process involved:

1. Preprocessing the cannabis dataset into a prompt-response format
2. Implementing LoRA with a rank of 4 to efficiently adapt the model
3. Training for a limited number of epochs with a small subset of data for demonstration purposes

## Usage

This model can be used to generate responses to cannabis-related queries. Example usage:

```python
import keras
import keras_nlp

# Load the model
model = keras.models.load_model("gemma_lm_model.keras")

# Set up the sampler
sampler = keras_nlp.samplers.TopKSampler(k=5, seed=2)
model.compile(sampler=sampler)

# Generate a response
prompt = "Instruction:\nWhat does OG Kush feel like\nResponse:\n"
response = model.generate(prompt, max_length=256)
print(response)
```

## Limitations

- The model was fine-tuned on a limited dataset for demonstration purposes. For production use, consider training on a larger dataset for more epochs.
- The current LoRA rank is set to 4, which may limit the model's adaptability. Experimenting with higher ranks could potentially improve performance.

## Future Improvements

To enhance the model's performance, consider:

1. Increasing the size of the fine-tuning dataset
2. Training for more epochs
3. Experimenting with higher LoRA rank values
4. Adjusting hyperparameters such as learning rate and weight decay

## License

Please refer to the Gemma model's original license for usage terms and conditions.

## Acknowledgements

This project uses the Gemma model developed by Google. We acknowledge the Keras and KerasNLP teams for providing the tools and frameworks used in this project.

Files changed (1) hide show
  1. README.md +7 -0
README.md ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-nd-4.0
3
+ language:
4
+ - en
5
+ base_model:
6
+ - google/gemma-2-2b
7
+ ---