---
license: cc-by-nc-nd-4.0
language:
- en
base_model:
- google/gemma-2-2b
---
# GemmaLM-for-Cannabis

This repository contains a fine-tuned version of the Gemma 2B model, specifically adapted for cannabis-related queries using Low Rank Adaptation (LoRA).

## Model Details

- **Base Model**: Gemma 2B
- **Fine-tuning Method**: Low Rank Adaptation (LoRA)
- **LoRA Rank**: 4
- **Training Data**: Custom dataset derived from cannabis strain information
- **Task**: Causal Language Modeling for cannabis-related queries

## Fine-tuning Process

The model was fine-tuned using a custom dataset created from cannabis strain information. The dataset includes details about various cannabis strains, their effects, flavors, and descriptions. The fine-tuning process involved:

1. Preprocessing the cannabis dataset into a prompt-response format
2. Implementing LoRA with a rank of 4 to efficiently adapt the model
3. Training for a limited number of epochs with a small subset of data for demonstration purposes

## Usage

This model can be used to generate responses to cannabis-related queries. Example usage:

```python
import keras
import keras_nlp

# Load the model
model = keras.models.load_model("gemma_lm_model.keras")

# Set up the sampler
sampler = keras_nlp.samplers.TopKSampler(k=5, seed=2)
model.compile(sampler=sampler)

# Generate a response
prompt = "Instruction:\nWhat does OG Kush feel like\nResponse:\n"
response = model.generate(prompt, max_length=256)
print(response)
```

## Limitations

- The model was fine-tuned on a limited dataset for demonstration purposes. For production use, consider training on a larger dataset for more epochs.
- The current LoRA rank is set to 4, which may limit the model's adaptability. Experimenting with higher ranks could potentially improve performance.

## Future Improvements

To enhance the model's performance, consider:

1. Increasing the size of the fine-tuning dataset
2. Training for more epochs
3. Experimenting with higher LoRA rank values
4. Adjusting hyperparameters such as learning rate and weight decay

## License

Please refer to the Gemma model's original license for usage terms and conditions.

## Acknowledgements

This project uses the Gemma model developed by Google. We acknowledge the Keras and KerasNLP teams for providing the tools and frameworks used in this project.