README.md

---
license: mit
datasets:
- DarwinAnim8or/grug
language:
- en
pipeline_tag: text-generation
tags:
- grug
- caveman
- fun
widget:
- text: "what does grug think of football?"
  example_title: "football"
- text: "what does grug like to do?"
  example_title: "hobbies"
---

# GPT-Grug-355m
A finetuned version of [GPT2-Medium](https://huggingface.co/gpt2-medium) on the 'grug' dataset.
A demo is available [here](https://huggingface.co/spaces/DarwinAnim8or/grug-chat)

If you're interested, there's a smaller model available here: [GPT-Grug-125m](https://huggingface.co/DarwinAnim8or/gpt-grug-125m)
Do note however that it is very limited by comparison.

# Training Procedure
This was trained on the 'grug' dataset, using the "HappyTransformers" library on Google Colab.
This model was trained for 4 epochs with learning rate 1e-2.
The notebook used to train has been included in this repo.

# Biases & Limitations
This likely contains the same biases and limitations as the original GPT2 that it is based on, and additionally heavy biases from the grug datasets.

# Intended Use
This model is meant for fun, please do not take anything this caveman says seriously.

# Sample Use
```python
#Import model:
from happytransformer import HappyGeneration
happy_gen = HappyGeneration("GPT2", "DarwinAnim8or/gpt-grug-355m")

#Set generation settings:
from happytransformer import GENSettings
args_top_k = GENSettings(no_repeat_ngram_size=2, do_sample=True,top_k=50, temperature=0.7, max_length=50, early_stopping=False)

#Generate a response:
result = happy_gen.generate_text("""Person: "Hello grug"
Grug: "hello person"
###
Person: "how are you grug"
Grug: "grug doing ok. grug find many berry. good for tribe."
###
Person: "what does grug think of new spear weapon?"
Grug: "grug no like new spear weapon. grug stick bigger. spear too small, break easy"
###
Person: "what does grug think of football?"
Grug: \"""", args=args_top_k)

print(result)
print(result.text)
```