Update README.md

README.md

@@ -9,6 +9,7 @@ pipeline_tag: text-generation
 tags:
 - llama
 - alpaca
+- Transformers
 ---
 
 # open_llama_7b_alpaca_clean_dutch_qlora
@@ -19,6 +20,34 @@ This adapter model is a fine-tuned version of [openlm-research/open_llama_7b](ht
 
 See [openlm-research/open_llama_7b](https://huggingface.co/openlm-research/open_llama_7b) for all information about the base model.
 
+## Model usage
+
+A basic example of how to use the finetuned model.
+
+```
+import torch
+from peft import PeftModel, PeftConfig
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "robinsmits/open_llama_7b_alpaca_clean_dutch_qlora"
+
+tokenizer =  AutoTokenizer.from_pretrained(model_name, use_fast = False, add_eos_token = True)
+
+config = PeftConfig.from_pretrained(model_name)
+
+model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, load_in_8bit = True, device_map = "auto")
+model = PeftModel.from_pretrained(model, model_name)
+
+prompt = "### Instructie:\nWat zijn de drie belangrijkste softwareonderdelen die worden gebruikt bij webontwikkeling?\n\n### Antwoord:\n" 
+
+inputs = tokenizer(prompt, return_tensors = "pt", truncation = True).input_ids.cuda()
+sample = model.generate(input_ids = inputs, max_new_tokens = 512, num_beams = 2, early_stopping = True, eos_token_id = tokenizer.eos_token_id)
+output = tokenizer.decode(sample[0], skip_special_tokens = True)
+
+print(output.split(prompt)[1])
+```
+For more extensive usage and a lot of generated samples (both good and bad samples) see the following [Inference Notebook](https://github.com/RobinSmits/Dutch-LLMs/blob/main/Open_Llama_7B_Alpaca_Clean_Dutch_Inference.ipynb)
+
 ## Intended uses & limitations
 
 The open_llama_7b model was primarily trained on the English language. Part of the dataset was a Wikipedia dump containing pages in 20 languages.