Update README.md

README.md

@@ -4,4 +4,78 @@ datasets:
 - teknium/openhermes
 language:
 - en
----
+tags:
+- llama
+- llama-2
+- instruct
+- finetune 
+- OpenHermes
+---
+
+## llama2-7b-openhermes-15k-mini
+
+- 4-bit qlora fine-tuning of llama-v2-guanaco with openhermes dataset.
+- It is finetuned on the Hermes dataset. The dataset had 15,000 rows.
+
+## Usage:
+
+```
+def text_gen_eval_wrapper(model, tokenizer, prompt, model_id=1, show_metrics=True, temp=0.7, max_length=200):
+    """
+    A wrapper function for inferencing, evaluating, and logging text generation pipeline.
+
+    Parameters:
+        model (str or object): The model name or the initialized text generation model.
+        tokenizer (str or object): The tokenizer name or the initialized tokenizer for the model.
+        prompt (str): The input prompt text for text generation.
+        model_id (int, optional): An identifier for the model. Defaults to 1.
+        show_metrics (bool, optional): Whether to calculate and show evaluation metrics.
+                                       Defaults to True.
+        max_length (int, optional): The maximum length of the generated text sequence.
+                                    Defaults to 200.
+
+    Returns:
+        generated_text (str): The generated text by the model.
+        metrics (dict): Evaluation metrics for the generated text (if show_metrics is True).
+    """
+    # Suppress Hugging Face pipeline logging
+    logging.set_verbosity(logging.CRITICAL)
+
+    # Initialize the pipeline
+    pipe = pipeline(task="text-generation",
+                    model=model,
+                    tokenizer=tokenizer,
+                    max_length=max_length,
+                    do_sample=True,
+                    temperature=temp)
+
+    # Generate text using the pipeline
+    pipe = pipeline(task="text-generation",
+                    model=model,
+                    tokenizer=tokenizer,
+                    max_length=200)
+    result = pipe(f"<s>[INST] {prompt} [/INST]")
+    generated_text = result[0]['generated_text']
+
+    # Find the index of "### Assistant" in the generated text
+    index = generated_text.find("[/INST] ")
+    if index != -1:
+        # Extract the substring after "### Assistant"
+        substring_after_assistant = generated_text[index + len("[/INST] "):].strip()
+    else:
+        # If "### Assistant" is not found, use the entire generated text
+        substring_after_assistant = generated_text.strip()
+
+    if show_metrics:
+        # Calculate evaluation metrics
+        metrics = run_metrics(substring_after_assistant, prompt, model_id)
+
+        return substring_after_assistant, metrics
+    else:
+        return substring_after_assistant
+
+
+prompt = "### Human: Why can camels survive for long without water? ### Assistant:"
+generated_text = text_gen_eval_wrapper(model, tokenizer, prompt, show_metrics=False, max_length=250)
+print(generated_text)
+```