Update README.md

README.md

@@ -20,62 +20,35 @@ tags:
 ## Usage:
 
 ```
-def text_gen_eval_wrapper(model, tokenizer, prompt, model_id=1, show_metrics=True, temp=0.7, max_length=200):
-    """
-    A wrapper function for inferencing, evaluating, and logging text generation pipeline.
-
-    Parameters:
-        model (str or object): The model name or the initialized text generation model.
-        tokenizer (str or object): The tokenizer name or the initialized tokenizer for the model.
-        prompt (str): The input prompt text for text generation.
-        model_id (int, optional): An identifier for the model. Defaults to 1.
-        show_metrics (bool, optional): Whether to calculate and show evaluation metrics.
-                                       Defaults to True.
-        max_length (int, optional): The maximum length of the generated text sequence.
-                                    Defaults to 200.
-
-    Returns:
-        generated_text (str): The generated text by the model.
-        metrics (dict): Evaluation metrics for the generated text (if show_metrics is True).
-    """
-    # Suppress Hugging Face pipeline logging
-    logging.set_verbosity(logging.CRITICAL)
-
-    # Initialize the pipeline
-    pipe = pipeline(task="text-generation",
-                    model=model,
-                    tokenizer=tokenizer,
-                    max_length=max_length,
-                    do_sample=True,
-                    temperature=temp)
-
-    # Generate text using the pipeline
-    pipe = pipeline(task="text-generation",
-                    model=model,
-                    tokenizer=tokenizer,
-                    max_length=200)
-    result = pipe(f"<s>[INST] {prompt} [/INST]")
-    generated_text = result[0]['generated_text']
-
-    # Find the index of "### Assistant" in the generated text
-    index = generated_text.find("[/INST] ")
-    if index != -1:
-        # Extract the substring after "### Assistant"
-        substring_after_assistant = generated_text[index + len("[/INST] "):].strip()
-    else:
-        # If "### Assistant" is not found, use the entire generated text
-        substring_after_assistant = generated_text.strip()
-
-    if show_metrics:
-        # Calculate evaluation metrics
-        metrics = run_metrics(substring_after_assistant, prompt, model_id)
-
-        return substring_after_assistant, metrics
-    else:
-        return substring_after_assistant
+from transformers import AutoTokenizer
+import transformers
+import torch
+
+model = "aloobun/llama2-7b-openhermes-15k-mini"
+prompt = "What are large language models?"
+
+tokenizer = AutoTokenizer.from_pretrained(model)
+pipeline = transformers.pipeline(
+    "text-generation",
+    model=model,
+    torch_dtype=torch.float16,
+    device_map="auto",
+)
+
+sequences = pipeline(
+    f'[INST] {prompt} [/INST]',
+    do_sample=True,
+    top_k=10,
+    num_return_sequences=1,
+    eos_token_id=tokenizer.eos_token_id,
+    max_length=200,
+)
+for seq in sequences:
+    print(f"Result: {seq['generated_text']}")
+```
 
+### Output:
 
-prompt = "### Human: Why can camels survive for long without water? ### Assistant:"
-generated_text = text_gen_eval_wrapper(model, tokenizer, prompt, show_metrics=False, max_length=250)
-print(generated_text)
+```
+Result: [INST] What are large language models? [/INST] Large language models are artificial intelligence systems that can be trained on vast amounts of text to generate human-like language. Libraries of natural language processing (NLP) algorithms like BERT and GPT have allowed these systems to learn and improve their capacity for language understanding and generation. These language models have found applications in natural language translation, text summarization, chatbots, and even creative writing. They can help in tasks like predicting the next word in a sentence or even generating a whole text based on a given topic or prompt. Large language models have the potential to revolutionize many industries, from customer support to content creation and beyond. However, their use and development raise important ethical and societal questions, such as the impact on employment or the potential misuse of generated content. As AI technology continues to advance, the role and capabilities of large language models will continue to evolve.
 ```