metadata

library_name: transformers
tags: []

Model Card for Model ID

Model Details

This model is being made to enhance our work within crewai. We started with a high context length (1048K) version of Llama 3. We then fine-tuned on top of that to get a base agent.

Model Description

Built on the following:

Models:
- Meta-Llama-3-8B-Instruct
- Llama-3-8B-Instruct-Gradient-1048k
Datasets:
- m-a-p/CodeFeedback-Filtered-Instruction
- RomanTeucher/awesome_topic_code_snippets
- dair-ai/emotion
- mzbac/function-calling-llama-3-format-v1.1
- gretelai/synthetic_text_to_sql

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Developed by: [More Information Needed]
Model type: [More Information Needed]
License: [More Information Needed]
**Finetuned from model [Llama-3-8B-Instruct-Gradient-1048k]

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

Code Examples

Use the following format when using the model for inference:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are the helpful assistant. <|eot_id|><|start_header_id|>user<|end_header_id|>

{prompt} <|eot_id|><|start_header_id|>assistant<|end_header_id|>

Example of multi turn

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|>

What is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Bonjour! The capital of France is Paris!<|eot_id|><|start_header_id|>user<|end_header_id|>

What can I do there?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Paris, the City of Light, offers a romantic getaway with must-see attractions like the Eiffel Tower and Louvre Museum, romantic experiences like river cruises and charming neighborhoods, and delicious food and drink options, with helpful tips for making the most of your trip.<|eot_id|><|start_header_id|>user<|end_header_id|>

Give me a detailed list of the attractions I should visit, and time it takes in each one, to plan my trip accordingly.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

How to Get Started with the Model

Use the code below to get started with the model.

import transformers
import torch

model_id = "meta-llama/Meta-Llama-3-70B-Instruct"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]

prompt = pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    prompt,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])