Model Card for Model ID
This model is a fine-tuned version of Llama-3-8B-Instruct on the BatchPrompting dataset, which spans 13 diverse NLP tasks. The model has been fine-tuned to effectively perform batch prompting - answering multiple questions concatenated into a single prompt in one inference pass.
Model Details
This model is a fine-tuned version of Llama-3-8B-Instruct on the BatchPrompting dataset, which spans 13 diverse NLP tasks. The model has been fine-tuned to effectively perform batch prompting - answering multiple questions concatenated into a single prompt in one inference pass.
Model Description
- Developed by: Alex Chandler, Sebastian Joseph
- Model type: Large Language Model (Llama-3 variant
- Language(s) (NLP): English
- License: MIT
- Finetuned from model [optional]: Llama-3-8B-Instruct
Model Sources [optional]
- Repository: Forthcoming
- Paper: Forthcoming
- Demo: Forthcoming
Uses
How to Use
Use with transformers See the snippet below for usage with Transformers:
import transformers
import torch
model_id = "achandlr/Llama-3-8B-Instruct-BatchPromptQA"
# Load the model pipeline
pipeline = transformers.pipeline("text-generation", model=model_id)
# Generate text using the pipeline
generated_text = pipeline("Hey how are you doing today?")
print(generated_text)
Direct Use
The model can be used for efficient question-answering on a variety of NLP tasks by concatenating multiple questions into a single prompt. It demonstrates strong generalization to unseen tasks and maintains performance with larger batch sizes compared to the non-fine-tuned model.
Out-of-Scope Use
The model should not be used for tasks that may cause harm or for generating factually incorrect or biased content. Caution should be exercised if using the model for high-stakes decision making.
Bias, Risks, and Limitations
The model may exhibit biases present in its pretraining data or the BatchPrompting dataset. It has not been extensively tested for fairness or potential misuse. Performance may degrade on out-of-distribution examples or tasks very dissimilar to the training data.
Recommendations
Users should be made aware of the model's potential limitations and biases. The model's outputs should be carefully monitored, especially when used for sensitive applications. More testing is needed to fully characterize its capabilities and shortcomings.
How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
Training Details
Training Data
The model was fine-tuned on our BatchPrompting dataset consisting of 13 NLP tasks:
- GLUE Benchmark Tasks: A collection of datasets used for evaluating the performance of models on a variety of natural language understanding tasks.
- Mathematical Reasoning Datasets:
- GSM8K: Focuses on numerical and logical reasoning challenges.
- GSM8K-Hard: Contains more complex problems from the GSM8K dataset.
- CommonsenseQA: Tests the model's commonsense reasoning ability through multiple-choice question answering.
- RACE Reading Comprehension Dataset: Consists of passages and questions designed to assess reading comprehension, derived from English exams.
Training Procedure
The model was fine-tuned using the LoRA method.
Training Hyperparameters
- Training regime: Forthcoming
Speeds, Sizes, Times [optional]
Forthcoming
[More Information Needed]
Evaluation
Testing Data, Factors & Metrics Evaluation was performed on tasks that were excluded from the training run. Key metrics included accuracy and BatchPrompt error rate (failure to answer a question or conform to the specified format). A table of our results is forthcoming.
Testing Data, Factors & Metrics
Forthcoming
Testing Data
Forthcoming
[More Information Needed]
Metrics
Forthcoming
[More Information Needed]
Results
Forthcoming [More Information Needed]
Summary
Forthcoming
Model Examination [optional]
Forthcoming
[More Information Needed]
Model Architecture and Objective
[More Information Needed]
Compute Infrastructure
[More Information Needed]
Hardware
[More Information Needed]
Software
[More Information Needed]
Citation
Forthcoming
BibTeX:
Forthcoming
APA:
[More Information Needed]
Glossary [optional]
[More Information Needed]
More Information [optional]
[More Information Needed]
Model Card Authors [optional]
[More Information Needed]
Model Card Contact
[More Information Needed]