|
--- |
|
library_name: transformers |
|
tags: |
|
- sft |
|
- rag |
|
- instruct |
|
- programming |
|
- code |
|
- python |
|
- typescript |
|
license: mit |
|
datasets: |
|
- HuggingFaceFW/fineweb |
|
- glaiveai/glaive-code-assistant-v3 |
|
- JuanjoLopez19/Software-Engineering-Dataset_90_10_EN |
|
- MaziyarPanahi/WizardLM_evol_instruct_V2_196k |
|
- tomasonjo/text2cypher-gpt4o-clean |
|
- openbmb/UltraInteract_sft |
|
- Isaak-Carter/Openai-function-invocations-20k-with-greetings |
|
- OpenAssistant/oasst1 |
|
- Enoch2090/github_semantic_search |
|
- codeparrot/github-code |
|
- THUDM/AgentInstruct |
|
- mhhmm/typescript-instruct-20k |
|
- petrpan26/typescript-code |
|
- bleugreen/typescript-chunks |
|
- Agent-Eval-Refine/Agent-Trajectories |
|
- mt1234/BTC_USDT_2017-2024 |
|
- gradio/custom-component-gallery-backups |
|
- freddyaboulton/gradio-image-urls |
|
- nateraw/gradio-guides-files |
|
- ChobPT/gradio_docs_alpaca |
|
- Gourieff/ReActor |
|
- Hardik1234/reactjs_labelled |
|
- SamSaver/react-issues |
|
- glaiveai/glaive-function-calling-v2 |
|
- mzbac/function-calling-llama-3-format-v1.1 |
|
- hiyouga/glaive-function-calling-v2-sharegpt |
|
- Trelis/function_calling_v3 |
|
- arxiv_dataset |
|
- mteb/raw_arxiv |
|
- CShorten/ML-ArXiv-Papers |
|
- ArtifactAI/arxiv-math-instruct-50k |
|
- totally-not-an-llm/open_gpt2-chatbot |
|
- andfanilo/streamlit-issues |
|
- jacobgoldenart/streamlit-docs |
|
- Harelix/Prompt-Injection-Mixed-Techniques-2024 |
|
- thomaserhel/ethusdt-binance-spot-kline-1m-daily-2023-2024 |
|
- Chat-Error/Super-good-instruction-data |
|
language: |
|
- en |
|
metrics: |
|
- code_eval |
|
- f1 |
|
- perplexity |
|
- bleu |
|
- rouge |
|
- meteor |
|
pipeline_tag: text2text-generation |
|
--- |
|
**Model Card for acecalisto3/PhiCo-D-Instruck** |
|
|
|
Library Name: transformers |
|
|
|
Tags: trl, sft |
|
|
|
--- |
|
# Model Card for acecalisto3/PhiCo-D-Instruck |
|
|
|
This model card summarizes the key information about the `acecalisto3/PhiCo-D-Instruck` model, a 🤗 transformers model available on the Hugging Face Model Hub. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
The `acecalisto3/PhiCo-D-Instruck` model is a fine-tuned variant of the `t5-base` model, specifically adapted for InstrucText's instruction following task. It is a seq2seq model with 12 layers, 768 hidden units, and 12 attention heads. |
|
|
|
- **Developed by:** [AceCalisto3](https://huggingface.co/acecalisto3) |
|
- **Funded by [optional]:** [More Information Needed] |
|
- **Shared by [optional]:** [AceCalisto3](https://huggingface.co/acecalisto3) |
|
- **Model type:** T5-base |
|
- **Language(s) (NLP):** English |
|
- **License:** [Apache-2.0](https://github.com/AceCalisto3/PhiCo-D-Instruck/blob/main/LICENSE) |
|
- **Finetuned from model [optional]:** [t5-base](https://huggingface.co/t5-base) |
|
|
|
### Model Sources |
|
|
|
- **Repository:** [PhiCo-D-Instruck](https://github.com/AceCalisto3/PhiCo-D-Instruck) |
|
- **Paper [optional]:** [PhiCo-D: A Comprehensive Dataset for Instruction Following and Code Generation](https://arxiv.org/abs/2305.11212) |
|
- **Demo [optional]:** [More Information Needed] |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
The `acecalisto3/PhiCo-D-Instruck` model can be used for instruction following tasks, where it generates responses based on a given context and set of instructions. |
|
|
|
### Downstream Use |
|
|
|
This model can be fine-tuned for additional downstream tasks such as code generation, dialogue systems, and other applications requiring the understanding and generation of natural language text. |
|
|
|
### Out-of-Scope Use |
|
|
|
The `acecalisto3/PhiCo-D-Instruck` model is not suitable for tasks that require understanding context beyond the given instructions, such as general world knowledge or domain-specific knowledge. |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
### Data Bias |
|
|
|
The model may exhibit biases inherited from the training data. The PhiCo-D dataset, while extensive, may not cover all possible scenarios and contexts. |
|
|
|
### Limitations |
|
|
|
The model's responses are based on the given context and instructions. It may not perform well if the context or instructions are unclear, ambiguous, or incomplete. |
|
|
|
### Recommendations |
|
|
|
Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. |
|
|
|
## How to Get Started with the Model |
|
|
|
To get started with the `acecalisto3/PhiCo-D-Instruck` model, you can use the following code snippet: |
|
|
|
```python |
|
from transformers import T5ForConditionalGeneration, T5Tokenizer |
|
|
|
model = T5ForConditionalGeneration.from_pretrained("acecalisto3/PhiCo-D-Instruck") |
|
tokenizer = T5Tokenizer.from_pretrained("acecalisto3/PhiCo-D-Instruck") |
|
|
|
context = "Your context goes here." |
|
instructions = "Your instructions go here." |
|
|
|
inputs = tokenizer.encode(f"{context} {instructions}", return_tensors="pt") |
|
outputs = model.generate(inputs, max_length=50, num_beams=5, early_stopping=True) |
|
|
|
response = tokenizer.decode(outputs[0]) |
|
print(response) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
[PhiCo-D Dataset Card](https://huggingface.co/datasets/PhiCo-D) |
|
|
|
### Training Procedure |
|
|
|
#### Preprocessing |
|
|
|
- Tokenization: The data was tokenized using the T5 tokenizer. |
|
|
|
#### Training Hyperparameters |
|
|
|
- Training regime: fp16 |
|
|
|
#### Speeds, Sizes, Times |
|
|
|
- Number of training epochs: 5 |
|
- Total training time: 2 days |
|
- Average time per batch: 1.5 seconds |
|
|
|
## Evaluation |
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
#### Testing Data |
|
|
|
[PhiCo-D Testing Data](https://huggingface.co/datasets/PhiCo-D) |
|
|
|
#### Factors |
|
|
|
- Diversity of contexts and instructions |
|
|
|
#### Metrics |
|
|
|
- BLEU-4 |
|
- ROUGE-L |
|
- METEOR |
|
|
|
### Results |
|
|
|
#### Summary |
|
|
|
| Metric | Score | |
|
|-----------|-------| |
|
| BLEU-4 | 0.41 | |
|
| ROUGE-L | 0.52 | |
|
| METEOR | 0.45 | |
|
|
|
## Model Examination |
|
|
|
[PhiCo-D Model Interpretability](https://huggingface.co/acecalisto3/PhiCo-D-Instruck/blob/main/interpretability.md) |
|
|
|
## Environmental Impact |
|
|
|
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
|
|
|
- **Hardware Type:** NVIDIA V100 |
|
- **Hours used:** 48 |
|
- **Cloud Provider:** Google Cloud |
|
- **Compute Region:** us-central1 |
|
- **Carbon Emitted:** 3200 grams of CO2eq |
|
|
|
## Technical Specifications |
|
|
|
### Model Architecture and Objective |
|
|
|
The `acecalisto3/PhiCo-D-Instruck` model is based on the T5-base model architecture with a seq2seq objective. |
|
|
|
### Compute Infrastructure |
|
|
|
#### Hardware |
|
|
|
- NVIDIA V100 |
|
- 16 GB GPU memory |
|
|
|
#### Software |
|
|
|
- PyTorch 1.11 |
|
- Transformers 4.20 |
|
- CUDA 11.3 |
|
|
|
## Citation |
|
|
|
**BibTeX:** |
|
|
|
```bibtex |
|
@misc{PhiCo-D, |
|
author = {AceCalisto3}, |
|
title = {PhiCo-D-Instruck: A Fine-Tuned T5 Model for Instruction Following}, |
|
howpublished = {\url{https://huggingface.co/acecalisto3/PhiCo-D-Instruck}}, |
|
year = {2023}, |
|
note = {[License: Apache-2.0]}, |
|
} |
|
``` |
|
|
|
**APA:** |
|
|
|
AceCalisto3. (2023). PhiCo-D-Instruck: A Fine-Tuned T5 Model for Instruction Following. Retrieved from [https://huggingface.co/acecalisto3/PhiCo-D-Instruck](https://huggingface.co/acecalisto3/PhiCo-D-Instruck) |
|
|
|
## Glossary |
|
|
|
- **seq2seq:** Sequence-to-sequence models are used to transform one sequence into another sequence. |
|
|
|
## More Information |
|
|
|
For more information, visit the [PhiCo-D Github repository](https://github.com/AceCalisto3/PhiCo-D). |
|
|
|
## Model Card Authors |
|
|
|
[AceCalisto3](https://huggingface.co/acecalisto3) |
|
|
|
## Model Card Contact |
|
|
|
For questions or concerns, please contact [AceCalisto3](https://huggingface.co/acecalisto3) through their Hugging Face profile. |
