🤗 Serve any model with Inference Endpoints + Custom Handlers
•
2
Alvaro Bartolome
alvarobartt
AI & ML interests
☁️ cloud machine learning @huggingface and open source passionate
Recent Activity
New activity
1 day ago
hf-vision/object_detection_leaderboard:🚩 Leaderboard down!
updated
a collection
2 days ago
🤗 Custom Inference Endpoints Handlers
Reacted to
danielhanchen's
post
with 🔥
4 days ago
Vision finetuning is in 🦥Unsloth! You can now finetune Llama 3.2, Qwen2 VL, Pixtral and all Llava variants up to 2x faster and with up to 70% less VRAM usage! Colab to finetune Llama 3.2: https://colab.research.google.com/drive/1j0N4XTY1zXXy7mPAhOC1_gMYZ2F2EBlk?usp=sharing
Articles
Organizations
Posts
5
Post
2753
🤗 Serving Meta Llama 3.1 405B on Google Cloud is now possible via the Hugging Face Deep Learning Containers (DLCs) for Text Generation Inference (TGI)
In this post, we showcase how to deploy https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct-FP8 on an A3 instance with 8 x H100 GPUs on Vertex AI
Thanks to the Hugging Face DLCs for TGI and Google Cloud Vertex AI, deploying a high-performance text generation container for serving Large Language Models (LLMs) has never been easier. And we’re not going to stop here – stay tuned as we enable more experiences to build AI with open models on Google Cloud!
Read the full post at https://huggingface.co/blog/llama31-on-vertex-ai
In this post, we showcase how to deploy https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct-FP8 on an A3 instance with 8 x H100 GPUs on Vertex AI
Thanks to the Hugging Face DLCs for TGI and Google Cloud Vertex AI, deploying a high-performance text generation container for serving Large Language Models (LLMs) has never been easier. And we’re not going to stop here – stay tuned as we enable more experiences to build AI with open models on Google Cloud!
Read the full post at https://huggingface.co/blog/llama31-on-vertex-ai
Post
3001
🔥 Prometheus 2 was recently released by Kaist AI as an alternative and closely mirroring both human and GPT-4 evaluation, and surpassing the former Prometheus!
prometheus-eval/prometheus-7b-v2.0
prometheus-eval/prometheus-8x7b-v2.0
🌬️Fine-tuned on top of mistralai/Mistral-7B-Instruct-v0.2 and mistralai/Mixtral-8x7B-Instruct-v0.1
🗂️The datasets used for fine-tuning have been publicly released i.e. prometheus-eval/Feedback-Collection and prometheus-eval/Preference-Collection
🤝🏻Unified LM evaluator for absolute (a single prompt-completion pair) and relative (two completions for a given prompt) due to model merging
❌No longer needs a mandatory reference / golden answer, but can still be provided optionally
🔝Surpasses the former version of Prometheus, and has a high correlation with human, GPT-4, and Claude 3 Opus scores when evaluating LMs
📝Apache 2.0 license
Long-story short, an amazing job from Kaist AI bridging the gap with LLM evaluators other than proprietary and bigger models!
This week at Argilla, we decided to add a new task to use Prometheus 2 as an LLM evaluator using
😱 Using
Find the generated dataset and the code at distilabel-internal-testing/instruction-dataset-prometheus
prometheus-eval/prometheus-7b-v2.0
prometheus-eval/prometheus-8x7b-v2.0
🌬️Fine-tuned on top of mistralai/Mistral-7B-Instruct-v0.2 and mistralai/Mixtral-8x7B-Instruct-v0.1
🗂️The datasets used for fine-tuning have been publicly released i.e. prometheus-eval/Feedback-Collection and prometheus-eval/Preference-Collection
🤝🏻Unified LM evaluator for absolute (a single prompt-completion pair) and relative (two completions for a given prompt) due to model merging
❌No longer needs a mandatory reference / golden answer, but can still be provided optionally
🔝Surpasses the former version of Prometheus, and has a high correlation with human, GPT-4, and Claude 3 Opus scores when evaluating LMs
📝Apache 2.0 license
Long-story short, an amazing job from Kaist AI bridging the gap with LLM evaluators other than proprietary and bigger models!
This week at Argilla, we decided to add a new task to use Prometheus 2 as an LLM evaluator using
distilabel, so we implemented PrometheusEval.😱 Using
PrometheusEval running their 7B variant with vLLM in a single L40 on top of
HuggingFaceH4/instruction-dataset, we got the 327 existing prompt-completion pairs evaluated and pushed to the Hub in less than 2 minutes!Find the generated dataset and the code at distilabel-internal-testing/instruction-dataset-prometheus
Collections
9
models
22
alvarobartt/NVLM-D-72B-IE-compatible
Image-Text-to-Text
•
Updated
•
37
alvarobartt/ghibli-characters-flux-lora
Text-to-Image
•
Updated
•
673
•
•
31
alvarobartt/ghibli-characters-sd3.5-lora
Text-to-Image
•
Updated
•
784
•
•
9
alvarobartt/bert-base-multilingual-cased-ner-spanish
Token Classification
•
Updated
•
87
•
2
alvarobartt/mistral-7b-orpo-capybara-reproduction
Text Generation
•
Updated
•
7
alvarobartt/mistral-7b-orpo-airoboros-pref-10k
Text Generation
•
Updated
•
12
alvarobartt/mistral-7b-orpo-alignment-handbook
Text Generation
•
Updated
•
10
alvarobartt/mistral-orpo-mix-b0.05-l1024-pl512-lr5e-7-cosine
Text Generation
•
Updated
•
6
alvarobartt/mistral-orpo-mix-b0.1-l2048-pl1792-lr5e-6-inverse-sqrt
Text Generation
•
Updated
•
7
alvarobartt/mistral-orpo-mix
Text Generation
•
Updated
•
9
datasets
55
alvarobartt/Magicoder-Vicuna-1.0
Viewer
•
Updated
•
75.2k
•
17
alvarobartt/SQL-OAI
Viewer
•
Updated
•
106k
•
46
alvarobartt/Magicoder-OAI
Viewer
•
Updated
•
75.2k
•
46
alvarobartt/ghibli-characters
Viewer
•
Updated
•
9
•
86
•
6
alvarobartt/Capybara-Preferences-Tiny
Viewer
•
Updated
•
10
•
41
alvarobartt/replacing-judges-with-juries-distilabel
Viewer
•
Updated
•
100
•
90
•
3
alvarobartt/prometheus-eval-distilabel-default
Viewer
•
Updated
•
2
•
116
alvarobartt/prometheus-eval-distilabel-ratings
Viewer
•
Updated
•
2
•
56
alvarobartt/prometheus-eval-distilabel-generation
Viewer
•
Updated
•
2
•
62
alvarobartt/prometheus-eval-distilabel-index
Viewer
•
Updated
•
2
•
59