Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
comaniac
/
Meta-Llama-3-70B-Instruct-FP8-v1
like
0
Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
fp8
Model card
Files
Files and versions
Community
Train
Deploy
Use this model
Edit model card
YAML Metadata Warning:
empty or missing yaml metadata in repo card (
https://huggingface.co/docs/hub/model-cards#model-card-metadata
)
Llama-3-70B-Instruct-FP8-v1
Evaluation
Llama-3-70B-Instruct-FP8-v1
Weights and activations are per-tensor quantized to float8_e4m3.
Quantization with AutoFP8.
Calibration dataset: Ultrachat (mgoin/ultrachat_2k)
Samples: 1024
Sequence length: 4096
Evaluation
TBA
Downloads last month
5
Safetensors
Model size
70.6B params
Tensor type
BF16
·
F8_E4M3
·
Inference Examples
Text Generation
Inference API (serverless) is not available, repository is disabled.