hugging-quants
/

Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

Resources

View closed (9)

add base_model

#15 opened 7 days ago by

add base_model metadata

#14 opened 29 days ago by

Issues to set the seed

#13 opened about 1 month ago by

could not run this in lmdeploy

#12 opened about 1 month ago by

How to use INT4 with vLLM?

#8 opened about 2 months ago by

Fixed tokenizer

#6 opened about 2 months ago by