Robert Shaw

robertgshaw2's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

upvoted a paper 25 days ago

"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization

Paper • 2411.02355 • Published 26 days ago • 45

New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 about 1 month ago

How to download the model with transformer library

5

#6 opened about 1 month ago by Rick10

New activity in mistralai/Pixtral-12B-2409 about 1 month ago

Update README.md

3

#25 opened about 1 month ago by robertgshaw2

updated a model about 2 months ago

robertgshaw2/llama-3-act-order

Updated Oct 9 • 2

upvoted a collection about 2 months ago

Llama-3.1 Quantization

Collection

Neural Magic quantized Llama-3.1 models • 22 items • Updated 8 days ago • 39

New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 about 2 months ago

Issue running on vLLM using FP8

2

#3 opened about 2 months ago by ffleandro

updated a model about 2 months ago

nm-testing/pixtral-fp8-test

Updated Oct 2

New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 3 months ago

vllm says the requested model does not exist

2

#1 opened 3 months ago by shivams101

New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 3 months ago

Storage format differs from other w4a16 models

2

#2 opened 3 months ago by timdettmers

New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 3 months ago

Model weights are not loaded

4

#3 opened 3 months ago by MarvelousMouse

updated 2 models 4 months ago

nm-testing/Meta-Llama-3-70B-Instruct-FBGEMM-nonuniform

Text Generation • Updated Jul 20 • 60 • 1

nm-testing/Meta-Llama-3-8B-Instruct-FBGEMM-nonuniform

Text Generation • Updated Jul 20 • 31

New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 4 months ago

Can not be inferenced with vllm openai server

1

#1 opened 4 months ago by jjqsdq

updated 4 models 5 months ago

nm-testing/llama-3-8b-instruct-fbgemm-test-model

Text Generation • Updated Jul 18 • 6

nm-testing/llama-3-8b-instruct-fp8

Updated Jul 18

nm-testing/Meta-Llama-3-8B-Instruct-W8A8-FP8-Non-Uniform-compressed-tensors

Text Generation • Updated Jul 17 • 5

nm-testing/DeepSeek-Coder-V2-Lite-Instruct-FP8

Text Generation • Updated Jul 14 • 6

New activity in neuralmagic/Meta-Llama-3-70B-Instruct-quantized.w4a16 5 months ago

Code example request with vllm

2

#1 opened 5 months ago by ShiningJazz

updated 2 models 5 months ago

neuralmagic/SparseLLama-2-7b-ultrachat_200k-pruned_50.2of4

Text Generation • Updated Jul 7 • 18

nm-testing/tiny-random-llama-test

Updated Jul 3 • 3