nisten
/

llama3-8b-instruct-32k-gguf

Inference Endpoints

Model card Files Files and versions Community

llama3-8b-instruct-32k-gguf

1 contributor

History: 16 commits

nisten's picture

4.2gb version for 8b macbook air with same pplx as iq4_xs

d51d312 verified 7 months ago

.gitattributes

2.21 kB

4.2gb quant specifically for macbook air, tiny bit better pplx than iq4_xs but 400mb smaller 7 months ago
LICENSE

7.8 kB

Upload folder using huggingface_hub 7 months ago
README.md

4.76 kB

Update README.md 7 months ago
USE_POLICY.md

4.7 kB

Upload folder using huggingface_hub 7 months ago
llama3-8GbMacbookAir-q4.gguf

4.2 GB
LFS

4.2gb version for 8b macbook air with same pplx as iq4_xs 7 months ago
llama3bf16imatrix.dat

4.99 MB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-f16.gguf

16.1 GB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-q1ns.gguf

2.72 GB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-q2ns.gguf

3.32 GB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-q3ns.gguf

4.14 GB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-q4ns.gguf

4.8 GB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-q5km.gguf

6.06 GB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-q6.gguf

6.85 GB
LFS

Upload folder using huggingface_hub 7 months ago
llama3ins-8b-32k-q8.gguf

8.54 GB
LFS

Upload folder using huggingface_hub 7 months ago
wiki.test.raw

1.29 MB

Upload folder using huggingface_hub 7 months ago