---
base_model: hpcgroup/hpc-coder-v2-6.7b
library_name: transformers
tags:
- code
- hpc
- parallel
- axonn
- llama-cpp
- gguf-my-repo
---

# HPC-Coder-v2-6.7b-Q8_0-GGUF
This is the [HPC-Coder-v2-6.7b](https://huggingface.co/hpcgroup/hpc-coder-v2-6.7b) model with 8 bit quantized weights in the GGUF format that can be used with llama.cpp.
Refer to the [original model card](https://huggingface.co/hpcgroup/hpc-coder-v2-6.7b) for more details on the model.

## Use with llama.cpp
See the [llama.cpp repo](https://github.com/ggerganov/llama.cpp) for installation instructions.
You can then use the model as:

```bash
llama-cli --hf-repo hpcgroup/hpc-coder-v2-6.7b-Q8_0-GGUF --hf-file hpc-coder-v2-6.7b-q8_0.gguf -r "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:" --in-prefix "\n" --in-suffix "\n### Response:\n" -c 8096 -p "your prompt here"
```