meta-llama

/

Llama-3.2-1B-Instruct

like 516

Follow

Meta Llama 7,536

@LiPengtao12138 please make sure you are using a GPU during inference

What I like to do is forcefully set the device to "cuda", for example device = torch.device("cuda") and then model.to(device) if model is an instance of, say, AutoModelForCausalLM(..) (I'm using huggingface's transformers library)