Model Loading Error

#40
by kcsham - opened

I have the following code

from transformers import pipeline
pipe = pipeline("feature-extraction", model="nvidia/NV-Embed-v1", trust_remote_code=True)

I got this error seeming loading the model to local cache.

Loading checkpoint shards:  25%|██▌       | 1/4 [00:01<00:03,  1.31s/it]/tmp/tmpwsi3vxvc: line 3: 12435 Killed                  python /mnt/c/Users/kcsha/Projects/learn/nvidia-embedding/embed2.py
ERROR conda.cli.main_run:execute(125): `conda run python /mnt/c/Users/kcsha/Projects/learn/nvidia-embedding/embed2.py` failed. (See above for error)

Any suggestion what could be the issue? Thanks.

NVIDIA org
edited Jul 5

Hi, @kcsham . Thanks for reporting the issue. However, we can execute the below code snippets from our side. Can you check your system has enough storage to load the model?

from transformers import pipeline
pipe = pipeline("feature-extraction", model="nvidia/NV-Embed-v1", trust_remote_code=True)

Hi, @kcsham . Thanks for reporting the issue. However, we can execute the below code snippets from our side. Can you check your system has enough storage to load the model?

from transformers import pipeline
pipe = pipeline("feature-extraction", model="nvidia/NV-Embed-v1", trust_remote_code=True)

I was trying to load a locally downloaded version of the model. I got the same error too. I'm not out of space in disk and memory when that happened.

However, I'm trying to load the above in Ubuntu 20.04 in WSL of Windows 11.

NVIDIA org

@kcsham . Can you download the model locally in the following way?

import torch
from sentence_transformers import SentenceTransformer

## save the model
model = SentenceTransformer('nvidia/NV-Embed-v1', trust_remote_code=True)
model.max_seq_length = 4096
model.tokenizer.padding_side="right"
model = model.to(torch.float16)
model.save("<your_local_directory>")  ## change path of <your_local_directory>

## load the model
model = SentenceTransformer("<your_local_directory>")

Sign up or log in to comment