Version Issue
Dear Tsu-Jui (Ray) Fu, We successfully reached the final step in the demo.ipynd file, but we encountered a roadblock with a KeyError: 'emb'. Specifically, after training with the dataset and pretrained models you provided, we imported the trained mllm.pt file. However, it seems that the trained model parameters lacks the 'emb' field, which is causing the following line to fail:
EMB = ckpt['emb'].cuda()
with T.inference_mode(): NULL = model.edit_head(T.zeros(1, 8, 4096).half().to('cuda'), EMB)
print('NULL:', NULL.shape)
The error traceback is as follows:
#####
KeyError Traceback (most recent call last)
Cell In[3], line 31
28 image_token_len = (vision_config.image_size//vision_config.patch_size)**2
30 _ = model.eval()
---> 31 EMB = ckpt['emb'].cuda()
32 with T.inference_mode(): NULL = model.edit_head(T.zeros(1, 8, 4096).half().to('cuda'), EMB)
33 print('NULL:', NULL.shape)
KeyError: 'emb'
#####
Also, the version of transformsers in the requirement file of this page(4.30.2) is not the same as the version in requirement of vicuna(https://huggingface.co/lmsys/vicuna-7b-delta-v1.1/blob/main/generation_config.json), which is 4.28.0.dev0 , we would like to know which version is the recommended one?