Inquiry Regarding Model Performance and Attention Mask
Good afternoon!
I have been working with the model locally, but I've noticed that the results I am obtaining are significantly worse compared to those produced by GOT_online.
I received the following warning while running the model:
"The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id: 151643 for open-end generation."
I believe this might be affecting the model's performance. Could you please provide guidance on how to resolve this issue? Any assistance would be greatly appreciated.
Thank you in advance for your help!
I am also having the same issue, I have to change the bfloat16 to float 32 to get it running in my EC2 instance(g3s.xlarge) since it's an older GPU, but it's performance is really bad compared to the one in the demo can you please help us out?
TIA