Spaces:
Runtime error
Reproducing paper results
In Figure 4 of the BLIP-2 paper, a selection of examples is given by the authors with the following legend: "Selected examples of instructed zero-shot image-to-text generation using a BLIP-2 model w/ ViT-G and FlanT5 XXL (...)".
So far, I could not reproduce those selected examples with the HF demo. I understand the model served is the right one (BLIP-2 ViT-G + FlanT5 XXL), but despite playing the different decoding strategies and parameters, I am not able to reproduce those selected examples with the corresponding images.
For instance, considering this image:
The paper reports a long and informative answer from the model, while the actual running model in the demo only produces short texts that are less informative. There are small discrepancies as well, like the answers starting with capital in the paper but not in the demo. Adding prompts like "Question: {} Answer: {}", or "Question: {} Long answer: {}" does not change the outcome.
Am I missing something?
The currently running demo backend has max_txt_len=30. The example shown we have a larger max_txt_len=100.
The demo does not allow change max txt length as for now. We ll rework to include this option.
That makes sense. Thanks a lot for the feedback.
Let me close this thread.