Fix the Image, Not the Prompt

TL;DR

Vision models can be significantly improved by adding a few well-placed labels directly on images rather than solely relying on prompt adjustments. This approach enhances model reasoning and performance more effectively.

Key Takeaways

• Assess what the model sees before changing prompts.
• Use targeted labels to clarify critical elements in images.
• Avoid cluttering images with unnecessary annotations.
• Focus on improving image representations for better accuracy.