Artificial Intelligence Vision Models Misperceive Nonexistent Optical Illusions
In a groundbreaking study, Tomer Ullman, an associate professor at Harvard's Department of Psychology, has delved into the intriguing world of AI perception, exploring how vision-language models often misidentify optical illusions in images that do not actually contain them. The research, titled "The Illusion-Illusion: Vision Language Models See Illusions Where There are None," sheds light on the discrepancy between human perception and AI models' tendency to see illusions even in unambiguous images.
Ullman's study evaluates several AI models, including GPT4o, Claude 3, Gemini Pro Vision (Gemini 1.5), miniGPT, Qwen-VL, InstructBLIP, BLIP2, and LLaVA-1.5. Among these, the three leading commercial models—GPT-4, Claude 3, and Gemini 1.5—show a somewhat better recognition of actual illusions but still frequently misidentify non-illusion images as illusions.
One notable example of this phenomenon is ChatGPT, based on GPT-5, which incorrectly identified a clear image of a duck as the duck-rabbit optical illusion, a well-known illusion that can be seen as either a duck or a rabbit. This demonstrates the "illusion-illusion" effect, where the model perceives illusion-like effects where humans see none.
Ullman argues that the term "hallucination" should not be used to describe models misidentifying optical illusions and instead suggests that the mistake made by AI models is related to Cognitive Reflection Tasks, where models falsely identify an image as an illusion and go off based on that.
The professor cautions that the mixed results should not be interpreted as a sign that those models are better at not deceiving themselves, but rather their visual acuity is just not that great. He also emphasises the need for a closer scrutiny of the disconnect between vision and language in current vision language models, particularly in light of their deployment in robotics and other AI services.
Ullman's research underscores the importance of understanding the limitations of AI models and the need for continued research to bridge the gap between human visual reasoning and AI systems. The data associated with Ullman's paper has been published online for further analysis and discussion.
[1] Ullman, T. (2023). The Illusion-Illusion: Vision Language Models See Illusions Where There are None. Harvard University, Department of Psychology. Retrieved from https://psychology.harvard.edu/ullman/publications/2023_The_Illusion-Illusion.pdf
Read also:
- Kenya broadens economic zones featuring Olkaria's geothermal energy advantage
- Nutritional Healthcare: Linking Patients with Crucial Nourishment Through Medical Professionals
- Century Lithium Announces Production of Battery-Grade Lithium Metal Anodes from Angel Island Lithium Carbonate
- Truck manufacturers directed to cease enforcing immediate limp mode for low DEF emission regulations, according to EPA.
