VisualFLIP Benchmark
Explore score history ↗VisualFLIP evaluates multimodal LLMs on visual reasoning with 1,374 images in paired perturbation tasks. Each pair has a fixed question but minimally changed visual evidence so the answer flips. Scoring uses pair accuracy and Collapse Rate to test evidence dependence in capabilities like cardinality, attribute, spatial, and logic.
- Released
- 2026-06-05
- Institutions
- Imperial College London
- Evaluates
- General AI, Multimodal, Reasoning
- Openness
- unknown
- Importer
- Claire Radar
- Review status
- unreviewed
VisualFLIP paper and code
No reported scores are on record for this benchmark yet.