Benchmark Radar™
Star —

VisualFLIP Benchmark

Explore score history ↗

VisualFLIP evaluates multimodal LLMs on visual reasoning with 1,374 images in paired perturbation tasks. Each pair has a fixed question but minimally changed visual evidence so the answer flips. Scoring uses pair accuracy and Collapse Rate to test evidence dependence in capabilities like cardinality, attribute, spatial, and logic.

Released
2026-06-05
Institutions
Imperial College London
Evaluates
General AI, Multimodal, Reasoning
Openness
unknown
Importer
Claire Radar
Review status
unreviewed

VisualFLIP paper and code

No reported scores are on record for this benchmark yet.

Which sources cite VisualFLIP?

Related General AI, Multimodal, Reasoning benchmarks