VisualFLIP Benchmark

What is VisualFLIP?

VisualFLIP evaluates multimodal LLMs on visual reasoning with 1,374 images in paired perturbation tasks. Each pair has a fixed question but minimally changed…

Released

2026-06-05

Evaluates

General AI, Multimodal, Reasoning

Openness

unknown

Importer

Claire Radar

Review status

unreviewed

Reported scores

0

Source provenance

• Original evidence https://arxiv.org/abs/2606.07872

VisualFLIP paper

• Paper https://arxiv.org/abs/2606.07872
• Website https://didizhu-judy.github.io/VisualFLIP/

Which sources cite VisualFLIP?

• VisualFLIP

Related General AI, Multimodal, Reasoning benchmarks

• $\tau^\tau$-Bench: An Environment for End-To-End, Realistic Agent Construction
• 3DCodeBench
• 5-Dialects-BN
• A-Bench
• A-OKVQA
• A3S-Bench
• AARRI-Bench
• ActBench