MMOOC Benchmark
Explore score history ↗MMOOC evaluates multimodal large language models on out-of-context (OOC) and shifted in-context (Shifted IC) visual question answering. It contains over 41K image-question pairs covering three question formats, eight shift types, and six visual scenarios. Responses are scored for accuracy and refusal rate, with an LLM-as-a-judge metric for reasoning correctness.
- Released
- 2026-07-30
- Institutions
- Hong Kong Polytechnic University · Harbin Institute of Technology · Eastern Institute of Technology, Ningbo
- Evaluates
- General AI, Multimodal
- Openness
- unknown
- Importer
- Claire Radar
- Review status
- unreviewed
MMOOC paper and code
No reported scores are on record for this benchmark yet.