SAFIRE Benchmark

What is SAFIRE?

Evaluates MLLMs on 193K multiple-choice VQA over 83K fire and smoke images across 20 scenarios and 10 reasoning dimensions.

Released

2026-09-07

Evaluates

General AI, Multimodal, Reasoning, Safety, Safety & Trustworthiness

Openness

unknown

Importer

Claire Radar

Review status

ai-reviewed

Reported scores

0

Source provenance

• Original evidence https://arxiv.org/abs/2609.07823

SAFIRE paper and code

• Paper https://arxiv.org/abs/2609.07823
• Code https://github.com/RISys-Lab/SAFIRE
• Website https://risys-lab.github.io/SAFIRE/

Which sources cite SAFIRE?

• SAFIRE

Related General AI, Multimodal, Reasoning, Safety, Safety & Trustworthiness benchmarks

• $\tau^\tau$-Bench: An Environment for End-To-End, Realistic Agent Construction
• 3DCodeBench
• 5-Dialects-BN
• A-Bench
• A-OKVQA
• A3S-Bench
• AARRI-Bench
• ActBench