SWE-Bench Multimodal

What is SWE-Bench Multimodal?

SWE-Bench Multimodal extends SWE-Bench to evaluate language models on software engineering tasks that involve visual inputs such as screenshots, UI mockups…

Modality

multimodal

Evaluates

multimodal, reasoning, agents, code, vision

Openness

unknown

Importer

LLM Stats

Reported scores

4

Source provenance

• Original evidence https://api.zeroeval.com/leaderboard/benchmarks/swe-bench-multimodal?top_n=500

SWE-Bench Multimodal results and reported scores

Which sources cite SWE-Bench Multimodal?

• SWE-Bench Multimodal

Related multimodal, reasoning, agents, code, vision benchmarks

• AA-AnalystAgent
• AA-Briefcase
• AA-LCR
• AA-LCR
• AA-Omniscience Index
• AbductionRules
• ACEBench
• ActivityNet