Benchmark Radar™
RSS Star —

Library

120 of 3,761 benchmarks

The B2B SaaS AI Visibility Benchmark 2026

Evaluates which of 94 B2B SaaS brands are named by ChatGPT, Claude, Perplexity, Google Gemini and Google AI Overviews on 50 real buying questions across five G2 categories.

Oct 13, 2026 · Claire Radar

Decision Layer Benchmark

A token-cost model for an AI agent's decision layer: estimating monthly spend for five decision questions across naive, cached, and tuned OpenAI, Anthropic, and Google baselines versus Jev.

Oct 12, 2026 · Claire Radar

Graduate Admissions Agent Benchmark

Evaluates Chinese-language agent products on graduate admissions advising, covering recommendation, planning, and two-stage information-update tasks with per-task rubrics.

Abstract Reasoning & Planning · Multilingual & Translation (provisional) · Oct 12, 2026 · Claire Radar

RoB 2 AI Benchmark

Unified frozen benchmark for AI risk-of-bias assessment using Cochrane RoB 2, built from three public expert-annotated sources, scored by agreement with human judgments at signalling, domain, and overall levels.

Health & Medicine · Oct 12, 2026 · Claire Radar

CausalCascade

A simulator-grounded benchmark for assessing causal cascade generation in image-to-video models through event-rich scenes, motion metrics, and diagnostic question answering.

World Models & Physical Understanding · Image & Video Generation · Oct 11, 2026 · Claire Radar

EmoTrace-Bench

EmoTrace-Bench evaluates text-to-video generation on emotional expression by controlling prompt cues and intensity, and scoring videos on target emotion, perceived intensity, and support across ten audiovisual cue channels.

Image & Video Generation · Chat, Writing & Preference · Oct 11, 2026 · Claire Radar

MedHarnessBench

Evaluates healthcare agents on 342 executable tasks spanning FHIR/EHR operations, healthcare administration interfaces, and multimodal clinical tools, measuring both outcome success and seven process-quality dimensions.

Health & Medicine · Professional & Enterprise Work · Oct 11, 2026 · Claire Radar

NepPlural

NepPlural is a dataset and evaluation suite for modeling societal personas in Nepali migration discourse, with human gold annotations, LLM protocol audit, encoder baselines, and human-judged steerability.

Multilingual & Translation · Chat, Writing & Preference · Oct 11, 2026 · Claire Radar

PhySimCode

PhySimCode evaluates video-to-code generation for physics simulations, requiring models to infer physical laws and parameters from videos and produce runnable Python code rendered and scored against the input video.

Code Generation · Video Understanding · World Models & Physical Understanding · Oct 11, 2026 · Claire Radar

RoboCombine

A benchmark for skill recombination in embodied manipulation without combined skill training trajectories, consisting of 19 compositional tasks with evaluation tracks for different skill execution paradigms.

Robotics & Embodied AI · Oct 11, 2026 · Claire Radar

DocumentVision

Evaluates OCR pipelines and vision-language models on dense tables, key-value extraction, charts, and layout grounding across high-resolution enterprise document images.

Document Understanding & OCR · Image & Visual Reasoning · Oct 10, 2026 · Claire Radar

Reg Affairs Bench

Evaluates AI systems on regulatory affairs deliverables from drugs, biologics, and medical devices using scenarios and dossier documents, scored against task-specific rubrics by a jury of LLM judges.

Health & Medicine · Professional & Enterprise Work · Oct 10, 2026 · Claire Radar

A Real-World CAD Segmentation Benchmark

CAD segmentation datasets are predominantly synthetic or automatically labelled, leaving it unclear how well existing methods transfer to real-world engineering tasks.

Engineering & Hardware Design · Image & Visual Reasoning · Oct 9, 2026 · Claire Radar

APEX-Harm

Professional agents increasingly work through command-line interfaces and multiple tools over shared workspaces containing files, messages, and business applications.

Safety & Alignment · Professional & Enterprise Work · Oct 9, 2026 · Claire Radar

AgentSWE

As the production of software passes into the hands of software itself, coding agents are increasingly commissioned to build agents fitted to their users' own workflows.

Agentic Coding & SWE · Oct 9, 2026 · Claire Radar

BASIS-Bench

Recent speech LLMs can leverage rich paralinguistic cues, enabling more complex and personalized interactions.

Speech & Audio · Oct 9, 2026 · Claire Radar

CRBugBench

With the advances of coding agents, software developers are spending more time reviewing code than writing code.

Agentic Coding & SWE · Oct 9, 2026 · Claire Radar

DRYLAB-Bench

As large language models (LLMs) acquire advanced biological design capabilities, they offer substantial research benefits but simultaneously introduce severe dual-use risks, such as engineering enhanced pathogens or drug-resistant bacteria.

Safety & Alignment · Science · Oct 9, 2026 · Claire Radar

EmbodiedCLBench

Continual learning requires models to accumulate past experience and transfer it to new tasks.

Robotics & Embodied AI · Oct 9, 2026 · Claire Radar

GeoSpatialBench

Understanding and manipulating spatial relations is fundamental to AI systems that reason about physical environments and construct grounded world models.

Spatial & 3D Reasoning · Image & Visual Reasoning · Oct 9, 2026 · Claire Radar

MAGMA-Bench

Long-horizon embodied agents must maintain task-relevant information, coordinate heterogeneous robots, and adapt their decisions as execution unfolds.

Robotics & Embodied AI · Oct 9, 2026 · Claire Radar

MVGraph

We introduce MVGraph, a benchmark for evaluating large vision-language models (LVLMs) on multi-graph reasoning where the evidence required by a query is distributed across multiple interrelated visual graph views.

Image & Visual Reasoning · Oct 9, 2026 · Claire Radar

MathVis

This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted reasoning.

Image & Visual Reasoning · Math Reasoning · Oct 9, 2026 · Claire Radar

MisVisAgentBench

Web agents increasingly browse, interpret, and act on web content on behalf of users.

Computer Use & GUI Agents · Image & Visual Reasoning · Oct 9, 2026 · Claire Radar

PPG-Bench

Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (FMs) have been developed to compute reusable representations for various downstream task…

Health & Medicine · Science · Oct 9, 2026 · Claire Radar

ReqClarifyBench

As LLMs become increasingly capable of implementing software, a critical bottleneck emerges: whether they truly understand what users want.

Agentic Coding & SWE · Instruction Following · Oct 9, 2026 · Claire Radar

SCCGBench

LLM-generated service-call code is increasingly important for service computing, where developers rely on large language models (LLMs) to compose services and invoke third-party services.

Code Generation · Tool Use & Function Calling · Oct 9, 2026 · Claire Radar

SWE-DeceptionBench

ArXiv paper introducing a benchmark for deceptive behaviors of LLMs across the software development lifecycle, with abstract stating tasks and results, but no public artifact available.

Safety & Alignment · Agentic Coding & SWE · Oct 9, 2026 · Claire Radar

SmartSalesBench

We present SmartSalesBench, the first multi-turn conversational benchmark explicitly designed to evaluate large language model (LLM) sales agents against psychologically grounded, resistant buyer personas.

Chat, Writing & Preference · Multi-Agent & Games · Oct 9, 2026 · Claire Radar

SqueezeBench

AI systems are making rapid progress in mathematics, but this progress is hard to measure.

Math Reasoning · Oct 9, 2026 · Claire Radar

TextVBench

ArXiv paper introducing a benchmark for text rendering and preservation in video generation, with abstract stating two tracks and scoring metrics, but no public artifact available.

Image & Video Generation · Oct 9, 2026 · Claire Radar

AgentHorizon

Computer-use agents are capable of completing complex tasks, increasing the use of automatic judges to determine success, either for training or for evaluation without human involvement.

Computer Use & GUI Agents · Oct 8, 2026 · Claire Radar

AgenticBBO-Bench

Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensive and limited.

AI R&D & ML Engineering · Science · Oct 8, 2026 · Claire Radar

DataSense-Bench

As claims about recursive self-improvement (RSI) and artificial general intelligence (AGI) proliferate, we ask a simple question: do frontier AI models have a sense of data, i.e., can they reliably select the right data for training?

AI R&D & ML Engineering · Oct 8, 2026 · Claire Radar

GameCommBench

Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge.

Chat, Writing & Preference · Video Understanding · Oct 8, 2026 · Claire Radar

LIVIN

Realistic household simulation must capture not only diverse environments but also the lived-in object arrangements and spatial constraints that shape robot motion and interaction.

Robotics & Embodied AI · Oct 8, 2026 · Claire Radar

MedBench

Large-scale construction of medical vision-language model (VLM) benchmarks is increasingly feasible with richly annotated imaging datasets and large language models (LLMs), yet existing automation largely focuses on generating evaluation i…

Health & Medicine · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar

Mine Odyssey

Advances in foundation models are driving efforts to introduce agents to assist people in the physical world.

Robotics & Embodied AI · Oct 8, 2026 · Claire Radar

MiniVer-V

A core challenge in short-video fact-checking is identifying which evidence is sufficient to support a verification conclusion.

Video Understanding · Factuality & Hallucination · Oct 8, 2026 · Claire Radar

ModeBench

A language model (LM) can usually answer the same question in more than one way, but reinforcement learning with verifiable rewards (RLVR) is indifferent to which correct answer a model produces.

Instruction Following · Oct 8, 2026 · Claire Radar

MultiWorldBench

Multiplayer world models must ensure that independently controlled views remain consistent with one shared and persistent world.

World Models & Physical Understanding · Oct 8, 2026 · Claire Radar

OmniCapBench

Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that expose their capability limits.

Speech & Audio · Video Understanding · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar

PathLang

Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characterized.

Health & Medicine · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar

PolyCodeEval

As large language models increasingly move toward repository-level software engineering, existing code-generation benchmarks remain fragmented across language coverage, task granularity, and evaluation protocols, impeding systematic compar…

Code Generation · Agentic Coding & SWE · Oct 8, 2026 · Claire Radar

Pumpire

We present Pumpire, a unified benchmark for evaluating metric point-pair distance estimation capability of both image- and video-level 3D foundation models, with or without depth priors.

Spatial & 3D Reasoning · Image & Visual Reasoning · Video Understanding · Oct 8, 2026 · Claire Radar

SWE-Journey

Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use, particularly in task horizon and interaction length.

Agentic Coding & SWE · Oct 8, 2026 · Claire Radar

SpaceCast-Bench

Existing spatial reasoning benchmarks mainly test spatial perception: reading off relations already visible in the input.

Spatial & 3D Reasoning · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar

TypedBench

System One models output calibrated probabilities over typed answers such as categorical choices, ordinal levels, or binary outcomes, via a non-generative interface.

Instruction Following · Oct 8, 2026 · Claire Radar

VenusRX-Bench

Existing reaction models primarily learn molecular transformations, whereas enzy- matic reactions depend jointly on molecular structure and catalytic function.

Science · Oct 8, 2026 · Claire Radar

WOVEN

Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning.

Spatial & 3D Reasoning · World Models & Physical Understanding · Oct 8, 2026 · Claire Radar

Workerville

LLM-based agents now interact with their environments continuously, shaped by such organizational channels as user instructions, peer messages, and long-term memory.

Professional & Enterprise Work · Agent Memory & Personalization · Oct 8, 2026 · Claire Radar

AdSpark

Product-centric advertisement video generation aims to create promotional videos that preserve fine-grained product identity while presenting selling points through coherent multi-shot narratives.

Image & Video Generation · Oct 7, 2026 · Claire Radar

AgentTime

An essential control of AI agents is their ability to manage runtime.

Tool Use & Function Calling · Oct 7, 2026 · Claire Radar

AirGroundVLN

Goal-oriented Vision-and-Language Navigation (VLN) requires agents to locate and reach targets described in natural language without prescribed routes.

Robotics & Embodied AI · Spatial & 3D Reasoning · Oct 7, 2026 · Claire Radar

CROSS-Bench

Vision-language models increasingly reason through crops, regions, and tool-produced observations.

Image & Visual Reasoning · Tool Use & Function Calling · Oct 7, 2026 · Claire Radar

DUDA-Bench

Urban diagnosis integrates heterogeneous observations to identify urban problems, localize affected areas, and investigate contributing factors, informing evidence-based urban planning and management.

Image & Visual Reasoning · Oct 7, 2026 · Claire Radar

InsClaimBench

Recent advances in reasoning-oriented large language models (LLMs) have motivated increasing evaluation of their ability to perform professional decision tasks.

Professional & Enterprise Work · Oct 7, 2026 · Claire Radar

InstanceBench

Referring Expression Segmentation (RES) links natural-language descriptions to pixel-level object masks.

Image & Visual Reasoning · Oct 7, 2026 · Claire Radar

LiveMACEBench

Evaluating agents by outcomes alone can obscure the capabilities that produce them.

Multi-Agent & Games · Oct 7, 2026 · Claire Radar

OcuBench

Eyeglass reflection removal is important across smartphone imaging, video conferencing, and other face-centric visual applications.

Image & Video Generation · Oct 7, 2026 · Claire Radar

PHRBench

Hallucinated information can propagate through multi-stage LLM systems and become part of the context for subsequent reasoning.

Factuality & Hallucination · Long Context · Oct 7, 2026 · Claire Radar

PatchBench

An LLM safety patch can pass a benchmark while still being a poor repair.

Cybersecurity & CTF · Safety & Alignment · Oct 7, 2026 · Claire Radar

RH-Detect

Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model systems.

Safety & Alignment · Agent Security & Prompt Injection · Oct 7, 2026 · Claire Radar

RT-Safe

Rapid progress in AI agents has brought growing attention to agent safety, with extensive evaluation focused on digital environments.

Agent Security & Prompt Injection · Safety & Alignment · Oct 7, 2026 · Claire Radar

RoboQuest

Recent advances in multimodal foundation models have made them capable generalist physical agents for a range of manipulation tasks.

Robotics & Embodied AI · Spatial & 3D Reasoning · Oct 7, 2026 · Claire Radar

RobotWorld

General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carry into the physical world.

Robotics & Embodied AI · World Models & Physical Understanding · Oct 7, 2026 · Claire Radar

SAVU-BENCH

Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modalities.

Speech & Audio · Image & Visual Reasoning · Spatial & 3D Reasoning · Oct 7, 2026 · Claire Radar

SoccerNet-FoulRet

Refereeing decisions in professional soccer remain inconsistent because referees cannot easily compare a contentious foul against similar past cases.

Video Understanding · Deep Research & Search · Oct 7, 2026 · Claire Radar

StoreBench

Reinforcement learning environments are now a primary lever for improving large language model (LLM) capabilities in post-training, yet most agentic benchmarks remain static: the world moves only when the agent acts, the reward is a termin…

Agentic Coding & SWE · Tool Use & Function Calling · Oct 7, 2026 · Claire Radar

TaoD2C-Bench

A key challenge for multimodal large language models (MLLMs) is moving beyond visual recognition to constraint-aware cross-modal reasoning.

Image & Visual Reasoning · Oct 7, 2026 · Claire Radar

BioStudyBench

We evaluate whether AI agents can match the reported findings of published biomedical studies using public data.

Science · Oct 6, 2026 · Claire Radar

CheckerBench

Static-analysis checker synthesis requires agents to interpret a defect specification, inspect a repository, implement analyzer-specific logic, and refine the checker through repeated compilation and analysis feedback.

Agentic Coding & SWE · Cybersecurity & CTF · Oct 6, 2026 · Claire Radar

DSV-Mem

Conversational MLLM agents are increasingly expected to assist in professional workflows, from AI research and engineering design to product management and business operations.

Agent Memory & Personalization · Oct 6, 2026 · Claire Radar

DecepEval

As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their reliable deployment.

Safety & Alignment · Oct 6, 2026 · Claire Radar

HarnessSecurity-Bench

Coding agent harnesses mediate tool use and authorize actions, yet their security mechanisms and runtime effects remain incompletely characterized.

Agent Security & Prompt Injection · Agentic Coding & SWE · Oct 6, 2026 · Claire Radar

Learn2Play Bench

Learning from experience is essential for LLM agents to adapt to unfamiliar and dynmaic environments.

Agent Skills & Self-Improvement · Oct 6, 2026 · Claire Radar

OpenWAM

World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult t…

World Models & Physical Understanding · Robotics & Embodied AI · Oct 6, 2026 · Claire Radar

ParanoiaEval

As coding agents increasingly undertake real-world work autonomously, judging whether their risk treatments are warranted has become important.

Agent Security & Prompt Injection · Oct 6, 2026 · Claire Radar

RAG-PIBench

Retrieval-Augmented Generation (RAG) systems are vulnerable to prompt-injection attacks embedded in retrieved content.

Agent Security & Prompt Injection · Deep Research & Search · Oct 6, 2026 · Claire Radar

RenderBench

Modern video models can generate realistic videos from real appearance references and proxy renders that specify scene structure, viewpoint changes, and motion.

Image & Video Generation · World Models & Physical Understanding · Oct 6, 2026 · Claire Radar

ST-Bench

The rapid progress of LLM-based multi-agent systems (MAS) has shown that they largely outperform single agents on coding, math, and QA tasks, where executable tests provide a binary success signal.

Multi-Agent & Games · Oct 6, 2026 · Claire Radar

SpeedrunBench

Frontier LLM agents have been shown to be capable of solving increasingly complex tasks for which humans have measurable solutions.

Agentic Coding & SWE · Code Generation · Oct 6, 2026 · Claire Radar

WorldSolver

LLM-based agents are increasingly advancing scientific and engineering problem solving, with physics simulation emerging as a challenging yet practical testbed for reproducing complex physical phenomena with application in embodied AI, gam…

World Models & Physical Understanding · Science · Oct 6, 2026 · Claire Radar

sk-bench

Multilingual LLM benchmarks omit Slovak, a morphologically rich West Slavic language of five million speakers, or cover it only by machine translation.

Multilingual & Translation · Oct 6, 2026 · Claire Radar

2d-fet-bench

Field-effect transistor (FET) layouts on exfoliated two-dimensional flakes are typically drawn by hand for each flake, placing contacts and gates to match its position and outline in optical micrographs.

Engineering & Hardware Design · Image & Visual Reasoning · Oct 5, 2026 · Claire Radar

ACG-Bench

Generalization in multi-arm collaboration can be studied as composing familiar atomic skills in new ways across arms.

Robotics & Embodied AI · Multi-Agent & Games · Oct 5, 2026 · Claire Radar

AgentMonBench

As agents take on long-horizon tasks, users shift from making individual decisions to overseeing autonomous execution.

Agent Skills & Self-Improvement · Oct 5, 2026 · Claire Radar

BazaarBench

In decentralized consumer-to-consumer (C2C) marketplaces, people list goods, negotiate with strangers, and rate one another, so trust rests on reputation.

Multi-Agent & Games · Oct 5, 2026 · Claire Radar

BrainTRACE

Brain MRI interpretation is a longitudinal clinical reasoning problem: radiologists compare serial studies, integrate information across MRI sequences, localize findings within volumetric anatomy, and translate this evidence into report-gr…

Health & Medicine · Image & Visual Reasoning · Oct 5, 2026 · Claire Radar

CavaBench

AI voice assistants often use Automatic Speech Recognition (ASR) with LLM-based reasoning, yet existing systems struggle with regional British accents, including Scottish, Irish, and Welsh accents, since most ASR models are trained predomi…

Speech & Audio · Oct 5, 2026 · Claire Radar

EMG-FM-Bench

Foundation models (FMs) are increasingly being developed for general time series and physiological signals, yet their transferability to downstream physiological tasks remains poorly understood.

Health & Medicine · Oct 5, 2026 · Claire Radar

FairRSFM

Remote sensing foundation models (RSFMs) are commonly evaluated using aggregate metrics, which can hide systematic performance disparities across ecological regions.

Safety & Alignment · Oct 5, 2026 · Claire Radar

Logbook

Audio benchmarks are built around short, pre-segmented clips, limiting model design to brief inputs or fixed vocabularies.

Speech & Audio · Oct 5, 2026 · Claire Radar

OR-Bench

Vision-language models (VLMs) can detect that an object has rotated across views, but cannot reliably tell by how much.

Spatial & 3D Reasoning · Image & Visual Reasoning · Oct 5, 2026 · Claire Radar

RMD-Bench

LLM-driven evolution enables iterative model development, but two practical goals remain underexplored: finding model designs that transfer across related tasks and sustaining improvement when training is expensive.

AI R&D & ML Engineering · Oct 5, 2026 · Claire Radar

TasteVal

We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models.

AI R&D & ML Engineering · Oct 5, 2026 · Claire Radar

AgentDoxx

As Large Language Models (LLMs) gain tool use capabilities such as web search, they can retrieve and cross-reference public information, creating privacy risks beyond memorization.

Agent Security & Prompt Injection · Oct 4, 2026 · Claire Radar

ColdDDI

Cold-start drug-drug interaction (DDI) prediction tests whether models can identify clinically significant interactions for drugs without training-time interaction history.

Health & Medicine · Science · Oct 4, 2026 · Claire Radar

DelegationBench

AI agents that send emails, edit files, and make purchases must decide when to act on their own and when to check with the user first.

Tool Use & Function Calling · Agent Security & Prompt Injection · Oct 4, 2026 · Claire Radar

EMBER-Bench

Lifelong physical agents must reason over extended interactions where past events continue to shape the world long after they disappear from view.

Robotics & Embodied AI · World Models & Physical Understanding · Oct 4, 2026 · Claire Radar

GNN-CB

Large language models (LLMs) have demonstrated strong performance on coding and reasoning benchmarks; however, their ability to solve graph-structured machine learning problems remains largely unexplored.

Code Generation · AI R&D & ML Engineering · Oct 4, 2026 · Claire Radar

MLE-InsightBench

Agentic machine learning engineering (MLE) is an emerging AI4SE application for complex ML tasks and a step toward recursive self-improvement of AI systems.

AI R&D & ML Engineering · Oct 4, 2026 · Claire Radar

MMPostTrainBench

Autonomous research seeks sustained model improvements through iterative experimentation and feedback.

AI R&D & ML Engineering · Oct 4, 2026 · Claire Radar

PortraitAes-Bench

Portrait aesthetic assessment assigns comparable scores according to how effectively human-centered images fulfill their photographic intent.

Image & Visual Reasoning · Chat, Writing & Preference · Oct 4, 2026 · Claire Radar

RMMBench

Although the advancement of vision-language models (VLMs) has endowed robots with enhanced environmental understanding and task reasoning, a comprehensive evaluation methodology is important to advance the integration of VLMs in robotic na…

Robotics & Embodied AI · Spatial & 3D Reasoning · Oct 4, 2026 · Claire Radar

UndoBench

Tool-using AI agents are increasingly deployed across enterprise software systems, yet widely used benchmarks primarily evaluate nominal task completion, conflating baseline planning competence with operational fault recovery.

Tool Use & Function Calling · Agent Security & Prompt Injection · Oct 4, 2026 · Claire Radar

VHDL-REPOBENCH

Large Language Models (LLMs) are increasingly applied in hardware design automation, demonstrating strong potential in generating and understanding hardware description languages.

Engineering & Hardware Design · Code Generation · Oct 4, 2026 · Claire Radar

AgentPersonaBench

We introduce AgentPersonaBench (APB), a benchmark evaluating whether persona conditioning faithfully steers downstream agent behavior.

Agent Memory & Personalization · Oct 3, 2026 · Claire Radar

CoPE-Bench

Reliable language models should resist unsupported user pressure while effectively using objective contextual information.

Safety & Alignment · Instruction Following · Oct 3, 2026 · Claire Radar

EgoExo-Next

Vision-language models (VLMs) are increasingly evaluated for egocentric and cross-view video reasoning, yet existing benchmarks largely focus on semantic event understanding, temporal relations, or correspondence between already observed v…

Video Understanding · Spatial & 3D Reasoning · Oct 3, 2026 · Claire Radar

IEC-Bench

Agents built on large language models (LLMs) build and run software through tool calls.

Agentic Coding & SWE · Tool Use & Function Calling · Oct 3, 2026 · Claire Radar

MASBench

Large language models (LLMs) have progressively evolved into the core of autonomous agents.

Multi-Agent & Games · Oct 3, 2026 · Claire Radar

MedImageOSWorld

Graphical consoles offer a practical interface for medical acquisition assistance, allowing agents to work through the controls and visual feedback used by human operators.

Computer Use & GUI Agents · Health & Medicine · Oct 3, 2026 · Claire Radar

RAGStress

Retrieval-Augmented Generation (RAG) is typically evaluated under the implicit assumption that the underlying knowledge base (KB) is clean, leaving the behaviour of RAG systems under realistic KB degradation poorly characterised.

Deep Research & Search · Oct 3, 2026 · Claire Radar

Spec2Game

Generating an executable program does not necessarily mean that it correctly implements the behavioral requirements specified in natural language.

Code Generation · Oct 3, 2026 · Claire Radar

AptMQL-Bench

Document databases such as MongoDB are core infrastructure for modern applications, and natural-language interfaces to them---text-to-MQL---would let non-experts query complex, semi-structured data without mastering the query language.

Code Generation · Oct 2, 2026 · Claire Radar

D2K-Bench

GPU kernels generated by large language model (LLM) agents can remain less efficient than expert implementations, but runtime alone does not reveal how the gap relates to design discovery and implementation.

AI R&D & ML Engineering · Oct 2, 2026 · Claire Radar

DexJoCo-X

As dexterous hands proliferate, collecting data and training policies separately for every morphology becomes increasingly impractical.

Robotics & Embodied AI · Oct 2, 2026 · Claire Radar

DyadMem

Long-term agents must remember not only what is true about a user, but also how a particular agent should work with that user as their shared history evolves.

Agent Memory & Personalization · Oct 2, 2026 · Claire Radar

Ego2World

Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses itself.

World Models & Physical Understanding · Video Understanding · Oct 2, 2026 · Claire Radar