Evaluates which of 94 B2B SaaS brands are named by ChatGPT, Claude, Perplexity, Google Gemini and Google AI Overviews on 50 real buying questions across five G2 categories.
Oct 13, 2026 · Claire Radar
Evaluates which of 94 B2B SaaS brands are named by ChatGPT, Claude, Perplexity, Google Gemini and Google AI Overviews on 50 real buying questions across five G2 categories.
Oct 13, 2026 · Claire Radar
A token-cost model for an AI agent's decision layer: estimating monthly spend for five decision questions across naive, cached, and tuned OpenAI, Anthropic, and Google baselines versus Jev.
Oct 12, 2026 · Claire Radar
Evaluates Chinese-language agent products on graduate admissions advising, covering recommendation, planning, and two-stage information-update tasks with per-task rubrics.
Abstract Reasoning & Planning · Multilingual & Translation (provisional) · Oct 12, 2026 · Claire Radar
Unified frozen benchmark for AI risk-of-bias assessment using Cochrane RoB 2, built from three public expert-annotated sources, scored by agreement with human judgments at signalling, domain, and overall levels.
Health & Medicine · Oct 12, 2026 · Claire Radar
A simulator-grounded benchmark for assessing causal cascade generation in image-to-video models through event-rich scenes, motion metrics, and diagnostic question answering.
World Models & Physical Understanding · Image & Video Generation · Oct 11, 2026 · Claire Radar
EmoTrace-Bench evaluates text-to-video generation on emotional expression by controlling prompt cues and intensity, and scoring videos on target emotion, perceived intensity, and support across ten audiovisual cue channels.
Image & Video Generation · Chat, Writing & Preference · Oct 11, 2026 · Claire Radar
Evaluates healthcare agents on 342 executable tasks spanning FHIR/EHR operations, healthcare administration interfaces, and multimodal clinical tools, measuring both outcome success and seven process-quality dimensions.
Health & Medicine · Professional & Enterprise Work · Oct 11, 2026 · Claire Radar
NepPlural is a dataset and evaluation suite for modeling societal personas in Nepali migration discourse, with human gold annotations, LLM protocol audit, encoder baselines, and human-judged steerability.
Multilingual & Translation · Chat, Writing & Preference · Oct 11, 2026 · Claire Radar
PhySimCode evaluates video-to-code generation for physics simulations, requiring models to infer physical laws and parameters from videos and produce runnable Python code rendered and scored against the input video.
Code Generation · Video Understanding · World Models & Physical Understanding · Oct 11, 2026 · Claire Radar
A benchmark for skill recombination in embodied manipulation without combined skill training trajectories, consisting of 19 compositional tasks with evaluation tracks for different skill execution paradigms.
Robotics & Embodied AI · Oct 11, 2026 · Claire Radar
Evaluates OCR pipelines and vision-language models on dense tables, key-value extraction, charts, and layout grounding across high-resolution enterprise document images.
Document Understanding & OCR · Image & Visual Reasoning · Oct 10, 2026 · Claire Radar
Evaluates AI systems on regulatory affairs deliverables from drugs, biologics, and medical devices using scenarios and dossier documents, scored against task-specific rubrics by a jury of LLM judges.
Health & Medicine · Professional & Enterprise Work · Oct 10, 2026 · Claire Radar
CAD segmentation datasets are predominantly synthetic or automatically labelled, leaving it unclear how well existing methods transfer to real-world engineering tasks.
Engineering & Hardware Design · Image & Visual Reasoning · Oct 9, 2026 · Claire Radar
Professional agents increasingly work through command-line interfaces and multiple tools over shared workspaces containing files, messages, and business applications.
Safety & Alignment · Professional & Enterprise Work · Oct 9, 2026 · Claire Radar
As the production of software passes into the hands of software itself, coding agents are increasingly commissioned to build agents fitted to their users' own workflows.
Agentic Coding & SWE · Oct 9, 2026 · Claire Radar
Recent speech LLMs can leverage rich paralinguistic cues, enabling more complex and personalized interactions.
Speech & Audio · Oct 9, 2026 · Claire Radar
With the advances of coding agents, software developers are spending more time reviewing code than writing code.
Agentic Coding & SWE · Oct 9, 2026 · Claire Radar
As large language models (LLMs) acquire advanced biological design capabilities, they offer substantial research benefits but simultaneously introduce severe dual-use risks, such as engineering enhanced pathogens or drug-resistant bacteria.
Safety & Alignment · Science · Oct 9, 2026 · Claire Radar
Continual learning requires models to accumulate past experience and transfer it to new tasks.
Robotics & Embodied AI · Oct 9, 2026 · Claire Radar
Understanding and manipulating spatial relations is fundamental to AI systems that reason about physical environments and construct grounded world models.
Spatial & 3D Reasoning · Image & Visual Reasoning · Oct 9, 2026 · Claire Radar
Long-horizon embodied agents must maintain task-relevant information, coordinate heterogeneous robots, and adapt their decisions as execution unfolds.
Robotics & Embodied AI · Oct 9, 2026 · Claire Radar
We introduce MVGraph, a benchmark for evaluating large vision-language models (LVLMs) on multi-graph reasoning where the evidence required by a query is distributed across multiple interrelated visual graph views.
Image & Visual Reasoning · Oct 9, 2026 · Claire Radar
This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted reasoning.
Image & Visual Reasoning · Math Reasoning · Oct 9, 2026 · Claire Radar
Web agents increasingly browse, interpret, and act on web content on behalf of users.
Computer Use & GUI Agents · Image & Visual Reasoning · Oct 9, 2026 · Claire Radar
Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (FMs) have been developed to compute reusable representations for various downstream task…
Health & Medicine · Science · Oct 9, 2026 · Claire Radar
As LLMs become increasingly capable of implementing software, a critical bottleneck emerges: whether they truly understand what users want.
Agentic Coding & SWE · Instruction Following · Oct 9, 2026 · Claire Radar
LLM-generated service-call code is increasingly important for service computing, where developers rely on large language models (LLMs) to compose services and invoke third-party services.
Code Generation · Tool Use & Function Calling · Oct 9, 2026 · Claire Radar
ArXiv paper introducing a benchmark for deceptive behaviors of LLMs across the software development lifecycle, with abstract stating tasks and results, but no public artifact available.
Safety & Alignment · Agentic Coding & SWE · Oct 9, 2026 · Claire Radar
We present SmartSalesBench, the first multi-turn conversational benchmark explicitly designed to evaluate large language model (LLM) sales agents against psychologically grounded, resistant buyer personas.
Chat, Writing & Preference · Multi-Agent & Games · Oct 9, 2026 · Claire Radar
AI systems are making rapid progress in mathematics, but this progress is hard to measure.
Math Reasoning · Oct 9, 2026 · Claire Radar
ArXiv paper introducing a benchmark for text rendering and preservation in video generation, with abstract stating two tracks and scoring metrics, but no public artifact available.
Image & Video Generation · Oct 9, 2026 · Claire Radar
Computer-use agents are capable of completing complex tasks, increasing the use of automatic judges to determine success, either for training or for evaluation without human involvement.
Computer Use & GUI Agents · Oct 8, 2026 · Claire Radar
Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensive and limited.
AI R&D & ML Engineering · Science · Oct 8, 2026 · Claire Radar
As claims about recursive self-improvement (RSI) and artificial general intelligence (AGI) proliferate, we ask a simple question: do frontier AI models have a sense of data, i.e., can they reliably select the right data for training?
AI R&D & ML Engineering · Oct 8, 2026 · Claire Radar
Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge.
Chat, Writing & Preference · Video Understanding · Oct 8, 2026 · Claire Radar
Realistic household simulation must capture not only diverse environments but also the lived-in object arrangements and spatial constraints that shape robot motion and interaction.
Robotics & Embodied AI · Oct 8, 2026 · Claire Radar
Large-scale construction of medical vision-language model (VLM) benchmarks is increasingly feasible with richly annotated imaging datasets and large language models (LLMs), yet existing automation largely focuses on generating evaluation i…
Health & Medicine · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar
Advances in foundation models are driving efforts to introduce agents to assist people in the physical world.
Robotics & Embodied AI · Oct 8, 2026 · Claire Radar
A core challenge in short-video fact-checking is identifying which evidence is sufficient to support a verification conclusion.
Video Understanding · Factuality & Hallucination · Oct 8, 2026 · Claire Radar
A language model (LM) can usually answer the same question in more than one way, but reinforcement learning with verifiable rewards (RLVR) is indifferent to which correct answer a model produces.
Instruction Following · Oct 8, 2026 · Claire Radar
Multiplayer world models must ensure that independently controlled views remain consistent with one shared and persistent world.
World Models & Physical Understanding · Oct 8, 2026 · Claire Radar
Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that expose their capability limits.
Speech & Audio · Video Understanding · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar
Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characterized.
Health & Medicine · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar
As large language models increasingly move toward repository-level software engineering, existing code-generation benchmarks remain fragmented across language coverage, task granularity, and evaluation protocols, impeding systematic compar…
Code Generation · Agentic Coding & SWE · Oct 8, 2026 · Claire Radar
We present Pumpire, a unified benchmark for evaluating metric point-pair distance estimation capability of both image- and video-level 3D foundation models, with or without depth priors.
Spatial & 3D Reasoning · Image & Visual Reasoning · Video Understanding · Oct 8, 2026 · Claire Radar
Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use, particularly in task horizon and interaction length.
Agentic Coding & SWE · Oct 8, 2026 · Claire Radar
Existing spatial reasoning benchmarks mainly test spatial perception: reading off relations already visible in the input.
Spatial & 3D Reasoning · Image & Visual Reasoning · Oct 8, 2026 · Claire Radar
System One models output calibrated probabilities over typed answers such as categorical choices, ordinal levels, or binary outcomes, via a non-generative interface.
Instruction Following · Oct 8, 2026 · Claire Radar
Existing reaction models primarily learn molecular transformations, whereas enzy- matic reactions depend jointly on molecular structure and catalytic function.
Science · Oct 8, 2026 · Claire Radar
Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning.
Spatial & 3D Reasoning · World Models & Physical Understanding · Oct 8, 2026 · Claire Radar
LLM-based agents now interact with their environments continuously, shaped by such organizational channels as user instructions, peer messages, and long-term memory.
Professional & Enterprise Work · Agent Memory & Personalization · Oct 8, 2026 · Claire Radar
Product-centric advertisement video generation aims to create promotional videos that preserve fine-grained product identity while presenting selling points through coherent multi-shot narratives.
Image & Video Generation · Oct 7, 2026 · Claire Radar
An essential control of AI agents is their ability to manage runtime.
Tool Use & Function Calling · Oct 7, 2026 · Claire Radar
Goal-oriented Vision-and-Language Navigation (VLN) requires agents to locate and reach targets described in natural language without prescribed routes.
Robotics & Embodied AI · Spatial & 3D Reasoning · Oct 7, 2026 · Claire Radar
Vision-language models increasingly reason through crops, regions, and tool-produced observations.
Image & Visual Reasoning · Tool Use & Function Calling · Oct 7, 2026 · Claire Radar
Urban diagnosis integrates heterogeneous observations to identify urban problems, localize affected areas, and investigate contributing factors, informing evidence-based urban planning and management.
Image & Visual Reasoning · Oct 7, 2026 · Claire Radar
Recent advances in reasoning-oriented large language models (LLMs) have motivated increasing evaluation of their ability to perform professional decision tasks.
Professional & Enterprise Work · Oct 7, 2026 · Claire Radar
Referring Expression Segmentation (RES) links natural-language descriptions to pixel-level object masks.
Image & Visual Reasoning · Oct 7, 2026 · Claire Radar
Evaluating agents by outcomes alone can obscure the capabilities that produce them.
Multi-Agent & Games · Oct 7, 2026 · Claire Radar
Eyeglass reflection removal is important across smartphone imaging, video conferencing, and other face-centric visual applications.
Image & Video Generation · Oct 7, 2026 · Claire Radar
Hallucinated information can propagate through multi-stage LLM systems and become part of the context for subsequent reasoning.
Factuality & Hallucination · Long Context · Oct 7, 2026 · Claire Radar
An LLM safety patch can pass a benchmark while still being a poor repair.
Cybersecurity & CTF · Safety & Alignment · Oct 7, 2026 · Claire Radar
Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model systems.
Safety & Alignment · Agent Security & Prompt Injection · Oct 7, 2026 · Claire Radar
Rapid progress in AI agents has brought growing attention to agent safety, with extensive evaluation focused on digital environments.
Agent Security & Prompt Injection · Safety & Alignment · Oct 7, 2026 · Claire Radar
Recent advances in multimodal foundation models have made them capable generalist physical agents for a range of manipulation tasks.
Robotics & Embodied AI · Spatial & 3D Reasoning · Oct 7, 2026 · Claire Radar
General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carry into the physical world.
Robotics & Embodied AI · World Models & Physical Understanding · Oct 7, 2026 · Claire Radar
Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modalities.
Speech & Audio · Image & Visual Reasoning · Spatial & 3D Reasoning · Oct 7, 2026 · Claire Radar
Refereeing decisions in professional soccer remain inconsistent because referees cannot easily compare a contentious foul against similar past cases.
Video Understanding · Deep Research & Search · Oct 7, 2026 · Claire Radar
Reinforcement learning environments are now a primary lever for improving large language model (LLM) capabilities in post-training, yet most agentic benchmarks remain static: the world moves only when the agent acts, the reward is a termin…
Agentic Coding & SWE · Tool Use & Function Calling · Oct 7, 2026 · Claire Radar
A key challenge for multimodal large language models (MLLMs) is moving beyond visual recognition to constraint-aware cross-modal reasoning.
Image & Visual Reasoning · Oct 7, 2026 · Claire Radar
We evaluate whether AI agents can match the reported findings of published biomedical studies using public data.
Science · Oct 6, 2026 · Claire Radar
Static-analysis checker synthesis requires agents to interpret a defect specification, inspect a repository, implement analyzer-specific logic, and refine the checker through repeated compilation and analysis feedback.
Agentic Coding & SWE · Cybersecurity & CTF · Oct 6, 2026 · Claire Radar
Conversational MLLM agents are increasingly expected to assist in professional workflows, from AI research and engineering design to product management and business operations.
Agent Memory & Personalization · Oct 6, 2026 · Claire Radar
As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their reliable deployment.
Safety & Alignment · Oct 6, 2026 · Claire Radar
Coding agent harnesses mediate tool use and authorize actions, yet their security mechanisms and runtime effects remain incompletely characterized.
Agent Security & Prompt Injection · Agentic Coding & SWE · Oct 6, 2026 · Claire Radar
Learning from experience is essential for LLM agents to adapt to unfamiliar and dynmaic environments.
Agent Skills & Self-Improvement · Oct 6, 2026 · Claire Radar
World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult t…
World Models & Physical Understanding · Robotics & Embodied AI · Oct 6, 2026 · Claire Radar
As coding agents increasingly undertake real-world work autonomously, judging whether their risk treatments are warranted has become important.
Agent Security & Prompt Injection · Oct 6, 2026 · Claire Radar
Retrieval-Augmented Generation (RAG) systems are vulnerable to prompt-injection attacks embedded in retrieved content.
Agent Security & Prompt Injection · Deep Research & Search · Oct 6, 2026 · Claire Radar
Modern video models can generate realistic videos from real appearance references and proxy renders that specify scene structure, viewpoint changes, and motion.
Image & Video Generation · World Models & Physical Understanding · Oct 6, 2026 · Claire Radar
The rapid progress of LLM-based multi-agent systems (MAS) has shown that they largely outperform single agents on coding, math, and QA tasks, where executable tests provide a binary success signal.
Multi-Agent & Games · Oct 6, 2026 · Claire Radar
Frontier LLM agents have been shown to be capable of solving increasingly complex tasks for which humans have measurable solutions.
Agentic Coding & SWE · Code Generation · Oct 6, 2026 · Claire Radar
LLM-based agents are increasingly advancing scientific and engineering problem solving, with physics simulation emerging as a challenging yet practical testbed for reproducing complex physical phenomena with application in embodied AI, gam…
World Models & Physical Understanding · Science · Oct 6, 2026 · Claire Radar
Multilingual LLM benchmarks omit Slovak, a morphologically rich West Slavic language of five million speakers, or cover it only by machine translation.
Multilingual & Translation · Oct 6, 2026 · Claire Radar
Field-effect transistor (FET) layouts on exfoliated two-dimensional flakes are typically drawn by hand for each flake, placing contacts and gates to match its position and outline in optical micrographs.
Engineering & Hardware Design · Image & Visual Reasoning · Oct 5, 2026 · Claire Radar
Generalization in multi-arm collaboration can be studied as composing familiar atomic skills in new ways across arms.
Robotics & Embodied AI · Multi-Agent & Games · Oct 5, 2026 · Claire Radar
As agents take on long-horizon tasks, users shift from making individual decisions to overseeing autonomous execution.
Agent Skills & Self-Improvement · Oct 5, 2026 · Claire Radar
In decentralized consumer-to-consumer (C2C) marketplaces, people list goods, negotiate with strangers, and rate one another, so trust rests on reputation.
Multi-Agent & Games · Oct 5, 2026 · Claire Radar
Brain MRI interpretation is a longitudinal clinical reasoning problem: radiologists compare serial studies, integrate information across MRI sequences, localize findings within volumetric anatomy, and translate this evidence into report-gr…
Health & Medicine · Image & Visual Reasoning · Oct 5, 2026 · Claire Radar
AI voice assistants often use Automatic Speech Recognition (ASR) with LLM-based reasoning, yet existing systems struggle with regional British accents, including Scottish, Irish, and Welsh accents, since most ASR models are trained predomi…
Speech & Audio · Oct 5, 2026 · Claire Radar
Foundation models (FMs) are increasingly being developed for general time series and physiological signals, yet their transferability to downstream physiological tasks remains poorly understood.
Health & Medicine · Oct 5, 2026 · Claire Radar
Remote sensing foundation models (RSFMs) are commonly evaluated using aggregate metrics, which can hide systematic performance disparities across ecological regions.
Safety & Alignment · Oct 5, 2026 · Claire Radar
Audio benchmarks are built around short, pre-segmented clips, limiting model design to brief inputs or fixed vocabularies.
Speech & Audio · Oct 5, 2026 · Claire Radar
Vision-language models (VLMs) can detect that an object has rotated across views, but cannot reliably tell by how much.
Spatial & 3D Reasoning · Image & Visual Reasoning · Oct 5, 2026 · Claire Radar
LLM-driven evolution enables iterative model development, but two practical goals remain underexplored: finding model designs that transfer across related tasks and sustaining improvement when training is expensive.
AI R&D & ML Engineering · Oct 5, 2026 · Claire Radar
We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models.
AI R&D & ML Engineering · Oct 5, 2026 · Claire Radar
As Large Language Models (LLMs) gain tool use capabilities such as web search, they can retrieve and cross-reference public information, creating privacy risks beyond memorization.
Agent Security & Prompt Injection · Oct 4, 2026 · Claire Radar
Cold-start drug-drug interaction (DDI) prediction tests whether models can identify clinically significant interactions for drugs without training-time interaction history.
Health & Medicine · Science · Oct 4, 2026 · Claire Radar
AI agents that send emails, edit files, and make purchases must decide when to act on their own and when to check with the user first.
Tool Use & Function Calling · Agent Security & Prompt Injection · Oct 4, 2026 · Claire Radar
Lifelong physical agents must reason over extended interactions where past events continue to shape the world long after they disappear from view.
Robotics & Embodied AI · World Models & Physical Understanding · Oct 4, 2026 · Claire Radar
Large language models (LLMs) have demonstrated strong performance on coding and reasoning benchmarks; however, their ability to solve graph-structured machine learning problems remains largely unexplored.
Code Generation · AI R&D & ML Engineering · Oct 4, 2026 · Claire Radar
Agentic machine learning engineering (MLE) is an emerging AI4SE application for complex ML tasks and a step toward recursive self-improvement of AI systems.
AI R&D & ML Engineering · Oct 4, 2026 · Claire Radar
Autonomous research seeks sustained model improvements through iterative experimentation and feedback.
AI R&D & ML Engineering · Oct 4, 2026 · Claire Radar
Portrait aesthetic assessment assigns comparable scores according to how effectively human-centered images fulfill their photographic intent.
Image & Visual Reasoning · Chat, Writing & Preference · Oct 4, 2026 · Claire Radar
Although the advancement of vision-language models (VLMs) has endowed robots with enhanced environmental understanding and task reasoning, a comprehensive evaluation methodology is important to advance the integration of VLMs in robotic na…
Robotics & Embodied AI · Spatial & 3D Reasoning · Oct 4, 2026 · Claire Radar
Tool-using AI agents are increasingly deployed across enterprise software systems, yet widely used benchmarks primarily evaluate nominal task completion, conflating baseline planning competence with operational fault recovery.
Tool Use & Function Calling · Agent Security & Prompt Injection · Oct 4, 2026 · Claire Radar
Large Language Models (LLMs) are increasingly applied in hardware design automation, demonstrating strong potential in generating and understanding hardware description languages.
Engineering & Hardware Design · Code Generation · Oct 4, 2026 · Claire Radar
We introduce AgentPersonaBench (APB), a benchmark evaluating whether persona conditioning faithfully steers downstream agent behavior.
Agent Memory & Personalization · Oct 3, 2026 · Claire Radar
Reliable language models should resist unsupported user pressure while effectively using objective contextual information.
Safety & Alignment · Instruction Following · Oct 3, 2026 · Claire Radar
Vision-language models (VLMs) are increasingly evaluated for egocentric and cross-view video reasoning, yet existing benchmarks largely focus on semantic event understanding, temporal relations, or correspondence between already observed v…
Video Understanding · Spatial & 3D Reasoning · Oct 3, 2026 · Claire Radar
Agents built on large language models (LLMs) build and run software through tool calls.
Agentic Coding & SWE · Tool Use & Function Calling · Oct 3, 2026 · Claire Radar
Large language models (LLMs) have progressively evolved into the core of autonomous agents.
Multi-Agent & Games · Oct 3, 2026 · Claire Radar
Graphical consoles offer a practical interface for medical acquisition assistance, allowing agents to work through the controls and visual feedback used by human operators.
Computer Use & GUI Agents · Health & Medicine · Oct 3, 2026 · Claire Radar
Retrieval-Augmented Generation (RAG) is typically evaluated under the implicit assumption that the underlying knowledge base (KB) is clean, leaving the behaviour of RAG systems under realistic KB degradation poorly characterised.
Deep Research & Search · Oct 3, 2026 · Claire Radar
Generating an executable program does not necessarily mean that it correctly implements the behavioral requirements specified in natural language.
Code Generation · Oct 3, 2026 · Claire Radar
Document databases such as MongoDB are core infrastructure for modern applications, and natural-language interfaces to them---text-to-MQL---would let non-experts query complex, semi-structured data without mastering the query language.
Code Generation · Oct 2, 2026 · Claire Radar
GPU kernels generated by large language model (LLM) agents can remain less efficient than expert implementations, but runtime alone does not reveal how the gap relates to design discovery and implementation.
AI R&D & ML Engineering · Oct 2, 2026 · Claire Radar
As dexterous hands proliferate, collecting data and training policies separately for every morphology becomes increasingly impractical.
Robotics & Embodied AI · Oct 2, 2026 · Claire Radar
Long-term agents must remember not only what is true about a user, but also how a particular agent should work with that user as their shared history evolves.
Agent Memory & Personalization · Oct 2, 2026 · Claire Radar
Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses itself.
World Models & Physical Understanding · Video Understanding · Oct 2, 2026 · Claire Radar
Recently released benchmarks, ranked by the attention they are receiving now: GitHub stars, Hugging Face paper upvotes and Hugging Face dataset downloads over the last 30 days.
Which difficult benchmarks have been tested most
Each mark is one benchmark record. Height counts distinct models with numeric scores. Source model IDs preserve evaluated configurations; repeated score rows add no models. Source documents counts distinct cited reports or registry pages through the same evidence collection. Both counts appear on hover.
The side view projects reported scores and the selected counts onto the left wall, leaving out time. The gold staircase compares only benchmarks with verified score scales. Within the dated 2024+ cohort, a benchmark is Pareto when no other eligible benchmark has an equal or lower normalized score and an equal or greater selected count, with at least one strict advantage. Dates select the cohort; they do not determine dominance. Scores and counts are never multiplied. The score slice cannot promote a dominated benchmark onto the frontier.
Height uses log1p(count); ticks, tooltips and Pareto use raw counts. The axis covers the full scored 2024+ cohort and stays fixed while filtering. Only declared percentage metrics with a known selected count enter Pareto; lower-is-better percentages become 100 minus the original value. Hollow points flag an unverified score scale or an unknown count, which is never treated as zero. Overlapping caps are spaced apart; hover or focus traces their exact coordinates. Arrow keys move through benchmarks. A shared scale does not establish equivalent test protocols or prove a benchmark is solved.
The timeline starts on January 1, 2024, inclusive. Dates use the benchmark's release first, then its earliest numeric LLM score report. When a source dates score entries by model release, the earliest entry supplies a clearly labelled model-release proxy; this is not a verified score-publication date. Crawl timestamps and adoption-only mentions never supply dates. Known dates before 2024 are excluded. Records with no date remain individually visible in a labelled area.
No scored benchmarks match these filters.
Each source document counts once per benchmark record. Repeated scores or mentions in the same document add no citations. This table covers all sources.
Source documents from every registry. Expand a document to inspect the benchmarks it records and open the original evidence.
Big picture
See what shows up most often and where it came from. Open the connections view when you want to look at a specific item.
There are a lot of dots here. Pick one to see what it connects to, or open the matching results.
Scores over time
Every value that could be read verbatim from a cited document, placed at the date that document was published rather than at any evaluation date. The line directly links actual observations that set a new reported record among the values shown; it neither holds a score between reports nor extends past the final record. Test versions and run conditions can differ, so this is a reported-record path, not a like-for-like trend. When no newer number could be read, the gap is marked rather than drawn through. Whether a benchmark has saturated stays a reading you make, not a score this panel prints.
Category tags overlap. Each bar is an independent count, not a part of a stacked total.
Releases only excludes records re-announced as an update to something already surfaced.
Source mix counts ranked evidence after scoring. Fetch health counts raw records returned before scoring, so a source can be ok and still empty.
| Date | Coverage (UTC) | Evidence | Source mix | Categories | Events | Attention | Fetch health |
|---|
Dashboard unavailable
Try refreshing, or inspect the latest daily Issue while the dashboard rebuilds.
Open daily Issues ↗