Daily AI benchmark brief: 2026-10-12
MultiAgentInject is a new security benchmark with 4,050 trials that independently varies task, agent-network topology, attack class, stealth, and…
What changed in AI evaluation, and why it matters
Daily AI benchmark brief: 2026-10-11
The new Two-Stage Measurement of Update Adherence benchmark tests whether a memory-enabled language-model agent follows a revised requirement rather than…
Daily AI benchmark brief: 2026-10-10
The new milipoint-runs artifact exposes split leakage in the MiliPoint radar benchmark: overlapping windows let most test examples share frames with…
Daily AI benchmark brief: 2026-10-09
TRACE is a new diagnostic protocol for checking whether an agent’s benchmark score changed because its behavior changed or because the evaluator changed…
Daily AI benchmark brief: 2026-10-08
A new runtime-verification study exposes a data-design gap in existing agent-safety benchmarks. It replayed AgentDojo and STAC trajectories through formal…
Daily AI benchmark brief: 2026-10-07
The new Agent Network Traffic benchmark (ANT) tests whether security teams can audit large language model agents through network traffic without…
Daily AI benchmark brief: 2026-10-05
The new “Passing the Test You Trained On” study evaluates 15 prompt-injection detectors on benign and injected outputs reconstructed from real agent…
Daily AI benchmark brief: 2026-10-04
The new 2026 Cryo-EM Benchmark Dataset provides 12 protein structures deposited after every test set disclosed for MICA, an artificial intelligence system…
Daily AI benchmark brief: 2026-10-03
The updated retinal benchmark audit provides image-level manifests for patient-grouped, source-matched, and leave-one-source-out evaluations, plus…
Daily AI benchmark brief: 2026-10-02
Video-Index is a new meta-benchmark that audits whether video tests actually require video understanding. Its five-level “attack pyramid” tests…
Daily AI benchmark brief: 2026-10-01
The new study “Dating the Model” reports that hidden current-date text injected into system prompts changed measured performance across nine large…
Daily AI benchmark brief: 2026-09-30
The newly released UserProxyBench tests a hidden dependency in interactive agent benchmarks: whether the language model playing the user follows its…
Daily AI benchmark brief: 2026-09-29
TraceDance is a new release that turns real agent deployment traces into targeted benchmarks for user-specified undesirable behaviors. It evaluates the…
Daily AI benchmark brief: 2026-09-28
The new Finance Agents Benchmark packages financial due diligence as an end-to-end agent test: 50 tasks, 160 documents, 231 grading criteria, an execution…
Daily AI benchmark brief: 2026-09-27
The new Kalyvox Voice Benchmark 2026 evaluates one voice agent through 240 controlled French and English calls across 12 scenario families. It reports…
Daily AI benchmark brief: 2026-09-26
The new repo2bench release converts Python repositories into coding-agent tasks mined from version history or created by removing functions. It admits…
Daily AI benchmark brief: 2026-09-25
The new PartHackBench release tests whether partial-credit scorers for long-running tool agents reward misleading progress. It compares honest and…
Daily AI benchmark brief: 2026-09-24
The new Uncheatable Eval benchmark evaluates base language models by measuring how efficiently they losslessly compress newly published text that is…
Daily AI benchmark brief: 2026-09-23
A new B3DB audit tests whether stereoisomers inflate blood–brain barrier prediction scores because common two-dimensional molecular representations map…
Daily AI benchmark brief: 2026-09-22
The new Curation-Bench release evaluates whether coding agents can improve training-data selection through an iterative workflow: agents inspect data…
Daily AI benchmark brief: 2026-09-20
The newly released TC-Slang Guardrail Benchmark evaluates Traditional Chinese prompt-injection defenses by slang year and pairs each slang example with a…
Daily AI benchmark brief: 2026-09-18
The new SonoCorpus and SonoBase release pairs 456,963 ultrasound images and 1,626,085 expert segmentation masks with an interactive segmentation model…
Daily AI benchmark brief: 2026-09-17
The new lexEN benchmark replaces disputed word-sense labels with a conservative, human-adjudicated correction layer: 211 labels changed and 56 removed…
Daily AI benchmark brief: 2026-09-16
A new audit of 254 SWE-bench coding-agent submissions reports that the leaderboard cannot reliably order its leading entries: the top two resolve the same…
Daily AI benchmark brief: 2026-09-15
The new Sophea release evaluates a production Greek-English speech recognizer against nine simultaneous gates covering both languages, language…
Daily AI benchmark brief: 2026-09-14
The new KNOWS Benchmark evaluates web agents on 110 tasks that combine web research with producing documents, spreadsheets, and slide decks in Google…
Daily AI benchmark brief: 2026-09-13
The newly captured MusicAI background-music benchmark tests audio-capable large language models on 2,000 questions under clean audio, white noise, and 55…
Daily AI benchmark brief: 2026-09-12
The newly released Benchmark Radar combines a searchable catalog of AI evaluations with links to datasets and code, mentions in model cards and technical…
Daily AI benchmark brief: 2026-09-11
The new oncology visual question answering benchmark builds test items from private, single-institution radiology reports paired with three-dimensional…
Daily AI benchmark brief: 2026-09-10
The new “Double Measurement Confound” paper identifies a recurring agent-evaluation problem in this feed: a fixed software scaffold may make execution…