What benchmarks, datasets, or evaluation methods did the radar first see today?
high confidenceNot enough evidence
Notable releases include OpenMAS-GApp for multi-agent application construction, MLIRBench for compiler reasoning, ExtremeDrive for critical driving, CataTQA for scientific-table question answering, Accelerate Bench for program optimization, EVIDENT for evidence-grounded GUI agents, and EconSafeBench for economic-agent safety. The feed separately discovered ReSpaceBench, CAE-Bench, DetourBench, SWE-CoWork, HospitalBench, and Omni-Workspace. These are examples rather than a complete inventory.
The captured feed added evaluations spanning agent construction, software and compiler work, driving, scientific tables, web evidence, economic safety, spatial memory, engineering solvers, workplace coding, hospitals, and mixed-media office tasks. It also surfaced statistical evaluation methods for agent reliability and benchmark uncertainty. Each artifact’s event label matters: some were recorded as releases, while others were merely discovered by the radar.
Takeaway: Today’s first observations show broad coverage, with agent evaluations especially visible among the highlighted records. Treat this as a map of what entered this keyword-filtered feed, not as a representative account of the field or proof that every artifact itself originated today. A complete artifact-level inventory was not supplied in the evidence packet.
Another reading: First observation by the radar is not equivalent to a new release. For example, SHAPE benchmark data carried a publication time before the capture day even though the feed classified its record as released. The packet also covers only selected evidence, so less prominent first observations may materially broaden or alter this summary.
- S022 artifacts first observed by the radar today: 310
Which of today's arrivals document how they score an answer?
high confidenceNot enough evidence
The clearest documented scoring arrivals are Accelerate Bench, which uses a separate final-submission verifier; VAHBench, which combines delivery, quality, reliability, and reporting checks; EVIDENT, which separates task completion from evidence-grounded success; SchedOps-Bench, which judges server end states; DetourBench, which compares trajectories with computable minimum-action bounds; SWE-CoWork, which uses private executable tests; and HospitalBench, which scores linked subtasks and diagnoses where a trajectory breaks.
Other concrete examples include the dettato benchmark, which publishes outputs and scoring scripts; the Gujarati speech leaderboard, which reports character and word error measures with uncertainty testing; and Ai_Evals, which combines expected-answer datasets, fixed checks, model-based judging, and pass-or-fail tests. These records explain more than a leaderboard result: they identify the checks, verifier, test procedure, or score components used to judge an answer.
Takeaway: For auditable evaluation work, prioritize arrivals that expose executable tests, scoring scripts, end-state checks, evidence requirements, or explicit trajectory bounds. The registry does not contain a computed count of all first-observed artifacts with documented scoring, and the selected packet is insufficient for an exhaustive list.
Another reading: Naming score dimensions is not the same as publishing a reproducible scoring rule. For example, VAHBench describes completeness, quality, reliability, and reporting checks, but the supplied summary does not expose their formulas or aggregation. Full papers, repositories, and evaluator code would be needed to determine whether every listed method can be independently reproduced.
- S022 artifacts first observed by the radar today: 310