What benchmarks, datasets, or evaluation methods did the radar first see today?
high confidenceNot enough evidence
The captured feed first observed a broad mix spanning agent behavior and safety, scientific vision, remote sensing, cybersecurity, education, and software engineering. Notable arrivals include AgentVQA, AgentPhishing, EpiSEM-Bench, DuoCook, WildDepth, CryoAtlas, LongRS-Bench, CellTrack-Bench, SecLog-Bench, and REGATTA. It also surfaced methods based on matched lifecycles, fault injection, specification audits, exact option-set grading, and structured rubric trees.
These arrivals test whether systems can understand agent situations, resist phishing, recover missing safety evidence, coordinate in games, process scientific images, reason over satellite changes, track cells, and interpret security logs. Several also change how evaluation is performed by injecting known faults, auditing hidden test requirements, or breaking complex reports into individually scored claims.
Takeaway: This is a selective overview of first observations in the captured keyword-filtered feed, not a complete inventory or evidence of field-wide novelty. The supplied packet covers only part of the first-observed registry, so the remaining artifact records would be needed for a comprehensive answer.
Another reading: First observed by this radar does not mean newly created or newly released worldwide. Some records were merely discovered today, while source outages and the packet’s partial selection could omit important artifacts or skew the apparent mix.
- S024 artifacts first observed by the radar today: 315
Which of today's arrivals document how they score an answer?
high confidenceNot enough evidence
The clearest documented examples are IFC-MCQ, with exact option-set grading; TerrierTA, with rubric-based short-answer grading against an instructor reference and human graders; the AI Professions prompt benchmark, with criterion-level verdicts; HeriKGQA, with typed reference values and a scorer; TutorCaseBench, with source-grounded rubrics; NASICON, with multiple judges and rescoring records; and FrontierCS, with development and held-out evaluation suites.
These artifacts describe more than a question set. They indicate what counts as correct, such as matching an allowed option set, satisfying rubric criteria, agreeing with typed reference values, or passing a hidden test suite. The Decision Index also documents a weighted composite and rank-tie rule rather than reporting an unexplained leaderboard score.
Takeaway: These are the strongest scoring-method examples visible in the supplied evidence, but the packet does not establish that every method is fully reproducible. Full dataset cards, judge prompts, rubrics, reference answers, and scorer code would be needed to verify all arrivals comprehensively.
Another reading: A scoring description is not necessarily an auditable scorer. NASICON mentions judges and rescoring outputs without supplying the complete judging procedure in this packet, while FrontierCS deliberately keeps its final evaluator and held-out suite private. Those designs limit independent verification despite documenting the scoring structure.
- S024 artifacts first observed by the radar today: 315