Benchmark Evasion, Verifiable Workflows, and The Real-World QA Push

News

METR Publishes New Agent Evaluations Including First Public Sandbagging Tests

METR just published a major evaluation that includes the first public tests for agent 'sandbagging,' where agents strategically underperform on benchmarks. This is critical intel because it shows how easily current evaluations can be gamed, making catastrophic risk assessments unreliable.

Tools

Composio Launches New Multi-Agent Workflow Features and Integrations

Composio made a push with new multi-agent workflow demos and tool integrations. It's a direct push toward making the 'agent as a service' layer more composable and production-ready for teams that don't want to build all the glue code themselves.

Research

Collab: Structured Task Management for Long-Horizon AI Agents

This paper from arXiv introduces 'Collab,' a framework for managing long-horizon agent tasks by breaking them into verifiable subtasks with explicit dependency tracking. It's a direct architectural response to the problem of agents losing the plot on complex, multi-step goals.

Towards Verifiable Transparency and Explainability in Agentic AI Systems

The paper tackles a key pain point in real-world deployment: how to make agents' cognition and actions auditable for compliance and debugging. It proposes methods for structured explanation, which is less about safety rhetoric and more about the practical plumbing needed for enterprise trust.

Analysis

Evaluating AI-Powered Agents for Automated Accessibility Testing

This arXiv paper highlights a concrete application bottleneck: using agents to test accessibility. It's a strong example of moving agents beyond coding and data analysis into specialized QA roles where their value is measured by real-world compliance and usability outcomes.