Global AI Night

From Intent to Enforcement: Open-Source Evaluation and Governance for AI Agents

AI agents are increasingly entrusted with complex tasks, sensitive data, and real-world actions...but most teams still evaluate and govern them using disconnected tools, generic benchmarks, and controls tied to a single platform or framework. In this talk, we introduce two open-source tools: ASSERT and the Agent Control Specification (ACS).

• ASSERT turns natural-language requirements )what an agent should and should not do) into executable, scenario-specific evaluations. It helps developers systematically generate tests, evaluate multi-turn agent behavior, identify failure modes, and produce evidence grounded in the agent’s intended purpose rather than generic benchmark scores.
• ACS carries those requirements into production. It provides an open specification for attaching policies to critical intervention points across an agent’s execution, including user input, model interaction, tool calls, and final output. This enables policy engines, classifiers, deterministic rules, and LLM-based judges to produce normalized decisions that agent platforms can consistently enforce.

Together, ASSERT and ACS establish a continuous governance loop: define intended behavior, evaluate whether the agent satisfies it, discover new risks through testing and red teaming, translate findings into runtime policies, and continuously strengthen those policies using production evidence. Attendees will learn how specification-driven evaluation and portable runtime controls can make governance part of the agent development lifecycle, without locking developers into a particular model, framework, or deployment platform.