AI Show and Tell - New York

From Intent to Enforcement: Open-Source Evaluation and Governance for AI Agents

AI agents are taking on increasingly complex tasks, but evaluation and governance are still often fragmented across disconnected tools and platform-specific controls. In this talk, we introduce two open-source tools that connect testing to runtime enforcement. ASSERT turns natural-language expectations about what an agent should and should not do into executable, scenario-specific evaluations. ACS (Agent Control Specification) carries those expectations into production as portable policies that can be enforced across key points in an agent’s execution. Together, they create a continuous governance loop: define intended behavior → test it → identify failures → turn findings into controls → enforce them at runtime → evaluate again. The result is a practical approach to governing agents across models, frameworks, and deployment environments without locking developers into a single platform.

by Mehrnoosh Sameki

Your New Teammates: Architecting a Multi-Agent Workforce

hamba.nl is a live Dutch travel magazine built with the GitHub Copilot App and run by seven Foundry-hosted agents, orchestrated with Agent Framework, covering the full editorial pipeline from draft to published article.

In this session I'll take you under the hood: how the agents are scoped and orchestrated, how they connect to tools and data through MCP, how content, moderation, and fact-checking roles hand off to each other, and how a human-in-the-loop stays in control of final approval. This is a working system with real failure modes, and you'll leave with a concrete reference architecture for agent teams that run a function, not just generate code.

by Henk Boelman

Your Product Was Built for Humans. Is It Ready for Agents? 

What happens when coding agents become users of your product? A product can work beautifully for humans while its documentation, SDK guidance, examples, and tools leave agents generating code that looks right but fails when it runs.

Over three months, the SCOPE and Azure Cosmos DB teams tested eight AI-generated applications across six technology stacks. SCOPE provided repeatable runtime evidence about how coding agents experienced Cosmos DB's product surfaces. Cosmos DB engineers traced failures back to broken SDK usage, expensive queries, unsafe data patterns, outdated examples, and gaps in Agent Kit rules.

In eight weeks, the findings led to 33 Agent Kit rule changes, including four new rules, and resolved 51 issues across 45 documentation pages. A tight collaboration between the teams accelerated the path from evidence to fixes, while feedback from Cosmos DB also made SCOPE's evaluation process more rigorous.

Cedric Vidal will introduce why SCOPE was built to evaluate agentic product surfaces. Jay Gordon will share how Cosmos DB used it to uncover and address actionable product gaps. Along the way, they will show how their collaboration improved the evaluation and remediation process, following real failures from initial finding to product fix and then returning to the same scenarios to see which changes actually worked.

by Jay Gordon & Cedric Vidal