MEETUP · SAN FRANCISCO
Agent evals night
Agents break in ways unit tests don't catch. They can pass the demo and still pick the wrong tool, misread an MCP server's schema, or lose track of what it learned three steps earlier.
- When
- Thu, Nov 12
6:30 PM to 9:00 PM PT - Where
- China Basin, San Francisco, CA
- Host
- Apify Events
- Entry
- Free to attend
About this event
Agents break in ways unit tests don't catch.
They can pass the demo and still pick the wrong tool, misread an MCP server's schema, or lose track of what it learned three steps earlier. Evals are how teams catch these failures before they reach production. The tooling is still young, so plenty of teams build their own harnesses, LLM judges, and trace reviews from scratch.
Join us for an evening of lightning talks and live demos from teams who run evals on agents every day. We'll cover why agents pick one product over another, how to test one MCP server across the different clients people connect it to, and how to measure what an agent learns in context.
Speakers
- Robert-George Crupa, AI Engineer at Apify, on how we evaluates 100k+ tools - Joshua Chung and Vivek Shukla, co-founders of Xulux, on evals for in-context learning - Ben Swerdlow, CEO of Freestyle, on how to eval sandboxed agents (in Minecraft) - Renzo Viale, CEO of Oqoqo, on why agents choose (or ignore) your product - Prathmesh Patel, CEO of MCPJam, on cross-client evals for MCP servers - Nick Gaspardone, Accound Executive at Braintrust, on active observability and evals for agents What to expect
- 10 minute lightning talks and live demos on agent evals - Food and drinks - Networking with AI engineers and agent builders Agenda
- 6:30 pm: Doors open, food and drinks - 7:00 pm: Lightning talks and live demos - 8:0...
Shooting Agent evals night?
We staff meetups across San Francisco with vetted, background-checked professionals. Our studio edits every frame and delivers in 48 to 72 hours, so your recap goes out while the room is still talking about it.
- Photo coverage for meetups from $200
- A vetted professional confirms within 24 hours
- Deposit is only charged once they confirm
- Backup professional on standby for every date
