Evaluating AI Models with Microsoft Foundry | MVP Unplugged

From the creator

Welcome to the next MVP Unplugged, where Microsoft MVPs share real-world projects and insights from the field! In this episode, host Justin Garrett sits down with Microsoft MVP Veronika Kolesnikova to explore how she picks the right AI model for common developer tasks using Evaluations in Microsoft Foundry, complete with query-context-results, Python SDK workflows, and measures for evaluation. We'll explore some fun datasets around hiking, acrobatics, and common developer terminology too! Veronika walks through her full process—from generating datasets with GitHub Copilot, to running multi-model evaluations, to analyzing outputs in the Microsoft Foundry portal. Whether you’re building agents, experimenting with models, or ensuring AI reliability at scale, this episode breaks down a repeatable and practical approach you can use today. In this episode, you’ll learn ✅How to build custom evaluation datasets using AI ✅How to compare outputs across models like GPT‑5, Grok-4, and Claude Sonnet 4.5 ✅How to run Evaluations programmatically using the new Microsoft Foundry SDK ✅How to measure AI performance using F1, METEOR, similarity scores, and thresholds ✅Tips for choosing the right model for your AI agent ✅Practical debugging and iteration strategies for model quality ✅How to store and version evaluation datasets in Microsoft Foundry 🔗 Chapter Markers 00:00 – Intro to MVP Unplugged 00:20 – Meet Microsoft MVP Veronika Kolesnikova 02:20 – Introducing Microsoft Foundry Evaluations

Published 2026-01-27 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.