MSBuild Lab 333 - Evaluate Reasoning Models For Your AI Apps

Sessions At Microsoft Build
https://build.microsoft.com/sessions/LAB333

GitHub Repo For Self-Guided Learning
https://github.com/microsoft/Build25-LAB333

Discussion Forum To #GetHelp
https://github.com/orgs/azure-ai-foundry/discussions

Abstract:
Advanced reasoning models excel at complex problem-solving tasks and nuanced analysis. But are they always the right fit for the task? Join us on a journey from catalog to cloud as we explore the capabilities and limitations of reasoning models using an enterprise catering application with complex scheduling and inventory constraints. Compare and contrast the reasoning approaches visually (with inference API) and qualitatively (with evaluators) and build your intuition for tradeoffs involved.

Transcript

1. Evaluate Reasoning Models For Your Generative AI Applications Gustavo Cordido AI Advcate II, Microsoft Nitya Narasimhan, PhD Senior AI Advocate, Microsoft #MSBuild 2025 | Lab 333
2. © Copyright Microsoft Corporation. All rights reserved.
3. Agenda  Welcome – Meet The Team  Overview – Introducing Reasoning Models  Getting Started – Launch Lab & Codespaces  Lab Outline – What You’ll Learn  Wrap-up – Teardown, Survey & Next Steps
4. Welcome – Introducing The Lab Team INSTRUCTOR Nitya Narasimhan, PhD Senior AI Advocate INSTRUCTOR Gustavo Cordido AI Advocate II Our Amazing Proctors Brian Benz Lino Tadros Treb Gatte
5. Azure AI Foundry Security • Identity • Management Foundry Models Foundry Agent Service Azure AI Search Foundry Observability Azure AI Services Azure Machine Learning Azure AI Content Safety Copilot Studio Visual Studio GitHub Foundry SDK Serverless Control Azure Kubernetes Service Azure Container Apps Azure App Service Azure Functions Cloud Azure Azure Arc Foundry Local Edge Your AI Engineer Journey Starts With Model Choice
6. Workshop Outline What Are Reasoning Models? What Will You Learn In this Lab
7. Hands-on With Reasoning Models What Are Reasoning Models? What Can Reasoning Models Do? What Are The Tradeoffs? Workshop – Explore Text-Based Reasoning Workshop – Explore Visual Reasoning Workshop – Explore Richer Capabilities Homework – Keep Exploring With Sandbox Survey & Teardown
8. Reasoning models are a new category of Large Language Models that are trained to think deeply before they respond. These reasoning abilities are achieved by a combination of techniques including • chain-of-thought • self-consistency • deliberative alignment. What are Reasoning Models?
9. Designed to tackle hard problems involving logic, strategy, complex reasoning and multi-step planning. Very effective in STEM. OpenAI’s latest o3 and o4-mini achieve state-of-the-art performance on a variety of STEM challenges (math, science, coding). What can reasoning models do?
10. Lab 2 / Foundry Models gpt-4o-mini o1 & o4-mini We will be using 1 GPT model and 2 Reasoning Models in This Lab
11. Lab 3 / GitHub Models Compare GPT with Reasoning Get intuitive sense for differences – token cost, response length & latency
12. Lab 4 / Manage Tokens Understand Reasoning Tokens Reasoning Tokens count against context window – “thinking” time and compute
13. Lab 5 / Prompt Guidance Explore new guidelines interactively Reasoning models need end-goals and persona – not step-by-step guidance
14. Lab 6 / Visual Reasoning Expand to new visual scenarios “Solve This” -- then “Create another puzzle like this”
15. Lab 7 / Code-First Usage Configure & Use Model API in Code Complete the exercises – then try your own ideas to build intuition on API
16. Lab 8 / Advanced usage Explore API features & scenarios Walk away with a repo that has notebooks for API-driven experimentation
17. Wrap-Up Teardown, Survey & Next Steps
18. Hands-on With Reasoning Models What Are Reasoning Models? What Can Reasoning Models Do? What Are The Tradeoffs? Workshop – Explore Text-Based Reasoning Workshop – Explore Visual Reasoning Workshop – Explore Richer Capabilities Homework – Keep Exploring With Sandbox Survey & Teardown
20. Call to Action Are You Ready To Create The Future Of AI? Explore Azure AI Foundry: ai.azure.com Download the SDK: aka.ms/aifoundrysdk Review documentation: aka.ms/AzureAI Take the Azure AI Learn Course: aka.ms/learnatbuild Read More About What’s New In Azure AI Foundry aka.ms/Build25/HeroBlog/Foundry Join our developer community channels:  Discord: aka.ms/ai/discord  Discussions: aka.ms/azureaifoundry/forum