Model Mondays S1:E6 - Hands-on With Local AI Development

Join us for Model Mondays – a weekly show where we round up all the latest AI model news and put one model in the spotlight for a deeper dive.

In this episode, we’re joined by Rong Lu as we dive into the word of local AI development. We put the spotlight on AI Toolkit for Visual Studio Code - and discuss core features from model discovery, model playground usage, prompt builder, bulk run and evaluations - with hands-on demos. Walk away with a sense for how the AI Toolkit extension can streamline your AI development from local device to cloud.

Session Resources:
https://aka.ms/model-mondays
https://aka.ms/model-mondays/discord
https://aka.ms/model-mondays/chat

Explore The Models
https://aka.ms/model-mondays/collection

Watch The Replay
https://developer.microsoft.com/en-us/reactor/events/25357/

Transcript

1. Hands on with Local AI Development https://aka.ms/model-mondays/RSVP Apr 14, 2025 – 10:30am PT | 1:30pm ET Principal PM Manager, Microsoft Rong Lu Tooling
2. Senior AI Advocate Nitya Narasimhan, PhD Sharmila Chockalingam Senior Product Marketing Mgr. Welcome to Season 1: Model Mondays Build Your Model IQ https://aka.ms/model-mondays
3. Model Mondays Highlights 5 Things To Know This Week Sharmila Chockalingam
4. ❶ · Try Mistral OCR https://aka.ms/model-mondays Mistral OCR Transform unstructured content into actionable insights with unmatched speed, precision and multilingual versatility
5. ❷ · Try Llama4 models https://aka.ms/model-mondays Llama4 on MaaS New Llama4 models now available as a service on Azure AI Foundry model catalog
6. ❸ · Try DeepSeek-V3 https://aka.ms/model-mondays DeepSeek-V3-0324 Latest version of DeepSeek now available on Azure
7. ❹ · Try Knowledge Tools https://aka.ms/model-mondays New Knowledge Tools Azure AI Agent Service offers a wide range of knowledge tools to address various customer use cases.
8. ❺ · Read MIT Tech Review https://aka.ms/model-mondays MIT Technology Review Learn how hundreds of global technology leaders are customizing AI to drive business value
9. Learn how the AI Toolkit for Visual Studio Code enables you to download and run AI models locally, with cutting edge tools to support testing, fine-tuning & deployment. Hands on with Local AI Development Rong Lu, Principal PM Manager - Microsoft Tooling
10. AI Toolkit for VS Code Streamlines generative AI app development by integrating tools and models. Iterate fast locally. • Explore a wide range of industry- leading GenAI Models • Interactive model playground with attachment support • Craft and iterate on prompts • Bulk Run to curate test datasets • Evaluate the impact of model or prompt changes • Finetune models
11. Local Remote Model Catalog  Includes a wide range of industry-leading models.  Small and Large Language Model  Run Remotely or Locally  Stay up-to-date with state-of- the-art models. All GitHub Models 3rd Party Models (OpenAI, Anthropic, Google) Custom OpenAI Compatible Model Endpoints ONNX Models (Phi, Mistral, DeepSeek) Ollama Models Setup Sign in with GitHub ID Bring in Provider’s API Key Bring in model endpoint and API Key Download models locally Install Ollama and Download models locally Install NIM and Download models locally Nvidia NIM Models
12. Model Playground  A feature-rich model playground in VS Code  Explore local and remote models, including custom model endpoints  Switch between models with a single click  Provide context instruction and adjust model parameters  View token usage  Compare model performance side by side  Attach images or files  Option to enable web search  Chat history preserved locally Microsoft Confidential
13. Prompt Builder  Prompt Builder – craft and iterate on prompts quickly  AI auto-generate system prompts  Test prompts with different models within clicks  Supports structured outputs  Supports follow-up user and assistant message support  Attach images or files  Prompt history preserved locally  Get to code snippet for prototyping Microsoft Confidential
14. Bulk Run and Evaluation  Bulk Run helps curate eval datasets  AI auto-generated datasets using system prompts and user examples  Run against a variety of models  Fill in model responses in bulk  Import/export datasets  Evaluate the impact of model/prompts quickly  Support built-in metrics and custom metrics  Can use LLMs as a judge  View evaluation results  Evaluation run results are preserved locally Microsoft Confidential
15. Try the AI Toolkit! Give us feedback https://aka.ms/aitoolkit Microsoft Confidential Thank You Rong Lu, Principal PM Manager Microsoft
16. Hands-on with Open-Source + Models How can open-source accelerate model customization and AI application development? Join us to learn about Llama Stack! Sanyam Bhutani Partner Engineer - Meta Next Week
17. Join the #model-mondays channel https://aka.ms/model-Mondays/chat Q&A Office hours every Friday on Discord – bring questions, share your projects!
18. Watch the Livestream Join the Conversation Explore the Resources Mon 1:30pm ET Microsoft Reactor https://aka.ms/model-mondays/RSVP Fri 1:30pm ET Azure AI Discord https://aka.ms/model-mondays/Discord Watch or Star GitHub Repo https://aka.ms/model-monday #ModelMondays