Copiloting Your Developer Journey With Azure AI

Keynote for the #CommunityDays NYC Conference on Jun 21.

Abstract
Copilot-ing Your Developer Journey With Azure AI
By now you've probably heard of Generative AI. But do you know why Responsible AI is needed? Or why Agentic AI is the next frontier? Why are you engineering prompts & fine-tuning models but need a RAG for your data? Let's enter the fascinating world of modern AI development and go from prompts to production with Azure and friends! Along the way, we'll learn useful tools & explore open-source communities. We may even make an AI comic or play a round of Prompt Charades! Bring your curiosity & creativity - and build your Copilot IQ!

Speaker Bio
Nitya Narasimhan is a PhD and Polyglot with 25+ years of experience in software research, engineering and advocacy across industry, startups and academia. Her interests span distributed systems, mobile & web development, cloud and AI. She is currently a member of the AI Advocacy team at Microsoft where she works on empowering the application developer ecosystem to build intelligent apps with Azure and AI. Follow her tech adventures @nitya and her visual storytelling journeys @SketchTheDocs.

Transcript

1. Community Days NYC 2024 Keynote Copiloting Your Developer Journey with Azure AI Nitya Narasimhan, PhD Senior AI Advocate, Microsoft @nitya | #in/nityan
2. Hi, I’m Nitya! PhD & Polyglot (Mobile · Web · Cloud · AI) Research & Development (25+ years) Community & Education (15+ years) Illustrator & Storyteller (10+ years) Cloud Advocacy & Microsoft (5+ years) @nitya | #in/nityan | github/nitya
3. I’m Developer 0 For Azure AI Product Feedback (Focus on Developer Experience) Content Creation (Focus on Developer Education) Sample Creation (Focus on Developer Productivity) Community Interactions (Focus on Developers) @nitya | #in/nityan | github/nitya Join the Azure AI Discord
4. How do you see yourself? AI Beginner (How to I skill up on AI fundamentals!) Low-Code Dev (How do I get productive with AI?) Pro-Dev (How can I get control over e2e workflows?) Entrepreneur (How can I prototype my idea quickly?) We’re All On A Journey! @nitya | #in/nityan | github/nitya
5. Keeping up with Tech is .. Challenging https://aka.ms/bookofnews
6. Can I help you to see the big picture? @SketchTheDocs
7. I love to share my insights in Generative AI Responsible AI Agentic AI Careers
8. I love to build & empower local communities .. https://meetup.com/aide-hub
9. Join me on my AI Adventure! Copiloting Your Developer Journey With Azure AI 1 | Generative AI – Prompt Engineering 2 | LLM Ops– From Prompt To Production 3 | Copilot Stack - Developer Tools & Workflow 4 | Contoso Chat – Retail Copilot OSS Sample 5 | Responsible AI – From Evaluation to Safety| 6 | Agentic AI – From Assistants to AutoGen 7 | Hugging Face – From Open Source to Ops 8 | Resources – Discord, Collections & More
10. Generative AI = How Prompts Work PROMPT user request received as a text input Open AI GPT-3.5-turbo RESPONSE generated content returned as user response LANGUAGE MODEL 4097 Tokens Trained to Sep 2021 | Demo – Open AI Tokenizer
11. Generative AI = What Prompts Do PROMPT user request received as a text input Open AI GPT-3.5-turbo RESPONSE generated content returned as user response LANGUAGE MODEL 4097 Tokens Trained to Sep 2021 | Demo – Hugging Chat (Elementary)
12. Generative AI = What Prompts Do PROMPT user request received as a text input Open AI GPT-3.5-turbo RESPONSE generated content returned as user response LANGUAGE MODEL 4097 Tokens Trained to Sep 2021 | Demo – Hugging Spaces (AI Comics)
13. Prompt Engineering = Make it Better! Large Language Model Model Parameters System Context USER PROMPT CHAT HISTORY MODEL PROMPT MODEL RESPONSE PROMPT ENGINEERING VARIABLES RAG DATA PROMPT TEMPLATE | Demo – Open AI (Fine Tuning)
14. Generative AI For Beginners – Chapter 4 | https://aka.ms/genai-beginners
15. Generative AI For Beginners – Chapter 18 | https://aka.ms/genai-beginners
16. Challenge: Grounding Responses In My Data Large Language Model Model Parameters System Context USER PROMPT CHAT HISTORY MODEL PROMPT MODEL RESPONSE PROMPT ENGINEERING VARIABLES RAG DATA PROMPT TEMPLATE | Demo – Hugging Chat (“Hiking Shoes”)
17. Copilot Stack: Microsoft Runs on Azure AI | Demo – Azure AI Models Hub
18. Contoso Chat: A RAG-based Retail Copilot Contoso Web: Frontend A web app for user interactions. It is a client for the chat AI API. Contoso Chat: Backend A generative AI app that takes user query & returns response
19. Experience: Grounded, Personalized, Responsible AI • Grounded in Your Data. Ask a question, verify that answer uses product data. • Validate the Copilot. Ask a question and verify that you get an answer. • Personalized to User. Ask a question, verify if answer uses purchase history. • Default Safety Checks. Try simple jailbreaking to modify rules. It should fail. | Demo – Contoso Chat (“Chat UI”)
20. Architecture: Retrieval Augmented Generation
21. App Lifecycle E2E: Prompt Engineering to LLM Ops
22. Code-First Platform: Azure AI Studio (SDK & Portal)
23. Workshop: { Build – Evaluate – Deploy – Use }
24. | Try It Out!  Pre-Reqs: GitHub, Azure, AOAI, Quota  Visit - https://aka.ms/aitour/contoso-chat  Fork it – uncheck box so you get branches  Launch Codespaces – wait till it is ready  Validate setup – azd version  Authenticate – azd auth login  Provision & Deploy – azd up
25. https://prompty.ai  “Agency with Observability”  New asset class – versionable, actionable  Specification – language agnostic templates  Tooling – developer experience (IDE as playground)  Runtime – engine agnostic execution “It just works” with Azure AI Studio | Demo – Prompty Extension (“Playground In Your IDE”)
26. | Demo – Flex Flow (“Bring Your Own Framework or Code”) https://aka.ms/promptflow
27. https://aka.ms/ai-studio/intelligent-apps | 5-part Blog Series Responsible AI: Evaluation Flows & Safety Filters
28. | Demo – AZD Template Gallery https://aka.ms/ai-studio/azd-templates
29. If Time Permits 1 | Generative AI – Prompt Engineering 2 | LLM Ops– From Prompt To Production ( 3 | Copilot Stack - Developer Tools & Workflow 4 | Contoso Chat – Retail Copilot OSS Sample 5 | Responsible AI – From Evaluation to Safety| 6 | Agentic AI – From Assistants to AutoGen 7 | Hugging Face – From Open Source to Ops 8 | Resources – Discord, Collections & More See Microsoft Build Session For More Details
30. Building Generative AI Solutions Can be .. Challenging Large Language Models for rich multi-media content generation Conversational Interactions driven by natural language processing capabilities End-to-End Workflows from prompt-engineering to operationalization Chat-based experiences require human guidance (prompts) and intervention (instructions) to execute complex tasks, creating challenges in workflow automation and optimization
31. Agentic AI Application Development Can Help Autonomous Agents capable of planning & executing decisions Task Execution Tools identify and execute the right tools for each task Conversational Workflows coordinate actions across agents, user, environment Agentic AI Applications use autonomous agents to execute tasks on behalf of users, interacting with their environment or remote services as needed, and coordinating actions with other agents for efficiency
32. Building Agentic AI requires new tools & mindset Conversational Agents configurable (LLM) customizable (prompt) chainable (planning) Execution Workflows infer required skill execute relevant tool behave responsibly Conversational Patterns agent-human agent-environment agent-agent Traditional Agentic Application Frameworks are not versed in natural language. We need conversational design patterns and intelligent task inference and execution for generative AI usage scenarios
33. AutoGen Framework: Multi-Agent Collaboration Open-Source Framework & Samples Docs: https://aka.ms/autogen/website Discord: https://aka.ms/autogen/discord Customizable Conversable Agents, LLMs Research-Driven Tools & Patterns No-Code and Code-First Development
34. AutoGen Studio: Get Started With Low-Code UI/UX Define Skills Create reusable functions, tools Docs: https://microsoft.github.io/autogen/blog/2023/12/01/AutoGenStudio/ Define Models Define & configure required LLMs Define Workflows Create agents, multi-agent conversations Create Sessions Test and validate agent workflows Define Agents Configure LLM, skills, behaviors Publish Sessions Share sessions to a gallery to revisit
35. Your Turn! 1 | Generative AI – Prompt Engineering 2 | LLM Ops– From Prompt To Production ( 3 | Copilot Stack - Developer Tools & Workflow 4 | Contoso Chat – Retail Copilot OSS Sample 5 | Responsible AI – From Evaluation to Safety| 6 | Agentic AI – From Assistants to AutoGen 7 | Hugging Face – From Open Source to Ops 8 | Resources – Discord, Collections & More Look For Slides Soon!
36. Join The Discord 1 | https://aka.ms/genai-beginners 2 | https://aka.ms/ai-studio/collection 3 | https://aka.ms/rai-hub/collection 4 | https://aka.ms/aitour/contoso-chat 5 | https://aka.ms/ai-studio/intelligent-apps 6 | https://aka.ms/ai-studio/code-first-blog 7 | https://aka.ms/ai-studio/azd-templates 8 | Find Me – @nitya #in/nityan github/nitya Look For Slides Soon!
37. © Copyright Microsoft Corporation. All rights reserved. Thank You!