Microsoft Build 2024 Updates

S/N Ratio > Microsoft Build 2024 Updates (Machine Learning 15minutes! Hybrid #91) (2024/07/29)
https://satonaoki.wordpress.com/2024/07/29/ml15min-build-2024-updates/

Transcript

1. Microsoft Build 2024 Updates SATO Naoki (Neo) Senior Software Engineer Microsoft Corporation
2. The age of AI transformation
3. Scaling laws Training compute of notable machine learning systems over time (FLOP) 1e+2 1e+10 1e+18 1e+26 1950 2010 Today Deep learning era Pre-deep learning era 1.4x/year 4.1x/year Source: Epoch, ‘Parameter, Compute and Data Trends in Machine Learning’
5. Indian teachers are drafting better lesson plans India Switzerland Amid healthcare crunch, AI apps provide more time for patients Indonesia To keep fish and shrimp healthy, farmers now have a copilot to help aka.ms/BuildAIStories
7. Microsoft Copilot Copilot+ PC Copilot stack
9. Windows Copilot Runtime Applications + Experiences Windows Copilot Library + On-device models AI frameworks + Toolchains Silicon
10. Applications + Experiences AI frameworks + Toolchains Windows Copilot Library + On-device models Coming June Windows Studio Effects Live Captions Translations Recall User Activity OCR Phi-Silica
11. Applications + Experiences AI frameworks + Toolchains Windows Copilot Library + On-device models Announcing Phi-Silica SOTA SLM included in every Copilot+ PC
12. Applications + Experiences AI frameworks + Toolchains Windows Copilot Library + On-device models Coming later this year RAG Vector Embeddings Text Summarization
13. AI frameworks + Toolchains Developer preview PyTorch now native on Windows powered by DirectML Developer preview Web Neural Network on Windows powered by DirectML Windows Copilot Library + On-device models
14. udio lets developers responsibly build and deploy custom copilots Azure AI Search features search relevance updates and new integrations N Service boosts performance and security for web app creation Azure Functions launches Flex Consumption Plan, extensions Azure Static W Khan Academy and Microsoft announce partnership Azure Compute Fleet now in preview Microsoft Copilot in Azure preview open to all Azure Logic Apps Announcing custom generative mode in preview soon New Azure virtual machine series optimized for AI and cloud-native pabilities in Azure Database for PostgreSQL Azure API Center and generative AI capabilities in Azure API Management now available Visual odel to Pulumi in Azure Deployment Environments Azure Service Bus updates now in preview Microsoft for Startups Founders Hub gains AI c work 2.0 now in preview SharePoint Embedded now generally available New enhancements for custom app experiences connected to Micro wered features and enhanced data protection in Microsoft Teams Premium New features in Microsoft Dataverse in preview Power Apps upd Improvements to the developer experience in Windows Wide array of Windows Developer AI updates Windows Security additions an or Visual Studio Code now in preview Microsoft adds multimodal Phi-3 model Phi-3-vision Real-time video translation in Microsoft Edge com 50+ new updates
15. Copilot stack Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge
16. Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge
17. 60+ Azure regions
18. France Largest investment to date to accelerate the adoption of AI, skilling, and innovation Wisconsin $3.3B investment to spur AI innovation and economic growth Mexico First hyper-scale cloud datacenter region Malaysia US$2.2B investme to fuel cloud and AI transformation
19. 100% of energy from zero-carbon sources by 2025
20. Microsoft Azure Datacenter CPU AI accelerator IO Network End-to-end systems optimization
21. #1 supercomputer for cloud Microsoft Azure 561.20 PFlop/s TOP500, May 2024
22. November 2023 May 2024 30x Microsoft Azure supercomputers
23. Best AI performance. Lowest cost. Microsoft Copilot Your copilot apps Azure AI
24. March 2023 November 2023 May 2024 GPT-4 GPT-4 Turbo GPT-4o ⅙ ⅟₁₂ Cost Speed 2x 6x
25. Full-stack partnership aka.ms/NVIDIAPartnership
26. Full-stack partnership NVIDIA Blackwell Platform NVIDIA NIM-optimized models NVIDIA Omniverse, NVIDIA DGX Cloud + Fabric Windows 365 and Copilot GPU acceleration Azure AI Platform Microsoft Apps NVIDIA on Azure Microsoft Azure aka.ms/NVIDIAPartnership
27. Expanded partnership aka.ms/AMDPartnership
28. General availability ND MI300X V5 Leading price/performance on GPT-4 Optimized for Microsoft Azure workloads aka.ms/AMDPartnership
29. Updates Azure Maia Integrated system of silicon, racks, and software Designed for Azure OpenAI workloads aka.ms/AzureMaia
30. Public preview Azure Cobalt Up to 40% higher performance Most power-efficient compute offering in Microsoft Azure aka.ms/AzureCobalt
31. Azure Cobalt Customers aka.ms/AzureCobalt
32. Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge
33. Azure AI
34. Azure AI 50,000+ organizations
35. Microsoft OpenAI aka.ms/OpenAI
36. GPT-4o GPT-4o achieves state-of-the-art performance on visual perception benchmarks 100% 0 50% MathVista AI2D ChartQA DocVQA ActivityNet EgoSchema MMMU GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro
37. Azure OpenAI Service GPT-4 Turbo with Vision GPT-4 GPT-3.5 Turbo DALL·E 3 Whisper aka.ms/AOAIUpdates
38. Azure OpenAI Service GPT-4o Generally available Multimodal input and output Real-time response Safe by design to-speech models Preview aka.ms/AOAIUpdates
39. https://azure.microsoft.com/en-us/blog/openais-fastest-model-gpt-4o-mini-is-now-available-on-azure-ai/
40. Best selection of models Jan 2024 Feb Mar Apr May Deci Mistral-large Databricks DBRX Cohere Command R+ Cohere Embed Meta Llama 3 Snowflake Arctic Phi-3-mini Mistral-small Phi-3-small Phi-3-medium Phi-3-vision
41. New Models-as-a-Service Core42 JAIS New Nixtla TimeGEN-1 New Bria AI Coming soon Gretel Coming soon NTT DATA Coming soon Stability AI Coming soon AI21 Coming soon Cohere Rerank Coming soon aka.ms/ModelCatalog
42. https://azure.microsoft.com/en-us/blog/announcing-phi-3-fine-tuning-new-generative-ai-models- and-other-azure-ai-updates-to-empower-organizations-to-customize-and-scale-ai-applications/ • Serverless fine-tuning for Phi-3-mini and Phi-3-medium models • Updates to Phi-3-mini including significant improvement in core quality, instruction-following, and structured output, • Same day shipping of the latest models from OpenAI (GPT-4o mini), Meta (Llama 3.1 405B), Mistral (Large 2)
43. Microsoft Open source
44. Expanded partnership aka.ms/ExtendedHF
45. aka.ms/ExtendedHF Hugging Face expanded partnership Expanded partnership Deeper integration with Azure AI Studio Additional models coming to Azure AI TGI enabled for optimized runtime
46. Microsoft SLMs
47. Phi-3 Quality vs. size in SLM Model quality measured on MMLU benchmark 0 4 8 28 12 16 20 24 Phi-3-small Phi-3-mini-4k Phi-3-medium Llama-3-8B-In Mixtral 8x7B Gemma 7B Mistral 7B 80 75 65 60 70 Size in billions of parameters (active) Model quality Phi-3-mini-128k Gemma 2
48. Phi-3 Phi-3-mini 3.8B parameters Available on MaaS Phi-3-vision 4.2B parameters New Phi-3-small 7B parameters New Phi-3-medium 14B parameters Available on MaaS New aka.ms/Phi3
49. aka.ms/Phi3 Phi-3 Phi-3-vision 4.2B parameters New Multimodal model with language and vision capabilities Reason over real-world images Optimized for chart and diagram understanding 3-mini arameters e on MaaS Phi-3-s 7B param New
50. aka.ms/Phi3 Phi-3 Phi-3-vision 4.3B parameters Optimized for chart and diagram understanding Reason over real-world images Multimodal model with language and vision capabilities 3-mini arameters e on MaaS Phi-3-s 7B param New New
51. Phi-3 Cloud Local AI PC • Mobile • Edge (iOS, Android)
53. Announcing aka.ms/KhanAcademy
54. Announcing Building the future of education with Khan Academy Providing free access to Khanmigo for teachers in the US Democratizing AI-powered math tutoring with Phi-3 Supporting learners with high-quality education experiences aka.ms/KhanAcademy
55. Azure AI Studio A comprehensive platform to develop and deploy custom copilots
56. Generally available Azure AI Studio API and model choice Complete AI toolchain Responsible AI tools & practices Enterprise-grade production at scale AI.Azure.com
58. Announcing Azure AI Content Safety Coming soon Custom categories Preview Prompt shields Preview Groundedness detection aka.ms/ContentSafetyUpdates
59. Azure AI custom models Least customized Prompt engineering Retrieval-Augmented Generation Announcing Co-innovation with Microsoft Most customized
60. Announcing Co-innovation with Microsoft Assisted fine tuning Train a custom language model Ingest new knowledge domains Improve proficiency for specific languages Azure AI custom models
61. Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge
62. Microsoft Intelligent Data Platform Azure SQL DB Azure Cosmos DB Azure for PostgreSQL MySQL Microsoft Fabric Azure Databricks Azure AI Microsoft Purview Databases • Analytics • AI • Governance
63. Microsoft Fabric Data platform for the era of AI Complete analytics platform Lake-centric and open Empower every business user AI-powered
64. 11,000+ organizations use Microsoft Fabric
66. Public preview Real-Time Intelligence in Microsoft Fabric Instant actionable insights on all streaming data across clouds aka.ms/RealTimeIntelligence
67. Public preview Real-Time Intelligence in Microsoft Fabric Accelerated decisions AI-powered Intuitive Unified data estate aka.ms/RealTimeIntelligence
68. Public preview Microsoft Fabric Workload Development Kit Seamless user experience OneLake integration Enhanced customer discoverability Marketplace-based monetization aka.ms/FabDevKit
69. Public preview Microsoft Fabric Workload Development Kit aka.ms/FabDevKit
71. Azure AI Search Revolutionary retrieval State-of-the-art search technology Feature-rich vector database Seamless data and platform integrations Up to 12x more scale and performance
73. Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge
74. GitHub Copilot
75. 1.8 million GitHub Copilot subscribers The world’s most widely adopted AI developer tool
76. 50,000 organizations
78. Private preview GitHub Copilot Workspace Copilot-native developer environment aka.ms/GitHubCopilotWorkspace
80. Announcing GitHub Copilot Extensions Creating an intelligent third-party ecosystem in GitHub Copilot aka.ms/GitHubCopilotExtensions
81. GitHub Copilot Extensions aka.ms/GitHubCopilotExtensions
82. Preview GitHub Copilot for Azure aka.ms/GitHubCopilotAzure
84. AI safety and security Microsoft Copilot Your copilots Copilot extensibility Cloud and Edge Your data Foundation models AI orchestration and toolchain AI infrastructure
85. Microsoft Copilot
86. 68% said Copilot helped them jump-start the creative process Marketing 70% of Copilot users said they were more productive Knowledge workers 12% faster at resolving customer issues Customer service
87. Copilot Studio Your Copilot, your way Customize Build Manage
88. Announcing Microsoft Copilot connectors Build and customize your copilot and Microsoft Copilot using your business data, apps, and workflows Knowledge grounding Real-time actions and workflows Q&A over analytical data aka.ms/TryCopilotConnectors
89. Microsoft Copilot Team Personal
90. Announcing Team Copilot A valuable new member of the team Meeting facilitator Group collaborator Project manager aka.ms/TeamCopilot
92. Microsoft Copilot Agents Team Personal
94. Microsoft Copilot Your copilots Copilot extensibility AI safety and security Cloud and Edge Your data Foundation models AI orchestration and toolchain AI infrastructure
95. Secure Future Initiative Security above all else Secure by design Secure by default Secure operations Protect tenants and isolate production systems Protect identities and secrets Protect networks Protect engineering systems Monitor and detect threats Accelerate response and remediation
96. Empower every person and every organization on the planet to achieve more
97. © Copyright Microsoft Corporation. All rights reserved.