Built-in AI & WebMCP: The AI Revolution Right in Your Browser

More and more developers are looking to integrate Generative AI features into their applications. Until now, this has almost always meant going to the cloud, but it doesn't have to be that way! Microsoft and Chromium are currently implementing built-in AI interfaces in Chrome and Edge that provide access to a locally installed large language model (LLM). The advantages are obvious: user data does not leave the device, and everything works even with a weak or no internet connection. Google and Microsoft are also currently specifying the WebMCP protocol: this allows a web application to make its tools known to an LLM and become part of an agentic workflow. In this session, Christian Liebel will demonstrate the use cases that the APIs cover and show you how you can make your web application smarter.

Transcript

1. Built-in AI & WebMCP The AI Revolution Right in Your Browser Christian Liebel @christianliebel Consultant
2. Hello, it’s me. Built-in AI & WebMCP Christian Liebel W3C WebApps, WebML & TAG christian.liebel @thinktecture.com Angular, PWA & Generative AI Microsoft MVP & Google GDE The AI Revolution Right in Your Browser
3. Rule-based algorithms are limited in their capabilities. Built-in AI & WebMCP Why should you care about AI? The AI Revolution Right in Your Browser
4. Use AI to implement use cases that are difficult or impossible to implement using rule-based algorithms. Built-in AI & WebMCP Why should you care about AI? The AI Revolution Right in Your Browser
5. Examples Built-in AI & WebMCP Generative AI Cloud Providers The AI Revolution Right in Your Browser
6. Drawbacks Built-in AI & WebMCP Generative AI Cloud Providers Require a (stable) internet connection Subject to network latency and server availability Data is transferred to the cloud service Require a subscription The AI Revolution Right in Your Browser
7. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Frameworks – Transformers.js – ONNX Runtime – TensorFlow.js – APIs – WebGPU, WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Proofreader API – Translator & Language Detector APIs – Prompt API – WebMCP Built-in AI & WebMCP Web AI Landscape The AI Revolution Right in Your Browser
8. – Grants web apps access to the device’s CPU, GPU and Neural Processing Unit (NPU) – In specification by the WebML Working Group at W3C – Implementation in progress in Chromium (behind a flag) – Positive standard positions by Chromium and Firefox Built-in AI & WebMCP WebNN Source: https://webmachinelearning.github.io/webnn-intro/ The AI Revolution Right in Your Browser
9. https://github.com/explainers-by-googlers/cross-origin-storage Built-in AI & WebMCP Cross-Origin Storage DEMO The AI Revolution Right in Your Browser
10. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Frameworks – Transformers.js – ONNX Runtime – TensorFlow.js – APIs – WebGPU, WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Proofreader API – Translator & Language Detector APIs – Prompt API – WebMCP Built-in AI & WebMCP Web AI Landscape The AI Revolution Right in Your Browser
11. Built-in AI & WebMCP Built-in AI Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano The AI Revolution Right in Your Browser
12. – Initiative by Google Chrome – Exploratory APIs for local experiments and use case determination – Downloads AI models into Google Chrome – Models are shared across origins – Uses native APIs directly (full performance) Built-in AI & WebMCP Built-in AI https://developer.chrome.com/docs/ai/built-in The AI Revolution Right in Your Browser
13. Incubated by the WebML CG Built-in AI & WebMCP Built-in AI APIs https://webmachinelearning.github.io/incubations/ DEMO The AI Revolution Right in Your Browser
14. Prompt API Standard Positions Built-in AI & WebMCP The AI Revolution Right in Your Browser Built-in AI https://github.com/WebKit/standards-positions/issues/495 https://github.com/mozilla/standards-positions/issues/1213
15. Prompt API const languageModel = await LanguageModel.create(); console.log(await languageModel.prompt("Capital of the UK?")); Built-in AI & WebMCP The AI Revolution Right in Your Browser Built-in AI
16. Built-in AI & WebMCP Multimodal Models The AI Revolution Right in Your Browser
17. Built-in AI & WebMCP Multimodal Models DEMO The AI Revolution Right in Your Browser
18. Built-in AI & WebMCP Structured Output DEMO https://bandarra.me/apps/flyby/ The AI Revolution Right in Your Browser
19. about://flags Enables optimization guide on device à EnabledBypassPerfRequirement (API) for Gemini Nano à Enabled Built-in AI & WebMCP Built-in AI APIs The AI Revolution Right in Your Browser
20. TypeScript Definitions Built-in AI & WebMCP Built-in AI APIs The AI Revolution Right in Your Browser
21. Built-in AI & WebMCP Agentic Browsers The AI Revolution Right in Your Browser
22. Built-in AI & WebMCP WebMCP – Allows websites to expose tools to the browser or external agents – Joint effort by Microsoft and Google https://github.com/webmachinelearning/webmcp The AI Revolution Right in Your Browser
23. Built-in AI & WebMCP The AI Revolution Right in Your Browser WebMCP https://webmachinelearning.github.io/webmcp/
24. API Imperative navigator.modelContext .provideContext({ tools: [{ "name": "start_game", "description": "Start a new game.", "inputSchema": {}, "execute": () => {}, } ]}); Declarative <form id="reservationForm" toolname="book_table_le_petit_bistr o" tooldescription=...> <input name="name" toolparamdescription="Customer's full name (min 2 chars)" /> </form> Built-in AI & WebMCP The AI Revolution Right in Your Browser WebMCP
25. Imperative Built-in AI & WebMCP The AI Revolution Right in Your Browser WebMCP Maze https://googlechromelabs.github.io/webmcp-tools/demos/pizza-maker/ DEMO
26. Declarative Built-in AI & WebMCP The AI Revolution Right in Your Browser WebMCP Maze https://bandarra.me/posts/from-pytorch-to-browser-creating-a-web-friendly-ai-model DEMO
27. Agent-only! Built-in AI & WebMCP The AI Revolution Right in Your Browser WebMCP Maze https://webmcp-maze.bandarra.me/ DEMO
28. Agent-only! Built-in AI & WebMCP The AI Revolution Right in Your Browser WebMCP Maze https://nearform.github.io/web-agents/ DEMO
29. Built-in AI & WebMCP WebMCP https://mcp-ui.mcp-b.ai/ The AI Revolution Right in Your Browser DEMO
30. Pros & Cons + Data does not leave the browser (privacy) + High availability (offline support) + Low latency + Stability (no external API changes) + Low cost – Lower response quality – Less capable – High system (RAM, GPU) and bandwidth requirements – Large model size, models cannot always be shared – Model initialization and inference are relatively slow – APIs are experimental Built-in AI & WebMCP Web AI The AI Revolution Right in Your Browser
32. Thank you for your kind attention! Christian Liebel @christianliebel christian.liebel@thinktecture.com