Smarter Angular Apps with WebNN & Prompt API — Angular Air with Christian Liebel

Can modern browser APIs make your Angular apps smarter? Christian Liebel (@christianliebel) joins Angular Air to show us how!

In this episode, we explore the exciting world of WebNN (Web Neural Network API) and the Prompt API, and how they can be used to enhance performance and add AI-powered capabilities to your Angular applications — all right in the browser.

Christian breaks down real use cases, performance considerations, and what’s just over the horizon for intelligent web apps.

Transcript

1. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Christian Liebel @christianliebel Consultant
2. Hello, it’s me. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Christian Liebel W3C WebML WG & CG TAG Associate X: @christianliebel Bluesky: @christianliebel.com Angular, PWA & Generative AI Microsoft MVP & Google GDE (Angular, Web)
3. Examples AI in the Browser Smarter Angular Apps With WebNN & Prompt API Generative AI Cloud Providers
4. Drawbacks AI in the Browser Smarter Angular Apps With WebNN & Prompt API Generative AI Cloud Providers Require a (stable) internet connection Subject to network latency and server availability Data is transferred to the cloud service Require a subscription
5. Can we run GenAI models locally? AI in the Browser Smarter Angular Apps With WebNN & Prompt API
6. Bring Your Own AI (BYOAI) – Libraries – WebLLM – Transfomers.js – Frameworks – ONNX Runtime – TensorFlow.js – APIs – WebNN – Cross-Origin Storage Built-in AI (BIAI) – Writing Assistance APIs – Summarizer API – Writer API – Rewriter API – Translator & Language Detector APIs – Prompt API AI in the Browser Smarter Angular Apps With WebNN & Prompt API Local AI Inference
7. https://webllm.mlc.ai/ AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM DEMO
8. On NPM AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM
9. Storing model files locally AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM Internet Website HTML/JS Cache with model files Hugging Face Note: Due to the Same-Origin Policy, models cannot be shared across origins.
10. Model Size Comparison Model:Parameters Size phi3:3b 2.2 GB mistral:7b 4.1 GB llama3:8b 4.7 GB gemma2:9b 5.4 GB gemma2:27b 16 GB llama3:70b 40 GB AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebLLM
11. https://huggingface.co/docs/transformers.js/index AI in the Browser Smarter Angular Apps With WebNN & Prompt API Transformers.js DEMO
12. – Grants web apps access to the device’s CPU, GPU and Neural Processing Unit (NPU) – In specification by the WebML Working Group at W3C – Implementation in progress in Chromium (behind a flag) – Better performance for specific workloads AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebNN Source: https://webmachinelearning.github.io/webnn-intro/ DEMO
13. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Why should you care? DEMO
14. about://flags Enables WebNN API à Enabled Enables experimental WebNN API features à Enabled AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebNN
15. Drawbacks AI in the Browser Smarter Angular Apps With WebNN & Prompt API WebNN Models can’t be shared across origins Inference is fast, but doesn’t reach full native speed
16. https://github.com/explainers-by-googlers/cross-origin-storage AI in the Browser Smarter Angular Apps With WebNN & Prompt API Cross-Origin Storage
17. – Initiative by Google Chrome – Exploratory APIs for local experiments and use case determination – Downloads AI models into Google Chrome – Models are shared across origins – Uses native APIs directly (full performance) AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI https://developer.chrome.com/docs/ai/built-in
18. Incubated by the WebML CG AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs https://webmachinelearning.github.io/incubations/ DEMO
19. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Multimodal Models
20. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano
21. about://on-device-internals https://www.google.com/chrome/canary/ about://flags Enables optimization guide on device à EnabledBypassPerfRequirement (API) for Gemini Nano à Enabled AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs
22. TypeScript Definitions AI in the Browser Smarter Angular Apps With WebNN & Prompt API Built-in AI APIs
23. Rule-based algorithms are limited in their capabilities. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Why should you care?
24. Use AI to implement use cases that are difficult or impossible to implement using rule-based algorithms. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Why should you care?
25. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Chatbots DEMO
26. AI in the Browser Smarter Angular Apps With WebNN & Prompt API Categorization DEMO
27. (Cloud only!) AI in the Browser Smarter Angular Apps With WebNN & Prompt API Multimodal Realtime Models DEMO
28. Pros & Cons + Data does not leave the browser (privacy) + High availability (offline support) + Low latency + Stability (no external API changes) + Low cost – Lower response quality – Less capable – High system (RAM, GPU) and bandwidth requirements – Large model size, models cannot always be shared – Model initialization and inference are relatively slow – APIs are experimental AI in the Browser Smarter Angular Apps With WebNN & Prompt API On-device AI Models
29. Thank you for your kind attention! Christian Liebel @christianliebel christian.liebel@thinktecture.com