Smart Form Filler: Spice up your Forms With Generative AI & LLMs

Discover how to enhance user experience with generative AI that automates form filling. Christian Liebel will show you how to implement this feature in your web applications for seamless and efficient data entry.

Transcript

1. Smart Form Filler Spice up your Forms With Generative AI & LLMs Christian Liebel | Consultant | Thinktecture AG | christian.liebel@thinktecture.com
2. Hello, it’s me. Smart Form Filler Spice up your Forms With Generative AI & LLMs Christian Liebel X: @christianliebel Email: christian.liebel @thinktecture.com Angular & PWA Slides: thinktecture.com /christian-liebel
3. Overview Smart Form Filler Spice up your Forms With Generative AI & LLMs Generative AI Text OpenAI GPT Mistral … Speech OpenAI Whisper tortoise-tts … Images DALL·E Stable Diffusion … Audio/Music Musico Soundraw …
4. Large: Trained on lots of data Language: Process and generate text in natural language Models: Programs/neural networks Examples: – GPT (ChatGPT, Bing Chat, …) – Gemini, Gemma (Google) – LLaMa (Meta AI) Smart Form Filler Spice up your Forms With Generative AI & LLMs Large Language Models
5. Prompts serve as the universal interface Unstructured text conveying specific semantics Paradigm shift in software architecture Natural language becomes a first-class citizen Caveats Non-determinism and hallucination, prompt injections Smart Form Filler Spice up your Forms With Generative AI & LLMs Large Language Models
6. Smart Form Filler Spice up your Forms With Generative AI & LLMs Use cases beyond chat bots…? Source: https://www.apple.com/chde/apple-intelligence/
7. Just transfer the 17.34 euros to me, my IBAN is DE02200505501015871393. I am with Hamburger Sparkasse (HASPDEHH). Data Extraction Smart Form Filler Spice up your Forms With Generative AI & LLMs Use Case Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz
8. Just transfer the 17.34 euros to me, my IBAN is DE02200505501015871393. I am with Hamburger Sparkasse (HASPDEHH). Data Extraction Smart Form Filler Spice up your Forms With Generative AI & LLMs Use Case Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz
9. protected readonly formGroup = this.fb.group({ firstName: [''], lastName: [''], addressLine1: [''], addressLine2: [''], city: [''], state: [''], zip: [''], country: [''], }); Smart Form Filler Spice up your Forms With Generative AI & LLMs Idea Nice, here is my address: Peter Müller, Rheinstr. 7, 04435 Schkeuditz Smart Form Filler (LLM)
10. Smart Form Filler Spice up your Forms With Generative AI & LLMs Idea DEMO
11. Form Field Prompt Generator Model Backend Response Parser Smart Form Filler Spice up your Forms With Generative AI & LLMs Architecture
12. Smart Form Filler Spice up your Forms With Generative AI & LLMs Form Field “Try to determine the country based on the user’s input.” “If present, put the department in this field.”
13. Async Clipboard API Allows reading from/writing to the clipboard in an asynchronous manner Reading from the clipboard requires user consent first (privacy!) Supported by Chrome, Edge, Safari and Firefox Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Generator
14. System message • The form has the following setup: { "firstName": "", "lastName": "" } User message • Passt, dann hier schonmal meine Adresse: Peter Müller Assistant message • { "firstName": "Peter", "lastName": "Müller" } Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Generator
15. Cloud backends Smart Form Filler Spice up your Forms With Generative AI & LLMs Model Backend
16. Local backend (WebLLM) Smart Form Filler Spice up your Forms With Generative AI & LLMs Model Backend
17. https://webllm.mlc.ai/ Smart Form Filler Spice up your Forms With Generative AI & LLMs WebLLM
18. Smart Form Filler Spice up your Forms With Generative AI & LLMs
19. Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt API Operating System Website HTML/JS Browser Internet Apple Intelligence Gemini Nano
20. Part of Chrome’s Built-In AI initiative – Exploratory API for local experiments and use case determination – Downloads Gemini Nano into Google Chrome – Model can be shared across origins – Uses native APIs directly – Fine-tuning API might follow in the future Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt API https://developer.chrome.com/docs/ai/built-in
21. Demo: Smart Form Filler Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt API DEMO
22. Comparison 22.98 33.96 568.06 909.62 0 100 200 300 400 500 600 700 800 900 1000 WebLLM (Mistral-7b, M1) WebLLM (Mistral-7b, M3) Groq (Mixtral-8x7b) Groq (Llama3-8b) Tokens/sec Smart Form Filler Spice up your Forms With Generative AI & LLMs Performance Own tests (WebLLM: 23.03.2024, Groq: 01.05.2024)
23. Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Parser Assistant message • { "firstName": "Peter", "lastName": "Müller" }
24. Assistant message Parsing the assistant message as text/JSON/… Tool calling Specifying a well-defined interface via a JSON schema called by the LLM (safer, growing support) Smart Form Filler Spice up your Forms With Generative AI & LLMs Prompt Parser
25. – Super accurate – Versatile thanks to layered architecture – Runs in the browser, locally, or in the cloud – Very fast execution is possible via Groq https://github.com/thinktecture-labs/smart-form-filler Smart Form Filler Spice up your Forms With Generative AI & LLMs Summary
26. Thank you for your kind attention! Christian Liebel @christianliebel christian.liebel@thinktecture.com