ABS2024: Implementing AI: Successes and Lessons from a Software Agency by Tobias Kluge

⭐️ Implementing AI: Successes and Lessons from a Software Agency#
In this talk, we explore our journey of implementing Artificial Intelligence (AI) within a software agency, detailing both our successes and the challenges we faced. We will share valuable lessons learned on communication, training, and change management for the internal rollout. Takeaways on governance and hands-on experiences from the rollout will be presented.

This talk is aimed at anyone interested in AI implementation within an organization. Join us for a candid insight into the transformation process and an open discussion on the challenges.
🙂 TOBIAS KLUGE ⚡️ Mr. AI @ Nexplore AG

Transcript

1. 2 • We will investigate the results and discuss them later. • 2 goodies will be raffled off among all participats. • Perfect if you are bored, the talk is not as expected – and to be allowed using your smartphone during the presentation. How are you using AI in your daily work? Participate & win Participate & win
3. Azure Bootcamp Switzerland 05/16/2024 Implementing AI: successes & lessons from a software agency 4
4. Tobias Kluge «Mr. AI», Nexplore AG We are hiring!
5. Agenda 6 1. State of AI 2. Implementation 3. Tooling 4. Results of you 5. Raffle
6. 7 State of AI
7. 8 Neuronal network – the brain of AI systems 0.722 0.124 0.542 0.218 0.322 0.876 0.473 0.146 AI model = billions of numbers = probabilities AI does not really understand what it does or knows (and what not)
8. 9 Training of AI systems AI [0.421, 0.141, …] [0.811, 0.321, …] Take care of YOUR data!
9. 10 • https://ourworldindata.org/brief-history-of-ai Development of AI AGI
10. 11 OpenAI GPT4o – AGI 0.9 https://openai.com/index/hello-gpt-4o/
11. 12 Levels of «AI» Products with build-in AI Configuration of ready to use AI-tools Building ai applications with base models, fine- tuning & training
12. 13 What is AI (in this session today)? LLM Data
13. 14 Implementation of AI inside of your organization Organisation readiness
14. 15 It’s so easy… Explore & define use cases Identify minimal solution for use cases Test in pilot Ship to user base
15. 16 • First things first: • Provide base rules (no PII and customer data to public cloud) • Gover your data – best with classification & labeling • Bottom up • Train your people about the base rules & let them experiment • Start small – experiments, «lean» approach • With management support, of course • Have a «AI user group» and «C-AI-O» that supports decisions in the organisation • Communicate well! • Invest! time, people, ressources and money Explore & define use cases
16. 17 • Check the product – LLM model, data base, update frequency, internet-connection, prototype vs ready for production • Check the privacy: monitoring, training, reselling, …? • Check the pricing: usage-based, per user, enterprise license required to keep your data? • Define the rules guidelines how to use the solution Identify minimal solution for use cases
17. 18 Free Excel Template by Vischer Risk assessment template
18. 19 • Before • What is the expected outcome? • How do you want to measure it? • Write it down! • Have a small, but active user group • Not too long – 2-4 weeks might be fine for most cases • Talk to your peple • Measure and decide • Fail is an option! Test with a pilot (or multiple)
19. 20 • Inform well – and often! • Provide training especially for «late majority» • Monitor • Usage • Data sharing • Costs • User feedback • Product lifecycle • Prevent a zoo of tools • Monitor other tools being used • Adapt if necessary Ship to user base
20. 21 Just a few examples… AI tools for daily work
21. 22 • Goal: ai chatbot for knowledge workers • Data base is limited to training data, no «internet search» by default; hallucination & no source provided • Features • Work with texts (marketing, sales, HR, …) • Generate images • Coding: ask for ideas & sample code, fix code, document code, write test cases • Typical users: devs & system enginers, office workers, sales, marketing, C-level, students, your kids • Status: available • Privacy: data stored in US and used for monitoring & training • Pricing: free or 20-30 USD/month for advanced features as GPT4 & image generation • Details OpenAI ChatGPT
22. 23 • Goal: AI pair programmer for engineers • Data: trained on public github repos & user data; beware of weak code and un-licensed code • Features • Generate & refactor code • Inside terminal/console • Chat • Integration (VSC, Neovim, VS, JetBrains) • Typical users: software & system engineers • Status: available • Privacy: data transfered to US (take care of files with secrets), used for training with Individual • Pricing: 10-39 USD/month/user • Details GitHub Copilot
23. 24 • Goal: AI-powered dev environment • Data: trained on public github repos & user data • Features • AI assistant that breaks down issues into plan based on the codebase «plan to code» • Typical users: devs, system engineers • Status: technical preview • Privacy: data transfered to US & ? • Pricing: unknown • Details GitHub Copilot Workspaces Will replace your job! Will replace your job! … handle the ugly tickets!
24. 25 Architecture of MS Copilot https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/ecosystem Data
25. 26 • Goal: ai chatbot powered by Bing • Data: same as GPT4, enriched with Bing search • Features • Ask and search in web – with references • Work with texts and uploaded documents – e.g. summarize long pfs • Generate images • Uses GPT4 / GPT4-turbo • Typical users: everybody • Status: available • Privacy: M365 tenant is boundary, not used for training (with M365 account) • Pricing: free (included in most M365 licenses) • Details Microsoft Copilot with commercial data protection It’s really free! It’s good! Try it!
26. 27 • Goal: your personal AI assistant for office work • Data: graph of your tenant, various MS products and data sources; internet • Features • Teams meetings • Summarize emails and teams chats • Typical users: «office worker» • Status: available • Privacy: M365 tenant is boundary • Pricing: 26.90 CHF/month/user • Details Microsoft Copilot for M365
27. 28 • Goal: ai assistant for Azure • Features • Design: create and configure the services needed while aligning with organizational policies • Operate: answer questions, author complex commands, and manage resources • Troubleshoot: orchestrate across Azure services for insights to summarize issues, identify causes, and suggest solutions • Optimize: improve costs, scalability, and reliability through recommendations for your environment • Typical users: system engineers • Status: limited access • Privacy: Azure tenant is boundary • Pricing: • Details Microsoft Copilot for Azure
28. 29 • Goal: chatbot for security engineers on Azure • Features • Integration into MS security products as Entra, Intune, Defender, Purview and Sentinel • Investigate incidents along tool chain • Generate reports • Typical users: security engineers • Status: available • Privacy: Azure tenant is boundary • Pricing: security compute unit 2’670 CHF/mo • Details Microsoft Copilot for Security
29. 30 Configuration of ready to use AI-tools
30. 31 • Goal: configure chatbots with ai models on your data • Features • Integration with Sharepoint, Jira, Confluence, … • Provide internal support portals – e.g. for IT, HR, • Publish public-facing portals • Of course – integrate with Copilot • Typical users: anybody • Status: available • Privacy: Azure tenant is boundary, specific data protection available • Pricing: 200 CHF / month for 20k messages • Details Microsoft Copilot Studio (aka Power Virtual Agents + OpenAI)
31. 32 Building ai applications with base models, fine-tuning & training
32. 33 • Goal: deploy easy chatbot in your azure tenant • Features • Free blueprints directly deployed in your azure tenant • Provides additional features as document, photo and voice • Typical users: internal ChatGPT similar chatbots • Status: available (blueprints not «prod-ready») • Privacy: Azure tenant is boundary • Pricing: Azure consumption, probably 50-100 CHF / month (depending on usage and data) • Details • Bonus: Azure OpenAI Chatbot with your data in your tenant – in 15min Private Chatbot on Azure OpenAI Blueprints on Azure AI Studio
33. 34 (Probably) related talks today – check the slides OpenAI models with your own data using Azure OpenAI FILIP WOJCIESZYN The Era of Copilots - PoV from Microsoft MIKE BLOECHLINGER RICHARD LAGRANGE Easy On the way to hard Mobi-ChatGPT & Friends - or how to integrate enterprise ready Gen-AI at scale MATTHIAS SCHRANZ ALEXANDER MEIER ChatGPT over your own data MARCO GERBER MICHAEL RÜEFLI Azure AI deep dive with Ausgleichskasse Basel Stadt JÖRG BIERI IVAN BABIC
34. 35 My 5 cent Groth and adjustment of AI market AI will be implemented in every day tools Supporting the users and humans
35. 36 Next steps
36. The results 37
41. And the winners are… (sorry, too late ☺)
42. Lets discuss
43. • Next meetup today at 5.45pm @ Puzzle ITC • Uphill Conf Special Meetup – firesidechat with speakers of uphill Applied AI conference • Lisa Carpenter: Lead Data Science & AI Instructor @ Digital Futures • Leandro von Werra: Machine Learning Engineer @ Hugging Face • Pablo Pernías & Dominic Rampas: ML Researchers @ Luma AI, prev. StabilityAI Interested in AI? Join the ML & AI Meetup Bern!
44. Nexplore AG C.F.L. Lohnerstrasse 24 3645 Gwatt (Thun) Telefon +41 33 334 02 00 info@nexplore.ch See you ☺