Digital Wednesday

Transcript

1. 1 Creating Conversations For Smart Assistants Lee Boonstra Customer Engineer, Google Twitter: @ladysign
2. 2 Lee Boonstra @ladysign
3. Use cases in the contact center and beyond • Customer service • E-commerce • Home entertainment • Auto • Organizational knowledge • Onboarding & Helpdesk
4. Consumers have high expectations Based on data collected in Fall 2017 from sample of 1,051 U.S. adults, ages 18-64.
5. 5 50% of enterprises spend more time on creating bots than on mobile app development by 2021 —Gartner
6. 6 Chatbots are expected to trim business costs by more than $8 billion per year by 2022. - Juniper Research
7. Confidential + Proprietary Confidential + Proprietary Chatbots exists since the 90’s… So why are they popular now?
8. 8 1994, the first chatbot was born 2016, Focus on AI First
9. 9 Machine Learning How did you learn your first language? or
10. 10 It’s easier to make computers learn than to build smarter computers Machine Learning Process of making computers learn Artificial Intelligence Process of building smarter computers
11. Dialogflow users benefit from world-class AI assets & capabilities, including: Google is an AI company Goal: Make AI easy, fast, and useful for enterprises and developers Machine learning leadership Natural language understanding Knowledge graph Search across unstructured data Speech recognition & synthesis TPUs and unrivaled compute infra for ML
12. Cloud AI Less ML expertise More ML expertise Cloud AI solutions Cloud Job Discovery Contact Center Document Understanding Cloud AI building blocks Cloud AI platform Cloud ML Engine Cloud Video Intelligence ML professionals & service partners ASL Professional Services Organization Cloud AutoML Vision Vision Cloud Vision Language Conversation Cloud Natural Language Cloud AutoML NL Dialogflow Enterprise Cloud Translation Cloud Speech-to-Text Cloud Text-to-Speech Cloud AutoML Translation New New New Cloud GPU Cloud TPU Cloud Dataflow Cloud Dataproc Machine & Deep Learning ML accelerators ML libraries Tensorflow Kubeflow Kaggle/datasets Datasets
13. 13 Storage Compute
14. 14 Bots to improve your customer experience Callbots in callcenters Chatbots in websites, apps or social media platforms. Voice Activated Speakers & smart assistants.
15. 15 70% of requests are in natural language Is it gonna rain tomorrow? Tomorrow it will be sunny In Amsterdam!
16. 16 A conversation between you and Google that helps you get more done in your world Meet the Google Assistant
17. 17 Terminology Google Assistant — The virtual assistant of Google. Out of the box on Android 6+. For iOS available as app. Action — A third party app, running on top of the Google Assistant. Google Home — Voice-activated speaker powered by Voice. Smart Display — Voice-activated speaker with screen powered by Voice and Touch.
18. 18 • Manage tasks Ok Google, what’s my confirmation number for my flight? • Plan the day Ok Google, how is my commute to the office? • Get answers: Ok Google, who is the King of the Netherlands? • Make memories Ok Google, remember I left my keys on the Kitchen table. • Control your home Ok Google, turn off the bedroom lights. • Enjoy Entertainment Ok Google, play the Pixies on Spotify. Things you can ask Google...
19. Google | Proprietary & Confidential 19 Idle Listening Recognizing Speaking Thinking
20. Google | Proprietary & Confidential 20 recognitionResult listenEnd recognitionResult voicestart listenStart listenEnd voiceEnd eventName
21. 21 3rd party integration Extend the Google Assistant with your own custom actions. Hey Google, let me talk to BookAMeetingRoom Welcome, how can I help you? I want to book a meeting room for 3 persons. Let’s get BookAMeetingRoom Sure, for when? Tomorrow, from 2pm to 3pm.
22. Confidential + Proprietary Confidential + Proprietary How can you build your own action on top of the Google Assistant?
23. 23 Sketch (+ Google Assistant UI Toolkit) Prototyping tools Twine
24. 24 • Write the conversation - Dialogflow (Enterprise) • Deploy on GA+ UX components - Actions on Google Optional: • Back-end integration - Fulfillment app (dialogflow/aog SDK) • Communication to back-ends - Your own APIs Tooling
25. 25 Testing tools Ngrok secure internet tunnels to localhost • Actions on Google Simulator • Device Testing • Dialogflow Simulator • Dialogflow Diagnostics • GCP Stackdriver • Cloud Functions Emulator • Ngrok Secure Tunneling • Dialogflow Training
26. Confidential + Proprietary Dialogflow
27. 27 • Previously known as API.AI ◦ (Sept 2016, acquired by Google) • Powered by Machine Learning: ◦ Natural Language Understanding (NLU) ◦ Intent Matching ◦ Conversation Training • Cross platform • Build faster with the Web UI • Scalable: separate your conversation text from code • Speech / Voice Integration • Multi-lingual bot support (20+ languages) • Part of Google Cloud (60+ cloud services) Development suite for building Conversational UIs.
28. 28 Dialogflow has been used by over 600,000+ developers
29. 29 Intents
30. 30 Intent Responses
31. 31 Entities
32. 32 • Google Assistant • Amazon Alexa • MS Cortana • Web • iOS • Android Integrations • Facebook Messenger • Twitter • Slack • Telegram • Skype • Cisco Spark • Dialogflow Phone • Twilio • Viber • Kik • Line
33. 33 Fulfillment
34. 34 • Your application back-end code. ◦ Cloud Function / App Engine / GKE ...? • Router to catch all intents, that have webhooks enabled. • Application code • Response messages • Calls to external / on-premise APIs Fulfillment App
35. Confidential + Proprietary Actions on Google
36. 36 • Bring your agents to smart speakers (Google Home) or phones (Android, iOS app) • Actions on Google includes: ◦ UI toolkit, ◦ Audio toolkit ◦ Account Linking API ◦ SDKs ◦ tutorial guides • UI components such as: ◦ Buttons, Images ◦ Cards, Carousels, ◦ Lists ◦ Tables Program for developers of Actions (“apps”) that run via Google Assistant Actions on Google
37. 37 SSML Speech Synthesis Markup Language (SSML) is an XML based markup language for defining speech. • By using SSML, you can make your agent's responses seem more life-like. • Tags for defining: ◦ Breaks ◦ Audio snippets ◦ Type of text constructs (date, time, alphabet..) ◦ Pitch, Speaking rates ◦ Emphasis
38. 38 • Formatting Numbers - ($100.000 - > 100000) http://numeraljs.com/ • Format JS internationalization (dates etc) - https://formatjs.io/ • Moment JS (dates, times) - https://momentjs.com/ • Pluralize - https://www.npmjs.com/package/pluralize • Lightweight Translation Module - (Multi languages in fulfillment apps) - https://github.com/mashpie/i18n-node Handy JS libraries
39. 39 Assistance is not just about voice
40. 40 Your experience can run across multiple devices.
41. 41 Suggestion Chips
42. 42 Suggestion Chips
43. 43 Basic Cards
44. 44 Basic Cards
45. 45 Lists
46. 46 Lists
47. 47 Carousels
48. 48 Carousels
49. 49 Media Responses
50. 50 Media Responses
51. 51 Table Responses
52. 52 Table Responses
53. 53 Action Directory Submit your action to production.
54. Confidential + Proprietary Confidential + Proprietary How to measure your app usage?
55. Confidential + Proprietary Which customers are unhappy and why?
56. Confidential + Proprietary How can I improve the user experience?
57. 57 Sure! I’d like to transfer 100 euro to my wife her bank account.” A customer communicates with the Google Assistant
58. 58 What happens under the hood...
59. 59 Collect real-time chats from Dialogflow SDK
60. 60 Mask sensitive Information with DLP API
61. 61 Understand the text with NLP API
62. 62 Store all data in a data-warehouse
63. 63 Optimize your agent
64. Confidential + Proprietary Confidential + Proprietary Demo!
65. Confidential + Proprietary Advanced Chatflow with machine learning bot analytics User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Container Registry Containers images can be stored in the Container Registry Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook Container Builder Building Dev Pipelines
66. Confidential + Proprietary Advanced Chatflow with machine learning and bot analytics User types to custom UI or channel Chatbot replies Dialogflow Enterprise Customer Client JS Angular 5 web front-end Kubernetes Engine Chat Server Dialogflow SDK / socket.io Kubernetes Engine Back-end CRM Python / Django Kubernetes Engine Subscription Cloud Function Sensitivity Filter DLP API Sentiment Detector NLP API Data Warehouse BigQuery Messaging Publisher Pub/Sub Webhook Router Cloud Function Webhook
67. Confidential + Proprietary Confidential + Proprietary Thank you!
68. Confidential & Proprietary Demo: A TV Guide Do it yourself: https://goo.gl/pH86aT Ok Google, let me talk to the TV Guide Welcome, I’m the Dutch TV Guide. You can ask me the following: What’s on a channel? What’s on RTL 4? Sure! Here’s The TV Guide Enter Earcon On RTL 4: At 7am: Nieuws At 7:30am: Nieuws ... Exit Earcon . . .
69. Confidential + Proprietary ElasticSearch live search results Architecture: Dialogflow with live search results Router Cloud Functions Webhook Workflow Management Calls to logics layers Dialogflow Enterprise Elastic Search User types in chat UI Chatbot or User replies On premise banking systems