Building Chat Bots for the Google Assistant / Google Home using API.AI

In this session, Rebecca will explain how to quickly and easily build a custom Google Actions for the Google Home by leveraging API.AI.

Transcript

1. Building Chat Bots for Google Home using API.AI Rebecca Franks Google Developer Expert - Android @riggaroo riggaroo.co.za
2. Who am I?
3. “Ok Google!”
5. Direct Actions • Google Assistant handles the entire user interaction • You only handle the fulfillment • Not yet widely available to build “Ok Google, Play Charlie Brown Movie on Chromecast using DStv Now.” Conversation Actions • Two-way dialog with users • Your action manages the conversation • Can be built and deployed today. “Ok Google, Let me talk to Spelling Master” 
 Types of Actions
6. Conversational Actions Source: https://developers.google.com/actions/develop/conversation
7. Two Options • Actions SDK • Manual understanding of requests - raw text parsed to your server. • API.AI SDK • Natural language Processing • Machine Learning AI • Conversation building features
8. API.AI Flow Source: https://developers.google.com/actions/develop/conversation
9. 5 Steps to build a Google Action Chat Bot 1. Create API.AI Agent & intents 2. (Optional) Create WebHook fulfillment 3. (Optional) Deploy Webhook 4. Test Action 5. Submit for Approval
10. DStv ChatBot Prototype DStv Hi! I’m Dave from DStv, how can I assist you today? When is the next Liverpool Game on? The next match is on Thursday at 12. Would you like me to set a reminder? Ok Google, let me talk to DStv!
11. API.AI Basic Concepts • Agent - correspond to applications. • Intent - a mapping between what a user says and what action should be taken • Entities - concepts that are often specific to a domain as a way of mapping natural language phrases. • Actions - the steps your application will take when specific intents are triggered by user inputs • Contexts -current context of the user expression. This is useful for differentiating phrases which might be vague and have different meaning depending on what was spoken previously. Source: https://docs.api.ai/docs/key-concepts
12. Create API.AI Agent
13. Create API.AI Intent- Startup Intent
14. Create API.AI Intent- Query content intent
15. API.AI and our Webhook Speech spoken by user POST request to Webhook with JSON API.AI extract { "id": "b4e86431-0336-416f-8d4d-fe1572207e58", "timestamp": "2017-01-07T15:25:29.686Z", "result": { "resolvedQuery": "when is the simpsons showing", "action": "query_content", "parameters": { "program_name": “the simpsons" }, "metadata": { "intentName": "query_content" } } }
16. Create Webhook Fulfillment 1.Download template project from here: https://github.com/ actions-on-google/apiai-webhook-template-nodejs (or create Node JS server project.) 2. Edit app.js file
17. 'use strict'; process.env.DEBUG = 'actions-on-google:*'; let Assistant = require('actions-on-google').ApiAiAssistant; let express = require('express'); let bodyParser = require('body-parser'); let app = express(); app.use(bodyParser.json({type: 'application/json'})); Template Code defined in app.js file app.post('/', function (req, res) { const assistant = new Assistant({request: req, response: res}); // Fulfill action business logic … }); if (module === require.main) { // Start the server let server = app.listen(process.env.PORT || 8080, function () { let port = server.address().port; console.log('App listening on port %s', port); }); } module.exports = app;
18. Create WebHook Fulfillment let request = require('request');
 let moment = require(‘moment'); const QUERY_CONTENT_ACTION = 'query_content';
 const ARGUMENT_PROGRAM_NAME = 'program_name';
 
 app.post('/', function (req, res) {
 const assistant = new Assistant({request: req, response: res});
 
 function getNextAiringOfProgram(assistant){
 let showName = assistant.getArgument(ARGUMENT_PROGRAM_NAME);
 request(‘https://gook.com;searchTerm='+ showName, function (error, response, body) {
 }) } 
 if (!error && response.statusCode == 200) {
 var event = JSON.parse(body);
 var dateTimeFormatted = new moment(event.startDateTime);
 assistant.tell(event.title + ' will air on ' + dateTimeFormatted.format('dddd Do [at] hh:mm a'));
 } else {
 assistant.tell('I\'m sorry, there was an error trying to process your request.’);
 let actionMap = new Map();
 actionMap.set(QUERY_CONTENT_ACTION, getNextAiringOfProgram);
 assistant.handleRequest(actionMap);

19. Deploy Webhook 1. Create Google Cloud Project at https://cloud.google.com 2. Run the node server locally from folder 3. Deploy to Google Cloud 4. Set web hook fulfilment URL $ npm install $ npm start $ gcloud init $ gcloud app deploy
20. Preview, Test + Train Action
21. Test on Web Simulator
22. Testing on Google Home
23. Webhook Quirks • Timeout for service response – 5 seconds. • No option to configure this. • Be aware - the timezone you are receiving on the server is not the timezone of the client. • Solution: Request users device location and determine timezone from there
24. Submit for approval 1.Go to Google Cloud Console (https:// console.cloud.google.com), search for “Google Actions API” 2. Enable the API 3. Go to Directory Listing and enter information
25. Submit for approval • Prepare Store Listing items • Choose unique invocation name • Rules are strict, be prepared to be rejected
26. VoiceLabs.co Analytics
27. VoiceLabs.co Analytics
28. Things to consider • Lots of guidelines to read online • Voice UI Guidelines • Authentication - Support for account linking via Google Home app. Using OAuth2. • Error States • VoiceLabs.co Analytics
29. Demo Time
30. Thank you! Rebecca Franks Google Developer Expert - Android @riggaroo / +RebeccaFranksSA riggaroo.co.za