Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build Assistant Actions With Dialogflow

Build Assistant Actions With Dialogflow

This talk shows how to utilize Dialogflow for your Google Assistant Actions. It includes a rather larger part of live demo which obviously cannot be shown in the slides.

Wolfram Rittmeyer

September 28, 2019
Tweet

More Decks by Wolfram Rittmeyer

Other Decks in Programming

Transcript

  1. Permissions • For some things you need the user‘s consent

    – User name – Fine or coarse location – Push actions – Repeated actions @RittmeyerW
  2. Visual Elements • Only on devices with visual interface –

    Cards with images, text and links – Lists – Carousel – Suggestion chips @RittmeyerW
  3. SSML • Makes text to speech more lively • Supports

    – breaks – cardinals / ordinals / date / characters – audio / media – emphasis • Doesn‘t support other languages • https://github.com/mandnyc/ssml-builder @RittmeyerW
  4. Fulfillment • It‘s just JSON going back and forth •

    It has to use TLS • For real projects: Use Cloud Functions / AWS Lambda • For the hackathon: – Use a local server – Expose it via ngrok • Your backend has roughly five seconds time @RittmeyerW
  5. Fulfillment • Google libs for JS: https://github.com/actions-on-google/actions-on-google-nodejs • Google libs

    for Java/Kotlin: https://github.com/actions-on-google/actions-on-google-java • Json Spec https://developers.google.com/actions/build/json/dialogflow-webhook-json @RittmeyerW
  6. Initialisation const { dialogflow, Image, Suggestions, Carousel, //... } =

    require('actions-on-google'); const app = dialogflow(); // or: const app = dialogflow( {debug: true} ); @RittmeyerW
  7. Normal speech app.intent('welcome', conv => { conv.ask('Hi, welcome back to

    Fast Journey?'); conv.ask('You can ask to review booked journeys or start booking a new one.'); conv.ask('Which would you like to do: Revisit an existing booking or booking a new trip?'); }); @RittmeyerW
  8. Parameters app.intent('suggestion', (conv, params) => { if (params.number !== '')

    { conv.ask(`Your lucky number is: ${params.number}`); } else { conv.ask('I still need your number'); } }); @RittmeyerW
  9. Rich responses conv.ask(new Carousel({ title: 'Carousel Title', items: { //

    Add the first item to the carousel 'SELECTION_KEY_ONE': { synonyms: [ 'synonym 1', 'synonym 2', ], title: 'Title of First Carousel Item', description: 'This is a description of a carousel item.', image: new Image(/*...*/) , }, // ... } } )); @RittmeyerW
  10. Rich responses app.intent('helper.option', (conv, params, option) => { const SELECTED_ITEM_RESPONSES

    = { 'SELECTION_KEY_ONE': 'You selected the first item', 'SELECTION_KEY_GOOGLE_HOME': 'You selected the Google Home!', 'SELECTION_KEY_GOOGLE_PIXEL': 'You selected the Google Pixel!', }; conv.ask(SELECTED_ITEM_RESPONSES[option]); conv.ask('Which response would you like to see next?'); }); @RittmeyerW
  11. Async work app.intent('welcome', conv => { return somePromise() .then(promiseResult =>

    { conv.ask(promiseResult); }); }); app.intent('welcome', async conv => { let result = await somePromise(); conv.ask(result); }); @RittmeyerW
  12. Integration • Probably best for the hackathon is to use

    Express const express = require('express') const bodyParser = require('body-parser') // ... app code here const expressApp = express().use(bodyParser.json()) expressApp.post('/fulfillment', app) expressApp.listen(3000) @RittmeyerW
  13. Account Linking • If you want to use existing accounts

    (like Hermes does) • Different methods (with or without Oauth) • Not recommended for the hackathon • But definitely for existing services when going live @RittmeyerW
  14. Books • Cathy Pearl: Designing Voice User Interfaces • Amir

    Shevat: Designing Bots • Erika Hall: Conversational Design https://abookapart.com/products/conversational-design @RittmeyerW
  15. Online • Recommendations by Google – https://developers.google.com/actions/design/ • Recommendations by

    Amazon – https://developer.amazon.com/ask-resources/guided/conversational- design-workshop#/ @RittmeyerW