Upgrade to Pro — share decks privately, control downloads, hide ads and more …

IoT with the Google Assistant

IoT with the Google Assistant

Introductory talk held at the Voice meets IoT Hackathon:
https://www.meetup.com/de-DE/Munich-Voice-Interfaces/events/246064221/

Wolfram Rittmeyer

April 13, 2018
Tweet

More Decks by Wolfram Rittmeyer

Other Decks in Programming

Transcript

  1. Voice UX • Voice UX is special – Discoverability of

    app – Discoverability of features – Onboarding • Avoid repetitions – Past behaviour of user
  2. Voice UX • You need to think about your app‘s

    persona • Try to keep to user expectations of a dialog – Script it – Practise script with a partner
  3. Voice UX • Some recommendations – provide feedback – elicit

    answers – Give helpful comments – Repair broken conversations – Provide help – Avoid the usage of the same phrases over and over again
  4. Voice UX • Loads of UX recommendations by Google •

    See the explanations about the IO Assistant app
  5. Assistant SDK • If you want to add the Assistant

    to your device • Enables you to control some aspects of the device
  6. Assistant SDK • Communicate with Google‘s backend using gRPC •

    Change the trigger for the assistant – No direct support for a different hotword (use snowboy or pocketSphinx) • On device support for different traits – Brightness, color... • Either use a lib (Python) or the service directly • For proper devices; you must (!) contact Google; it‘s for development only
  7. Actions on Google – Smart Home • If you want

    your users to control your device using the Google Assistant – „Turn on the light in the living room“
  8. Actions on Google – Smart Home • Consists of two

    parts: – OAuth2 server to authenticate users and issue a token – Some backend to • Query the state of the device • Trigger changes to the device • User adds a device via Google Home app – Here the Oauth flow comes into play • Digg into sample app – https://github.com/actions-on-google/smart-home-nodejs
  9. Actions • If you want your users to use a

    voice app • „Ask MVV, when‘s the next bus due at Hofmannstraße“ • Use one of those options – Actions SDK – A service like Dialogflow
  10. Actions • If you want your users to use a

    voice app • „Ask MVV, when‘s the next bus due at Hofmannstraße“ • Use Dialogflow
  11. Base concepts • Intents – Predefined and custom ones –

    Special ones for smart home • Entities • Fulfillment • Actions • Context
  12. Permissions • For some things you need the user‘s consent

    – User name – Fine or coarse location – Push actions – Repeated actions
  13. Visual Elements • Only on devices with visual interface –

    Cards with images, text and links – List selectors – Carousel – Suggestion chips
  14. Fulfillment • It‘s just JSON going back and forth •

    Use Cloud Functions / Lambda or whatever you like • You can use ngrok to make your localhost server available
  15. Limitations of the Platform • No implicit intents – Probably

    the main beef on G+ • Limited options for visual UI elements • SSML missing important features – Though some have been added recently • Testing • Account switching is a PITA, language switching as well • No private actions (store apps, inhouse apps)
  16. But • Google is eagerly changing and adding stuff –

    what might be a limitation now might work tomorrow
  17. Resources • Talks at Google I/O https://events.google.com/io/schedule/?section=may-18&track=assistant https://events.google.com/io/schedule/?section=may-19&track=assistant • Actions

    on Google Dev Home: https://developers.google.com/actions/ • Actions on Google Console: https://console.actions.google.com • Dialogflow Console https://console.dialogflow.com