Future of the Web with Conversational Interface (in JavaScript!)

Future of the Web with Conversational Interface Tomomi Imura (@girlie_mac)

@ • advocate open web & technology • code JavaScript
& Node.js • write & speak about tech • hack useless stuff • A cat lady of the InterWeb • dev relations at Slack tomomi imura

Talking to non-human

@ Photo by Jeopardy Productions, Inc.

-- Satya Nadella, Microsoft

@ -- Sundar Pichai, Google

@ CC BY-SA: nextdayblinds.com

@ Traditional Web & App Interactions

@ Modern Web & Apps with Social Interactions

@ Conversational User Interactions: Siri and Alexa (Voice Assistants) Alexa,
how is the weather? Hey Siri In various form-factors

@ Conversational User Interactions for Kids - with Voice CogniToys
Dino - https://cognitoys.com

@ Conversational User Interactions in a robot shape - with
Voice

@ Conversational User Interactions: Google Assistant (Voice & Text)

@ Conversational User Interactions: Slack Bots (Text) !

Slack Integrations & Bots for Better Productivity + + +

Graphic Interface to Conversational Interface

@ Deliver me a large margherita pizza! What kind of
crust do you want? 1. Regular crust 2. Thin crust 3. Gluten free crust Thin crust cha ching!

Conversational Interface achieves: Natural user interactions with a minimal visual
interface.

No UI Clutter. Less Time Spent.

@ Where is the address of your home? It’s 325
Ruby Street. Request a ride Come over to my place!

@ Yes, get me a ride now Your driver, Sam
will pick you up in 6 minutes. Look for the red Toyota Prius!

@ Alexa UX (“Voice Chrome” Examples) https://developer.amazon.com/docs/alexa-voice-service/ux-design-overview.html Listening Speaking

Old concepts. Technology caught up with the ideas.

@ HAL9000: “I'm sorry Dave, I'm afraid I can't do
that”

@ Nuh, we’re still in control over AI. (I hope)

Conversational Interface is:

• Intuitive • Accessible • Productive

@ Messaging Platforms • Slack • Facebook Messenger • Telegram
• WeChat • Kik • Viver • LINE etc.

Messaging + Bots for More Interactive Communications

Slack bots at

TacoBot by Taco Bell https://www.tacobell.com/feed/tacobot

@ Natural Conversation APIs • DialogFlow (API.ai /Google) • Wit.ai
(Facebook) • Microsoft Bot Framework • Motion.ai • Chatbots.io • Converse AI • Recast.ai • Flow XO • ManyChat • Fluent.ai (Voice UI) etc.

AIaaS Motion.ai DialogFlow (API.ai) Artificial Intelligence as a Service

@ Hi, I want to book a flight! Yes, from
SFO. Where are you flying to? London Hi Linda! Are you flying from San Francisco, as usual? An airline customer service bot Linda may not know she is talking to a bot!

@ NLP Platforms / APIs Natural Language Processing & Cognitive
platforms: • IBM Watson • Google Cloud Natural Language API • Microsoft LUIS • Amazon Lex • Baidu UNIT

Build Your Own Conversational Interface

...with JavaScript

@ { REST } The APIs are mostly accessible with
JS

@ Example: API.ai + FB Messenger const app = require('apiai')(CLIENT_ACCESS_TOKEN);
function sendMessage(event) { let sender = event.sender.id; let text = event.message.text; let ai = app.textRequest(text, { sessionId: SESSION_STRING }); ai.on('response', (response) => { // Got a response. Let's POST to Facebook Messenger }); ai.end(); } API.ai Node.js SDK

@ Example: IBM Watson + Slack

@ Example: IBM Watson + Slack The bot workflow: 1.
Read each message on a Slack channel 2. Send the message to IBM Watson for examination 3. If the likelihood of an emotion is above the given confidence threshold post the most prominent emotion

@ Example: IBM Watson + Slack app.post('/events', (req, res) =>
{ let q = req.body; if (q.type === 'event_callback') { if(!q.event.text) return; analyzeTone(q.event); } }); Use Slack Events API to grab the text when a user post a message Pass the text data to Watson to analyze HTTP POST using ExpressJS

@ Example: IBM Watson + Slack const watson = require('watson-developer-cloud');
let tone_analyzer = watson.tone_analyzer({ username: process.env.WATSON_USERNAME, password: process.env.WATSON_PASSWORD, }); const confidencethreshold = 0.55; tone_analyzer.tone({text: text}, (err, tone) => { tone.document_tone.tone_categories.forEach((tonecategory) => { if(tonecategory.category_id === 'emotion_tone'){ tonecategory.tones.forEach((emotion) => { if(emotion.score >= confidencethreshold) { postEmotionOnSlackChannel(emotion); }}); }}); }); Returns emotions score in 0 to 1 Just initializing it w/ your API credentials Post the result on Slack

@ Example: IBM Watson + Slack + Raspberry Pi (for
fun) function colorEmotion(emotion) { if (emotion.tone_id === 'anger') { setLED(red); } else if(emotion.tone_id === 'joy') { setLED(yellow); } else if(emotion.tone_id === 'fear') { setLED(purple); } else if(emotion.tone_id === 'disgust') { setLED(green); } else if(emotion.tone_id === 'sadness') { setLED(blue); } } Change the LED color to match the emotion

@ Ack, this sucks! I want my money back! An
angry customer detected. Connect the customer with a human!

Conversational Interface with Voice in Browser?

@ Project: Artificial Voice Chat Hello! 1 2 Howdy 2.
Generate Artificial reply Using the 3rd party API 1. User talk to browser Voice command: Voice -> Text 3 3. Browser speaks back Text -> Synthetic Voice Text

@ Web Speech API & http://caniuse.com/#feat=speech-recognition http://caniuse.com/#feat=speech-synthesis 1 3

@ Web Speech API: Speech Recognition const SpeechRecognition = window.SpeechRecognition
|| window.webkitSpeechRecognition; const recognition = new SpeechRecognition(); Get an instance of the SpeechRecognition, the controller interface In the current Chromium, it is still prefixed

@ Web Speech API: Speech Recognition (Cont’d) recognition.lang = 'en-US';
recognition.interimResults = false; recognition.start(); recognition.addEventListener('result', (e) => { let last = e.results.length - 1; let text = e.results[last][0].transcript; }); Some properties Methods: start(), stop(), abort() Events: onresult, onerror, onaudiostarted, onaudioend, etc.

@ Web Speech API: Speech Recognition Hello! VOICE from a
user TEXT “hello”

@ Web Speech API: Speech Synthesis const synth = window.speechSynthesis;
const utterance = new SpeechSynthesisUtterance(); utterance.text = 'I am a robot'; utterance.pitch = 1.5; utterance.voice = 'Alex'; synth.speak(utterance); No vendor prefix Properties of the SpeechSynthesisUtterance interface Get available voices with synth.getVoices() method

@ Web Speech API: Speech Synthesis (voice by Alex or
whoever browser is) TEXT to VOICE “howdy” Howdy

Demo on Chrome https://webspeech.herokuapp.com/

@ https://www.smashingmagazine.com/2017/08/ai-chatbot-web-speech-api-node-js/

Conversational Interface is for human.

A bots interface really is a human interface.

Clippy 1997 - 2007

@ @girlie_mac girliemac.com github.com/girliemac speakerdeck.com/girlie_mac

@ Attribution: Open Emoji by Emoji-One (CC-BY 4.0)

Future of the Web with Conversational Interface...

Future of the Web with Conversational Interface (in JavaScript!)

More Decks by Tomomi Imura

Other Decks in Technology

Featured

Transcript