Points of VUI apps from implementing to releasing after designing it

Transcript

1. Points of VUI apps from implementing to releasing after designing it Yoichiro Tanaka, Google Developers Expert
2. Yoichiro Tanaka Software Engineer / IT Architect Google Developers Expert (Web) twitter.com/yoichiro google.com/+YoichiroTanaka
3. What does describe in this doc File points while developing the "Bulls and Cows" number-guessing app as an app for Google Assistant. • Points while designing • Points while implementing • Points at submitting for a review
4. https://assistant.google.com/services/a/uid/000000b349693dc4?hl=en
5. Happy path of Bulls and Cows
13. Points during designing
14. Tools for Design Currently, most nothing's tools for VUI designing. • Many companies and communities are finding a way. Many tools might be provided from now. • We don't have any tools, but we need some tool to design VUI apps. It is necessary to find some method by own (or refer to recommendations on Actions on Google)
15. My method 1. Write scenarios. 2. Find contexts. 3. Draw navigation between contexts. 4. Find intents, and add them to the context navigation. 5. Apply each dialog before and after each intent.
16. 1. Write scenarios
17. Color alternate rows in Google Spreadsheet in advance. Convenience!
18. First, write a Happy Path. Then, write other scenarios. Happy Path Scenario 2 Scenario 3
19. Happy Path
20. Play again
21. Help
22. Hint
23. Mistakes three times
24. 2. Find contexts
25. 2. Find contexts Search for a place where the theme of the conversation changed. Then extract before and after change as "context". • "Listen to a three-digit number, and answer the judgment result" • "After answering correctly, ask whether play again one more time" • "After three mistakes, ask whether want to hear the hint"
26. Happy Path - The theme changed from "Now, gaming" to "Ask whether plays again" -> The context changed.
27. Help - Heard the help in "Gaming", but the theme not changed from "Gaming". As the result, the context is still "Gaming".
28. Mistakes 3 times - By mistaken 3 times, the theme changed to "Asking to hear a hint". That is, the context changed.
29. 3. Draw navigation between contexts
30. "Now gaming" "Ask whether play again" "Ask whether want to hear a hint" Can understand an abstract of the conversation stream by drawing navigation between contexts.
31. 4. Find intents
32. 4. Find intents Find user's "intents (= declaration of intent, requirements)". • "Context" is for apps a bit. • "Intent" is for users.
33. Happy Path - The "input.numbers" intent changes a context (from "game" to "play_again")
34. Hint - "Hear a hint" = "help_hint" intent
35. Mistakes 3 times - The "input.numbers" intent also changes a context to "hear_hint"
36. 4. And add them to the context navigation.
37. Can understand an abstract of a stream of the conversation by drawing a chart of changing contexts.
40. 5. Apply each dialog before and after each intent.
42. Ex: Insert intents and apply dialogs between the "game" context and the "hear_hint" context.
43. Ex: Insert intents and apply dialogs between the "game" context and the "hear_hint" context. 789 1 Bull 0 Cow. Do you need a hint? Yes The sum of three digit numbers is 16.
44. Ex: Insert intents and apply dialogs between the "game" context and the "hear_hint" context. 789 Hint 1 Bull 0 Cow. Do you need a hint? Yes The sum of three digit numbers is 16. The sum of three digit numbers is 16. I see. Please say three digit numbers. 1 Bull and 0 Cow I didn't understand well. Please say three digit numbers. No 123 ??? * Rounded corners are Context, Rectangles are Intent.
45. Scenarios • Google SpreadSheet VUI design starts with scenarios VUI design • Google SpreadSheet • UML: State-chart diagram • UML: Activity diagram
46. Points during implementing
47. Points during implementing • Mapping VUI design to Dialogflow • Find actions from intents, and create a implementation code • Can end a conversation anytime
48. Mapping to Dialogflow
49. 789 1 Bull and 0 Cow. Do you need a hint? 1イート0バイトです。 Congratulation! You got three Bulls! Do you want to play again? Training phrases Input context Output context Response Intent
50. Intent Input context Output context Training phrases Response
51. Actions and Fulfillment
52. Find actions What one response is decided against one intent is rare. The response will be changed dynamically depending on user's phrases, contexts and app's statuses. → Fulfillment decides them. • Output contexts • Response To specify a process to the fulfillment, define actions.
53. Action name Use Fulfillment
54. Fulfillment The target code which developers write is this fulfillment code. • Can use something which can handle HTTP and JSON. • Generally, actions-on-google-nodejs for Node.js is used. ◦ https://github.com/actions-on-google/actions-on-google-nodejs • Fulfillment has an affinity for Cloud Functions. ◦ GCP/Firebase Cloud Functions (Free plan can be used if no external API accesses) ◦ Other alike solutions should also be no problem.
55. Dialogflow Fulfillment • Action name • Arguments (= Parameters) • ... HTTPS request
56. const DialogflowApp = require("actions-on-google").DialogflowApp; const ACTION_INPUT_WELCOME = "input.welcome"; exports.fulfillment = (req, res) => { const app = new DialogflowApp({request: req, response: res}); const inputWelcome = app => { app.ask("Hello, this is ***. ..."); }; const actionMap = new Map(); actionMap.set(ACTION_INPUT_WELCOME, inputWelcome); app.handleRequest(actionMap); }; This is a template code for Fulfillment.
57. The code which is used to decide the response. // Continue the conversation, and say a next phrase from the user. app.ask("Great! You got three bulls. Do you want to play again?"); // Specify re-prompts as 2nd argument for when the user doesn't say anything. app.ask( "Great! You got three bulls. Do you want to play again?", ["Sorry, I can't hear you well. Play again?", ...] ); // End the conversation app.tell("I see. See you again.");
58. The code which is used to decide Output Contexts // Specify the output Context. const CONTEXT_PLAY_AGAIN = "play_again"; app.setContext(CONTEXT_PLAY_AGAIN, 1); // The value "1" is for Lifespan // Delete the input Context const CONTEXT_GAME = "game"; app.setContext(GAME, 0); // Deleted by specifying "0"
59. The code which is used to keep information via multiple conversations. // Keep information within the conversation session. app.data.answer = "123"; // Keep information for a next conversation after ending the conversation. app.userStorage.answer = "123";
60. const inputNumber = app => { const number = app.getArgument("number"); const answer = app.data.answer; if (number === answer) { app.setContext("game", 0); app.setContext("play_again", 1); app.ask("Bingo! Do you want to play again?"); } else { app.setContext("game", 1); app.ask("1 bull and 0 cow."); } }; Actual code sample.
61. Must respond within 5 sec. Currently, cannot respond multiple times at once. Points of implementing Fulfillment
62. Can end the conversation anytime
63. The previous diagram is actually wrong.
64. The correct diagram is above.
65. Must be able to end the conversation by the user in all contexts. → In the review process, the reviewer will confirm whether the conversation can be ended by the phrase "Finish". → Note that the reviewer may reject the application because the reviewer cannot end the conversation with the phrase in the context answering "yes" or "no" with app.askForConfirmation()... Can end the conversation anytime
66. Points at submitting for a review
67. Registration of app information • App name, pronunciation • App description • Image, icon • Invocation phrases • Developer information • Privacy policy, Terms of use • Category • Other... Basically, need to register information to satisfied the page of the right image.
68. App name Recommend that should confirm whether your app can be invoked with the app name by Google Assistant as your expectation in advance. → Try calling the app name on Google Assistant, and know the text recognized by the assistant. → And, app name is first come, first served.
69. Image, icon The small icon is "rounded corners" icon. → Should design the icon which all corners will not be drawn.
70. Invocation phrases The reviewer confirms the app actually can be invoked with all registered phrases. → Also, on all devices. If not invoked, the application will be rejected. Especially, it is important to confirm Implicit Invocation on all devices.
71. Privacy policy, Terms of use It seems that the content of them is not checked well at the review. → However, it is important to protect me, therefore you must write them with responsibility. The review checks whether "a correct app name" is written in them.
72. Conclusion
73. Conclusion To design, implement and release smoothy your app for Google Assistant, you need to know below described in this material in advance: • Points during designing • Points during implementing • Points at submitting for a review
74. Fin.