Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Smart Applications with Conversational...

Building Smart Applications with Conversational Interfaces and Language Services [AWS Summit @ Berlin]

How do you build an application that is at the same time scalable, maintainable, cost-effective, smart, and conversational? The Cloud now brings natural language processing, automatic speech recognition, text-to-speech, and neural machine translation technologies within reach of every developer. In this session, I will discuss how you can easily add intelligence to any application with solution-oriented machine learning services that provide speech, language, and chatbot functionalities on AWS.

Alex Casalboni

February 26, 2019
Tweet

More Decks by Alex Casalboni

Other Decks in Technology

Transcript

  1. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Building Smart Applications with Conversational Interfaces and Language Services Alex Casalboni Technical Evangelist, AWS
  2. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T About me • Software Engineer & Web Developer • Data Science background • Startupper for 4.5 years • ServerlessDays Organizer • AWS Customer since 2013
  3. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Put machine learning in the hands of every developer Our mission
  4. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Some of our machine learning customers…
  5. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T M L F R A M E W O R K S & I N F R A S T R U C T U R E Amazon ML Stack A I S E R V I C E S R E K O G N I T I O N I M A G E P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X R E K O G N I T I O N V I D E O Vision Speech Chatbots A M A Z O N S A G E M A K E R B U I L D T R A I N F O R E C A S T T E X T R A C T P E R S O N A L I Z E D E P L O Y M L O P S & A L G O S F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e E C 2 P 3 & P 3 d n E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C I N F E R E N C E Language Forecasting Recommendations NEW NEW NEW NEW
  6. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Language capabilities Vision (OCR) Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Chatbot Interface
  7. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Language capabilities on AWS Vision (OCR) Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Chatbot Interface Amazon Rekognition & Textract Amazon Transcribe Amazon Polly Amazon Comprehend Amazon Translate Amazon Lex
  8. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Audio Input Example: End-to-end audio analysis store Amazon S3 trigger AWS Lambda call Amazon Transcribe in cascade Amazon Comprehend aggregate Amazon Athena analyze Amazon QuickSight
  9. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Example: Media analysis solution S3: Web UI aws.amazon.com/answers/media-entertainment/media-analysis-solution/ S3: Media storage Step Functions: Orchestrate analysis Elasticsearch: Search index Amazon Rekognition Video: Detect objects, scenes, faces, & celebrities AWS Elemental MediaConvert: Transcode videos Transcribe Comprehend Lambda API Gateway: REST API Lambda
  10. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Amazon Rekognition + Amazon Textract
  11. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Rekognition – Image and Video Analysis
  12. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Optical Character Recognition (OCR)
  13. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Textract – OCR++
  14. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Textract – How it works
  15. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Turn text into lifelike speech using deep learning
  16. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Polly • Content creation • Mobile & desktop applications • Internet of Things (IoT) • Education & e-learning • Telephony • Game development Use cases • 58 voices across 28 languages • Lip-syncing & text highlighting • Fine-grained voice control • Custom vocabularies • Available in 18 AWS Regions Key features
  17. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Polly “Hallo! Mein Name ist…” Text-to-speech (TTS)
  18. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Synthesize Speech API $ aws polly synthesize-speech --text "hello" --voice-id Matthew --output-format mp3 [--lexicon-names mylex1 mylex2] output.mp3 { "ContentType": "audio/mpeg", "RequestCharacters": "11" }
  19. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T “With Amazon Polly our users benefit from the most lifelike Text-to-Speech voices available on the market.” Severin Hacker CTO, Duolingo
  20. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Turn speech into text
  21. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T “Hello, this is Allan speaking” Amazon Transcribe Speech-to-text (STT)
  22. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T English Italian French Spanish Portuguese New! Supported languages * * more languages coming soon!
  23. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Use cases Call centers Subtitles for VOD Broadcast closed captions Transcribe meetings
  24. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T ringDNA End-to-end communications platform for sales teams Enterprise organizations use RingDNA to dramatically increase productivity, engage in smarter sales conversations, gain predictive sales insights and improve their win rate Speech to Text "A critical component of RingDNA’s Conversation AI requires best of breed speech-to-text to deliver transcriptions of every phone call. RingDNA is excited about Amazon Transcribe since it provides high-quality speech recognition at scale, helping us to better transcribe every call to text " Howard Brown, CEO & Founder, RingDNA https://www.youtube.com/watch?v=1ZJ_f1bDdog
  25. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Discover insights and relationships in text
  26. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Sentiment Entities Languages Keyphrases Topic modeling Syntax Natural Language Understanding (NLU)
  27. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T English Italian French German Portuguese New! Supported Languages
  28. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Syntax detection $ aws comprehend detect-syntax --language-code 'en' --text 'I love cloud!’ { "SyntaxTokens": [ { "TokenId": 1, "Text": "I", "BeginOffset": 0, "EndOffset": 1, "PartOfSpeech": { "Tag": "PRON", "Score": 0.9999802112579346 } }, ...
  29. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Sentiment Analysis $ aws comprehend detect-sentiment --language-code 'en' --text 'I love cloud!’ { "Sentiment": "POSITIVE”, "SentimentScore": { "Mixed": 0.012617903761565685, "Positive": 0.9599817991256714, "Neutral": 0.021758323535323143, "Negative": 0.005641999188810587 } }
  30. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T « Amazon Comprehend helps us analyze the key sentiments, objects, and geos in our 30 million plus reviews & testimonies. Now we are able to discover new insights into the unique experiences available at each property, so our customers can make the best decision possible for their travel.” Matt Fryer, VP and Chief Data Science Officer, Hotels.com
  31. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Extract health information from patient notes, clinical trial reports, and other electronic health records
  32. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Comprehend Medical
  33. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Yes, natural language translation J
  34. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Supported languages * Arabic Simplified Chinese French German Spanish Portuguese Japanese Traditional Chinese Italian Russian Turkish Czech * 417 translation combinations Danish Dutch Finnish Swedish Polish Hebrew
  35. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T «We operate 90 localized websites in 41 languages. (…) Having evaluated Amazon Translate and several other solutions, we believe that Amazon Translate presents a quick, efficient and most importantly, accurate solution. » Matt Fryer, VP and Chief Data Science Officer, Hotels.com
  36. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Conversational interfaces for your applications powered by the same deep learning technologies as Alexa
  37. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Lex End to End Speech to Intent ASR+NLU integrated into one API Dialog Management Native support & maintains context Text to Speech Amazon Polly integrated into API Business Logic Native integration with AWS Lambda Deployment One click deployment Security Encrypted data in transit & at rest Scale Completely managed service Analytics Monitor and improve End to End
  38. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Connect Self-service, cloud-based contact center service Real time and historical analytics High-quality voice capability Call recording Skills-based routing [Automatic Call Distribution (ACD)]
  39. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Intelligent call center chatbot Amazon Connect Customer Amazon Lex Lambda: Fulfillment DynamoDB: Customer Data SNS: SMS Messaging Customer calls Connect to reschedule an appointment Connect calls Lex chatbot Lex chatbot calls Lambda function to get customer preferences and fulfil Intents Lambda function sends text message confirmation via SNS Customer receives appointment confirmation text message Lambda function writes updates to DynamoDB
  40. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved.
  41. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved.
  42. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved.