Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using AWS Lambda with Ruby on a large-scale system

Using AWS Lambda with Ruby on a large-scale system

Ruby Conf TW 2019
Official site
https://2019.rubyconf.tw/#codingluka

The serverless framework is very popular for the last 5 years. AWS Lambda is the most important service in the Serverless framework.

AWS Lambda supports Ruby runtime on November 29, 2018. Now we can develop AWS lambda with Ruby. But what situation should we use AWS Lambda?

The key point in this talk is "Why we choose AWS Lambda as a solution on a large-scale system?".

The first question is "What are pain points when we want to implement a new feature on a large-scale system?".

And then let's think about "How AWS Lambda solve these pain points?".

I will share some experiences about developing features with AWS Lambda. Discussing what situation should we use AWS Lambda.

Here’s what you will learn:
- What is AWS Lambda?
- What is the serverless framework?
- What is event-driven programming?
- What are pain points when we develop features on large-scale system?
- How AWS Lambda solves these pain points?
- How to develop AWS Lambda with ruby?

Luka Huang

July 27, 2019
Tweet

More Decks by Luka Huang

Other Decks in Programming

Transcript

  1. REMOTE DESKTOP SOFTWARE SOLUTIONS Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  2. CURRENTLY HAVE MORE THAN 
 20 MILLION USERS Using AWS

    Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  3. ONLINE COMPUTERS > ? MILLION Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  4. I AM A DEVELOPER
 IN BACK-END TEAM. Using AWS Lambda

    with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  5. BACK-END = Using AWS Lambda with Ruby on a large-scale

    system, Luka Huang ©RubyConf TW 2019
  6. BACK-END = + CURRENTLY Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  7. WHY AWS LAMBDA? Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  8. 10 WHY AWS LAMBDA? WHEN SYSTEM BECAME LARGE, THERE ARE

    MANY PAIN POINTS TO SOLVE Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  9. SO I PREPARED THIS TALK TO INTRODUCE LARGE-SCALE SYSTEM. 


    AND POINT OUT THE PAIN POINTS. Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  10. HOW TO SOLVE PAIN POINTS VIA AWS LAMBDA Using AWS

    Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  11. IN THIS TALK. I’ LL SHARE PRINCIPLES TO USE +

    Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  12. INTRODUCTION TO AWS LAMBDA AND SERVERLESS Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  13. CORE OF SERVERLESS Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  14. EVERY INSTANCE HAS FULL FEATURES MONOLITHIC Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  15. WHAT IS SERVERLESS? Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  16. Monolithic Micro Service Function MONOLITHIC Micro Service Micro Service Micro

    Service Function Function Function Function Function Function Function Function MICRO SERVICE SERVERLESS Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  17. 28 
 GOOD NEWS Using AWS Lambda with Ruby on

    a large-scale system ,Luka Huang ©RubyConf TW 2019
  18. IF YOU ARE NOT USING AWS… Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  19. NO WORRIES Using AWS Lambda with Ruby on a large-scale

    system, Luka Huang ©RubyConf TW 2019
  20. PROVIDERS SUPPORT SERVERLESS Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  21. Azure Cloud Function AWS Lambda Google Cloud Function DIFFERENT NAMES,

    SAME THING IBM OpenWhisk Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  22. LAMBDA EXECUTION MODEL Synchronous Asynchronous (Event) Stream-based Amazon API Gateway

    Amazon S3 Amazon SQS /orders Amazon DynamoDb Amazon 
 Kinesis If data changed Lambda function Lambda function Lambda service Lambda function Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  23. MODEL 1 - SYNCHRONOUS Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  24. SERVERLESS WEB APPLICATION /orders Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  25. VISIT /ORDERS PAGE GET /orders /orders Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  26. CREATE ORDER POST /order /order Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  27. MODEL 2 - ASYNCHRONOUS Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  28. EVENT DRIVEN Event Source Lambda function Services Node.js Python
 Java

    C#
 Golang Ruby!! Request to endpoints Changes in resource state Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  29. EVENT SOURCES THAT TRIGGER LAMBDA Data Stores Endpoints Configuration Repositories

    Event / Message Service https://www.slideshare.net/BoazZiniman/introduction-to-serverless-computing-and-aws-lambda-floor28 Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  30. PROS OF SERVERLESS No Servers to provision or manage Scales

    with usage Never pay for idle Built-in availability and fault tolerance Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  31. WAYS TO DEVELOP LAMBDA WITH RUBY Using AWS Lambda with

    Ruby on a large-scale system ,Luka Huang ©RubyConf TW 2019
  32. #1 VIA AWS WEB CONSOLE Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  33. WHEN CLICK ADD TRIGGER Click Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  34. SELECT EVENT SOURCE AS TRIGGER Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  35. A TRIGGER IS ADDED Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  36. LAMBDA FUNCTION WITH RUBY Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  37. TEST LAMBDA VIA EVENT Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  38. EXECUTION RESULT Using AWS Lambda with Ruby on a large-scale

    system, Luka Huang ©RubyConf TW 2019
  39. #2 VIA AWS CLI Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  40. EDIT THE FUNCTION Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  41. ZIP IT Using AWS Lambda with Ruby on a large-scale

    system, Luka Huang ©RubyConf TW 2019
  42. UPDATE LAMBDA VIA AWS CLI Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  43. TEST ON AWS CONSOLE Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  44. #3 VIA SERVERLESS FRAMEWORK Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  45. SERVERLESS The most popular Serverless framework Node.js Python
 Java C#


    Golang Ruby Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  46. RUBY ON JETS Serverless framework build by Ruby Using AWS

    Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  47. #4 VIA TERRAFORM Infrastructure as code Lambda development + Using

    AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  48. INTRODUCTION TO LARGE- SCALE SYSTEM Using AWS Lambda with Ruby

    on a large-scale system ,Luka Huang ©RubyConf TW 2019
  49. WHAT IS LARGE-SCALE SYSTEM? Using AWS Lambda with Ruby on

    a large-scale system ,Luka Huang ©RubyConf TW 2019
  50. “IS DISTRIBUTED CRAWLER SYSTEM A KIND OF A LARGE-SCALE SYSTEM?”

    My friend ask me a question: Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  51. DDOS sites to get information Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  52. Also DDOS his Database Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  53. QPS is really high. So is it a large-scale system?

    Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  54. Crawler system could have some downtime. Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  55. Large-scale system must be reliable, right? Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  56. INTRODUCTION TO LARGE-SCALE SYSTEM Using AWS Lambda with Ruby on

    a large-scale system ,Luka Huang ©RubyConf TW 2019
  57. A SIMPLE ARCHITECTURE Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  58. MAKE IT FASTER Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  59. AND FASTER Using AWS Lambda with Ruby on a large-scale

    system, Luka Huang ©RubyConf TW 2019
  60. DB TOPIC 1 - READ REPLICA Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  61. READ REPLICA TROUBLESHOOTING Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  62. DB TOPIC 2 - HOT STANDBY Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  63. DB TOPIC 3 - REDIS Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  64. WORKER h%FMBZFEKPC %BUBCBTF MJOL h4JEFLJR1SP 3FEJT MJOL h4IPSZVLFO 424 MJOL

    h0UIFST Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  65. PERFORMANCE TUNING FOR API SERVERS Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  66. CHALLENGES ON LARGE- SCALE SYSTEM Using AWS Lambda with Ruby

    on a large-scale system ,Luka Huang ©RubyConf TW 2019
  67. KINDS OF HIGH THROUPUT Using AWS Lambda with Ruby on

    a large-scale system ,Luka Huang ©RubyConf TW 2019
  68. INCREASE 30% PER YEAR Throughput Time Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  69. RAPIDLY INCREASING IN SHORT TIME Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  70. INCREASE 500% IN 10 MINUTES Throughput Time Using AWS Lambda

    with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  71. We want to get rid heavy-load API of our main

    service Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  72. Rails is Monolithic. Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  73. A single purpose API still load all code of Rails.

    It is not efficient. Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  74. We’ll use AWS Lambda to solve it. Using AWS Lambda

    with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  75. REAL WORLD USE CASE Using AWS Lambda with Ruby on

    a large-scale system ,Luka Huang ©RubyConf TW 2019
  76. USE CASE 1 - UPLOAD FILE AND SEND NOTIFICATION Using

    AWS Lambda with Ruby on a large-scale system ,Luka Huang ©RubyConf TW 2019
  77. GENERAL FILE UPLOAD Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  78. DIRECTLY UPLOAD TO S3 ( ActiveStorage supports directly upload to

    S3. ) Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  79. NOTIFY USER - POLLING Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  80. NOTIFY USER - POLLING Heavy load Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  81. NOTIFY USER 
 BY LAMBDA Using AWS Lambda with Ruby

    on a large-scale system, Luka Huang ©RubyConf TW 2019
  82. USE CASE 2 - IMAGE RESIZE Using AWS Lambda with

    Ruby on a large-scale system ,Luka Huang ©RubyConf TW 2019
  83. RESIZE IMAGE Using AWS Lambda with Ruby on a large-scale

    system, Luka Huang ©RubyConf TW 2019
  84. RESIZE IMAGE Heavy load Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  85. DIRECTLY UPLOAD Using AWS Lambda with Ruby on a large-scale

    system, Luka Huang ©RubyConf TW 2019
  86. IMAGE RESIZE BY LAMBDA Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  87. USE CASE 3 - WEBSOCKET Using AWS Lambda with Ruby

    on a large-scale system ,Luka Huang ©RubyConf TW 2019
  88. WEBSOCKET IS A PAIN POINT h-POHDPOOFDUJPO h)BSEUPJNQMFNFOU h1FSGPSNBODFJOSVCZJTOPUHPPE Using AWS

    Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  89. WEBSOCKET PERFORMANCE Both machines are bare metal 4ghz i7 4790Ks

    with 16GB of RAM Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  90. AWS WEBSOCKET API GATEWAY Using AWS Lambda with Ruby on

    a large-scale system, Luka Huang ©RubyConf TW 2019
  91. WEBSOCKET API GATEWAY Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019
  92. ALL OF LAMBDA IS USING RUBY Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  93. SERVERLESS + WEBSOCKET + RUBY = :&4 Using AWS Lambda

    with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  94. IS RAILS + AWS LAMBDA MAKE SENSE ? Using AWS

    Lambda with Ruby on a large-scale system ,Luka Huang ©RubyConf TW 2019
  95. IS RAILS + AWS LAMBDA MAKE SENSE ? + =

    :&4 Using AWS Lambda with Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  96. IF YOU’RE INTERESTED IN THIS TALK Using AWS Lambda with

    Ruby on a large-scale system, Luka Huang ©RubyConf TW 2019
  97. WE’ RE HIRING Using AWS Lambda with Ruby on a

    large-scale system, Luka Huang ©RubyConf TW 2019