Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Field Notes from Expeditions in the Cloud

Field Notes from Expeditions in the Cloud

Keynote from Cloud Expo 2014, in NYC.

Matt Wood

June 10, 2014
Tweet

More Decks by Matt Wood

Other Decks in Technology

Transcript

  1. from Dr. Matt Wood matthew@amazon.com General Manager, Data Science a

    presentation by Matt Wood N E S W @mza FIELD NOTES EXPEDITIONS IN THE CLOUD
  2. 14th March 2006 S3 25th AUGUST 2006 EC2 22nd OCTOBER

    2009 RDS 15th FEBRUARY 2013 Redshift 18th JANUARY 2012 DYNAMODB RETAIL MERCH AWS 280
  3. 14th March 2006 S3 25th AUGUST 2006 EC2 22nd OCTOBER

    2009 RDS 15th FEBRUARY 2013 Redshift 18th JANUARY 2012 DYNAMODB RETAIL MERCH AWS 2008 2009 2010 2011 2012 2013 159 82 61 48 24 280
  4. 14th March 2006 S3 25th AUGUST 2006 EC2 22nd OCTOBER

    2009 RDS 15th FEBRUARY 2013 Redshift 170 launches in 2014 18th JANUARY 2012 DYNAMODB RETAIL MERCH AWS 2008 2009 2010 2011 2012 2013 159 82 61 48 24 280
  5. SOLAR PANELS Notes on CAN we make A better solar

    panel?” “ Synthesize PURIFY ANALYZE PHOTO CELL?
  6. SOLAR PANELS Notes on CAN we make A better solar

    panel?” “ Synthesize PURIFY ANALYZE PHOTO CELL? GRAD STUDENT EQUIPMENT +
  7. SOLAR PANELS Notes on CAN we make A better solar

    panel?” “ 205,000 organic molecules Synthesize PURIFY ANALYZE PHOTO CELL? GRAD STUDENT EQUIPMENT +
  8. 1.21 PFLOPS 16,788 instances 264 years of compute time <18

    hours $33k COST MODELS ENABLE INNOVATION UTILIZATION
  9. 1.21 PFLOPS 16,788 instances 264 years of compute time <18

    hours $33k COST MODELS ENABLE INNOVATION UTILIZATION RESERVED CAPACITY
  10. 1.21 PFLOPS 16,788 instances 264 years of compute time <18

    hours $33k COST MODELS ENABLE INNOVATION UTILIZATION RESERVED CAPACITY ON DEMAND
  11. 1.21 PFLOPS 16,788 instances 264 years of compute time <18

    hours $33k COST MODELS ENABLE INNOVATION UTILIZATION RESERVED CAPACITY ON DEMAND
  12. VIDEO GAMES Notes on “WHO BUYS VIDEO GAMES?” 3.5 billion

    records 13 TB of click stream logs 71 million unique cookies C luster
  13. VIDEO GAMES Notes on “WHO BUYS VIDEO GAMES?” 3.5 billion

    records 13 TB of click stream logs 71 million unique cookies C luster HUGE TVs
  14. RETARGETING Notes on HOW DO WE SERVE THE BEST POSSIBLE

    AD?” ” AD SERVERS BIDDING ENGINE
  15. RETARGETING Notes on HOW DO WE SERVE THE BEST POSSIBLE

    AD?” ” AD SERVERS BIDDING ENGINE DYNAMODB
  16. RETARGETING Notes on HOW DO WE SERVE THE BEST POSSIBLE

    AD?” ” AD SERVERS BIDDING ENGINE 10ms 20MS 20MS 20MS HARD LIMIT: 100ms 10k advertisers 100 countries 1.5PB in S3 DYNAMODB
  17. DYNAMODB THROUGHPUT AVERAGE LATENCY # REQUESTS “THE LAW OF LARGE

    NUMBERS IS YOUR FRIEND UNTIL YOU REACH LARGE NUMBERS” 11/1/2013 11/7/2013
  18. DYNAMODB THROUGHPUT “THE LAW OF LARGE NUMBERS IS YOUR FRIEND

    UNTIL YOU REACH LARGE NUMBERS” PRECISE DESCRIPTION FORMAL METHODS TEST, TEST, TEST AVERAGE LATENCY # REQUESTS 11/1/2013 11/7/2013
  19. SOCIAL GAMINg Notes on CAN PHYSICAL AND VIRTUAL WORLDS INTERACT?”

    ” SOCIAL EXPERIENCE BUILT RIGHT INTO THE GAMES
  20. SOCIAL GAMINg Notes on CAN PHYSICAL AND VIRTUAL WORLDS INTERACT?”

    ” SOCIAL EXPERIENCE BUILT RIGHT INTO THE GAMES LOW LATENCY HIGH AVAILABILITY ANALYTICS
  21. CLINICAL TRIALs Notes on BUSINESS DRIVERS Speeding server provisioning for

    R&D apps Extending capacity for internal grid environments Slowing internally hosted compute infrastructure growth On-boarding security, validation services and compliance Reducing cost while extending capabilities
  22. CLINICAL TRIAL SIMULATIONS Individual Clinical Trial Simulation Run Time (Min)

    Total number of trials Number of servers Number of CPUS TOTAL ANALYSIS RUN TIME (hours) COST INTERNAL AWS 56 56 2000 2000 2 32 60 ??? $336 1.2 2048 256
  23. REDUCED BURDEN ON pEDIATRIC SUBJECTS NUmber of subjects NUmber of

    blood samples per subject length of stay per subject length of study total study cost traditional DESIGN DESIGN OPTIMIZED USING CLINICAL TRIAL SIMULATION 60 40 12 5 72 hours 2.5 years $700k $250k 1.7 years 26 hours
  24. CONNECTED DEVICES GENOMICS CLINICAL TRIALS GAMING MOBILE DEVELOPMENT ADVERTISING HOSPITALITY

    RETAIL E-COMMERCE MANUFACTURING TRAVEL FREE STEAK SOCIAL APPS FINANCIAL ANALYTICS MEDIA STREAMING DISASTER RECOVERY ARCHIVAL INTERACTIVE TV APPS WEB APPS MARS EXPLORATION
  25. SUPPLEMENT EXISTING WORKFLOWS with on-premises infrastructure AWS serves up application

    content & data Integration back to Samsung Data Centers for financial transactions
  26. ALL IN “WE HAVE TO BE GREAT AT A NUMBER

    OF THINGS... OPERATING DATA CENTERS IS NOT ONE OF THOSE THINGS” “WE ARE A HOSPITALITY MANAGEMENT COMPANY, NOT AN IT SERVICES COMPANY” “FRIENDS DON’T LET FRIENDS BUILD DATA CENTERS” INFOR CLOUDSUITE micro-vertical BUSINESS APPS
  27. IN SUMMARY... DATA DRIVES INNOVATION WHEN COMBINED WITH UTILITY COMPUTING

    Broad and deep services enable new business AGILITY REMAINS THE #1 REASON FOR ADOPTING THE AWS CLOUD ENTERPRISES LEAD WITh DEV, TEST & NEW WORKLOADS MIGRATION AND SUPPLEMENTS QUICKLY FOLLOW Increasingly ‘ALL IN’
  28. More sessions from AWS today 11.00am: Transforming Applications into SaaS

    11.40am: Big Data Is Best Served Fresh: Real-time Analytics in the Cloud 1.55pm: What is the Internet of Things – and how can it help my business? 4.00pm: Moving Enterprises to the Cloud the Right Way and for the Right Reasons 4.40pm: Creating a Cloud Based DR Solution for your Enterprise