Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Full Stack Fest 2016: Shopify in Multiple Datac...

Full Stack Fest 2016: Shopify in Multiple Datacenters

How did Shopify evolve from a single tenant platform supporting a single snowboard store, to the massive beast it is today boasting 300,000 stores? What infrastructure challenges did we have to solve along the way? This is the story of the evolution of Shopify's infrastructure from one store to 300,000, sprinkled with the hoops of celebrities with devastating amounts of traffic.

Video: https://www.youtube.com/playlist?list=PLe9psSNJBf76DOOKMkDpyo_A5PfZk7JWc

Simon Hørup Eskildsen

September 05, 2016
Tweet

More Decks by Simon Hørup Eskildsen

Other Decks in Technology

Transcript

  1. 300,000+ SHOPS $20 BILLION+ 1500+ EMPLOYEES 1000+ SERVERS 2.5 DATACENTRES

    RUBY ON RAILS 10Ks PEAK RPS 10,000 CHECKOUTS/M PEAK 30+ DAILY DEPLOYS
  2. 8

  3. 11 Multi Tenant Single Tenant Large Capacity Low Capacity Good

    Utilization Poor Utilization Good for Flash Sales Poor for Flash Sales Cheaper Expensive No Isolation Complete Isolation Poor Scalability Great Scalability
  4. Resiliency Maturity Pyramid 23 No resiliency effort Testing with mocks

    Toxiproxy tests and resiliency matrix Resiliency Patterns Production Practise Days (Games) Kill nodes Latency Application-Specific Fallbacks Kill DC
  5. 24 https://github.com/shopify/toxiproxy Toxiproxy[:mysql_master].downstream(:latency, latency: 1000).apply do Shop.first # this takes

    at least 1s end Toxiproxy[/redis/].down do session[:user_id] # this will throw an exception end Simulate TCP conditions with Toxiproxy
  6. datacenter 1 shard 1 shard 3 shard 5 shard 2

    shard 4 shard 6 datacenter 2 shard 1 shard 3 shard 5 shard 2 shard 4 shard 6
  7. datacenter 1 shard 1 shard 3 shard 5 shard 2

    shard 4 shard 6 datacenter 2 shard 1 shard 3 shard 5 shard 2 shard 4 shard 6
  8. datacenter 1 pod 1 pod 3 pod 5 pod 2

    pod 4 pod 6 pod 1 pod 3 pod 5 pod 2 pod 4 pod 6 datacenter 2
  9. 34 Multi Tenant Single Tenant Large Capacity Low Capacity Good

    Utilization Poor Utilization Good for Flash Sales Poor for Flash Sales Cheaper Expensive No Isolation Complete Isolation Poor Scalability Great Scalability Excellent Utilization Pods in Multiple Datacenters Excellent Capacity Great for Flash Sales Cheap Pod Isolation Good Scalability
  10. pod 1 pod 3 pod 2 pod 4 pod 1

    pod 3 pod 2 pod 4 GET /products/beautiful-shoe HTTP/1.1 Host: myshop.com sorting hat
  11. rule 1: any request must be annotated with a pod

    or shop rule 2: any request can only touch one pod
  12. 42 if Shitlist.include?(klass) super else error = <<-EOE New usage

    of this API is deprecated. Please come talk to the Pods team in #pods and we'll help you out! EOE raise ShitList::Error, error end
  13. post “/webhooks/amazon?shop_id=\d+”, “webhooks#amazon”, routing_method: :shop_from_param post “/signup”, “signup#create”, routing_method: :signup

    post “/check_availability”, “signup#check_availability” routing_method: :try_all_pods post “/recover_password”, “accounts#recover”, routing_method: :try_all_pods get “*“, “storefront#index”, routing_method: :shop_from_host
  14. [ { “pattern”: “^/webhooks/amazon$”, “http”: [“POST”], “method”: “shop_from_param” }, {

    “pattern”: “^/signup/create$”, “http”: [“POST”], “method”: “signup” }, { “pattern”: “”, “http”: [“GET”, “HEAD”, “POST”, ..], “method”: “shop_from_host” } ]
  15. pod 1 pod 3 pod 2 pod 4 pod 1

    pod 3 pod 2 pod 4 GET /products/beautiful-shoe HTTP/1.1 Host: myshop.com sorting hat
  16. rule 1: any request must be annotated with a pod

    or shop rule 2: any request can only touch one pod
  17. resiliency through isolation resiliency through multi-dc multi-dc through isolation scalability

    through multi-dc, isolation and sharding simplification through isolation isolation through resiliency
  18. Gear by Alexandr Cherkinsky from the Noun Project snowboard by

    Andrew Cooper from the Noun Project database by Creative Stall from the Noun Project Bomb By Arthur Shlain from the Noun Project Line Graph by Jules Dominic from the Noun Project people pattern by Eliricon from the Noun Project online shopping by creative outlet from the Noun Project magnify by Tim Smith from the Noun Project Lightning by Sergey Demushkin from the Noun Project lever by Nick Abrams from the Noun Project train tracks by Guilhem from the Noun Project JSON File by useiconic.com from the Noun Project Gandalf by Abel Tan from the Noun Project pause by Chad Pennings from the Noun Project Truck by Lemon Liu from the Noun Project Cargo container by Icon Fair from the Noun Project