Scale Oriented Architecture

This talk looks at the science of building web services, thinking about APIs from their most basic primitives, and the lessons learned at Twilio and Runscope for supporting APIs at massive scale. (Presented at Silicon Valley Code Camp 2013)

Transcript

1. Scale-Oriented Architecture Frank Stratton @FrankRStratton
2. Runscope
3. Runscope API Client HTTP API Server HTTP
4. Runscope
5. Runscope
6. Runscope API Client HTTP API Server HTTP
9. Dashboard Traffic Gateway API
10. Dashboard Traffic Gateway API Request Vault Identity
11. Dashboard Traffic Gateway API Request Vault Identity Eventador Calculon Courier Warp Pipe Mission Control Prometheus Vivaldi Passageway
12. Dashboard Traffic Gateway API Request Vault Identity Eventador Calculon Courier Warp Pipe Mission Control Prometheus Vivaldi Passageway
13. “The largest architectural change we made was to move from our monolithic Ruby application to one that is more services oriented.” https://blog.twitter.com/2013/new-tweets-per-second-record-and-how
15. S O A ERVICE RIENTED RCHITECTURE S
16. S O A RIENTED RCHITECTURE S
17. S O A RIENTED RCHITECTURE CALE S
18. SOA 2.0
19. SOA 2.0 Scalable Architectures
20. SOA 2.0 Scalable Architectures Scalable Teams
21. Scalable Architectures Dashboard Traffic Gateway API Request Vault Identity Dashboard Dashboard Dashboard Dashboard Identity Identity Identity Identity Identity Request Vault Request Vault Traffic Gateway Scale Horizontally API API
22. Scalable Architectures Identity Identity Identity Identity Scale Horizontally HaProxy Nginx etc...
23. Scalable Architectures Dashboard Traffic Gateway API Request Vault Identity Dashboard Dashboard Dashboard Dashboard Identity Identity Identity Identity Identity Request Vault Request Vault Traffic Gateway Scale Horizontally API API
24. Scalable Architectures Dashboard Traffic Gateway Request Vault Identity Dashboard Dashboard Dashboard Dashboard Identity Identity Identity Identity Identity Request Vault Request Vault Traffic Gateway Fail Independently API API API
25. Scalable Architectures Fail Independently Request Vault Identity Eventador Calculon Courier Warp Pipe Mission Control Prometheus Vivaldi Passageway Dashboard Traffic Gateway API
26. master
27. master
28. master
29. master
30. master
31. master
32. master
33. master
34. master
35. master
36. master
37. Scalable Teams Request Vault Identity Dashboard Traffic Gateway API
38. Scalable Teams Deploy Independently Request Vault Identity Dashboard Traffic Gateway API
39. Scalable Teams Deploy Independently Request Vault Identity Dashboard Traffic Gateway API v3 v5 v20 v45 v2
40. Dashboard Traffic Gateway API Request Vault Identity Eventador Calculon Courier Warp Pipe Mission Control Prometheus Vivaldi Passageway Scalable Teams
41. Dashboard Traffic Gateway API Postgres
42. Identity Postgres Memcache Flask Scalable Teams Dashboard Traffic Gateway API Request Vault
43. Identity GET /teams/<id> PUT /teams/<id> GET /buckets/<id> GET /buckets ... Scalable Teams Dashboard Traffic Gateway API Request Vault
44. GET /teams/<id> PUT /teams/<id> GET /buckets/<id> GET /buckets ...
45. have you built an HTTP/REST API?
46. GET
47. POST
48. PUT
49. PUT is idempotent
50. PUT import requests resp = requests.put(“http://127.0.0.1/buckets/1234”, data={ ... }) if resp.ok: return resp.json() # else, raise an error?! raise Exception(“Bucket didn’t return 200 OK”)
51. PUT import requests retries = 3 for i in range(retries): resp = requests.put(“http://127.0.0.1/buckets/1234”, data={ ... }) if resp.ok: return resp.json()
52. Dashboard Traffic Gateway API Request Vault Identity PUT /requests/<uuid> PUT /responses/<uuid>
53. Fail Fast & Retry
54. Dashboard Traffic Gateway API Request Vault Request Vault Request Vault
55. Queue All The Things
56. Redis as a Queue import redis r = redis.StrictRedis() def queue_item(item): r.lpush(“work_queue”, item) def get_item(): return r.rpop(“work_queue”) class Worker(Greenlet): def _run(): while True: item = get_item() if item: # do work else: sleep(1)
57. Redis as a Queue import redis r = redis.StrictRedis() def queue_item(item): r.lpush(“work_queue”, item) def get_item(): return r.rpoplpush( “work_queue”, “in_progress”) def finish_task(item): r.lrem(“in_progress”, 1, item) class Worker(Greenlet): def _run(): while True: item = get_item() if item: # do work, then finish_task(item) else: sleep(1)
58. Redis as a Queue import redis r = redis.StrictRedis() def queue_item(item): r.lpush(“work_queue”, item) def get_item(): return r.rpoplpush( “work_queue”, “in_progress”) def finish_task(item): r.lrem(“in_progress”, 1, item) def cleanup(): items = r.lrange( “in_progress”, 0, -1) r.rpush(“work_queue”, *items) r.delete(“in_progress”)
59. Redis as a Queue class Worker(Greenlet): def _run(): while True: item = get_item() if item: do_work(item) finish_task(item) else: sleep(1) def do_work(item): item[‘formatted_name’] = string.capwords(item[‘name’]) requests.put(“http://my-service/items/item[‘id’]”, data=item)
60. Dashboard Traffic Gateway API Request Vault Identity Redis
62. Dashboard Traffic Gateway API Request Vault Identity Eventador Calculon Courier Warp Pipe Mission Control Prometheus Vivaldi Passageway
63. See Your Data
64. Dashboard Traffic Gateway API Request Vault Identity Eventador Calculon Courier Warp Pipe Mission Control Prometheus Vivaldi Passageway
65. Runscope
66. Questions? Frank Stratton Twitter: @FrankRStratton