Building Scalable Stateful Services

Slides from Scaling Stateful Services Talk given at Nike Tech Talk 8/11/16

Transcript

1. Building Scalable Stateful Services
2. Caitie McCaffrey Distributed Systems Engineer @caitie caitiem.com
3. Stateless Services Service
4. Stateless Services Service
5. Stateless Services Service Service
6. Stateless Services Service Service
7. Stateless Services Service Service
8. Stateless Services Service Service Service
9. Stateless Services Service Service Service
10. Stateless Services Service Service Service
11. Stateless Services Service Service Service Service
12. Stateless Services Service Service Service Service
13. Stateless Services Service Service Service Service
14. Stateless Services Service Service Service Service Service
15. Stateless Services Service Service Service Service Service
16. Data Shipping Paradigm Service Service Service
17. Data Shipping Paradigm Service Service Service
18. Data Shipping Paradigm Service Service Service
19. Data Shipping Paradigm Service Service Service
20. Data Shipping Paradigm Service Service Service
21. Data Shipping Paradigm Service Service Service
22. Data Shipping Paradigm Service Service Service
23. Data Shipping Paradigm Service Service Service
24. Data Shipping Paradigm Service Service Service
25. Data Shipping Paradigm Service Service Service
26. Data Shipping Paradigm Service Service Service
27. Data Shipping Paradigm Service Service Service
28. Data Shipping Paradigm Service Service Service
29. Data Shipping Paradigm Service Service Service
30. Data Shipping Paradigm Service Service Service
31. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
32. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
33. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
34. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
35. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
36. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
37. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
38. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
39. Overview Benefits Building Real World Caution
40. Data Locality For Low Latency & Data Intensive Services
41. Function Shipping Paradigm Service Service Service
42. Function Shipping Paradigm Service Service Service
43. Function Shipping Paradigm Service Service Service
44. Function Shipping Paradigm Service Service Service
45. Function Shipping Paradigm Service Service Service
46. Function Shipping Paradigm Service Service Service
47. Function Shipping Paradigm Service Service Service
48. Function Shipping Paradigm Service Service Service
49. Function Shipping Paradigm Service Service Service
50. Function Shipping Paradigm Service Service Service
51. Function Shipping Paradigm Service Service Service
52. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
53. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
54. Service Service Service Service Service Service Service Front End Data Shipping Paradigm
55. Just Put A Cache On It?
56. Service Service Service Service Service Service Service Front End Cache
57. Service Service Service Service Service Service Service Front End Cache
58. Service Service Service Service Service Service Service Front End Cache
59. Concurrency Issues Greater Operational Burden Problems with Caching
60. Sticky Connections & Consistency Additional Available Consistency Models
61. Sticky Connections
62. Sticky Connections
63. Linearizable Sequential Causal Pipelined Random Access Memory Read Your Write Monotonic Read Monotonic Write Write From Read Consistency Models CP Consistency AP Consistency
64. Linearizable Sequential Causal Pipelined Random Access Memory Read Your Write Monotonic Read Monotonic Write Write From Read CP Consistency AP Consistency AP Consistency w/ Sticky Connections Consistency Models
65. - Werner Vogel 2007 “Whether or not read-your-write, session and monotonic consistency can be achieved depends in general on the "stickiness" of clients to the server that executes the distributed protocol for them… Using sessions, which are sticky, makes this explicit and provides an exposure level that clients can reason about.”
66. Building Sticky Connections For Low Latency & Data Intensive Services
67. Building Sticky Connections
68. Building Sticky Connections
69. Persistent Connections Load Balancing Problems No Stickiness Once Connection Breaks Problems
70. Persistent Connections Load Balancing Problems No Stickiness Once Connection Breaks Problems
71. Routing Logic • Cluster Membership • Work Distribution Problems to Solve
72. Routing Logic • Cluster Membership • Work Distribution Problems to Solve
73. Routing Logic • Cluster Membership • Work Distribution Problems to Solve
74. Cluster Membership Dynamic Static
75. Static Cluster Membership Machine Outage means Partial Service Outage Downtime to Add Capacity
76. Static Cluster Membership Machine Outage means Partial Service Outage Downtime to Add Capacity
77. Static Cluster Membership Machine Outage means Partial Service Outage Downtime to Add Capacity
78. Dynamic Cluster Membership Fault Tolerance
79. Dynamic Cluster Membership Fault Tolerance
80. Dynamic Cluster Membership Fault Tolerance
81. Dynamic Cluster Membership Fault Tolerance
82. Dynamic Cluster Membership Fault Tolerance
83. Dynamic Cluster Membership Gossip Protocols Consensus Systems Availability vs Consistency
84. Consensus Systems for Consistency Paxos ETCD Zookeeper Raft
85. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
86. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
87. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
88. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
89. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
90. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
91. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
92. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
93. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information Dissemination • Pairwise Communication
94. Work Distribution Consistent Hashing Distributed Hash Tables Random Placement
95. Random Placement Write Anywhere Read from Everywhere
96. Random Placement Write Anywhere Read from Everywhere
97. Random Placement Write Anywhere Read from Everywhere
98. Random Placement Write Anywhere Read from Everywhere
99. Random Placement Write Anywhere Read from Everywhere
100. Random Placement Write Anywhere Read from Everywhere
101. Random Placement Write Anywhere Read from Everywhere
102. Consistent Hashing Deterministic Placement Node A Node B Node C NodeD Consistent Hashing & Random Trees: Distributed caching protocols for relieving hot spots on the World Wide Web
103. Consistent Hashing Deterministic Placement Node A Node B NodeD Consistent Hashing & Random Trees: Distributed caching protocols for relieving hot spots on the World Wide Web
104. Consistent Hashing Deterministic Placement Node A Node B NodeD Consistent Hashing & Random Trees: Distributed caching protocols for relieving hot spots on the World Wide Web
105. Distributed Hash Table Non- Deterministic Placement Node B Node A Node C
106. Distributed Hash Table Non- Deterministic Placement Node B Node A Node C
107. Distributed Hash Table Non- Deterministic Placement Node B Node A Node C
108. Distributed Hash Table Non- Deterministic Placement Node B Node A Node C
109. Distributed Hash Table Non- Deterministic Placement Node B Node A Node C
110. Stateful Services In the Real World
111. Scuba is a fast, scalable, distributed, in-memory database built at Facebook. It is the workhorse behind code regression analysis & bug report, revenue, and performance debugging Fan-out request to all machines in the cluster Compose Results Return Results and Completeness
112. Scuba is a fast, scalable, distributed, in-memory database built at Facebook. It is the workhorse behind code regression analysis & bug report, revenue, and performance debugging Fan-out request to all machines in the cluster Compose Results Return Results and Completeness
113. Scuba is a fast, scalable, distributed, in-memory database built at Facebook. It is the workhorse behind code regression analysis & bug report, revenue, and performance debugging Fan-out request to all machines in the cluster Compose Results Return Results and Completeness
114. Uber Ringpop is an open- source Node.js library that brings application-layer sharding to many of their dispatching platform services. Swim Gossip Protocol Consistent Hashing +
115. Uber Ringpop is an open- source Node.js library that brings application-layer sharding to many of their dispatching platform services. Swim Gossip Protocol Consistent Hashing +
116. Uber Ringpop is an open- source Node.js library that brings application-layer sharding to many of their dispatching platform services. Swim Gossip Protocol Consistent Hashing +
117. Orleans Cluster Orleans is a runtime and Programming model for building distributed systems based on the Actor Model from the eXtreme Computing Group at MSR Gossip Protocol Consistent Hashing + + Distributed Hash Table Actor Actor Actor Actor Actor
118. Orleans Cluster Orleans is a runtime and Programming model for building distributed systems based on the Actor Model from the eXtreme Computing Group at MSR Gossip Protocol Consistent Hashing + + Distributed Hash Table Actor Actor Actor Actor Actor
119. Orleans Cluster Orleans is a runtime and Programming model for building distributed systems based on the Actor Model from the eXtreme Computing Group at MSR Gossip Protocol Consistent Hashing + + Distributed Hash Table Actor Actor Actor Actor Actor
120. Orleans Cluster Orleans Distributed Hash Table
121. Orleans Cluster Orleans Distributed Hash Table
122. Orleans Cluster Orleans Distributed Hash Table
123. Orleans Cluster Orleans Distributed Hash Table
124. Orleans Cluster Orleans Distributed Hash Table
125. Orleans Cluster Orleans Distributed Hash Table
126. Orleans Cluster Orleans Distributed Hash Table
127. Orleans Cluster Orleans Distributed Hash Table
128. Caution Stateful Services Ahead
129. Unbounded Data Structures Implicit Assumptions are the Killer of Distributed Systems
130. Memory Management Get Ready to Make Friends with the Garbage Collector Profiler
131. Reloading State • First Connection • Recovering From Crashes • Deploying New Code
132. Fast Restarts at Facebook “Our Key Observation is that we can decouple the memory lifetime from the process lifetime. When we shutdown a server for a planned upgrade.”
133. Make Assumptions Explicit for Reliable Distributed Systems
134. Conclusion Data Locality & Available Consistency Cluster Membership & Work Distribution Successful Stateful Real World Systems Caution: Some New Challenges
135. https://github.com/CaitieM20/ScalingStatefulServices @Caitie Questions