Building a Realtime WebSocket API (BEAM Edition)

Sometimes consumers of your APIs require near-realtime communication because regular RESTful HTTP apis can be a few milliseconds too slow. These performant and scalable APIs can be made over websocket TCP connections where events are pushed from client and server in near-realtime fashion.

This talk is a story of how I built such an API. We'll look at why this decision to create a websocket API was made and we will take a look at the data that supported this decision. We will take a deep dive into Phoenix websockets, channels, and transports to expose the underlying architecture. Finally, we look at how we tested the API, how we authenticated users over the channels, and how Phoenix helped this all happen with relative ease.

Transcript

1. Websocket API CodeBEAM SF 2020 Building a real time ⋆ ⋆ in Phoenix
2. @jwright #CodeBEAMSF
3. jamie @brilliantfantastic.com #CodeBEAMSF
4. @jwright http://speakerdeck.com/jwright
5. Background Some #CodeBEAMSF
6. Background Some #CodeBEAMSF
7. Background Some #CodeBEAMSF
8. Background Some http://chronic.io
9. Background Some
10. Clients Multiple
11. Source One #CodeBEAMSF
12. Server is gold copy Source One #CodeBEAMSF
13. Server is gold copy Server pushes events to clients Source One #CodeBEAMSF
14. Server is gold copy Server pushes events to clients Clients displays event Source One #CodeBEAMSF
15. Tick Events at any second #CodeBEAMSF
16. Tick { remaining_time: 1497 } { remaining_time: 1497 } #CodeBEAMSF
17. Plugins Events at any second #CodeBEAMSF
18. Plugins Events at any second  #CodeBEAMSF
19. Timers Team { timers: { id: 724 } }
20. Timers Team Suzie’s iMac Leon’s MacBook Sandra’s iPhone Sandra’s Slack App { timers: { id: 724 } }
21. Background Some Demo #CodeBEAMSF
24. Websockets #CodeBEAMSF
25. Websocket What’s a #CodeBEAMSF
26. Websocket What’s a #CodeBEAMSF
27. Websocket What’s a Protocol #CodeBEAMSF
28. Websocket #CodeBEAMSF
29. Websocket Full-duplex connection #CodeBEAMSF
30. Websocket Full-duplex connection Both sides can chat at same time #CodeBEAMSF
31. Http #CodeBEAMSF
32. Http Simplex connection #CodeBEAMSF
33. Http Simplex connection One side can chat at a time #CodeBEAMSF
34. Websocket What’s a https://tools.ietf.org/html/rfc6455
35. Websocket What’s a https://tools.ietf.org/html/rfc6455 RFC
36. Websocket What’s a https://www.w3.org/TR/websockets/
37. Websocket What’s a https://www.w3.org/TR/websockets/ W3C Spec
38. Websocket What’s a
39. Websocket What’s a
40. Websocket What’s a
41. Websocket What’s a
42. Websocket What’s a #CodeBEAMSF
43. Websocket What’s a Starts with a Handshakes via HTTP #CodeBEAMSF
44. #CodeBEAMSF
45. #CodeBEAMSF
46. #CodeBEAMSF
47. #CodeBEAMSF
49. TCP (C’est une pipe)
50. TCP (C’est une pipe) HTTP
51. TCP (C’est une pipe) HTTP W ebSocket
52. Phoenix Why
53. Phoenix Why #CodeBEAMSF
54. Phoenix Why WebSockets #CodeBEAMSF
55. Phoenix Why #CodeBEAMSF
56. Phoenix Why Elixir #CodeBEAMSF
57. Phoenix Why #CodeBEAMSF
58. Fast Phoenix Why #CodeBEAMSF
59. Phoenix Why #CodeBEAMSF
60. Phoenix Why Stateful #CodeBEAMSF
61. Phoenix Why #CodeBEAMSF
62. Phoenix Why Hot Code Reloading #CodeBEAMSF
63. Example APIs #CodeBEAMSF
64. Example APIs https://api.slack.com/rtm
65. Example APIs https://api.slack.com/rtm Send messages in channel
66. Example APIs https://api.slack.com/rtm Send messages in channel Receive team events
67. Example APIs https://api.slack.com/rtm Send messages in channel Receive team events Receive user presence
68. Example APIs https://getstream.io/docs/#realtime
69. Example APIs https://getstream.io/docs/#realtime Receive feed changes
70. Works How it #CodeBEAMSF
71. Authenticating
72. Authenticating
73. Authenticating
74. Authenticating
75. Authenticating
76. Connecting #CodeBEAMSF
77. Connecting
78. Connecting
79. Connecting
80. Connecting
81. Connecting #CodeBEAMSF
82. Connecting Give clients a starting point #CodeBEAMSF
83. Connecting Give clients a starting point Identify events from a single endpoint (client) #CodeBEAMSF
84. Connecting Give clients a starting point Identify events from a single endpoint (client) Opportunity to authenticate #CodeBEAMSF
88. Connecting
89. Connecting
90. Connecting
91. Connecting
92. Connecting
93. Connecting
94. Connecting
95. Connecting
96. Connecting
97. Connecting
98. Connecting
99. Channels Joining #CodeBEAMSF
100. Channels Joining #CodeBEAMSF
101. Channels Joining Custom Protocol #CodeBEAMSF
102. Channels Joining #CodeBEAMSF
103. Channels Joining Multiplexed #CodeBEAMSF
104. Channels Joining #CodeBEAMSF
105. Channels Joining PubSub #CodeBEAMSF
106. Channels Joining #CodeBEAMSF
107. Channels Joining Built In #CodeBEAMSF
108. Channels The Problem with #CodeBEAMSF
109. Problems WS Joining Channels #CodeBEAMSF
110. Problems { topic: “events:1234”, event: “phx_join” } WS WS Joining Channels #CodeBEAMSF
111. Problems { topic: “events:1234”, event: “phx_join” } { topic: “events:1234”, event: “phx_reply”, payload: { ok: true, token: “1234” }}} WS WS WS WS Joining Channels #CodeBEAMSF
112. Problems Joining Channels #CodeBEAMSF
113. Problems Joining Channels Remove complicated setup from clients #CodeBEAMSF
114. Problems Joining Channels Remove complicated setup from clients Build reconnection/ heartbeat logic #CodeBEAMSF
115. Problems Joining Channels Remove complicated setup from clients Build reconnection/ heartbeat logic :cowboy_websocket https://bit.ly/2WjtKxZ #CodeBEAMSF
116. Messages #CodeBEAMSF
117. Messages #CodeBEAMSF
118. Messages Meat #CodeBEAMSF
119. Messages #CodeBEAMSF
120. Messages Event name #CodeBEAMSF
121. Messages Event name Event data #CodeBEAMSF
122. Messages Incoming #CodeBEAMSF
123. Messages Incoming #CodeBEAMSF
124. Messages Incoming #CodeBEAMSF
125. Messages Incoming
126. Messages Incoming
127. Messages Incoming
128. Messages Incoming
129. Messages Incoming
130. Messages Incoming
131. Messages Incoming
132. Authenticating
133. Authenticating
134. Authenticating
135. Authenticating
136. Authenticating
137. Authenticating
138. Authenticating
139. Messages Incoming #CodeBEAMSF
140. Messages Incoming #CodeBEAMSF
141. Messages Outgoing #CodeBEAMSF
142. Messages Outgoing
143. Messages Outgoing
144. Messages Outgoing
145. Messages Outgoing
146. Messages Outgoing
147. Outgoing
148. Outgoing
149. Outgoing
153. Responses Creating #CodeBEAMSF
154. Responses Creating
155. Responses Creating
156. Responses Creating
157. Responses Creating
158. Responses Creating
159. Responses Creating
160. without Channels Dispatching #CodeBEAMSF
161. without Channels Dispatching
162. without Channels Dispatching
163. without Channels Dispatching #CodeBEAMSF
164. Review #CodeBEAMSF
165. Review HTTP Connecting #CodeBEAMSF
166. Review POST /graphql HTTP HTTP Connecting { mutation: { connect: { … } } #CodeBEAMSF
167. Review POST /graphql { url: “ws://…”, token: “1234” } HTTP HTTP HTTP HTTP Connecting { mutation: { connect: { … } } #CodeBEAMSF
168. Review WS Sending Messages #CodeBEAMSF
169. Review [“timers:start”, { duration: 1500, timestamp: 1273485 } ] WS WS Sending Messages #CodeBEAMSF
170. Review [“timers:start”, { duration: 1500, timestamp: 1273485 } ] [“timers:started”, { id: 1234, duration: 1500, remaining_time: 1500, status: “running” } ] WS WS WS WS Sending Messages #CodeBEAMSF
171. Thank You! #CodeBEAMSF
172. Questions? #CodeBEAMSF
173. @jwright #CodeBEAMSF