Building Microservice Architectures with Go - VoxxedDays Berlin 2016

Presented at VoxxedDays Berlin 2016

Traditionally applications have been built as monoliths; single applications which become larger and more complex over time, which limit our ability to react to change. An example of this is the banking industry where mergers and acquisitions between banks have lead to a patchwork of different systems & technologies that cost billions of dollars per year to maintain. As a result, the pace of innovation in the banking industry has slowed to a crawl.

At Mondo we're building a new kind of bank, a smart bank that belongs in the 21st century, and we’re building it almost entirely in Go. This talk will cover how we’re developing new core banking systems from scratch backed by a microservice platform written in Go, running across multiple data centres using open source frameworks and tools including Docker and Mesos. We'll look at why Go is perfectly suited to this, our architectural decisions, common pitfalls to avoid, and how microservice architectures can vastly increase both the velocity of development teams and the scalability and fault tolerance of our systems.

Transcript

1. @mattheath #VoxxedBerlin Platinum Sponsor Building Microservice Architectures with Go Matt Heath Mondo
2. @mattheath
6. monoliths traditional dev
16. ?
17. DATABASE APPLICATION
18. DATABASE APPLICATION
19. DATABASE DATABASES APPLICATION
20. DATABASE DATABASES APPLICATION SEARCH
21. DATABASE DATABASES APPLICATION CACHE SEARCH
22. DATABASE DATABASES APPLICATION CACHE SEARCH CAT GIFS
23. ALL HAIL THE MONOLITH
24. DO NOT WANT
25. DATABASE DATABASES APPLICATION CACHE SEARCH CAT GIFS
26. APPLICATION
29. Microservices at Mondo?
30. Single Responsibility Principle
31. Bounded Context
32. Well defined Interfaces
33. Composability
34. Getting started
35. LOAD BALANCER
36. LOAD BALANCER HTTP API & ROUTING LAYER
37. TRANSPORT LOAD BALANCER HTTP API & ROUTING LAYER
38. SERVICE SERVICE SERVICE TRANSPORT LOAD BALANCER HTTP API & ROUTING LAYER
39. SERVICE SERVICE SERVICE TRANSPORT DATABASE DATABASE DATABASE LOAD BALANCER HTTP API & ROUTING LAYER
40. SERVICE SERVICE SERVICE TRANSPORT DATABASE DATABASE DATABASE LOAD BALANCER API GATEWAY
41. APPLICATION LOAD BALANCER
42. API GATEWAY APPLICATION LOAD BALANCER
43. API GATEWAY APPLICATION LOAD BALANCER
44. API GATEWAY APPLICATION LOAD BALANCER SERVICE SERVICE SERVICES
46. Simple Static typing Static linking
47. Comprehensive stdlib eg. Networking
48. Concurrency
49. Interfaces
50. Go Kit micro gRPC Kite
51. mondough/typhon mondough/mercury
52. LOAD BALANCER
53. LOAD BALANCER HTTP API & ROUTING LAYER
54. API SERVICE LOAD BALANCER HTTP API & ROUTING LAYER
56. /webhooks —-> Webhook API
57. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER
58. WEBHOOK API AUTH SERVICE WEBHOOK SERVICE LOAD BALANCER HTTP API & ROUTING LAYER
59. WEBHOOK API AUTH SERVICE WEBHOOK SERVICE LOAD BALANCER HTTP API & ROUTING LAYER DATABASE
60. WEBHOOK API AUTH SERVICE WEBHOOK SERVICE LOAD BALANCER HTTP API & ROUTING LAYER DATABASE DATABASE
61. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER DATABASE DATABASE EXTERNAL PROVIDER
62. SERVICE
63. Logic Handlers Storage SERVICE
64. mercury Logic Handlers Storage SERVICE
65. type Handler func(request) (response, error)
66. type Transport interface { ... }
67. type Transport interface { Listen(serviceName string, inbound chan<- request) error StopListening(serviceName string) bool Respond(req request, resp response) error Send(req request, timeout time.Duration) (response, error) ... }
68. type Transport interface { Listen(serviceName string, inbound chan<- request) error StopListening(serviceName string) bool Respond(req request, resp response) error Send(req request, timeout time.Duration) (response, error) ... }
69. mercury Logic Handlers Storage SERVICE
70. mercury Logic Handlers Storage libraries SERVICE
71. mercury Logic Handlers Storage libraries SERVICE Deployment Service Discovery Configuration Monitoring Authentication Authorisation Storage Circuit Breaking
74. Small Images
75. Statically Compiled
76. Root CA Certs
78. Making our service reliable
79. Event Driven Architecture
80. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER
81. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER
82. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER
83. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER SERVICE C SERVICE D E
84. API SERVICE SERVICE A SERVICE B LOAD BALANCER HTTP API & ROUTING LAYER SERVICE C SERVICE D G E F
85. ? ? ? LOAD BALANCER HTTP API & ROUTING LAYER ? ? ?? ? ? ? ? ? ? ?
86. Topology Management
87. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
88. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
89. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
90. WEBHOOK API LOAD BALANCER HTTP API & ROUTING LAYER WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE SLOW / ERRORS
91. Fanout & Cancellation
92. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
93. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
94. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
95. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
96. WEBHOOK API WEBHOOK SERVICE WEBHOOK SERVICE WEBHOOK SERVICE
97. Context Propagation
98. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
99. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
100. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
101. Testing
102. Load Failure Degradation
103. Monitoring
104. Monitor your Business Logic
105. ♥ customers
106. In-Band vs Out of Band
107. type Checker func() (error, map[string]string)
108. Distributed Tracing
109. api api api.customer api.customer service.customer service.customer
110. api api api.customer api.customer service.customer service.customer
111. 8096820c-3b7b-47ec-bce6-1c239252ab40
112. api api api.customer api.customer service.customer service.customer
113. api api api.customer api.customer service.customer service.customer
114. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
115. package context type Context interface { Deadline() (deadline time.Time, ok bool) Done() <-chan struct{} Err() error Value(key interface{}) interface{} }
116. api api api.customer api.customer service.customer service.customer
117. api api api.customer api.customer service.customer service.customer SEND RECV SEND RECV RECV SEND RECV SEND
118. api api.customer service.customer SEND RECV SEND RECV RECV SEND RECV SEND phosphor
120. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
121. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
122. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
123. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
124. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
126. API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns API card-api card-processing cards transactions balance transaction-enrichment merchant feed-generator feed apns
130. Small Simple Easy to learn
131. Concurrency Interfaces Networking
132. Downsides?
133. Starting with Microservices?
134. #voxxedberlin Thanks! @mattheath @getmondo
135. ATM: Thomas Hawk
 Bank of Commerce: ABQ Museum Archives IBM System/360: IBM Absorbed: Saxbald Photography Orbital Ion Cannon: www.rom.ac Go Gopher: Renee French Control Room: NASA ATM Failure: George Redgrave Credits