Building Distributed Systems On the Shoulders of Giants

Recording is here https://youtu.be/rctYpZqIT2Y

Developing a distributed system is one of the hardest things you will do as a software developer. You will end up having to deal with topics like network inconsistencies, load balancing and service discovery etc. Only to find that solving these problems require expert level knowledge. What if I told you there was a way to leverage the expertise of industry leaders like Google/Microsoft and benefit from their research so you don’t have to reinvent the wheel? Let’s investigate some tools you can use today to build modern, resilient and scalable microservices. Technologies like Service Fabric, Knative, Istio and DAPR can give you the right foundation to build on top of so you can concentrate on solving the business domain problems and be productive.

API Days Live Australia 15-16 September 2020

Transcript

1. Building Distributed Systems On the Shoulders of Giants Dasith Wijes
2. I am here because I have a love hate relationship with distributed systems. HELLO! I AM Dasith Wijes @dasiths dasith.me
3. Modern technology landscape Unique challenges of distributed systems Agenda Azure Service Fabric Distributed Application Runtime (Dapr) Istio Service Mesh Knative Recap and Conclusion
4. Unique Challenges of Distributed Systems
5. var myService = new MyService(); var result = myService.GetProductPrice(“abc123”);
6. System running in single process var myService = new MyService(); var result = myService.GetProductPrice(“abc123”);
7. System running in single process var myService = new MyService(); var result = myService.GetProductPrice(“abc123”);
8. // var myService = new MyService(); var myRestService = new MyRestService(“192.168.x.x”); var result = myRestService.GetProductPrice(“abc123”);
9. System running distributed across multiple nodes var myRestService = new MyRestService(“192.168.x.x”); var result = myRestService.GetProductPrice(“abc123”);
10. System running distributed across multiple nodes var myRestService = new MyRestService(“192.168.x.x”); var result = myRestService.GetProductPrice(“abc123”);
11. System running distributed across multiple nodes var myRestService = new MyRestService(“192.168.x.x”); var result = myRestService.GetProductPrice(“abc123”);
12. • Network is reliable • Latency is zero • Bandwidth is infinite •Connection is secure • Network is homogeneous Did you assume?
13. • Service discovery • Load balancing • Fault tolerance and recovery •Security • Scalability • Telemetry What about…
14. “ “First Law of Distributed Object Design: don't distribute your objects"
15. Looking at the Modern Landscape
16. Istio Service Mesh Pick Your Poison Azure Service Fabric Distributed Application Runtime (Dapr) Knative
17. START FROM BUILDING BLOCKS
18. Azure Service Fabric Distributed systems platform that makes it easy to package, deploy, and manage scalable and reliable microservices and containers.
19. Azure Service Fabric Distributed systems platform that makes it easy to package, deploy, and manage scalable and reliable microservices and containers.
20. Azure Service Fabric Cloud On Premise Developer Machine Environments
21. Azure Service Fabric Environments
22. Azure Service Fabric • Reliable Services • Stateful & Stateless • Reliable Actors • Persisted state options • Guest Executables • Containers • ASP.NET Core Programming Models
23. Azure Service Fabric public interface IHelloWorld : IActor { Task<string> GetHelloWorldAsync(); } Actor Example – C# Interface
24. Azure Service Fabric [StatePersistence(StatePersistence.Persisted)] internal class HelloWorld : Actor, IHelloWorld { public HelloWorld(ActorService actorService, ActorId actorId) : base(actorService, actorId) { } public Task<string> GetHelloWorldAsync() { return Task.FromResult("Hello from my reliable actor!"); } } Actor Example – C# Implementation
25. Azure Service Fabric class Program { static void Main(string[] args) { IHelloWorld actor = ActorProxy.Create<IHelloWorld>( ActorId.CreateRandom(), new Uri("fabric:/MyApplication/HelloWorldActorService")); Task<string> retval = actor.GetHelloWorldAsync(); Console.Write(retval.Result); Console.ReadLine(); } } Actor Invocation – C# Sample
26. Azure Service Fabric class Program { static void Main(string[] args) { IHelloWorld actor = ActorProxy.Create<IHelloWorld>( ActorId.CreateRandom(), new Uri("fabric:/MyApplication/HelloWorldActorService")); Task<string> retval = actor.GetHelloWorldAsync(); Console.Write(retval.Result); Console.ReadLine(); } } Actor Invocation – C# Sample
27. Azure Service Fabric class Program { static void Main(string[] args) { IHelloWorld actor = ActorProxy.Create<IHelloWorld>( ActorId.CreateRandom(), new Uri("fabric:/MyApplication/HelloWorldActorService")); Task<string> retval = actor.GetHelloWorldAsync(); Console.Write(retval.Result); Console.ReadLine(); } } Actor Invocation – C# Sample • Service discovery and routing • Availability • Reliability • Fault Tolerance • Auto scaling
28. OPINIONATED RUNTIME A MORE
29. Distributed Application Runtime (Dapr) Portable, event-driven runtime that makes it easy for developers to build resilient, microservice stateless and stateful applications that run on the cloud and edge and embraces the diversity of languages and developer frameworks.
30. Distributed Application Runtime (Dapr)
31. Building Blocks https://github.com/dapr/docs/tree/master/concepts Distributed Application Runtime (Dapr)
32. Components https://github.com/dapr/docs/tree/master/concepts Distributed Application Runtime (Dapr)
33. • Service discovery • State • Pub/sub • Bindings • Middleware • Secret stores • Tracing exporters Components https://github.com/dapr/docs/tree/master/concepts Distributed Application Runtime (Dapr) 1 or more
34. • Service discovery • State • Pub/sub • Bindings • Middleware • Secret stores • Tracing exporters Some Pub/Sub Implementations • Redis Streams • Kafka • Azure Service Bus • RabbitMQ • Azure Event Hubs • GCP Pub/Sub • MQTT https://github.com/dapr/components-contrib/tree/master/pubsub Distributed Application Runtime (Dapr) Pluggable Implementations
35. • Service discovery • State • Pub/sub • Bindings • Middleware • Secret stores • Tracing exporters Picking a Component Implementation apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: messagebus namespace: default spec: type: pubsub.redis metadata: - name: redisHost value: <HOST> - name: redisPassword value: <PASSWORD> - name: enableTLS value: <bool> https://github.com/dapr/components-contrib/tree/master/pubsub Distributed Application Runtime (Dapr) pubsub.yaml (Redis Streams )
36. • Service discovery • State • Pub/sub • Bindings • Middleware • Secret stores • Tracing exporters Pub/Sub Interface type PubSub interface { Init(metadata Metadata) error Publish(req *PublishRequest) error Subscribe(req SubscribeRequest, handler func(msg *NewMessage) error) error } https://github.com/dapr/components-contrib/tree/master/pubsub Distributed Application Runtime (Dapr)
37. Endpoints For Building Blocks Building Block Endpoint Service-to-Service Invocation /v1.0/invoke State Management /v1.0/state Publish and Subscribe /v1.0/publish + /v1.0/subscribe Resource Bindings /v1.0/bindings Actors /v1.0/actors Secrets /v1.0/secrets https://github.com/dapr/docs/tree/master/concepts Distributed Application Runtime (Dapr)
38. Example – Node.js App app.post('/neworder', (req, res) => { const data = req.body.data; const orderId = data.orderId; console.log("Got a new order! Order ID: " + orderId); //todo: persist order res.sendStatus(200); }); Distributed Application Runtime (Dapr)
39. Example – Node.js App app.post('/neworder', (req, res) => { const data = req.body.data; const orderId = data.orderId; console.log("Got a new order! Order ID: " + orderId); //todo: persist order res.sendStatus(200); }); dapr run --app-id nodeapp --app-port 3000 --dapr-http-port 3500 node app.js Distributed Application Runtime (Dapr)
40. Service Invocation – Endpoint Distributed Application Runtime (Dapr)
41. Service Invocation – Endpoint http://localhost:3500/v1.0/invoke/nodeapp/method/neworder Distributed Application Runtime (Dapr)
42. Cont. Invocation of Node.js App from a Python App dapr_port = os.getenv("DAPR_HTTP_PORT", 3500) dapr_url = "http://localhost:{}/v1.0/invoke/nodeapp/method/neworder".format(dapr_port) n = 0 while True: n += 1 message = {"data": {"orderId": n}} try: response = requests.post(dapr_url, json=message) except Exception as e: print(e) time.sleep(1) Distributed Application Runtime (Dapr)
43. Cont. Invocation of Node.js App from a Python App dapr_port = os.getenv("DAPR_HTTP_PORT", 3500) dapr_url = "http://localhost:{}/v1.0/invoke/nodeapp/method/neworder".format(dapr_port) n = 0 while True: n += 1 message = {"data": {"orderId": n}} try: response = requests.post(dapr_url, json=message) except Exception as e: print(e) time.sleep(1) Distributed Application Runtime (Dapr)
44. Cont. Invocation of Node.js App from a Python App dapr_port = os.getenv("DAPR_HTTP_PORT", 3500) dapr_url = "http://localhost:{}/v1.0/invoke/nodeapp/method/neworder".format(dapr_port) n = 0 while True: n += 1 message = {"data": {"orderId": n}} try: response = requests.post(dapr_url, json=message) except Exception as e: print(e) time.sleep(1) Distributed Application Runtime (Dapr) • Service discovery and routing • mTLS • Retry logic • Tracing
45. Side-Car Process https://cloudblogs.microsoft.com/opensource/2019/10/16/announcing-dapr-open-source-project-build-microservice-applications/ Distributed Application Runtime (Dapr)
46. Side-Car Container in Kubernetes Pod https://cloudblogs.microsoft.com/opensource/2019/10/16/announcing-dapr-open-source-project-build-microservice-applications/ Distributed Application Runtime (Dapr)
47. Dapr on Kubernetes https://itnext.io/simplifying-microservices-on-kubernetes-with-microsofts-dapr-distributed-application-runtime-9aece5723484 Dapr control-plane components are deployed as Kubernetes deployments. Dapr can be easily initialized on the Kubernetes cluster using Dapr-CLI or can be deployed using Helm. • dapr-operator: Watches for creation, deletion and updating events of services. • dapr-sidecar-injector: Injects a Dapr sidecar to all the services. • dapr-placement: Connects all the Dapr sidecars and adds the IP address to route the communication. Distributed Application Runtime (Dapr)
48. https://github.com/dapr/docs/tree/master/overview
49. CONTAINERS? ALREADY HAVE
50. Istio Service Mesh to Connect, secure, control, and observe services. Istio provides behavioural insights and operational control over the service mesh as a whole, offering a complete solution to satisfy the diverse requirements of microservice applications. Istio Service Mesh
51. https://tud-ccc.github.io/yauhau-doc/ Why Do We Need a Service Mesh?
52. Istio Service Mesh A Primer https://istio.io/latest/docs/ops/deployment/architecture/ • Istiod provides service discovery, configuration and certificate management. • Envoy proxies are deployed as sidecars to services, logically augmenting the services with Envoy’s many built-in features.
53. • Developers can focus on adding business value, instead of connecting services. • Apps are more resilient to downtime, since a service mesh can reroute requests away from failed services. • Performance metrics can suggest ways to optimize communication in the runtime environment. Istio Service Mesh Elevator Pitch https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh
54. Istio Service Mesh Elevator Pitch var myRestService = new MyRestService(“svc1.ns.local”); var result = myRestService.GetProductPrice(“abc123”);
55. Istio Service Mesh Elevator Pitch var myRestService = new MyRestService(“svc1.ns.local”); var result = myRestService.GetProductPrice(“abc123”);
56. Istio Service Mesh Elevator Pitch var myRestService = new MyRestService(“svc1.ns.local”); var result = myRestService.GetProductPrice(“abc123”); • Dynamic service discovery • Load balancing • TLS termination • HTTP/2 and gRPC proxies • Circuit breakers
57. WHAT ABOUT SERVERLESS?
58. Kubernetes-based platform to deploy and manage modern serverless workloads. Knative components builds on top of Kubernetes, abstracting away the complex details and enabling developers to focus on what matters. Knative
59. Knative • Focused API with higher level abstractions for common app use-cases. • Stand up a scalable, secure, stateless service in seconds. • Knative is portable: run it anywhere Kubernetes runs, never worry about vendor lock-in. • Idiomatic developer experience, supporting common patterns such as GitOps, DockerOps, ManualOps. https://knative.dev/ Elevator Pitch
60. Knative apiVersion: serving.knative.dev/v1 # Current version of Knative kind: Service metadata: name: helloworld-go # The name of the app namespace: default # The namespace the app will use spec: template: spec: containers: - image: gcr.io/knative-samples/helloworld-go # Reference to the app image env: - name: TARGET # The environment variable printed out by the sample app value: "Go Sample v1" Sample Configuration - service.yaml
61. Knative apiVersion: serving.knative.dev/v1 # Current version of Knative kind: Service metadata: name: helloworld-go # The name of the app namespace: default # The namespace the app will use spec: template: spec: containers: - image: gcr.io/knative-samples/helloworld-go # Reference to the app image env: - name: TARGET # The environment variable printed out by the sample app value: "Go Sample v1" Sample Configuration - service.yaml
62. Knative apiVersion: serving.knative.dev/v1 # Current version of Knative kind: Service metadata: name: helloworld-go # The name of the app namespace: default # The namespace the app will use spec: template: spec: containers: - image: gcr.io/knative-samples/helloworld-go # Reference to the app image env: - name: TARGET # The environment variable printed out by the sample app value: "Go Sample v1" Sample Configuration - service.yaml
63. Knative Sample - Deploy # To Deploy services.yaml kubectl apply --filename service.yaml
64. Knative Sample - Deploy # To Deploy services.yaml kubectl apply --filename service.yaml # To Invoke curl http://helloworld-go.default.svc.cluster.local
65. Knative Sample - Deploy # To Deploy services.yaml kubectl apply --filename service.yaml # To Invoke curl http://helloworld-go.default.svc.cluster.local Hello World: Go Sample v1!
66. Knative Sample - Deploy # To Deploy services.yaml kubectl apply --filename service.yaml # To Invoke curl http://helloworld-go.default.svc.cluster.local Hello World: Go Sample v1! • Routing • Load balancing • TLS termination • Blue/green deployments • Smart Scaling
67. Picking a Winner
68. Service Fabric Dapr Istio Knative • Distributed systems platform. • Any cloud, any vendor. • Orchestrate containers or executables. • Windows or Linux clusters.* • Small community and little support. • Opinionated programming model. • Support for many languages and frameworks. • Extensible components. • Runs as side-car. • Native support for K8s. • Service mesh with focus on routing. • Doesn’t enforce programming model. • Fits nicely with existing container workloads. • Serverless model. • Developer productivity. • Higher level abstraction of K8s. • Easily configurable smart scaling. Recap
69. STAND ON THE SHOULDERS OF GIANTS Why reinvent the wheel?
70. Any questions? THANKS! @dasiths dasith.me https://www.nationalgeographic.com/travel/destinations/asia/sri-lanka/
71. Presentation template designed by powerpointify.com Special thanks to all people who made and shared these awesome resources for free: CREDITS Photographs by unsplash.com Free Fonts used: https://www.fontsquirrel.com/fonts/oswald