Learnings from Implementing Microservices Architecture with Kubernetes

Ray has been on a 6-months rotation with an internal Google team to help bringing a project to public Cloud using cloud-native technology stack and Kubernetes. Ray will share the architecture, development environment technicals, devops tools, and some tough decisions that needed to be made to move the project along while being prepared for changes in the future.

This session to learn the journey including development environment tools choices (Docker Compose, Skaffold, Kustomize, Jib), to the stack (Gradle, Spring Boot, Kafka, PostgreSQL, gRPC, gRPC-Web), to mono-repo vs multi-repo, to the runtime infrastructure (Kubernetes, Istio, Prometheus, Grafana). With hindsight 20-20, we’ll visit some best practices, lessons learned, and how decisions/compromises are being made.

Transcript

1. Learnings from Implementing Microservices w/ Kubernetes
2. 2 @saturnism @gcpcloud Ray Tsang Developer Advocate Google Cloud Platform Java Champion @saturnism | saturnism.me
3. 3 @saturnism @gcpcloud 3 The Project...
4. 4 @saturnism @gcpcloud Microservices on Kubernetes!
5. 5 @saturnism @gcpcloud Help Alphabet companies to adopt Cloud
6. 6 @saturnism @gcpcloud Move from internal technologies To open source technologies
7. 7 @saturnism @gcpcloud 2 implementation teams Infrastructure, CI/CD, cloud practices, monitoring Application development, business logic
8. 8 @saturnism @gcpcloud 2 teams - working together Discuss business requirements, architecture, operations needs Application implementation team takes over operation
9. 9 @saturnism @gcpcloud 9 Infrastructure
10. 10 @saturnism @gcpcloud Kubernetes cluster / Private Cluster Istio Namespace Networking Istio Ingress Istio Project Namespace Virtual Services Istio Virtual Services Istio Frontend Deployment Backends Deployment Cloud Load Balancing Identity-Aware Proxy Istio Egress Istio Cloud NAT Third-Party Services PostgreSQL Cloud SQL Images Container Registry Prometheus Monitoring Grafana Monitoring Jaerger Distributed Trace
11. 11 @saturnism @gcpcloud 11 #1 ClickOp → GitOp In the cloud, it's every easy to click It's hard to reproduce!
12. 12 @saturnism @gcpcloud Infrastructure as Code Terraform for all cloud infrastructure - GKE, Cloud SQL, Static IPs, VPC... Laydown other Kubernetes infrastructure - Istio, OPA Gatekeeper...
13. 13 @saturnism @gcpcloud Kubernetes cluster / Private Cluster Istio Namespace Networking Istio Ingress Istio Project Namespace Virtual Services Istio Virtual Services Istio Frontend Deployment Backends Deployment Cloud Load Balancing Identity-Aware Proxy Istio Egress Istio Cloud NAT Third-Party Services PostgreSQL Cloud SQL Images Container Registry Prometheus Monitoring Grafana Monitoring Jaeger Distributed Trace Terraformed
14. 14 @saturnism @gcpcloud Kubernetes cluster / Private Cluster Istio Namespace Networking Istio Ingress Istio Project Namespace Virtual Services Istio Virtual Services Istio Frontend Deployment Backends Deployment Cloud Load Balancing Identity-Aware Proxy Istio Egress Istio Cloud NAT Third-Party Services PostgreSQL Cloud SQL Images Container Registry Prometheus Monitoring Grafana Monitoring Jaeger Distributed Trace Ready for deployment
15. 15 @saturnism @gcpcloud There is a lot of YAML But at least, the process is repeatable
16. 16 @saturnism @gcpcloud Check-in the final configuration helm template \ istio-${ISTIO_VERSION}/install/kubernetes/helm/istio \ --name istio \ --namespace istio-system \ -f dev-values.yaml >> istio.yaml istio.yaml change → PR (review, see diffs) → merge → CI/CD (apply) (Depends on how much you trust the templating engine)
17. 17 @saturnism @gcpcloud CI Pipeline (triggered by code commit) 1. Build, test application 2. Create container image 3. Commit deployment manifests with new container image tag/sha CD Pipeline (triggered by manifest commit) 1. Render full manifest if necessary 2. Apply the full manifest
18. 18 @saturnism @gcpcloud 18 #2 Adopt incrementally Understand the requirements for production Have a roadmap to know what technology to adopt, when
19. 19 @saturnism @gcpcloud Initial Learning Click-op to GKE Repeatable Infrastructure Infrastructure as code Terraform Networking / Ingress POC with L4 LB Kubernetes Moved to L7 LB with Ingress Kubernetes SSL Let's Encrypt? GCP Managed Certificates? URL mapping/routing Envoy → Istio Security Enable mTLS Istio Deny/allow Egress Istio Add security policies Kubernetes Deny/allow container images OPA Gatekeeper Monitoring Collect Metrics Prometheus Scrape the metrics Dashboards Grafana Alerts SLOs Infrastructure-level alerts Uptime checks
20. 20 @saturnism @gcpcloud Somethings are is hard to change Be careful of one way doors Istio sidecar requires privileges → Reevaluate/reinstall Istio w/ CNI Public cluster to private cluster → Delete and recreate
21. 21 @saturnism @gcpcloud 21 #3 Adopt carefully Don't go back and say "I need everything on that slide!" Consider what you really want to achieve Explore and _make_ sure everything works as advertised
22. 22 @saturnism @gcpcloud Problem Statement Goal / Scope Solutions / Alternatives Pros / Cons Recommendation / Decision
23. 23 @saturnism @gcpcloud 23 #4 - Consider non-technical factors Write a doc! Consider reality Weigh the risks Maintenance / Operations
24. 24 @saturnism @gcpcloud Mono Repo or Multi Repo
25. 25 @saturnism @gcpcloud Distributed Monolith or Distributed Services?
26. 26 @saturnism @gcpcloud It Depends…!
27. 27 @saturnism @gcpcloud Mono Repo Multi Repo Project Structure Multi-module/multi-project Single module/project Dependency Management Parent / Includes All dependencies are up to date Common Parent/BOM Automate dependency version updates Artifact Management Can avoid initially Need to publish artifacts Where to publish? Testing Easy Against Snapshots, Flaky CI Just one pipeline Builds everything Copy of pipeline per repo Build only service that changed CD Which service to deploy? Deploy service that changed Initial Velocity Fast Slow... Long Term Velocity Slow down over time Long builds Faster
28. 28 @saturnism @gcpcloud We still chose Mono Repo... Team is already familiar with Mono Repo Fast ramp up and velocity Lack of existing infrastructure for dependency and artifact management Setting up one repo and pipeline was difficult enough...
29. 29 @saturnism @gcpcloud We do this analysis for everything Every service have their own database? gRPC or REST? Kafka? Knative?
30. 30 @saturnism @gcpcloud 30 #5 Anticipate changes Choices made today is made. Design to expect changes tomorrow. Avoid one way doors.
31. 31 @saturnism @gcpcloud Anticipate Multi Repo We anticipate to out grow the Mono Repo Make sure the Mono Repo is still splittable!
32. 32 @saturnism @gcpcloud project/ +-- build.gradle +-- services/ +-- common.gradle +-- auth/ +-- user/ +-- email/ Project Structure
33. 33 @saturnism @gcpcloud project/ +-- build.gradle +-- services/ +-- common.gradle +-- auth/ +-- src/main/proto/auth.proto +-- user/ +-- email/ Project Structure
34. 34 @saturnism @gcpcloud apply from: '../common.gradle' group = 'com.example.services' mainClassName = 'com.example.services.auth dependencies { implementation project(':common:user') protobuf project(path: ':services:user, configuration: 'proto') }
35. 35 @saturnism @gcpcloud Anticipate Multi Repo As the team grows, and new teams comes to take over services... Successfully split out 3 services from the Mono Repo
36. 36 @saturnism @gcpcloud 36 #6 Focus on your application Architecture and design - it has nothing to do with Kubernetes If you design well, you can almost always deploy into Kubernetes 12factor.net
37. 37 @saturnism @gcpcloud Adopt Carefully Anticipate Changes Microservices architecture is not the answer to everything Monolith works too, as long as it is designed well!
38. 38 @saturnism @gcpcloud 38 #7 Local != Production Do not bring Slide 10 to local development Focus on velocity of well-designed application Rely on self-encapsulating unit/integration tests
39. 39 @saturnism @gcpcloud Why not Istio Locally? A lot to learn and troubleshoot Use less compute resource
40. 40 @saturnism @gcpcloud Test Locally - without Kubernetes Unit tests Integrations tests Wiremocks Testcontainers
41. 41 @saturnism @gcpcloud If you need to test something... Simple Envoy Proxy Local Kubernetes (k3s, minikube, ...)
42. 42 @saturnism @gcpcloud Cloud Code / Skaffold After you test everything… want to see end-to-end result Continuous development loop
43. 43 @saturnism @gcpcloud Kustomize Every environment is different Single source of truth Kustomize for different environments
44. 44 @saturnism @gcpcloud 44 #8 Friends don't let friends ______ Write Dockerfiles or YAML!
45. 45 @saturnism @gcpcloud Just jib It Or pack it
46. 46 @saturnism @gcpcloud kubectl create deployment myservice --image=... --dry-run -oyaml > k8s/deployment.yaml kubectl create svc clusterip myservice --tcp=8080:8080 --dry-run -oyaml > k8s/service.yaml
47. 47 @saturnism @gcpcloud Automate best practices whenever possible
48. 48 @saturnism @gcpcloud In fact, automate the entire platform!
49. 49 @saturnism @gcpcloud 49 #9 Contracts with the runtime environment When is your application ready to serve traffic? When is it in trouble? How do you shutdown gracefully?
50. 50 @saturnism @gcpcloud Resources Resource Request and Resource Limits
51. 51 @saturnism @gcpcloud When to use? Failure Means... Practices Example Liveness Probe If application is alive. Application will be restarted, and that a restart will help recover. Runs on serving port of the application, e.g., 8080. Don't check dependency. E.g., don't check dependent database connection, etc. A simple /alive URL that returns 200. Readiness Probe Ready to serve requests. Take the pod instance out of load balancer. Flip to ready when application has done all the initializations (cache preloaded). Upon SIGTERM, flip readiness to false. See Graceful Shutdown. /actuator/health on the management port.
52. 52 @saturnism @gcpcloud Anatomy of a Graceful Shutdown 1. Receive SIGTERM or PreStop Lifecycle Hook 2. Fail Readiness Probe 3. Receive requests until Kubernetes detects readiness probe failure 4. Kubernetes removes pod endpoint from Service 5. Finish serving in-flight requests 6. Shutdown
53. 53 @saturnism @gcpcloud 53 We now speak the same language
54. 54 @saturnism @gcpcloud All the cross-cutting concerns are the same Monolith, microservices, Kubernetes, not Kubernetes... But the 2 teams now speak with the same nouns: Deployment, Service, Ingress, Virtual Service, ...
55. 55 @saturnism @gcpcloud 55 Thanks! saturnism.me/talk/kubernetes-microservices-lessons-learned/ bit.ly/k8s-lab | bit.ly/istio-lab | bit.ly/spring-gcp-lab gcplab.me/spring @saturnism | saturnism.me