Fastly Yamagoya Meetup: Leveraging Cloud Portability with Fastly

Transcript

1. Leveraging Cloud Portability with Fastly Jun Sakata / @sakajunquality Ubie, inc. October 23, 2019. Fastly Yamagoya Meetup
2. Ubie, Inc. - Medical Startup - Founded in 2017 - 30+ employee - 15+ engineers - Business - Clinical decision making support - Operation efficiency support
3. Agenda How Fastly helps - Migrating services across vendors - Migrating services within a cloud - Making more portability in architecture
4. - Kubernetes: Open-source container orchestration platform. - Istio: Open-source Service Mesh platform. - Envoy: Open-source service proxy. Used in Istio dataplane. - GKE: GCP’s managed version of GKE. - Istio on GKE: GKE’s addon to install Istio. Terminologies
5. Migrating services across vendors
6. Migrating services across vendors - OnPrem to cloud - One cloud to another - etc...
7. 1st Migration in Ubie - Migrated from Heroku to GCP - Microservices w/ Kubernetes (GKE) - Data and ML services - Google’s Startup Support - etc...
8. 1st Migration in Ubie - Two existing service - Frontend service - Old backend service - Older than the company! - One new service - New backend service - in Kotlin
9. 1st Migration in Ubie Frontend Old Backend Frontend Old Backend New Backend
10. Why Fastly? - Fast Configuration Activation - L7 Load Balancing - CDN Interconnect with GCP
11. Fast Configuration Activation - DNS could take much time - Required a quick rollout and rollback - Not enough e2e test (at that time) - Personally not enough knowledge with applications (at that time) - Rolled-back once actually
12. L7 Load Balancing - Outgoing traffic is sometimes limited in medical institutions - e.g. Domain name, IP address etc… - Meanwhile we want to use multiple services - Several backends are running and called from client-side
13. L7 Load Balancing - With Fastly, we’re using L7 path based routing Service A Service B Service C / /log /static Client
14. L7 Load Balancing - With Fastly, we’re using L7 path based routing - Can choose the best backend Service A Service B Service C / /log /static Client
15. How we migrated Frontend Old Backend Frontend Old Backend New Backend
16. How we migrated Frontend Old Backend Frontend Old Backend New Backend Changing the origin
17. Good points with Fastly - Quick Rolled-out/back - Dev Experience - Support
18. Dev Experience - Logging - Access log export to Google BigQuery - Most of logs are stored in BigQuery - Configuration via API - terraform
19. Support Case - Satisfactional technical support - Both in English and Japanese
20. Migrating services within a cloud
21. https://twitter.com/kelseyhightower/status/935252923721793536
22. Architecture Migration - Increase in # of users - Increase in # of services - Increase in # of developers - Changing softwares - Insufficient design - etc...
23. 2nd Migration in Ubie - Changing Kubernetes (GKE) cluster - Installing Istio - Public to Private Cluster - Some new features - Changing logical deployment - namespaces
24. Istio - Open-source Service Mesh platform - Originally from Google and etc… - Service Discovery / Traffic Control / Observability
25. Why Istio? - Single Ingress-Gateway - More and more services are deployed - 2 services -> ~10 services - Client-side load balancing and its telemetry - Internal traffic has increased
26. Why Istio? - Ingress Gateway Service A Service B Service C Service A Service B Service C Ingress Gateway ... ... Redundant origin config... Single origin config Separated Ingress LBs...
27. Why Istio? - Internal traffic Service A Service B Service A Service B ILB It used to be using internal GCLB Need an additional configurations...
28. Why Istio? - Internal traffic
29. Logical Deployment - namespace = virtual separation of cluster - Frequently used with access control - namespace is separated into teams at first - => startup’s teams change often!! - Change to 1 namespace for 1 service
30. 2nd Migrations namespace Services Services Service Service Separate namespaces Istio installed
31. 2nd Migrations namespace Services Services Service Service Changed several backends to single backend (Istio Ingress Gateway)
32. Good points with Fastly - Quick Rolled-out/back - Dev Experience - Support - Security
33. Security - WAF - Same rule for different backend infra - +2 - Customer on-boarding process - Not white-boxed rules - Access Restriction with VCL - Restrictions are not affected by backend infra
34. Making more portability
35. The “God” Cluster - Create a single big cluster - Maintain it carefully and nicely
36. The “God” Cluster - Pros - Resource Utilization - Less things to consider e.g. CI/CD Canary release ... - Less things to manage - Cons - Version updates - Testing new features - Cluster outage directly goes to service disruption
37. Updates - Kubernetes (OSS) - Quortaly major updates - Security Patches - GKE (Managed) - OSS + GCP specific updates - Istio (OSS) - Frequent updates - Istio on GKE (Half-managed)
38. Updates - Kubernetes (OSS) - Quortaly major updates - Security Patches - GKE (Managed) - OSS + GCP specific updates - Istio (OSS) - Frequent updates - Istio on GKE (Half-managed) So many chance to destroy the whole service!!
39. 3rd Migration in Ubie - Migrating to multi-clusters - Like a East-West traffic - Migrating to OSS Istio - “Istio on GKE” has limitations… - Test alpha/experimental features in production safely
40. 3rd Migration in Ubie Service Service Service Service Tenant 1 Tenant 2 Round Robin Do not allow cross-cluster internal requests
41. If a disruption happens in either of clusters... Service Service Service Service Tenant 1 Tenant 2
42. Remove the whole cluster Service Service Service Service Tenant 1 Tenant 2 The whole service keeps running with the rest of clusters!
43. Traffic Management - External Requests - Internal Requests
44. External Requests - Load Balance with Fastly - Origin to Istio Ingress Gateway Origin configuration
45. External Requests Still working with the following problems... - Health Check - Kubernetes cluster level? - Each service level? - Session Stickiness - Definition of client... - Canary-based rollouts
46. Not Multi Cluster Ingress with GKE? - https://cloud.google.com/kubernetes-engine/docs/how-to/multi-cluster-ingress - Ingress = L7 LB, Istio use L4 LB - Still in beta - Small chance to use another cloud
47. Internal Requests - Client-side load balancing with envoy in Istio - All the internal requests stays in the cluster
48. Good points with Fastly - Quick Rolled-out/back - Dev Experience - Support - Security - Custom VCL
49. Custom VCL - Implement domain specific features - e.g. sticky load balancing, WAF bypassing etc...
50. Access Log - Fastly’s external access log - + envoy’s internal access log
51. 4th Migration in Ubie - Frontend performance? maybe
52. Takeaways
53. Takeaways - With Fastly, it’s really easy to migrate services and improve architecture. - Even if architecture design is not good enough - In Ubie, external traffic management w/ Fastly + internal traffic management w/ Istio combination works well
54. So many Good points with Fastly! - Quick Rolled-out/back - Dev Experience - Support - Security - Custom VCL
55. Thank You