Upgrade to Pro — share decks privately, control downloads, hide ads and more …

KUBECON EU 2019: Ingress V2 and Multicluster Se...

KUBECON EU 2019: Ingress V2 and Multicluster Services

Kubecon EU 2019: Ingress V2 and Multicluster Services

Rohit Ramkumar & Bowei Du, Google

Bowei Du

May 21, 2019
Tweet

More Decks by Bowei Du

Other Decks in Programming

Transcript

  1. API models: Ingress GA (proposed) k8s.Ingress host.com/path ... k8s.Service k8s.Service

    k8s.IngressClass cloud.Bucket Key attributes: “flavor” of Ingress via class, heterogenous backends.
  2. API models: Istio Gateway k8s.Service k8s.Service istio.VirtualService host.com/path istio.VirtualService istio.DestinationRule

    istio.DestinationRule host match subset ... Key attributes: Separate roles for proxy infrastructure, application definition; rich feature set based on Envoy
  3. API models: IngressRoute contour.IngressRoute foo.com bar.com k8s.Service contour.IngressRoute foo.com/path ...

    k8s.Service k8s.Service subset contour. IngressRoute Key attributes: Separate roles, recursive delegation + composability, additional features such as traffic splitting.
  4. API models: Others Lots of annotations for features (Everybody) Decorators

    on k8s.Service instead of k8s.Ingress (Ambassador) Use Custom Resources (Gloo)
  5. What have we learned? Multi-role environments • Infrastructure vs App

    dev How much portability? • Should be a user choice. Support future API growth • Claim: providers/features will converge over time (but quite slowly).
  6. A modest proposal Warning: this is very early… Takes from

    heavily from existing work... Shape of the resources (model) Portability and extensibility Future directions
  7. A modest proposal: the model Mesh Cloud LB Proxy (deployment)

    Infrastructure Load-balancer implementation Router Application description k8s.Service (protocol, host, path) (protocol, host, path) ... Backends Application implementation cloud.Bucket
  8. A modest proposal: the model Infrastructure Router Backends *Names are

    temporary GatewayClass Gateway VirtualHost k8s.Service cloud.Bucket VirtualHost
  9. Gateway GatewayClass Gateway Gateway Represents actual instance of LB/proxy infrastructure,

    capacity Protocol termination (<IP:port>, TLS) Deployment specific options - e.g. mergeable Abstracts available implementations
  10. Gateway ↔ VirtualHost Gateway foo.com *.bar.com VirtualHost foo.com/path1 foo.com/path2 ...

    filtering gateways: [g1, g2] How to match Gateway and VirtualHost? • VirtualHost attaches to a Gateway • Gateway filters VirtualHosts -- wildcard allows for self-service.
  11. request VirtualHost: Match → Action host, protocol, port protocol specific

    predicate path == “/foo” protocol specific predicate path ~ “/bar.*” action send to service “foo” action reply with 404 ... Match Action
  12. VirtualHost # VirtualHost spec: routes: - http: host: "foo.com" rules:

    - path: "/" backend: service: foo-app servicePort: www - tcp: port: 9000 rules: - backend: service: tcp-app servicePort: my-protocol type VirtualHost struct { Routes []Route } type Route struct { HTTP *HTTPRoute TCP *TCPRoute } type HTTPRoute struct { Host string Rules []HTTPRouteRule } type HTTPRouteRule struct { // Match Path string ... // Action Backend *HTTPRouteActionBackend StatusCode *HTTPRouteActionStatusCode ... } <blink> just a sketch </blink>
  13. VirtualHost extensibility Complex syntax tree -- needs a decorator pattern

    • Better API machinery? • map[string]string vs Raw Objects (inline CRDs) vs CRD link Work through examples and UX for users
  14. Extended API 100% portable IF supported Portability Core API 100%

    portable Core MUST be supported. Extended Feature by feature. MAYBE supported, but MUST be portable. Part of k8s API schema.
  15. Custom API Extended API 100% portable IF supported Portability Core

    API 100% portable Core MUST be supported. Extended Feature by feature. MAYBE supported, but MUST be portable. Part of k8s API schema. Custom No guarantee for portability, No k8s API schema. gravity...
  16. Custom API Extended API 100% portable IF supported Portability Core

    API 100% portable Enforcement by conformance tests. Extended feature definition requires self-contained conformance. Require all extended features be checked statically? gravity...
  17. Portability Custom API Extended API Core API Neato Feature ®

    Ingress GA Traffic splitting Regex? Rewrite
  18. Multi-Cluster Services: Use Cases • Canary - Deploy new version

    of application in one Kubernetes cluster in one locality before rolling it out worldwide. • Low latency - Users get routed to the application backends that are closest to them geographically. • High availability - Users requests are served even if one cluster holding application backends is completely down. • Hybrid - Application spans multiple cloud providers or on-prem
  19. Multi-Cluster Services: User Journey Persona: Admin • I want L7

    load balancing across a global service that is deployed on Kubernetes clusters. • I want the ability to add / remove clusters from this load balancing based on business requirements. • I want teams in my organization (service owners) to manage their own ingress traffic but maintain a clear boundary between the “admin” and the “Kubernetes end user”. • I want my teams to use a native Kubernetes API to leverage default features (e.g namespacing, labels) & to reduce the learning curve.
  20. kind: MultiClusterIngress metadata: name: my-mci spec: template: spec: backend: serviceName:

    my-mc-service servicePort: 80 rules: - host: foo.bar.com http: paths: - backend: serviceName: my-mc-service servicePort: 80 kind: MultiClusterService metadata: name: my-mc-service spec: template: selector: app: shopping ports: - name: web protocol: TCP port: 80 targetPort: 80 clusters: - selector: matchLabels: region: us Multi-Cluster Services: APIs
  21. kind: MultiClusterIngress metadata: name: my-mci spec: template: spec: backend: serviceName:

    my-mc-service servicePort: 80 rules: - host: foo.bar.com http: paths: - backend: serviceName: my-mc-service servicePort: 80 kind: MultiClusterService metadata: name: my-mc-service spec: template: selector: app: shopping ports: - name: web protocol: TCP port: 80 targetPort: 80 clusters: - selector: matchLabels: region: us Multi-Cluster Services: APIs
  22. kind: MultiClusterIngress metadata: name: my-mci spec: template: spec: backend: serviceName:

    my-mc-service servicePort: 80 rules: - host: foo.bar.com http: paths: - backend: serviceName: my-mc-service servicePort: 80 kind: MultiClusterService metadata: name: my-mc-service spec: template: selector: app: shopping ports: - name: web protocol: TCP port: 80 targetPort: 80 clusters: - selector: matchLabels: region: us Multi-Cluster Services: APIs
  23. MCI MCS “Config Source” app app app app app app

    MCI MCS Admin Developer Teams Cluster turnup / turndown app app app app app app app app app app app app Multi-Cluster Services: Workflow
  24. MCI MCS “Config Source” app app app app app app

    MCI MCS Admin manages Developer Teams Cluster turnup / turndown app app app app app app app app app app app app Multi-Cluster Services: Workflow
  25. MCI MCS “Config Source” app app app app app app

    MCI MCS Admin manages manages Developer Teams Cluster turnup / turndown app app app app app app app app app app app app Multi-Cluster Services: Workflow
  26. MCI MCS “Config Source” app app app app app app

    MCI MCS Admin manages manages Developer Teams Cluster turnup / turndown app app app app app app app app app app app app manage workloads manage primitives Multi-Cluster Services: Workflow
  27. Multi-Cluster Services: FAQ • Why not have a MultiClusterDeployment? That

    alleviates the need for having to do the manual work of replicating their workload across clusters. ◦ Do not want to be opinionated on how users deploy their workloads. It’s not really the spirit of what the API is trying to provide. ◦ Best practices in this space are divided, no one-size fits all approach.
  28. Multi-Cluster Services: FAQ What about Federation v2? • Multi-cluster models

    and assumptions are slightly different. • Federation v2 is opinionated on cross-cluster DNS & SD, deployment of workloads, etc. • Federation v2 is trying to target a much wider set of use cases, multi-cluster services are an “à la carte” offering Where does Istio’s Multicluster story fit? • Potential for existing API’s to support use case.
  29. Conclusion • Initial KEP proposal to come shortly, although we

    have to get V1 GA wrapped up first. • Feedback from community is key • Reference implementation