abstraction. • Nodes are in a single cluster • Scheduling is considered per-cluster • Clusters have their own network configs • Service discovery is per-cluster • Volumes and LBs are tied to cluster • Each cluster is its own ID provider • Each cluster does its own authn and authz In the beginning...clusters
as possible • Jurisdiction: Required to keep user-data in the country • Data gravity: Large amounts of data already exists and would cost too much to move Why so many clusters?
the whole app • Blast radius: Unplanned problems have bounded impact • Upgrades: Do one part at a time, or even avoid in-place upgrades • Scale: App is too big for one cluster Why so many clusters?
not impact each other • Security: Sensitive data, untrusted code, or very-high-value services • Organization: Different management • Cost: Teams get their own bills Why so many clusters?
• Goal: API compatible with k8s • Runs a “control-plane” cluster • Adds a “cluster” API resource • Adds cross-cluster controllers Kubefed aka “Übernetes”
multi-cluster controllers without defining or implementing them ourselves • Human- and machine-friendly • Enable Single-Pane-Of-Glass UX ClusterRegistry
• Only specific types are federated • API leaves room for placement and overrides • Many of v1’s problems persist • Proposed to be archived Kubefed v2 v2
clusters • Care about governance • Care about TCO 2) Application operators • Shouldn’t care about clusters • Care about functionality • Care about UX Who are we solving for?
payload Target selection Payload specialization per target Payload delivery member cluster Might be a git repository, a k8s API, a kcp instance, or other. Might be label selectors, a list of cluster names, or other. Might be push or pull. Might have policies applied (e.g. rate limits). Might be unilateral or reconciled bi-directionally. Might be templates, helm, kustomize, or other. Cluster registry Might be individual resources or some form of “package”. Target sequencing Might be none or highly orchestrated or something in-between. NB: Arbitrary processing can occur at various stages E.g “package” expansion could be: • before commit • at payload specialization • at payload delivery
Capturing tenancy more fully MC Scheduling • Pick the best cluster for my workload MC Stateful apps • Move/share disks between clusters • DR, active-passive, or active-active Future projects (?)