Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Stateful Applications in Kubernetes: Ready for ...

Niraj Tolia
September 25, 2018

Stateful Applications in Kubernetes: Ready for Production!

This presentation addresses the widespread misconception that Kubernetes isn’t ready for stateful applications. Historically, Kubernetes storage arrived late and stateful workloads were missing the equivalent platform primitives that enabled ease-of-use and flexibility with stateless applications. Not only has this been addressed, but we will show how Kubernetes can become the preferred platform for stateful applications because of its developer-first approach.

Readers will walk away with a clear idea of:
* Support at every layer for building stateful cloud-native applications including the Container Storage Interface (CSI) and concepts such as StatefulSets
* How developers can effectively leverage Helm charts and use CRDs+Operators to manage applications at scale
* Knowledge of upcoming functionality (eg, LocalStorage) and how next-gen data stores (eg, CockroachDB) fit into cloud-native environments

Niraj Tolia

September 25, 2018
Tweet

More Decks by Niraj Tolia

Other Decks in Technology

Transcript

  1. kubernetes philosophy page 04 Developer and Application Focused Puts the

    needs of the application and developer first and optimizes for agility Enforces Good DevOps Hygiene Immutability, config as code, automation makes it easy to repave all infrastructure Declarative Approach A robust systems approach where the state of the world is reconciled with the expectation
  2. key kubernetes features page 05 Self-Healing Auto restart of unhealthy

    containers to match service levels Auto Scaling Scale applications up and down in response to load Resource Utilization Better bin packing for higher resource utilization Portability Isolates developers and applications from infrastructure Deployment Options Variety of upgrade deployment strategies w/ rollback options Service Discovery Familiar IP and DNS-based service discovery and load balancing
  3. dynamic storage provisioning for persistent storage page 014 01Self Service

    Allow high developer velocity, no admin in the loop 02Portable No references to underlying storage provider. Allows application portability 03On-Demand Provisioned at time of use. Lifecycle can be tied to the application.
  4. dynamic storage provisioning persistent volume (pv) page 015 A Persistent

    Volume (PV) represents provisioned storage in the cluster (e.g., NFS, iSCSI, other block, etc.). A PV’s lifecycle is independent of the container/pod that uses it.
  5. dynamic storage provisioning persistent volume claim (pvc) page 016 kind:

    PersistentVolumeClaim apiVersion: v1 metadata: name: my-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 8Gi storageClassName: ssd
  6. page 017 kind: Deployment apiVersion: v1 metadata: name: my-app spec:

    template: spec: containers: - name: app-container image: alpine:3.7 command: ["my-app.sh"] args: ["--datadir", "/data/my-app"] volumeMounts: - name: data-volume mountPath: /data volumes: - name: data-volume persistentVolumeClaim: claimName: my-claim kind: PersistentVolumeClaim apiVersion: v1 metadata: name: my-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 8Gi storageClassName: ssd dynamic storage provisioning persistent volume claim (pvc)
  7. dynamic storage provisioning putting it all together page 018 StorageClass

    (SC) PersistentVolumeClaim (PVC) Volume mounted on node where Pod is scheduled (based on Pod -> PVC -> PV mapping) Bind PVC to PV PersistentVolume (PV) Select SC Select Provisioner Create PV for new Volume Storage Infrastructure Volume Create Volume Node App Application Definition
  8. container storage interface the path forward page 019 Out of

    Tree Independent Development and Release Cycles, Easier to Maintain Standard Deployment Common deployment interface using native Kubernetes primitives File & Block Standardized implementation APIs for using file and block Cross-Orchestrator Vendor friendly. Kubernetes, Mesos, CloudFoundry, See Managing Disk Volumes in Kubernetes SDC 2018 talk by Saad and Nikhil for more info!
  9. page 020 data is important • How does backup, recovery,

    and migrate work? See Kasten’s K10 as an example! • Resource contention concerns • High-availability depends on instance coordination • Frequent restarts/pre-empts destabilize service resiliency is complex • Instances are unique and are not interchangeable • Access to persistent data is needed across restarts state is meaningful other operational concerns scheduling, backup, restore, migration
  10. StatefulSets support for stateful applications page 022 Stable Identifiers Stable

    network identifiers for applications that depend on this Stable Persistence Includes persistent mapping across pod restarts and reschedules Ordered Operations Ordered and graceful deployment, scaling, termination Update Operations Rolling updates with restrictions
  11. Observe Analyze Act the operator design pattern to deploy and

    manage apps page 023 human ops knowledge → software Support Complex Ops Backups, Recovery, Scaling, Upgrades Active Reconciliation Reconcile desired vs. actual state SDK-based Easy to get started with multiple SDKs. Still a few sharp edges though. Extensible Developer-extensible via CustomResourceDefinitions
  12. page 024 kanister: A framework for application-level data management •

    Supports complex distributed applications • Separates mechanism from policy/orchestration • Allows for unified schedulers and monitoring • Clean API allows for developer extensions https://github.com/kanisterio
  13. kanister operator example postgresql backup page 026 1. Object Creation

    2. Base Backup + Env Setup KubeExec 4. Status Update Kanister Controller Backup Request Object (Custom Resource) PostgreSQL + WAL-E 3. Base + WAL Shipping Object Storage Kanister Blueprints
  14. kanister operator example postgresql backup page 027 1. Object Creation

    2. Base Backup + Env Setup KubeExec 4. Status Update Kanister Controller Backup Request Object (Custom Resource) PostgreSQL + WAL-E 3. Base + WAL Shipping Object Storage Kanister Blueprints
  15. kanister actionset (abridged) page 028 apiVersion: cr.kanister.io/v1alpha1 kind: ActionSet spec:

    actions: - name: backup blueprint: postgresql object: kind: StatefulSet name: postgresql-cluster namespace: default configMaps: ...
  16. kanister blueprint (abridged) page 029 apiVersion: cr.kanister.io/v1alpha1 kind: Blueprint actions:

    backup: type: StatefulSet phases: - func: KubeExec args: - '{{ .StatefulSet.Namespace }}' - '{{ index .StatefulSet.Pods 0 }}' - postgresql-tools-sidecar - bash - -c - wal-e ... - func: ... restore: ...
  17. other awesome stateful operators page 030 Look at the extensive

    list at https://github.com/operator-framework/awesome-operators and more…
  18. packaging your applications helm: the kubernetes package manager page 031

    off-the-shelf stateful “charts” Multiple community charts available for databases, NoSQL systems, and more. supports composability Enhance or restrict based on your goals. Compose stateful services within your apps. organize settings Easy-to-use mechanisms and a single place to codify your application’s configuration options. $ helm install stable/postgresql --set persistence.size=40Gi --set persistence.storageClass=ssd <your-app>/requirements.yaml dependencies: -name: postgresql
  19. cloud-native databases cockroachdb, vitess, yugabyte, and more… page 033 Reduces

    ops overhead by automatically handling system management tasks self-managing Fault-tolerance built in to support transparent self-healing infra resilient Auto-scaling built to respond to load and deliver predictable performance scalable
  20. local persistent volumes (beta) local disks “done right” page 034

    Leverage Local Disks For systems (Ceph, Cassandra, etc.) that work best on local storage Common Primitives Uses well-know PersistentVolume, PersistentVolumeClaim, StorageClass Smarter Scheduling Smarter pod scheduling and volume binding compared to hostPath Optionally Expose as Block Not just file system access anymore
  21. kubernetes and state wrapping up page 35 01Platform Support Equivalent

    features and concepts that made stateless successful 04Increased Production Usage 50%+ users using stateful applications - SIG-APPs Survey, Apr’18 02Storage Vendor Choices Large number of storage provider choices, CSI, Portability Abstractions 03Relational / NoSQL Systems Support from traditional relational and NoSQL systems. First-class operators. Cloud-Native DBs. Stateful is Ready for Production!