Upgrade to Pro — share decks privately, control downloads, hide ads and more …

#PGConf.Asia 2019 Building PostgreSQL as a Serv...

tzkoba
September 02, 2019

#PGConf.Asia 2019 Building PostgreSQL as a Service with K8s

This slide is for PGConf.Asia 2019.
You can check how to build "PostgreSQL on Kubernetes" with production-grade.

tzkoba

September 02, 2019
Tweet

More Decks by tzkoba

Other Decks in Technology

Transcript

  1. 3 My Actitvities PGConf.Asia 2018 @Tokyo A guide of PostgreSQL

    on Kuberntes - In terms of Storage - CloudNativeDays Tokyo 2019 The Future of Database on Kubernetes - What run with Cloud Native Storage -  + =∞
  2. 4 Agenda 1. Recap: What is Kubernetes? 2. The Issues

    for Database on Kubernetes 3. How to run your PostgreSQL on K8s 4. Kubernetes becomes The Platform
  3. 7 What is Kubernetes? Pod Pod Pod Pod Pod •

    Kubernetes(K8s) is the orchestrator tool for containers. It has 3 features below. • Declarative config • Auto-healing • Immutable Database is not Immutable.
  4. 8 Better to handle the database system by Kubernetes? Node

    Node Node Master Slave Replicate • The database usually has a state that is not easy to maintain by Kubernetes. <Disadvantage for DB> • Necessary to startup in turn. • Must never lose their data. • Handle the database as pets.
  5. 9 Example of Database on Kubernetes: Vitess VTtablet VTtablet VTtablet

    VTgate app app app SQL SQL SQL • Vitess that used on YouTube is the CNCF incubating project. • Vitess provides MySQL sharding in K8s. • VTgate and VTtablet can scale by K8s. • When terminating a component abnormally, Kubernetes repair it automatically.
  6. 10 The choice: How to manage your database Compute Storage

    Managed Amazon Aurora Amazon Redshift Amazon RDS on Cloud on Kubernetes • You can choose to manage the database by yourself or else.
  7. 18 Kubernetes is the Distributed Systems • Developed as following

    a distributed architecture. • When doesn’t a node reply – Network partition? – Process failure? – Node failure? • If the disk resource attached, harder to determine. FailOver?
  8. 20 Database Architects are familiar with Clustering  “If you

    don’t know the status, it’s okay. We act on the premise of failsafe.”  “No need to share resources. Right?”  “Both have long been known for database clustering.”
  9. 21 Basic: Database Clustering HA (Active/Standby) 1 Sharding Replication (Active/Active)

    2or more Instances Redundancy 2 or more Shared Disk Log Shipping --- × Scaleout? Read Read/ Write Failover (Fencing) Availability Promotion (Election) --- • There are differences to build a DB cluster with some nodes.
  10. 22 Clustering #1: HA • With Linux-HA • Use high-available

    shared storage <Worst Case> • Multiple writes to storage <Solution> • Fencing VIP Linux-HA Controller Controller • It's been used since before Linux but helpful.
  11. 23 Note: Fencing VIP Linux-HA Controller Controller < When Detecting

    Node Failure > 1. Forced node power off i. Definite processes stop ii. Unmount storage iii. Detach virtual IP 2. PostgreSQL starts to run on the standby node. • Failed node is isolated from resources = Fencing
  12. 24 Clustering #2: Replication WAL • The master can Read/Write,

    Slaves are Read-Only. • Data synchronization by WAL transmission <Worst Case> • 2 or more Masters <Solution> • Leader Election • Redundancy built into PostgreSQL = Streaming Replication Master Slave Slave
  13. 25 Note: Leader Election WAL Be promoted as a master,

    The other is still a slave. • Always one master • The former master joins as a slave. <Master in unknown state> 1. The remaining one slave is elected as the leader 2. The leader is promoted as a master. • Algorithms such as Paxos and Raft are used. Master Slave
  14. 26 Clustering #3 Sharding • Divide data between nodes and

    operates as one DB. • Dispatches queries to relevant nodes. • Basically no availability. • Problems with the transaction. • For rather scalability than availability. Coordinator
  15. 28 Implemetation Overview : on Kubernetes # Category OSS used

    Description ⅰ HA • Use Rook/Ceph as Shared Storage. ⅱ • Use LINSTOR/DRBD as Shared Storage. ⅲ Replication • Use Streaming Replication, without Shared Storage. ⅳ Operator • Building and Operating Replication automatically. • We can see following four patterns.
  16. 29 • K8s manages everything(DB,storage) • Shared-Storage: Ceph • Fenced

    by kube-fencing < Disadvantage > • Complicated • Insufficient IO HA (i): Replicas:1 • is deployed as StatefulSet using Rook/Ceph. kube-fencing
  17. 30 Note: Without Fencing Replicas:1 • When a node goes

    down, never failover. • To avoid network partition. • It is by design.
  18. 31 Note: What is • Rook is Kubernetes Operator managing

    Ceph or others. operator agent/discover agent/discover agent/discover osd osd osd mon mon mon CSI csi-provisioner csi-rbdplugin csi-rbdplugin csi-rbdplugin Rook • Rook makes easy to build Ceph cluster. • Also easy to deploy CSI modules. • CSI: Containar Storage Interface
  19. 32 HA (ii): Replicas:1 kube-fencing • LINSTOR is Software-Defined Storage

    based on DRBD. • K8s manages everything(DB,storage) • Redundancy: DRBD • Simple, Read IO without Network < Disadvantage > • Limited to Scale
  20. 33 Benchmark Results Single(with EBS) Rook/Ceph DRBD 1nodes 5nodes 2nodes

    100 37.8 77.1 • Measured by pgbench for 3 patterns. TPS
  21. 34 Replication : proxy proxy proxy keeper keeper keeper sentinel

    sentinel sentinel • Builds Streaming Replication on top of Kubernetes. • 3 types of processes have different roles • Without Shared- Resources < Disadvantage > • Not builtin Read Off- loading
  22. 36 Operator : • KubeDB operates not only but also

    others. kubedb-operator -0 -1 -2 postgres snapshot dormantdabases • Database Operator for – PostgreSQL – MySQL – Redis • Kubedb-operator builds SR. • Able to get/restore snapshot easily.
  23. 37 Example : PostgreSQL Configration by KubeDB apiVersion: kubedb.com/v1alpha1 kind:

    Postgres metadata: name: ha-postgres namespace: demo spec: version: “10.6-v2" replicas: 3 storageType: Durable storage: storageClassName: "standard" accessModes: - ReadWriteOnce resources: requests: storage: 100Gi  spec.version – Choose PostgreSQL version.  spec.replicas – The number of Instances.  spec.storage – Define storage type/size, etc. • Allows to define Streaming Replication with a simple YAML.
  24. 38 Example : Snapshot by KubeDB apiVersion: kubedb.com/v1alpha1 kind: Snapshot

    metadata: name: snapshot-to-s3 labels: kubedb.com/kind: Postgres spec: databaseName: ha-postgres storageSecretName: s3-secret s3: endpoint: 's3.amazonaws.com' bucket: kubedb-qa prefix: demo • Write declarative Snapshot settings by YAML. • Simple backup that applies only this YAML. • You can select storage, – S3 – Swift – Kubernetes Persistent Volume
  25. 39 Note : Backup with PostgreSQL + Ceph $ kubectl

    exec -it -n rook-ceph rook-ceph-tools-seq -- rbd -p replicapool ls pvc-bdbc6e53-f6e9-11e8-b0d9-02f062df6b48 $ kubectl exec -it pg-rook-sf-0 -- psql -h localhost -U postgres -c "SELECT pg_start_backup(now()::text);" pg_start_backup ----------------- 0/C000028 (1 row) $ kubectl exec -it -n rook-ceph rook-ceph-tools-seq -- rbd snap create replicapool/img@snap $ kubectl exec -it pg-rook-sf-0 -- psql -h localhost -U postgres -c "SELECT pg_stop_backup();" NOTICE: pg_stop_backup complete, all required WAL segments have been archived pg_stop_backup ---------------- 0/D000050 (1 row) • Need to know PostgreSQL & Ceph Commands.
  26. 42 To Recap  The components of database clustering with

    Kubernetes Native are already available.  You can see some operators for DBA task automation.  However, it is not over yet. Cloud Native Storage + + = ???
  27. 43 The Signs I. Pluggable Storage  Optimized Storage system

    for DB on K8s? II. Forked and Cloud-Oriented PostgreSQL  AWS Aurora, Azure Hyperscale
  28. 44 THE LOG IS THE DATABASE. SQL Transactions Caching Storage

    Logging Storage Logging Storage Logging CPU Memory Cache(SSD) Page Cache(SSD) Log AWS Aurora(PostgreSQL) Azure Hyperscale • Both divide RDBMS functions and are extended by each cloud.
  29. 45 As the platform for PostgreSQL as a Service DBaaS

    by Kubernetes STaaS by Kubernetes What we got for DBaaS • HA • Streaming Replication • DB Operator Also for STaaS • Simple Redundancy • Distributed Storage • Interoperable IF(CSI) • Kubernetes will be "The Platform for Platforms."