Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to NoSQL Solutions with Couchbase ...

Introduction to NoSQL Solutions with Couchbase and .NET Core

Baris Ceviz

April 20, 2019
Tweet

More Decks by Baris Ceviz

Other Decks in Programming

Transcript

  1. Baris Ceviz | Software Developer @Trendyol Twitter – Github –

    LinkedIn : @PeaceCwz Mail: [email protected] Introduction to NoSQL Solutions with Couchbase and .NET Core
  2. AGENDA 01/ 02/ 03/ 04/ What’s Couchbase Clusters and nodes

    Key-Value Database Couchbase .NET SDK
  3. A Cluster and Sync Gateway Application SDK App Server Application

    SDK App Server Application SDK App Server Application SDK App Server Sync Gateway Application SDK App Server Application SDK App Server Sync Gateway 4 CLUSTER of COUCHBASE NODES
  4. App Server SDK Application Application SDK App Server Application SDK

    App Server Sync Gateway Sync Gateway Application SDK App Server Application SDK App Server Application SDK App Server 5 A Cluster and Sync Gateway CLUSTER of COUCHBASE NODES
  5. Application SDK App Server Application SDK App Server Application SDK

    App Server Sync Gateway Sync Gateway Application SDK App Server Application SDK App Server Application SDK App Server 6 A Cluster and Sync Gateway Kubernetes CLUSTER of COUCHBASE NODES
  6. Application SDK App Server Application SDK App Server Application SDK

    App Server Sync Gateway Sync Gateway Application SDK App Server Application SDK App Server Application SDK App Server A Cluster and Sync Gateway CLUSTER of COUCHBASE NODES 7
  7. Kubernetes Autonomous Operator A Cluster and Sync Gateway and Kubernetes

    Autonomous Application SDK App Server Application SDK App Server Application SDK App Server Application SDK App Server Sync Gateway Application SDK App Server Application SDK App Server Sync Gateway 8 CLUSTER of COUCHBASE NODES
  8. Nodes COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing

    Key Value Cluster Manager COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing Key Value Cluster Manager COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing Key Value Cluster Manager COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing Key Value Cluster Manager
  9. Nodes COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing

    Key Value Cluster Manager COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing Key Value Cluster Manager COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing Key Value Cluster Manager COUCHBASE SERVER NODE Analytics Query Indexing Full Text Eventing Key Value Cluster Manager
  10. Mix & match subsets of services (Multidimensional Scaling) Analytics Query

    Indexing Full Text Eventing Key Value Cluster Manager Analytics Query Indexing Full Text Eventing Key Value Cluster Manager Analytics Query Indexing Full Text Eventing Key Value Cluster Manager Analytics Query Indexing Full Text Eventing Key Value Cluster Manager COUCHBASE SERVER NODE COUCHBASE SERVER NODE COUCHBASE SERVER NODE COUCHBASE SERVER NODE
  11. Mix & match subsets of services (Multidimensional Scaling) Analytics Query

    Indexing Full Text Eventing Key Value Cluster Manager Analytics Query Indexing Full Text Eventing Key Value Cluster Manager Analytics Query Indexing Full Text Eventing Key Value Cluster Manager Analytics Query Indexing Full Text Eventing Key Value Cluster Manager COUCHBASE SERVER NODE COUCHBASE SERVER NODE COUCHBASE SERVER NODE COUCHBASE SERVER NODE
  12. Cluster Manager Cluster Manager Cluster Manager Cluster Manager Cluster Manager

    Cluster Manager Cluster Manager Cluster Manager Analytics Search Key Value Query Indexing Key Value Key Value Mix & match subsets of services (Multidimensional Scaling) Analytics
  13. The Cluster Manager The control plane of the cluster: •

    Admin APIs & User Interface • Authentication / RBAC • Master Services • Rebalance and Auto-Failover • Automatic shard placement • Bucket Services • Per-node Services • Generic facilities per-node-&-bucket services Generic distributed facilities Generic local facilities Logging and other services Distributed node discovery Master Services • Cluster-wide operations • Rebalance • Auto-failover • Automatic shard placement Global config (gossip replication) Local config Store Per-node Services • Heartbeats • Babysitter Cluster Manager Bucket services • Replica stream management • Stats collection & monitoring 14 Admin Console & REST API Authentication
  14. KEY VALUE DATA SERVICE App Server SDK Application HASH("user_profile-1234") ==>

    27 VB Node 0 A 1 B 2 C … 27 C 1023 D { … } Automatic Sharding and Rebalancing NODE A NODE B NODE C NODE D CLUSTER MANAGER 16 user_profile-1234
  15. App Server SDK Application VB Node 0 A 1 B

    2 C … 27 C 1023 D Automatic Sharding and Rebalancing CLUSTER MANAGER KEY VALUE DATA SERVICE NODE A NODE B NODE C NODE D CLUSTER MANAGER 17 user_profile-1234 { … }
  16. App Server SDK Application VB Node 0 A 1 B

    2 C … 27 C 1023 D Automatic Sharding and Rebalancing KEY VALUE DATA SERVICE NODE A NODE B NODE C NODE D NODE E CLUSTER MANAGER 18 user_profile-1234 { … }
  17. C L DATA SERVICE App Server SDK Application VB Nod

    e 0 A 1 B 2 C … 27 C 1023 D { … } NODE A NODE B NODE C NODE D VB Nod e 0 A 1 B 2 C … 27 E 1023 D NODE E Automatic Sharding and Rebalancing CLUSTER MANAGER KEY VALUE DATA SERVICE HASH(“user_profile-1234”) ==> 27 user_profile-1234 19
  18. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable 21 The Database Change Protocol
  19. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable Ke 22 y Value D ata Service The Database Change Protocol
  20. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable Ke y Value Data Service The Database Change Protocol DCP 23
  21. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable Ke y Value Data Service The Database Change Protocol DCP 24
  22. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable Ke y Value D ata Service The Database Change Protocol DCP DCP 25
  23. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable The Database Change Protocol DCP Key Value Data Service DCP 26
  24. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable Ke y Value D ata Service The Database Change Protocol OTHER FEATURES & SERVICES • XDCR & Backup • Sync Gateway DCP DCP 27
  25. • High Performance / In-Memory • De-Duplication • Ordered, predictable

    and consistent • Restartable Key Value Data Service DCP The Database Change Protocol INTEGRATIONS • hadoop, kafka & more OTHER FEATURES & SERVICES • XDCR & Backup • Sync Gateway DCP 28
  26. Key Value … Networking I/O & handler threads VBucket Manager

    Item Pager Expiry Pager Checkpoint Manager Cache . . Partition Hash Tables Fetcher Scheduler Reader IO Writer IO Generic Tasks Flusher The Key Value Data Service Authentication 29 Analytics Query Indexing Full Text Eventing Key Value Cluster Manager
  27. Key Value … Networking I/O & handler threads VBucket Manager

    Item Pager Expiry Pager Checkpoint Manager Cache . . H Partition ash Tables Fetcher Scheduler Reader IO Writer IO Generic Tasks Flusher The Key Value Data Service The KV "Data Plane" of a Couchbase cluster: • High performance memcached DNA • With data partitioning ("vbuckets") • Sub-Document operations • Authentication / RBAC And also... • Asynch task workers and scheduler • For I/O: persistence, backfill • For expiry & cleanup • For replication • The source of Database Change Protocol • A consumer of Database Change Protocol 30 Authentication