in database Scaling up Scaling out NoSQL workloads and scalability model Core data operations, indexing and querying Homogenous HW scaling Introducing multi-dimensional scaling Isolation of workloads through a services architecture Independent HW scaling Q & A
views expressed in this presentation are our own and do not necessarily reflect the views of Couchbase Most of the content in this presentation is originally created by Anil, Product Manager, Couchbase Thanks Anil, for allowing me to re-use your slides 3
M Tauro – [email protected] - Senior Instructor, Couchbase - ~11 Years Professional Teaching and Consulting Experience - Worked at Pivotal – Instructor/Consultant for Spring/Spring Security/Spring Web/Enterprise Integration with Spring/Spring JMS/Spring Web/Spring Batch, Pivotal Hadoop/Cloud Foundry - PhD in Computer Science from Christ University [thesis accepted] - Hard-core Dog lover 4
Pros Can result in major performance improvement Machines can now support having many cores and terabytes of RAM Cons Expensive Requires downtime Performance bounded – at some point the database engine itself becomes the bottleneck Limited in scalability and elasticity
Pros Simple and easily scalable Data evenly split across cluster of nodes Scales linearly with throughput Highly available No single point of failure Cons Not great for all workloads – data, index and query
workloads Core data processing: GETs & SETs for a given key Indexing: Index maintenance and lookups Querying: Combine index and data with complex just-in-time data re-shaping, ordering, grouping, aggregations, and more Varying resource requirements - CPU, RAM, I/O, Network Varying methods to optimize latency & throughput for each
Each node get a slice of the workload Simple to do… But... Workloads compete and interfere with each other Can’t fine tune each workload Core Data operation are partition-able so great with wider fan-out Indexing and queries aren’t always partitionable, so worse with wider fan-out Index Workload Couchbase Cluster Query Workload Data Workload node1 node8
MDS is the architecture that enables independent scaling of data, query and indexing workloads. Index Service Couchbase Cluster Query Service Data Service node1 node8
Isolated Service for minimized interference Independent “zones” for Query, Index and Data Services Minimize indexing and query overhead on core key-value operations. Index Service Query Service Data Service
Best Computational Capacity per Service Heavier indexing (index more fields) : scale up index service nodes More RAM for query processing: scale up query service nodes Couchbase Cluster node1 node8 node9 Data Service Index Service Query Service
1 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 2 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service
1 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 2 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 3 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 4 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 5 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service STORAGE Couchbase Server 6 SHARD 7 SHARD 9 SHARD 5 SHARD SHARD SHARD Managed Cache Cluster Manager Cluster Manager Managed Cache Storage Data Service Index Service Query Service
Projector & Router New Services in Couchbase Server 4.0 Query Service Index Service Supervisor Index maintenance & Scan coordinator Index#2 Index#1 Query Processor cbq-engine Bucket#1 Bucket#2 DCP Stream Index#4 Index#3 ... B u c k e t # 2 B u c k e t # 1 18093 11211 18901 Managed Cache
of scalability with Couchbase Server Separate out competing workloads to independent services Independently scale each service “zone” within the cluster Couchbase Server with MDS maximizes scalability and performance Improves scale and performance to degrees not possible with other NoSQL or big-data engines on premise or in the cloud Improved price/performance and squeezes more performance and throughput for mission-critical systems
Couchbase: http://training.couchbase.com CD220: Developing Couchbase NoSQL Applications Oct 20 – Oct 23 2015 CS300: Couchbase NoSQL Server Administration Nov 17 – Nov 20 Enroll Today!