Share
Embed
Supercharge your JVM App with Couchbase
This talk with demo was given at the EJUG Austria Java User Group in December 2013. Most of the slides shown here are introduction since the latter part focussed on UI and code demos.
Michael Nitschinger
December 03, 2013
Transcript
31 slides
-
Supercharge your JVM App with Couchbase Michael
Nitschinger -
Who‘s this guy? • SDK Engineer at Couchbase
Maintaining • Official Java SDK • Spymemcached • Spring-‐Data-‐Couchbase • @daschl on TwiCer • Core Team of NeCy.io • ContribuGng to Reactor and others -
Instagram gained nearly 1 million users overnight when then expanded
to Android Growth is the New Reality -
Draw Something Viral Growth 19 17 15
13 11 9 7 5 3 3/1 28 26 24 22 20 18 16 14 12 10 8 2/6 Draw Something by OMGPOP Daily Ac)ve Users (millions) 21 2 4 6 8 10 12 14 16 -
How do you take this growth? RDBMS is good
for many thing, but hard to scale RDBMS Scales Up Get a bigger, more complex server Users ApplicaUon Scales Out Just add more commodity web servers Users System Cost ApplicaUon Performance RelaUonal Database Web/App Server Tier System Cost ApplicaUon Performance Won’t scale beyond this point -
Scaling out RDBMS • Run Many SQL Servers • Data
could be shared -‐ Done by the applicaUon code • Caching for faster response Ume Web/App Server Tier Memcached Tier MySQL Tier -
Scaling out fla]ens the cost and performance curves NoSQL
Database Scales Out Cost and performance mirrors app Uer Users NoSQL Distributed Data Store Web/App Server Tier ApplicaUon Scales Out Just add more commodity web servers Users System Cost ApplicaUon Performance ApplicaUon Performance System Cost NoSQL Technology Scales Out -
Couchbase Open Source Project • Leading NoSQL database project
focused on distributed database technology and surrounding ecosystem • Supports both key-‐value and document-‐oriented use cases • All components are available under the Apache 2.0 Public License • Obtained as packaged so`ware in both enterprise and community ediUons. -
Easy Scalability Consistent High Performance Always
On 24x365 Grow cluster without applicaUon changes, without downUme with a single click Consistent sub-‐millisecond read and write response Umes with consistent high throughput No downUme for so`ware upgrades, hardware maintenance, etc. Flexible Data Model JSON document model with no fixed schema. Couchbase Server Core Principles -
New in 2.0+ JSON support Indexing and Querying
Cross data center replicaGon Incremental Map Reduce JSON JSON JSON JSON JSON -
Couchbase Server Architecture Heartbeat Process monitor Global
singleton supervisor ConfiguraUon manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replicaUon manager hCp REST management API/Web UI HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 -‐ 21199 Erlang/OTP storage interface Couchbase EP Engine 11210 Memcapable 2.0 Moxi 11211 Memcapable 1.0 Memcached New Persistence Layer 8092 Query API Query Engine Data Manager Cluster Manager -
Couchbase Server Architecture ReplicaUon, Rebalance, Shard State
Manager REST management API/Web UI 8091 Admin Console Erlang /OTP 11210 / 11211 Data access ports Object-‐managed Cache MulG-‐threaded Persistence Engine 8092 Query API Query Engine hCp Data Manager Cluster Manager -
3 3 2 Single node -‐ Couchbase
Write OperaGon Managed Cache Disk Queue Disk ReplicaUon Queue App Server Couchbase Server Node Doc 1 Doc 1 Doc 1 To other node -
COUCHBASE SERVER CLUSTER Cluster-‐wide Basic OperaGon •
Docs distributed evenly across servers • Each server stores both acGve and replica docs Only one server acUve at a Ume • Client library provides app with simple interface to database • Cluster map provides map to which server doc is on App never needs to know • App reads, writes, updates docs • MulGple app servers can access same document at same Gme User Configured Replica Count = 1 READ/WRITE/UPDATE ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 ACTIVE Doc 4 Doc 7 Doc Doc Doc SERVER 2 Doc 8 ACTIVE Doc 1 Doc 2 Doc Doc Doc REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc REPLICA Doc 6 Doc 3 Doc 2 Doc Doc Doc REPLICA Doc 7 Doc 9 Doc 5 Doc Doc Doc SERVER 3 Doc 6 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 -
Add Nodes to Cluster • Two servers added
One-‐click operaGon • Docs automaGcally rebalanced across cluster Even distribuUon of docs Minimum doc movement • Cluster map updated • App database calls now distributed over larger number of servers REPLICA ACTIVE Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc 8 Doc Doc 9 Doc Doc 2 Doc Doc 8 Doc Doc 5 Doc Doc 6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 -
Fail Over Node REPLICA ACTIVE
Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc 9 Doc 8 Doc Doc 6 Doc Doc Doc 5 Doc Doc 2 Doc 8 Doc Doc • App servers accessing docs • Requests to Server 3 fail • Cluster detects server failed Promotes replicas of docs to acUve Updates cluster map • Requests for docs now go to appropriate server • Typically rebalance would follow Doc Doc 1 Doc 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 User Configured Replica Count = 1 COUCHBASE SERVER CLUSTER -
COUCHBASE SERVER CLUSTER Indexing and Querying
User Configured Replica Count = 1 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 • Indexing work is distributed amongst nodes • Large data set possible • Parallelize the effort • Each node has index for data stored on it • Queries combine the results from required nodes ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 2 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 3 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 Query -
SERVER 3 SERVER 1
SERVER 2 Couchbase Server – San Francisco SERVER 3 SERVER 1 SERVER 2 Couchbase Server – New York OpGmisGc replicaGon Per replicaGon Tunable Parameters OpGmized protocol based on memcached Reliability and performance at scale Cross Data Center ReplicaGon (XDCR) -
Official Java SDK • Current Release: 1.2.3 (December 2013)
• Fully asynchronous (Futures & Callbacks) • Builds on the (very mature) Spymemcached Library • Does the heavy liping for you Knows the cluster topology „Routes“ operaUons to target servers Abstracts protocol semanUcs (memcache, h]p,...) -
Spring IntegraGon • IntegraGon with Spring Data
Templates Repositories ExcepUon Mapping • Transparent Caching support (@Cacheable) • Maps POJO EnGGes to JSON (and back) • Milestone 2 released, next up: RC1 • Planned: Spring Boot, Spring XD -
And more... • Play Framework IntegraGon Community
driven h]ps://github.com/mathieuancelin/play2-‐couchbase • Working on: Hibernate OGM support N1QL Query IntegraUon Next GeneraUon SDKs (2.0 API and performance)