Share
Embed
Couchbase in the Enterprise with Spring Data
Given at SpringExchange 2013 in London, this talk is an introduction into Couchbase, followed by a Demo on Spring Data Couchbase.
See the video here: http://skillsmatter.com/podcast/java-jee/couchbase-in-the-enterprise-with-spring-data
Michael Nitschinger
November 15, 2013
Transcript
34 slides
-
1 Couchbase in the Enterprise with Spring Data
Michael Nitschinger -
2 {“about“: “me“} • Developer Advocate at Couchbase
Java SDK Maintainer Spring-‐Data-‐Couchbase Maintainer • @daschl • Core dev on NeAy.io • ContribuDng to Reactor, and others • Living in Austria (Vienna) -
4 Couchbase Open Source Project • Leading NoSQL
database project focused on distributed database technology and surrounding ecosystem • Supports both key-‐value and document-‐oriented use cases • All components are available under the Apache 2.0 Public License • Obtained as packaged soHware in both enterprise and community ediJons. Couchbase Open Source Project -
5 Easy Scalability Consistent High Performance
Always On 24x365 Grow cluster without applicaJon changes, without downJme with a single click Consistent sub-‐millisecond read and write response Jmes with consistent high throughput No downJme for soHware upgrades, hardware maintenance, etc. JSON JSON JSON JSON JSON PERFORMANCE Flexible Data Model JSON document model with no fixed schema. Couchbase Server -
6 Since 2.* JSON support Indexing and
Querying Cross data center replicaDon Incremental Map Reduce JSON JSON JSON JSON JSON -
7 High-‐Level Architecture Heartbeat Process monitor
Global singleton supervisor ConfiguraJon manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replicaJon manager hAp REST management API/Web UI HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 -‐ 21199 Erlang/OTP storage interface Couchbase EP Engine 11210 Memcapable 2.0 Moxi 11211 Memcapable 1.0 Memcached New Persistence Layer 8092 Query API Query Engine Data Manager Cluster Manager -
8 Lower-‐Level Architecture ReplicaJon, Rebalance, Shard
State Manager REST management API/Web UI 8091 Admin Console Erlang /OTP 11210 / 11211 Data access ports Object-‐managed Cache MulD-‐threaded Persistence Engine 8092 Query API Query Engine hAp Data Manager Cluster Manager -
9 3 3 2 WriDng Data
Managed Cache Disk Queue Disk ReplicaJon Queue App Server Couchbase Server Node Doc 1 Doc 1 Doc 1 To other node -
10 GET Doc 1 3 3
2 Reading Data Disk Queue ReplicaJon Queue App Server Doc 1 Doc 1 Doc 1 Managed Cache Disk To other node Couchbase Server Node -
11 COUCHBASE SERVER CLUSTER Basic OperaDon
• Docs distributed evenly across servers • Each server stores both acDve and replica docs Only one server acJve at a Jme • Client library provides app with simple interface to database • Cluster map provides map to which server doc is on App never needs to know • App reads, writes, updates docs • MulDple app servers can access same document at same Dme User Configured Replica Count = 1 READ/WRITE/UPDATE ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 ACTIVE Doc 4 Doc 7 Doc Doc Doc SERVER 2 Doc 8 ACTIVE Doc 1 Doc 2 Doc Doc Doc REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc REPLICA Doc 6 Doc 3 Doc 2 Doc Doc Doc REPLICA Doc 7 Doc 9 Doc 5 Doc Doc Doc SERVER 3 Doc 6 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 -
12 Add Nodes to Cluster • Two servers
added One-‐click operaDon • Docs automaDcally rebalanced across cluster Even distribuJon of docs Minimum doc movement • Cluster map updated • App database calls now distributed over larger number of servers REPLICA ACTIVE Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc 8 Doc Doc 9 Doc Doc 2 Doc Doc 8 Doc Doc 5 Doc Doc 6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 -
13 Fail Over Node REPLICA
ACTIVE Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc 9 Doc 8 Doc Doc 6 Doc Doc Doc 5 Doc Doc 2 Doc 8 Doc Doc • App servers accessing docs • Requests to Server 3 fail • Cluster detects server failed Promotes replicas of docs to acJve Updates cluster map • Requests for docs now go to appropriate server • Typically rebalance would follow Doc Doc 1 Doc 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 User Configured Replica Count = 1 COUCHBASE SERVER CLUSTER -
14 COUCHBASE SERVER CLUSTER Indexing and Querying
User Configured Replica Count = 1 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 • Indexing work is distributed amongst nodes • Large data set possible • Parallelize the effort • Each node has index for data stored on it • Queries combine the results from required nodes ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 2 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 3 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 Query -
15 SERVER 3 SERVER
1 SERVER 2 Couchbase Server – San Francisco SERVER 3 SERVER 1 SERVER 2 Couchbase Server – New York OpDmisDc replicaDon Per replicaDon Tunable Parameters OpDmized protocol based on memcached Reliability and performance at scale Cross Data Center ReplicaDon (XDCR) -
16 3 3 2 XDCR Data
Flow 2 Managed Cache Disk Queue Disk ReplicaJon Queue App Server Couchbase Server Node Doc 1 Doc 1 Doc 1 To other node XDCR Engine Doc 1 To other cluster -
17 www.couchbase.com/develop Go Clojure Community SDKs
Python Ruby Official SDKs Couchbase SDKs -
19 Documents within Couchbase • A (mostly) JSON-‐oriented
Document Database. • Every document has • a unique idenJfier (“key”) • JSON or binary content (“value”) • Couchbase uses JSON values to build indexes, query data and perform advanced lookups. Couchbase stores the ‘Meta’ of each Document, and the Body (Content)… -
20 meta { “id”: “[email protected]”, “rev”:
“1-‐0002bce0000000000”, “flags”: 0, “expiraJon”: 0, “type”: “json” } document { “uid”: 1234, “firstname”: “Foo”, “lastname”: “Bar”, “age”: 22, “favorite_colors”: [“green”, “red”], “email”: “[email protected]” } Meta InformaDon Including Key (ID) All Keys Unique and Kept in RAM at all Jmes. Document Value Most Recent In Ram And Persisted To Disk JSON Document Structure -
21 User Object string uid string
firstname string lastname int age array favorite_colors string email u::[email protected] { “uid”: 1234, “firstname”: “Foo”, “lastname”: “Bar”, “age”: 22, “favorite_colors”: [“green”, “red”], “email”: “[email protected]” } User Object string uid string firstname string lastname int age array favorite_colors string email set() get() Objects Serialized to JSON and Back u::[email protected] { “uid”: 1234, “firstname”: “Foo”, “lastname”: “Bar”, “age”: 22, “favorite_colors”: [“green”, “red”], “email”: “[email protected]” } -
23 Spring Data Couchbase • IntegraDon with Spring
Data Template access Repositoriy access ExcepJon TranslaJons Transparent Caching through @Cacheable • Main purpose: POJOs <-‐> JSON • M2 Released! Thanks to David Harrigan! Now on the way to a Release Candidate -
24 Spring Data Couchbase • Dependencies
Spring-‐Data-‐Commons Jackson Official Couchbase Java Client • Supports Java & XML based ConfiguraDon • Works with Couchbase Server 2.* • Grab snapshots and milestones from the spring repositories! -
25 Spring & Couchbase • Future Plans
Spring Data • In 1.0 First class async support • 1.1+ N1QL integraJon (dynamic querying) Ideas from SpringExchange • Spring XD Couchbase Sink and others • Spring Boot Autoconfig & Starters -
28 Couchbase Lite The only NaDve NoSQL
Database for Mobile -
30 Architecture SERVER 3
SERVER 1 SERVER 2 Couchbase Server Channel Sync Gateway Channel Channel Sync Gateway Channel Couchbase Lite for iOS and Android On Premise In the cloud -
31 N1QL The next generaDon
Query language -
32 N1QL • A SQL-‐like query language for
JSON documents. • Supports Expressions AggregaJons Nested document structures • Currently in DP2 • Go try it out! hAp://query.couchbase.com -
34 Thank you! [email protected] @daschl Download
Couchbase Server 2.2 htp://www.couchbase.com/download