Share
Embed
Couchbase in the Enterprise with Spring Data and Hibernate OGM
I gave this talk at ConFESS 2013 (https://2013.con-fess.com/). It contains both the introduction to Couchbase Server 2.0, how to work with the Java SDK and how the current state of Spring Data & Hibernate OGM integration is.
Michael Nitschinger
April 05, 2013
Transcript
55 slides
-
Couchbase in the Enterprise with Spring Data and Hibernate OGM
Michael Nitschinger Developer Advocate, Couchbase Inc. -
• Developer Advocate at Couchbase, Inc. • Maintainer of
the Couchbase Java SDK • Speaking at Conferences and Meetups • Living and Working here in Vienna, Austria {“about”: “me”} -
1. IntroducCon to Couchbase Server 2.0 2. OperaCons &
Document Design 3. Developing in Java with Couchbase 4. Spring Data & Hibernate OGM IntegraCon Agenda -
Workshop Reminder • Tomorrow a<er Lunch -‐ Room 4
(13:00 – 17:00): Discover NoSQL Development with Couchbase Server 2.0 • Hands-‐On & In-‐Depth • More on 2.0 Views! -
Couchbase Open Source Project • One of the leading
NoSQL database projects focused on distributed database technology and surrounding ecosystem • Supports both key-‐value and document-‐oriented use cases • All components are available under the Apache 2.0 Public License • Obtained as packaged soSware in both enterprise and community ediCons. -
Easy Scalability Consistent High Performance Always
On 24x365 Grow cluster without applicaCon changes, without downCme with a single click Consistent sub-‐millisecond read and write response Cmes with consistent high throughput No downCme for soSware upgrades, hardware maintenance, etc. JSON JSON JSON JSON JSON PERFORMANCE Flexible Data Model JSON document model with no fixed schema. Core Principles -
New in 2.0 JSON support Indexing and Querying
Cross data center replicacon Incremental Map Reduce JSON JSON JSON JSON JSON -
Couchbase Server 2.0 Architecture Heartbeat Process monitor
Global singleton supervisor ConfiguraCon manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replicaCon manager hdp REST management API/Web UI HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 -‐ 21199 Erlang/OTP storage interface Couchbase EP Engine 11210 Memcapable 2.0 Moxi 11211 Memcapable 1.0 Memcached New Persistence Layer 8092 Query API Query Engine Data Manager Cluster Manager -
3 3 2 Single node -‐ Couchbase
Write Operacon Managed Cache Disk Queue Disk ReplicaCon Queue App Server Couchbase Server Node Doc 1 Doc 1 Doc 1 To other node -
GET Doc 1 3 3 2
Single node -‐ Couchbase Read Operacon Disk Queue ReplicaCon Queue App Server Doc 1 Doc 1 Doc 1 Managed Cache Disk To other node Couchbase Server Node -
COUCHBASE SERVER CLUSTER Basic Operacon • Docs
distributed evenly across servers • Each server stores both accve and replica docs Only one server acCve at a Cme • Client library provides app with simple interface to database • Cluster map provides map to which server doc is on App never needs to know • App reads, writes, updates docs • Mulcple app servers can access same document at same cme User Configured Replica Count = 1 READ/WRITE/UPDATE ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 ACTIVE Doc 4 Doc 7 Doc Doc Doc SERVER 2 Doc 8 ACTIVE Doc 1 Doc 2 Doc Doc Doc REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc REPLICA Doc 6 Doc 3 Doc 2 Doc Doc Doc REPLICA Doc 7 Doc 9 Doc 5 Doc Doc Doc SERVER 3 Doc 6 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 -
Add Nodes to Cluster • Two servers added
One-‐click operacon • Docs automaccally rebalanced across cluster Even distribuCon of docs Minimum doc movement • Cluster map updated • App database calls now distributed over larger number of servers REPLICA ACTIVE Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc 8 Doc Doc 9 Doc Doc 2 Doc Doc 8 Doc Doc 5 Doc Doc 6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 -
Fail Over Node REPLICA ACTIVE
Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc 9 Doc 8 Doc Doc 6 Doc Doc Doc 5 Doc Doc 2 Doc 8 Doc Doc • App servers accessing docs • Requests to Server 3 fail • Cluster detects server failed Promotes replicas of docs to acCve Updates cluster map • Requests for docs now go to appropriate server • Typically rebalance would follow Doc Doc 1 Doc 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 User Configured Replica Count = 1 COUCHBASE SERVER CLUSTER -
Fundamentals • Every Document has a Key assigned to
it • Keys must be max. 255 UTF-‐8 chars long must be unique in a bucket (“database”) are completely under the control of the applicaCon • Values can be any binary blob (but bonus points when JSON!) can be up to 20MB in size -
Basic Store & Retrieve Operacons • get(key) Retrieve
a document • set(key, value) Store a document or replace if it exists • add(key, value) Store a document and fail if it exists • replace(key, value) Replace a document and fail if it does not exist -
Lots of other Operacons • View operacons •
Atomic Counters (increment, decrement) • Append/Prepend • CAS (Compare and Set) OpCmisCc Locking • “Get with Lock” Write Lock on Objects • Bulk Operacons Saves network overhead • Stats -
Couchbase 2.0 Bonus Points: JSON • JSON is a
lightweight format to represent document structure in a language-‐independent manner. • If JSON documents are stored, the View engine can be used. • Allows to build secondary indexes on your datasets. • Makes it possible to ask quesCons like “Give me all user documents by lastname”. -
The BIG mental adjustment • In SQL, we tend
to avoid hilng the database as much as possible. Use JOINs and let the DB opCmizer figure out what to do • In Couchbase, get’s and set‘s are so fast they are trivial, not boglenecks, this is hard for many people to accept at first; Mulcple get statements are commonplace, don’t avoid it! -
meta { “id”: “u::[email protected]”, “rev”: “1-‐0002bce0000000000”,
“flags”: 0, “expiraCon”: 0, “type”: “json” } document { “uid”: 123456, “firstname”: “Michael”, “lastname”: “Nitschinger”, “age”: 25, “favorite_colors”: [“blue”, “black”], “email”: “michael.nitschinger@...” } Meta InformaCon Including Key All Keys Unique and Kept in RAM Document Value Most Recent In Ram And Persisted To Disk Document Structure with JSON -
Views & Indexes • Views can cover a few different
use cases Simple secondary indexes (the most common) Complex secondary, terCary and composite indexes AggregaCon funcCons (reducCon) • Example: count the number of North American Ales Organizing related data • Built using Map/Reduce Map funcCon creates the index (b-‐tree) Reduce funcCon summarizes (reduces) informaCon Wrigen using superfast Javascript (V8) -
www.couchbase.com/develop Python Ruby Go Clojure
Official SDKs Community & Libraries -
• Current Version: 1.1.5 (supports 1.8 and 2.0) •
Extends the Spymemcached Library • Asynchronous where possible • Dependencies: Negy, Apache HTTPCore (+NIO), Jetson • Either Download (hdp://www.couchbase.com/develop/java/current) • Or include through Maven (hdp://files.couchbase.com/maven2/) About the SDK (1) -
• Hides the complexity of different protocols used
(memcached, hgp) cluster configuraCon changes (add/remove) failover scenarios • Provides a unified API for key-‐ and view-‐based access APIs for bucket and design document management extended debugging and logging faciliCes About the SDK (2) -
• Either directly through CouchbaseClient or customize opCons by
using the CouchbaseConnecconFactoryBuilder. You need at least: List of Nodes, Bucket Name and Password Simple Instancacon -
View Access • Steps to Follow: 1. Fetch
the View 2. Init Query object 3. Query the View -
Spring Data • Umbrella project to provide
unified access pagerns on a variety of databases/frameworks without loosing flexibility and beger integraCon into the Spring ecosystem • Support for JPA Hadoop MongoDB Neo4J .... -
Spring Data Couchbase • Integracon with Spring Data
Templates Repositories ExcepCon Mapping also @Cacheable • Maps POJO Encces to JSON (and back) • Scll under development! -
Transparent Caching • Very easy and transparent Caching mechanism
• Use @Cacheable, @CacheEvict or @CachePut • Support for SpEL in those annotaCons • Translates to get/set/delete calls behind the scenes • Get distributed, high performance caching in a mager of minutes! -
Hibernate OGM • Hibernate „Object/Grid“ Mapper • JPA
for NoSQL • Hibernate Sub Project • Mulcple Persistence Store Infinispan EHCache MongoDB Couchbase Neo4J -
Hibernate OGM + Couchbase • ParCal Query Support
• RelaConships • SCll under heavy development • Work with your EnCCes (POJOs) in a tradiConal and natural way • Integrated ConfiguraCon for your Persistence.xml -
What got stored in Couchbase? Key: {tbl:Dog,id:1} ! {!
"id": 1,! "name": "Dina",! "breed_id": "fc5ba547-b86a-47a4-ad84-9aed4b8c7024"! }! Key: {tbl:Breed,id:fc5ba547-b86a-47a4- ad84-9aed4b8c7024} ! {! "id": "fc5ba547-b86a-47a4-ad84-9aed4b8c7024",! "name": "Collie"! }! -
Shameless Plug • Couchbase DeveloperDays Munich, April
23th Vienna, April 24th (use „MICHAEL“ as the discount code!) Prague, April 26th • Couchbase [UK]: April 11th -
Thank you! [email protected] @daschl Get Couchbase
Server at hgp://www.couchbase.com/download