Share
Embed
Scaling PHP Applications with Couchbase
This talk was given at the PHP User Group Vienna at 12th December. After a short introduction of Couchbase we jumped right into demos.
Michael Nitschinger
December 12, 2013
Transcript
29 slides
-
Who‘s this guy? • SDK Engineer at Couchbase
Maintaining • Official Java SDK • Spymemcached • Spring-‐Data-‐Couchbase • Former Lithium Core Contributor • @daschl on TwiGer -
Instagram gained nearly 1 million users overnight when then expanded
to Android Growth is the New Reality -
Draw Something Viral Growth 19 17 15
13 11 9 7 5 3 3/1 28 26 24 22 20 18 16 14 12 10 8 2/6 Draw Something by OMGPOP Daily Ac)ve Users (millions) 21 2 4 6 8 10 12 14 16 -
How do you take this growth? RDBMS is good
for many thing, but hard to scale RDBMS Scales Up Get a bigger, more complex server Users ApplicaUon Scales Out Just add more commodity web servers Users System Cost ApplicaUon Performance RelaUonal Database Web/App Server Tier System Cost ApplicaUon Performance Won’t scale beyond this point -
Scaling out RDBMS • Run Many SQL Servers • Data
could be shared -‐ Done by the applicaUon code • Caching for faster response Ume Web/App Server Tier Memcached Tier MySQL Tier -
Scaling out fla]ens the cost and performance curves NoSQL
Database Scales Out Cost and performance mirrors app Uer Users NoSQL Distributed Data Store Web/App Server Tier ApplicaUon Scales Out Just add more commodity web servers Users System Cost ApplicaUon Performance ApplicaUon Performance System Cost NoSQL Technology Scales Out -
Couchbase Open Source Project • Leading NoSQL database project
focused on distributed database technology and surrounding ecosystem • Supports both key-‐value and document-‐oriented use cases • All components are available under the Apache 2.0 Public License • Obtained as packaged so`ware in both enterprise and community ediUons. -
Easy Scalability Consistent High Performance Always
On 24x365 Grow cluster without applicaUon changes, without downUme with a single click Consistent sub-‐millisecond read and write response Umes with consistent high throughput No downUme for so`ware upgrades, hardware maintenance, etc. Flexible Data Model JSON document model with no fixed schema. Couchbase Server Core Principles -
New in 2.0+ JSON support Indexing and Querying
Cross data center replica-on Incremental Map Reduce JSON JSON JSON JSON JSON -
Couchbase Server Architecture Heartbeat Process monitor Global
singleton supervisor ConfiguraUon manager on each node Rebalance orchestrator Node health monitor one per cluster vBucket state and replicaUon manager hGp REST management API/Web UI HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 -‐ 21199 Erlang/OTP storage interface Couchbase EP Engine 11210 Memcapable 2.0 Moxi 11211 Memcapable 1.0 Memcached New Persistence Layer 8092 Query API Query Engine Data Manager Cluster Manager -
Couchbase Server Architecture ReplicaUon, Rebalance, Shard State
Manager REST management API/Web UI 8091 Admin Console Erlang /OTP 11210 / 11211 Data access ports Object-‐managed Cache Mul--‐threaded Persistence Engine 8092 Query API Query Engine hGp Data Manager Cluster Manager -
3 3 2 Single node -‐ Couchbase
Write Opera-on Managed Cache Disk Queue Disk ReplicaUon Queue App Server Couchbase Server Node Doc 1 Doc 1 Doc 1 To other node -
COUCHBASE SERVER CLUSTER Cluster-‐wide Basic Opera-on •
Docs distributed evenly across servers • Each server stores both ac-ve and replica docs Only one server acUve at a Ume • Client library provides app with simple interface to database • Cluster map provides map to which server doc is on App never needs to know • App reads, writes, updates docs • Mul-ple app servers can access same document at same -me User Configured Replica Count = 1 READ/WRITE/UPDATE ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 ACTIVE Doc 4 Doc 7 Doc Doc Doc SERVER 2 Doc 8 ACTIVE Doc 1 Doc 2 Doc Doc Doc REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc REPLICA Doc 6 Doc 3 Doc 2 Doc Doc Doc REPLICA Doc 7 Doc 9 Doc 5 Doc Doc Doc SERVER 3 Doc 6 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 -
Add Nodes to Cluster • Two servers added
One-‐click opera-on • Docs automa-cally rebalanced across cluster Even distribuUon of docs Minimum doc movement • Cluster map updated • App database calls now distributed over larger number of servers REPLICA ACTIVE Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc Doc 8 Doc Doc 9 Doc Doc 2 Doc Doc 8 Doc Doc 5 Doc Doc 6 READ/WRITE/UPDATE READ/WRITE/UPDATE APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 -
Fail Over Node REPLICA ACTIVE
Doc 5 Doc 2 Doc Doc Doc 4 Doc 1 Doc Doc SERVER 1 REPLICA ACTIVE Doc 4 Doc 7 Doc Doc Doc 6 Doc 3 Doc Doc SERVER 2 REPLICA ACTIVE Doc 1 Doc 2 Doc Doc Doc 7 Doc 9 Doc Doc SERVER 3 SERVER 4 SERVER 5 REPLICA ACTIVE REPLICA ACTIVE Doc 9 Doc 8 Doc Doc 6 Doc Doc Doc 5 Doc Doc 2 Doc 8 Doc Doc • App servers accessing docs • Requests to Server 3 fail • Cluster detects server failed Promotes replicas of docs to acUve Updates cluster map • Requests for docs now go to appropriate server • Typically rebalance would follow Doc Doc 1 Doc 3 APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 User Configured Replica Count = 1 COUCHBASE SERVER CLUSTER -
COUCHBASE SERVER CLUSTER Indexing and Querying
User Configured Replica Count = 1 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 1 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc APP SERVER 1 COUCHBASE Client Library CLUSTER MAP COUCHBASE Client Library CLUSTER MAP APP SERVER 2 Doc 9 • Indexing work is distributed amongst nodes • Large data set possible • Parallelize the effort • Each node has index for data stored on it • Queries combine the results from required nodes ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 2 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 ACTIVE Doc 5 Doc 2 Doc Doc Doc SERVER 3 REPLICA Doc 4 Doc 1 Doc 8 Doc Doc Doc Doc 9 Query -
SERVER 3 SERVER 1
SERVER 2 Couchbase Server – San Francisco SERVER 3 SERVER 1 SERVER 2 Couchbase Server – New York Op-mis-c replica-on Per replica-on Tunable Parameters Op-mized protocol based on memcached Reliability and performance at scale Cross Data Center Replica-on (XDCR) -
Official PHP SDK • Current Release: 1.2.1 (October 2013)
• Based on top of libcouchbase • Does the heavy liping for you Knows the cluster topology „Routes“ operaUons to target servers Abstracts protocol semanUcs (memcache, h]p,...) -
Integra-on • Doctrine Cache Provider Since 2.4
• Fully featured ODM in the making • For now, try Basement! h]ps://github.com/Basement/Basement