Introducing Riak

Transcript

1. Introducing Riak Von Stark Friday, March 22, 13
2. Me Co-Founder @ NoSQL Taiwan I use : ruby, riak Friday, March 22, 13
3. BASHO RIAK Friday, March 22, 13
4. About Basho Founded Jan 2008 Eric Brewer (father of CAP theorem) is the director of board They are agile Friday, March 22, 13
5. Concept Distributed Scalable Influenced Amazon Dynamo Paper Friday, March 22, 13
6. Design Masterless, Scale predictably , easily , simplifies Flexible and powerful Fault Tolerant High Available Partitions More nodes, more fast ( powerful ) Friday, March 22, 13
7. Use it as & when... Hybrid Database Main Database Cache Cross Machine Session Keeper Can’t lose any data Friday, March 22, 13
8. WE LOVE ERLANG Friday, March 22, 13
9. Erlang Erlang is a programming language used to build massively scalable soft real-time systems with requirements on high availability. Some of its uses are in telecoms, banking, e-commerce, computer telephony and instant messaging. Erlang's runtime system has built-in support for concurrency, distribution and fault tolerance. OTP is set of Erlang libraries and design principles providing middle- ware to develop these systems. It includes its own distributed database, applications to interface towards other languages, debugging and release handling tools. Friday, March 22, 13
10. Because Erlang, so.... Erlang/OTP provides an ideal platform for developing systems like Riak because it provides inter-node communication, message queues, failure detectors, and client-server abstractions out of the box. What's more, most frequently-used patterns in Erlang have been implemented in library modules, commonly referred to as OTP behaviors. They contain the generic code framework for concurrency and error handling, simplifying concurrent programming and protecting the developer from many common pitfalls. Behaviors are monitored by supervisors, themselves a behavior, and grouped together in supervision trees. A supervision tree is packaged in an application, creating a building block of an Erlang program. A complete Erlang system such as Riak is a set of loosely coupled applications that interact with each other. Some of these applications have been written by the developer, some are part of the standard Erlang/OTP distribution, and some may be other open source components. They are sequentially loaded and started by a boot script generated from a list of applications and versions. HTTP://WWW.AOSABOOK.ORG/EN/RIAK.HTML Friday, March 22, 13
11. BEAUTIFUL RING Friday, March 22, 13
12. Magic Ring Friday, March 22, 13
13. Clustering 160-bit integer space 32 partitions Each Node is : equal size, partition, read/write able Dynamic add / remove Node without any reboot, config Consistent Hash Friday, March 22, 13
14. Beautiful Ring Friday, March 22, 13
15. Replication Hinted Handoff Eventually Consistent Failure Tolerance Conflict Resolution Friday, March 22, 13
16. Storage Bitcask Level DB Memory Multiple-Storage or.. write your own with Backend-API Friday, March 22, 13
17. RIAK CHOOSES TO EMPHASIZE A & P Friday, March 22, 13
18. BASIC KNOWLEDGE OF RIAK Friday, March 22, 13
19. Details Key-Value OpenSource Meta Data Vector Clock Bucket Full Text Search, Secondary Indexes, Link Walk , Map/Reduce Restful / Protocol Buffer Interface Friday, March 22, 13
20. Bucket It’s a collection of keys It’s not a table Please do not search entire me (bucket) Friday, March 22, 13
21. SEARCHING WITH RIAK Friday, March 22, 13
22. Riak Basic Query Query : Get /riak/bucket/key Store : Put /riak/bucket/key -Content-Type -X-Riak-Vclock Delete : Delete /riak/bucket/key Example key : member1 Friday, March 22, 13
23. Riak Secondary Index tag an object with 1 or more values type : Integer, String Search can be exact match or range results can be the input of MapReduce Friday, March 22, 13
24. Riak Full Text Search Built on RiakCore Advanced search tool, Concise , Easy to use Various mine types : JSON, XML, plain text, Erlang, Erlang binaries Various analyzers : white space, integer, no-op... Queries : Wildcards, In/Exclude and/or/not range, Grouping, Prefix match, Proximity search, Term boosting Scoring or Ranking for results Can be the the input of Map/Reduce Friday, March 22, 13
25. Riak Solr Yokozuna Will probably replace Riak Search ( soon? ) Tight integration with Solr (Active) Anti-entropy Yokozuna has what Solr has Friday, March 22, 13
26. Overview Riak 2i : limited, and has resource issues Riak Search : bad performance & resource usage for certain queries, lack of search language Riak Map/Reduce : too general and resource hungry Friday, March 22, 13
27. Riak Map/Reduce Parallel, Distributed Queries Operates in isolation Key - Filters Friday, March 22, 13
28. Mapper Input the key-data ( regular expression available) Return a list of values Parallel results are aggregated into a single list What you can do with it ? Count words, Extract data Friday, March 22, 13
29. Reducer Input the list of results Merge results Two processes per reducer What you can do with it ? Sort, Aggregate Friday, March 22, 13
30. How Riak Map/Reduce works? Friday, March 22, 13
31. Hadoop (similarities) Provide data locality Phases runs beside data Distributed across multiple machines Friday, March 22, 13
32. Hadoop (differences) Used for large , long-run jobs Target is different 3 phases ( map, combine, reduce ) Higher level language Friday, March 22, 13
33. Riak with nosqls Riak-Hadoop Riak-Redis Riak-Mongo Riak-TokyoCabinet AND MORE.... Friday, March 22, 13
34. DEVELOPER FRIENDLY Friday, March 22, 13
35. Riak-Control (Rekon) Restart, Start, Shutdown Nodes Snapshot Ring Status Lookup Objects Run Map/Reduce Live Graph Console HTTPS://GITHUB.COM/BASHO/RIAK_CONTROL Friday, March 22, 13
36. Some Screenshots Friday, March 22, 13
37. SCALE IN SECONDS Friday, March 22, 13
38. Add Node riak-admin cluster join dev1@127.0.0.1 ======================= Membership ======================== Status Ring Pending Node ----------------------------------------------------------- valid 25.0% -- 'dev1@127.0.0.1' valid 25.0% -- 'dev2@127.0.0.1' valid 25.0% -- 'dev3@127.0.0.1' valid 25.0% -- 'dev4@127.0.0.1' ----------------------------------------------------------- Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 Friday, March 22, 13
39. Remove Node riak-admin cluster leave dev1@127.0.0.1 ======================= Membership ======================== Status Ring Pending Node ----------------------------------------------------------- valid 33.3% -- 'dev2@127.0.0.1' valid 33.3% -- 'dev3@127.0.0.1' valid 33.3% -- 'dev4@127.0.0.1' ----------------------------------------------------------- Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 Friday, March 22, 13
40. ACTIVE COMMUNITY Friday, March 22, 13
41. Riak to csv export otto - s3 clone. Build with Cyclone & riak ldapjs - riak backend for ldapjs ebot - web crawler use Riak as backend riakbloom - use bloom filter with riak mr riaktivity - a ruby interface to store timelime in riak riakfuse - distributed filesystem use riak AND MORE.... Friday, March 22, 13
42. WHAT’S NEW WITH 1.3 ? Friday, March 22, 13
43. Active Anti-Entropy Health checker Improved data center replication performance Improved Riak Control User Experience Support IPV6 now Improved MapReduce back-pressure Can optionally send log message to syslog Friday, March 22, 13
44. QUESTIONS ? NEED CONSULT? FIND ME @ FACEBOOK : facebook.com/vonstark TWITTER : vonstark32 Friday, March 22, 13