Advancing Riak CS

Ricon East 2013:

Riak CS has come a long way since it was first released in 2012, and then open sourced in March 2013. We'll take a look at some of the features and improvements in the recently released Riak CS 1.3.0, and planned for the future, like better integration with CloudStack and OpenStack. Next, we'll go over some of the Riak CS guts that deployers should understand in order to successfully deploy, monitor and scale Riak CS.

Transcript

1. Advancing Riak CS Reid Draper Thursday, May 16, 13
2. Riak CS dev github.com/reiddraper @reiddraper Thursday, May 16, 13
3. working on Riak CS since the beginning Thursday, May 16, 13
4. •so far •what’s coming •service integration •performance tips Thursday, May 16, 13
5. 1.0.0 1.0.1 1.0.2 1.1.0 1.2.0 1.2.1 1.2.2 1.3.0 1.3.1 March 2012 April 2013 Thursday, May 16, 13
6. Public Release! Garbage Collection DTrace Multi-DC Replication Open Source! Multi-part Upload March 2012 April 2013 Thursday, May 16, 13
7. Since Open Sourcing Thursday, May 16, 13
8. git shortlog --summary | wc -l 29 Thursday, May 16, 13
9. easier to deploy Thursday, May 16, 13
10. How We Work Thursday, May 16, 13
11. when a feature is hard to add, we know it’s time for some refactoring Thursday, May 16, 13
12. Thursday, May 16, 13
13. gitflow (+ some variations) Thursday, May 16, 13
14. Future Work Thursday, May 16, 13
15. Keylisting (1.4) Thursday, May 16, 13
16. 3 iterations Thursday, May 16, 13
17. Riak and s3 key listing is different Thursday, May 16, 13
18. Riak CS needs to be like s3 Thursday, May 16, 13
19. aardvark aardwolf Aaron Aaronic Aaronical Aaronite Aaronitic Aaru Ab aba Ababdeh Ababua abac abaca abacate abacay Riak aardvark 102 application/json aardwolf 1024 application/json Aaron 1024 application/json Aaronic 67 application/json Aaronical 462 text/plain Aaronite 105235 image/png Aaronitic 462 text/plain Aaru 1024 application/json Ab 102 application/json aba 67 application/json Ababdeh 67 application/json Ababua 462 text/plain S3 Thursday, May 16, 13
20. Fault-tolerance (1.4) Thursday, May 16, 13
21. Thursday, May 16, 13
22. Admin improvements Thursday, May 16, 13
23. Object Copy Thursday, May 16, 13
24. refcounting vs deep copy Thursday, May 16, 13
25. complexity vs. storage Thursday, May 16, 13
26. Object versioning Thursday, May 16, 13
27. immutable blocks and mvcc makes this much simpler Thursday, May 16, 13
28. BNW integration Thursday, May 16, 13
29. replication policies Thursday, May 16, 13
30. fine-grained control over replication for legal reasons Thursday, May 16, 13
31. a new backend? Thursday, May 16, 13
32. what are the backend needs? Thursday, May 16, 13
33. manifests and blocks Thursday, May 16, 13
34. Manifests sorted <10Kb Thursday, May 16, 13
35. Blocks locality only amongst same blocks in a file > 1MB Thursday, May 16, 13
36. Cloud Integration Thursday, May 16, 13
37. Open Stack Thursday, May 16, 13
38. Thursday, May 16, 13
39. Cloud Stack Thursday, May 16, 13
40. Eucalyptus Thursday, May 16, 13
41. Gotchas and Tips Thursday, May 16, 13
42. leveldb on ssd Thursday, May 16, 13
43. Blocks Manifests Thursday, May 16, 13
44. Bitcask Leveldb Thursday, May 16, 13
45. 2GB Manifests / 1TB Blocks Thursday, May 16, 13
46. 2TB Manifests / 1PB Blocks Thursday, May 16, 13
47. different access pattern Thursday, May 16, 13
48. archive and access tuning Thursday, May 16, 13
49. access_log_flush_factor access_log_flush_size access_log_flush_size access_archive_period access_archiver_max_backlog storage_schedule storage_archive_period usage_request_limit Thursday, May 16, 13
50. bucket sizing Thursday, May 16, 13
51. Garbage Collection Thursday, May 16, 13
52. leeway_seconds gc_interval gc_retry_interval Thursday, May 16, 13
53. 10GbE Thursday, May 16, 13
54. Thursday, May 16, 13
55. zdbbl Thursday, May 16, 13
56. +zdbbl 131072 (128MB) goes in Riak vm.args Thursday, May 16, 13
57. OS config Thursday, May 16, 13
58. swappiness 0 noop or deadline Thursday, May 16, 13
59. dirty_background_ratio to dirty_background_bytes Thursday, May 16, 13
60. Thanks Kelly McLaughlin Scott Lystig Fritchie Andrew J. Stone + a bunch more Thursday, May 16, 13
61. Questions? @reiddraper Thursday, May 16, 13