The fundamentals of scaling rails applications

8 hours workshop I taught at the Google Campus in Israel.

Transcript

1. FullStack Developers Israel Hosted by: !1 Fundamentals of scaling rails application ! With real world examples 15.12.2013! Google
2. WHO AM I?
3. Avi Tzurel, Ruby domain leader @KensoDev github - KensoDev
4. Tikal Expertise
5. text • Scaling a monolithic rails application • Caching • Key Value stores (Focusing on Redis) • Search indexing with NoSql enhancements (Focusing on SOLR) Agenda
6. text • Slides -> talk -> questions • Code together (moving at a normal pace) • Difficulties -> join someone else • More slides, more code, same process goes on How is this going to work?
7. text • Computer • Ruby • Vagrant • Virtual Box • Don’t have something? Please join someone else Prerequisites
8. text • Chef • Vagrant • Chef server • Redis • Memcached • SOLR (3.6) • Offloading from user thread • Micro services • Ruby & Rails (Obviously!) Tools & Concepts
9. Configuration management like a boss! Chef • Chef is a part of the “scaler” daily life • Easily (well, that’s a big fucking fat lie) manage and configure server • Chef-server with chef-client are a devops dream • Your code is your server documentation • Ruby!!!!!
10. up, hack, damn it!, destroy Vagrant • Cost effective way to “play” around with your environment configurations • Working with the same exact images as you would on your production env • Having a production env on your own machine is great for bug catching (princess env)
11. workshop-awesome Redis • Super fast key-value store on steroids • Has sets support, sorted sets, arrays and more • Not full ACID • If you don’t turn on journaling, redis may lose data on writes • Love to hate and lean the fsync() process
12. Date=12.12.2012 Memcached / Membase • Super fast memory (with disk backup if it’s membase) store • Storing values in binary • Simple yet effective API for what you need • Still running tons of caching solutions at all sizes of companies from your average 2-3 person startup to Facebook • Absolutely great for the average web app that has 80% reads with 20% writes
13. go look if i’m there Solr • Lucene based search index • Fully blown NoSql solution • The most mature solution out there • Can run on almost any Java server, Jetty, Tomcat (we’ll be running jetty) • The most misunderstood solution out there, people under-use it, we will learn how NOT to do it
14. text The basic rails application
16. text Real world scaled rails application
18. text • Genuine full stack development • Chef for configuration (chef server) • Caching layers (CDN, object cache, page cache) • Key Value storage • Messaging layers • Micro Services written in other languages • Multiple monitoring systems, multiple logs The modern scaled rails application
19. text Step back ! What is scaling?
20. “ “ Scaling - Replacing all components of a car while driving it at 100mph ! Mike Krieger - Instagram
21. Story Time Scaling a rails app, real life story
22. text Setting out to scale an app What do you want to achieve? • Highly available application • Great performance both for api users and app users • Easily manageable and configureable • Cost effective
23. text Mixing up concepts • Scaling is NOT optimization / hardware / storate • Not about less lines of code or specific piece of software • We are going to talk about DESIGNING for scale, not actually scaling • If you don’t design to scale, you won’t scale • If you design to scale, scaling will be painful • There is NO pain free scaling
24. Story Time Pains of scaling 5X in traffic ! App DESIGNED to scale
26. Questions?
27. text Lets get started
28. http://www.bit.ly/scaling-workshop-google
29. www.bit.ly/scaling-workshop-chef
30. text Having a look into our application
31. text Adding features
32. We are going to add & scale features EXACTLY like we would do in real like ! Well, not “we” but regular developers :)
33. text How do we measure? Primitively!
34. text Adding the like count
35. text Adding the review count
36. text Adding the most recent review snippet
37. Now, we need to scale this, it’s blowing up in production
38. Caching
39. Basics • Base class is ActiveSupport::Cache::Store • Has multiple adapters • Redis • Memcached • File (don’t) • Memory (don’t) Caching
40. Types • Full page caching • Action caching • Object caching • Partial caching Caching
41. API • Read • Write • Fetch (decorator for read/write) • Exist? ! NO Get all keys Search for keys Append, Prepend ! Nothing, learn to love it Caching
42. “ “ There are only two hard things in Computer Science: cache invalidation and naming things. ! ! -- Phil Karlton
43. I DO NOT recommend using anything other than partial caching in your rails application ! Caching solution explained
44. text Adding partial caching
45. text Explaining Cache Keys
46. text product:#{id}:review_snippet Model Name id field/property/item
47. text Invalidating the cache
48. after_commit on the model
49. Problems?
50. • Magic strings • Knowledge of keys are in multiple places • Typos break cache invalidation Problems
51. text Using the model for the key
52. text [product, ‘review_snippet’] Model Instance field / property
53. text Invalidating the cache
54. after_commit on the model
55. Exercise ! Cache (partial) all the products on the page, invalidate the cache when a new product is created or when an existing product is updated ! 30 minutes
56. checkout step2 from remote ! git fetch origin ! git checkout -t origin/step2
57. API • http://redis.io • get/set • sets • hashes • sorted sets • Best practice for keys “object_name:#{id}:#{key}” • product:2:likes • product:2:like_count • product:2:like Redis
58. text Adding counts to redis (using hashes)
59. text Adding search
60. text Searching using SOLR
61. text Why use SOLR only for search?
62. text SOLR ALL THE THINGZ!!!
63. THANK YOU Avi Tzurel Email: avit@tikalk.com