Using Ruby & Rails in the wild - Gogobot use case

Talk I gave at an Israeli meetup about Ruby and Rails vs other languages/frameworks.

1. ݆ૉ Ruby & Rails In the wild Gogobot use-case Avi Tzurel Gogobot Monday, September 10, 12
2. Who Am I? Avi Tzurel http://avi.io http://twitter.com/kensodev http://kensodev.com http://github.com/kensodev Monday, September 10, 12
3. What is Gogobot? Monday, September 10, 12
4. Social Recommendation Engine For Travel Monday, September 10, 12
5. Personalized Recommendations From Friends, not strangers Monday, September 10, 12
6. The Gogobot Architecture Monday, September 10, 12
7. How we make shit work? Monday, September 10, 12
8. CDN + Reverse Proxy Monday, September 10, 12
9. CDN • Edge servers across the world (hundreds) • Caches static assets • CSS, JS, Images • Caches full pages (with smart expire API) Monday, September 10, 12
10. Reverse Proxy • Super fast connection from edges means users get a local experience, with minimal latency • Cache miss? get from load balancer • Logged out traffic almost never hits or affects logged in traffic • scale differently, control differently Monday, September 10, 12
11. CDN + Reverse Proxy Monday, September 10, 12
12. CDN + Reverse Proxy Front End Back End Monday, September 10, 12
13. Front End • Serves user facing content • Communicates with cache layer and the DB • No heavy lifting, respond to the user as fast as possible Monday, September 10, 12
14. Back End • Serves realtime services (Facebook, Twitter) • Heavy lifting thrown at it from the Front End Machines • Hosts workers for the queue service Monday, September 10, 12
15. CDN + Reverse Proxy Front End Back End Monday, September 10, 12
16. Front End Back End Cache Monday, September 10, 12
17. Cache • Memcached cluster • Memcached 1.4+ • 6+ Machines • 4.5 - 15K operations per second • Hosted on Amazon ElastiCache • Solved tons of problems with memcached dying Monday, September 10, 12
18. Front End Back End Cache Monday, September 10, 12
19. Cache MySql MongoDB Redis SOLR Monday, September 10, 12
20. MySql • Master + 2 Slaves • 64G memory for each machine with 400G storage • Hourly backups • Used as the main persistence layer for the site • Reads are from slaves, writes are from master • Logged out traffic will never have access the master Monday, September 10, 12
21. MySql • EBS snapshots are used as backups • Multi region support for Amazon (1a, 1b, 1c) Monday, September 10, 12
22. MongoDB • 6 Shard • 3 Replica sets in each shards • 16HD (100G) raid on each machine • 64G memory for each machine to provide out of memory index for all queries • Used for the Graph Engine + Scoring system Monday, September 10, 12
23. Monday, September 10, 12
24. Redis • Used for key+value store • Cache tagging solution on top of memcached • Queue services is hosted on Redis • Master / Slave replica • different redis cluster for different things Monday, September 10, 12
25. Redis • Different Redis clusters for each need • Indexer • Cache tagging • Realtime push with Node.js to the client • When one down, others behave normally Monday, September 10, 12
26. SOLR • Search index • NoSQL (Schema Less) • Master/Slave • Slave on each app machine, single master • Eventual consistency Monday, September 10, 12
27. Numbers • 400m+ graph users • 10K triggers per user in the scoring system • Grew ~170X last 18 months • Announced 1m registered users a month ago • hit 2m registered users 2 weeks agter Monday, September 10, 12
28. Numbers • HUGE growth in a relatively short time Monday, September 10, 12
29. 99.9% uptime Monday, September 10, 12
30. ~200ms avg server response time Monday, September 10, 12
31. Logged out user gets a page in 30ms Monday, September 10, 12
32. How can you manage all of this? Monday, September 10, 12
33. Not too technical Monday, September 10, 12
34. Full Ruby & Rails shop Monday, September 10, 12
35. Ruby will increase developer happiness (That’s what it was designed for) Monday, September 10, 12
36. Community Drives you towards Best Practices Monday, September 10, 12
37. TDD Monday, September 10, 12
38. “Sometimes I want to do Ruby because it means I’m more likely to be able to TDD.” Greg Malcolm Monday, September 10, 12
39. Continuos Integration Monday, September 10, 12
40. Continuous Deployment Monday, September 10, 12
41. Monday, September 10, 12
42. Machine Configuration using Chef Monday, September 10, 12
43. SOA Application with Rails Engines Monday, September 10, 12
44. EVERYTHING IS OPEN SOURCE! Monday, September 10, 12
45. Laptop Monday, September 10, 12
46. Perform Later Monday, September 10, 12
47. Rails fixes & features Monday, September 10, 12
48. Resque Monday, September 10, 12
49. Methodologies and sharing ideas Monday, September 10, 12
50. CI Configuration Monday, September 10, 12
51. Full pages crawler Monday, September 10, 12
52. and more... http://github.com/gogobot http://github.com/kensodev Monday, September 10, 12
53. Super small team Monday, September 10, 12
54. Gets a lot done Monday, September 10, 12
55. No QA! Monday, September 10, 12
56. Rails is awesome Monday, September 10, 12
57. “Ruby on Rails is astounding. Using it is like watching a kung-fu movie, where a dozen bad-ass frameworks prepare to beat up the little newcomer only to be handed their asses in a variety of imaginative ways.” Nathan Tarkington, O'Reilly Publishing Monday, September 10, 12
58. No other language / framework is able to do such things Monday, September 10, 12
59. Best community in the world Monday, September 10, 12
60. Tons of open source projects Monday, September 10, 12
61. Github Monday, September 10, 12
62. Heroku EngineYard Amazon Monday, September 10, 12
63. Thank you! Monday, September 10, 12
64. Questions? Monday, September 10, 12