[2017.05 Meetup #13] [TALK #1] André Cruz - Building Scalable Systems

When building scalable systems certain decisions have to be made beforehand. In this talk I'll discuss some of them that I had to make while working at SAPO, and also relate some ways to better deal with and prevent scalability failures.

André Cruz is a systems engineer with a passion for functional languages and information security.

1. Building scalable systems André Cruz
2. Agenda • Intro and motivation • Scalability decisions • Storing state • Concurrency model • Scalability failures • Q&A
3. Intro and motivation
4. Scalability • What does it mean?
5. –Wikipedia “Scalability is the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged in order to accommodate that growth.”
6. Throughput vs Latency • ++ Throughput • ~~Latency
7. Is scalability important? • Google: increasing page load time from 0.4 seconds to 0.9 seconds decreased traffic and ad revenues by 20%. • Amazon: every 0.1 second increase in load time decreases sales by 1%. "Website Optimization", Andrew B. King
8. Performance vs Scalability • Performance problem: the system is slow for a single user. • Scalability problem: the system is fast for a single user, but slow with lots of concurrent users.
9. Handling scalability • Doing nothing: degraded app • Throttling: deny the service to new users to better serve existing or important users • Optimize resource usage: better programming model • Adding resources: scale up or out • Delegating scalability: serverless • Caching: mitigation
10. –Leon Bambrick “There are only 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.”
11. Example applications • SAPO Blogs - 2005 - http://blogs.sapo.pt/ • SAPO ID/SSO - 2008 - https://id.sapo.pt/ • MEO Cloud - 2012 - https://meocloud.pt/
12. Scalability decisions 1 Storing state
13. Storing state • Stateless application servers • shared database • RDBMS vs NOSQL • ACID vs BASE
14. RDBMS Scale out - queries • Scaling out a RDBMS is hard • reads - difficult • writes - impossible
15. RDBMS Scale out - data • Scaling a RDBMS -> sharding • Partitioning • Replication • NOSQL
16. Which one? • Different data may have different needs • ACID -> RDBMS • Volume of data does not fit master -> NOSQL • Volume of write queries too big -> NOSQL
17. How high can you go? Azure Standard_GS5 CPU cores Memory : GiB Local SSD: GiB Max data disks Max cached and local disk throughput: IOPS / MBps (cache size in GiB) Max uncached disk throughput: IOPS / MBps 32 448 896 64 160,000 / 1,600 (4,224) 80,000 / 2,000
18. Example applications • SAPO Blogs - Sharded relational database • SAPO ID/SSO - Relational database • MEO Cloud - NoSQL store
19. Scalability decisions 2 Concurrency model
20. Processes 1 Request <-> 1 Process Blogs • Apache prefork mpm + mod_perl
21. Asynchronous the easy way • Async reverse proxy • Perlbal • nginx
22. Threads 1 Request <-> 1 Thread SAPO ID • Apache worker mpm + mod_python/mod_jk
23. Asynchronous 1 Request <-> 1 Coroutine MEO Cloud • uWSGI + gevent
24. Asynchronous model • event loop • callbacks • promises • coroutines
25. Threads function isUserTooYoung(id) { db = openDatabase() col = getCollection(db, 'users') user = find(col, {'id': id}) return user.age < cutoffAge }
26. Callbacks function isUserTooYoung(id, callback) { openDatabase(function(db) { getCollection(db, 'users', function(col) { find(col, {'id': id},function(result) { result.filter(function(user) { callback(user.age < cutoffAge) }) }) }) }) }
27. Promises function isUserTooYoung(id) { return openDatabase(db) .then(getCollection) .then(find.bind(null, {'id': id})) .then(function(user) { return user.age < cutoffAge; }); }
28. Coroutines async function isUserTooYoung(id) { const db = await openDatabase() const col = await getCollection(db, 'users') const user = await find(col, {'id': id}) return user.age < cutoffAge }
29. In practice • Async + Threads
30. Scalability failures Preventing/dealing with them
31. Monitoring/Alerting • Graphical • Historical • No real-time data Reactive
32. Dashboard • Near real-time data • Instrumentation Proactive
33. What does your failure look like?
34. OK Error pages
35. Error page • Fail fast
36. Disaster recovery • Document everything • Time it
37. Questions?
38. Thank you André Cruz <andre@brpx.com>