Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Resilient Software Design
Search
Swanand Pagnis
January 21, 2015
Technology
1
420
Resilient Software Design
Lessons learned from Michal Nygard's "Release It!"
Swanand Pagnis
January 21, 2015
Tweet
Share
More Decks by Swanand Pagnis
See All by Swanand Pagnis
Principles of Project Management Flow Part 1 and 2
swanandp
0
270
The Road to Continuous Deployments
swanandp
0
670
Fearless Joins
swanandp
0
340
The Deep Work Divide: Restoring the balance between collaboration and deep work
swanandp
1
970
The Well Rounded Engineer
swanandp
24
14k
The Deep Work Mentality
swanandp
7
2.1k
Combinator Parsing
swanandp
0
850
It's 2017, and I still want to sell you a graph database
swanandp
2
940
Architecture Over Framework: Rethink Your App Structure
swanandp
1
260
Other Decks in Technology
See All in Technology
AI時代に非連続な成長を実現するエンジニアリング戦略
sansantech
PRO
1
410
現場が抱える様々な問題は “組織設計上” の問題によって生じていることがある / Team-oriented Organization Design 20250827
mtx2s
6
50k
AIエージェントの開発に必須な「コンテキスト・エンジニアリング」とは何か──プロンプト・エンジニアリングとの違いを手がかりに考える
masayamoriofficial
0
450
Goでマークダウンの独自記法を実装する
lag129
0
230
実践アプリケーション設計 ②トランザクションスクリプトへの対応
recruitengineers
PRO
4
1k
Yahoo!広告ビジネス基盤におけるバックエンド開発
lycorptech_jp
PRO
1
300
Kubernetes における cgroup v2 でのOut-Of-Memory 問題の解決
pfn
PRO
0
210
Vault meets Kubernetes
mochizuki875
0
130
Lessons from CVE-2025-22869: Memory Debugging and OSS Vulnerability Reporting
vvatanabe
2
100
トヨタ生産方式(TPS)入門
recruitengineers
PRO
5
1.1k
【 LLMエンジニアがヒューマノイド開発に挑んでみた 】 - 第104回 Machine Learning 15minutes! Hybrid
soneo1127
0
170
VPC Latticeのサービスエンドポイント機能を使用した複数VPCアクセス
duelist2020jp
0
320
Featured
See All Featured
GraphQLとの向き合い方2022年版
quramy
49
14k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Music & Morning Musume
bryan
46
6.8k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
Typedesign – Prime Four
hannesfritz
42
2.8k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
131
19k
Building an army of robots
kneath
306
46k
Statistics for Hackers
jakevdp
799
220k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
21k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
Art, The Web, and Tiny UX
lynnandtonic
302
21k
Transcript
Resilient Software Design Building software that doesn’t give up!
1. What’s all this about? 2. Anti-patterns: What not to
do! 3. Patterns: Make your life easier.
Find the odd one 1. :cloud_factory 2. "CloudFactory" 3. CloudFactory
4. CLOUD_FACTORY
What will this code print? if fork puts "I won
the lottery!" else puts "I am bankrupt!" end
What’s all this about?
Stability
It Just Works™
It Should Work™
Bad things happen
Good things happen as well
But those are rare!
Anything that can go wrong, will go wrong. - Murphy’s
Law
Developers think positive
Too much so!
Be negative.
Our goal is to build software
Our goal is also to minimise pain
Our goal is also to save money
Resilient software saves money by not breaking when needed
Resilient software saves money by using optimum infrastructure
Resilient software saves money by keeping developers happy
Anti-patterns
1. Integration points
Integration is not what you think™
Database is an integration
Third party services are integration.
Your cache layer is an integration
None
Networks fail more often
Socket based protocols have a special way of failing
Refused connection is bad.
Hanged connection is worse.
Micro-services that talk to each other, will stop talking abruptly
2. Unbalanced Capacities
Specially applicable to micro-services
3. Slow responses
4. Unbounded Result Sets
Major anti-pattern, overlooked by many
What is the size of an HTTP cookie?
Patterns
1. Use timeouts!
What is the default timeout on Ruby’s net/http?
Now and forever, networks will always be unreliable. - Michael
T Nygard
Every network call in your system must have a timeout
This includes database calls
This includes API calls
This includes cache lookups
What to do when the timeout occurs depends on where
it occurred
Highly context specific, so the dev team should make that
decision
ProTip: Do not use Ruby’s “timeout” module
Instead, depend on libraries for the timeout
If you’re a library author, just use net/http’s timeout
2. Circuit Breaker
None
3. Bulkheads
A ship is divided into several water- tight compartments
In case there is a leakage in one section, water
doesn’t flood into other sections
Same principle!
Use resource pools
Use rate limiting
Consideration: Capacity
Bulkheading often conflicts increasing or variable capacity
Consideration: Performance
Bulkheading often results in slightly reduced performance
It’s worth it, trust me™
4. Fail fast
Remember guard clauses in Ruby?
# class Event def closest_event return unless self.location # …
# … end
Same principle!
Any Ruby libraries?
Not a lot :(
shopify/semian
Thank you!
Questions?
Swanand Pagnis Principal Engineer @ First
Swanand Pagnis @_swanand on Twitter
Swanand Pagnis @swanandp on GitHub