Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Servers are doomed to fail
Search
JBD
May 17, 2019
Technology
3
1.5k
Servers are doomed to fail
JBD
May 17, 2019
Tweet
Share
More Decks by JBD
See All by JBD
eBPF in Microservices Observability at eBPF Day
rakyll
1
2.1k
eBPF in Microservices Observability
rakyll
1
1.7k
OpenTelemetry at AWS
rakyll
1
1.8k
Debugging Code Generation in Go
rakyll
5
1.6k
Are you ready for production?
rakyll
8
2.8k
Serverless Containers
rakyll
1
250
Critical Path Analysis
rakyll
0
600
Monitoring and Debugging Containers
rakyll
2
1.1k
CPDD
rakyll
0
4.2k
Other Decks in Technology
See All in Technology
データ戦略部門 紹介資料
sansan33
PRO
1
3.1k
いまさら聞けない Git 超入門 〜Gitって結局なに?から始める第一歩〜
devops_vtj
0
160
会社紹介資料 / Sansan Company Profile
sansan33
PRO
6
360k
ゴリラ.vim #36 ~ Vim x SNS ~ スポンサーセッション
yasunori0418
1
350
KMP導⼊において、マネジャーとして考えた事
sansantech
PRO
1
210
ITエンジニアを取り巻く環境とキャリアパス / A career path for Japanese IT engineers
takatama
4
1.5k
Streamline Cloud-Native App Development Using CDEs
saeedzf
0
850
技術書典18結果報告
mutsumix
2
180
NW運用の工夫と発明
recuraki
1
780
Oracle Database オプティマイザ・ヒントの活用
oracle4engineer
PRO
1
140
大手企業のAIツール導入の壁を越えて:サイバーエージェントのCursor活用戦略
gunta
14
2.5k
Machine Intelligence for Vision, Language, and Actions
keio_smilab
PRO
0
490
Featured
See All Featured
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.6k
Agile that works and the tools we love
rasmusluckow
329
21k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Adopting Sorbet at Scale
ufuk
76
9.4k
It's Worth the Effort
3n
184
28k
How to train your dragon (web standard)
notwaldorf
92
6k
A Modern Web Designer's Workflow
chriscoyier
693
190k
Raft: Consensus for Rubyists
vanstee
137
7k
Git: the NoSQL Database
bkeepers
PRO
430
65k
Balancing Empowerment & Direction
lara
1
84
The Art of Programming - Codeland 2020
erikaheidi
54
13k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
15
890
Transcript
Servers are doomed to fail Jaana B. Dogan
[email protected]
@rakyll
Serverless is also doomed to fail Jaana B. Dogan
[email protected]
@rakyll
Systems are doomed to fail Jaana B. Dogan
[email protected]
@rakyll
Is failure OK? Is failure an unexpected case?
Failure is not an exception. Systems change all the time.
“I haven’t touched the code for a century, it should
just work.” Said no one ever.
Failure is expected. Yes, it is.
None
@rakyll monitoring debugging postmortem
Monitoring is about saying if something is broken.
“99.99% of the requests should return in 100ms.”
@rakyll
@rakyll
Debugging
Debugging is collaborative.
Debugging comes in flavors. Logs Traces Metrics ...
Postmortems
Postmortems
Postmortems
Blameless? Focus on identifying problems.
Collaboration Design for collaboration.
Design for failure Set SLOs, plan for instrumentation, plan for
debugging.
Cross-stack debugging Accountability across stack with high cardinality data. speakerdeck.com/rakyll/rpc-metrics-at-google
Correlation Jump from monitoring/debugging data to data.
On-call debugging Jump from distributed tracing data to on-call information.
who to page?
Dynamic collection Capability to enable more collection in production when
needed.
Continuous collection Continuously collect signals, generate fleet-wide analysis reports.
Introspection Introspection pages provided from the services.
@rakyll monitoring debugging postmortem
Thank you Jaana B. Dogan Google
[email protected]