Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Visibility for web developers
Search
Bruno Renié
May 13, 2014
Technology
3
450
Visibility for web developers
Bruno Renié
May 13, 2014
Tweet
Share
More Decks by Bruno Renié
See All by Bruno Renié
Decentralization & real-time with PubSubHubbub
brutasse
1
160
Deployability of Python Web Applications
brutasse
17
2.3k
Stop writing settings files
brutasse
21
2.6k
Class-based Views: patterns and anti-patterns
brutasse
9
1.6k
Packager son projet Django
brutasse
4
560
Staticfiles : tout ce qu'il faut savoir, rien que ce qu'il faut savoir
brutasse
4
540
Introduction to Django
brutasse
3
420
Other Decks in Technology
See All in Technology
Behind Postgres 18: The People, the Code, & the Invisible Work | Claire Giordano | PGConfEU 2025
clairegiordano
0
130
AI-Readyを目指した非構造化データのメダリオンアーキテクチャ
r_miura
1
310
GraphRAG グラフDBを使ったLLM生成(自作漫画DBを用いた具体例を用いて)
seaturt1e
1
130
あなたの知らない Linuxカーネル脆弱性の世界
recruitengineers
PRO
3
150
会社を支える Pythonという言語戦略 ~なぜPythonを主要言語にしているのか?~
curekoshimizu
3
660
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
13
82k
ハノーファーメッセ2025で見た生成AI活用ユースケース.pdf
hamadakoji
1
460
スタートアップの現場で実践しているテストマネジメント #jasst_kyushu
makky_tyuyan
0
130
SRE × マネジメントレイヤーが挑戦した組織・会社のオブザーバビリティ改革 ― ビジネス価値と信頼性を両立するリアルな挑戦
coconala_engineer
0
200
Kubernetes self-healing of your workload
hwchiu
0
510
オブザーバビリティが育むシステム理解と好奇心
maruloop
2
1k
現場の壁を乗り越えて、 「計装注入」が拓く オブザーバビリティ / Beyond the Field Barriers: Instrumentation Injection and the Future of Observability
aoto
PRO
1
380
Featured
See All Featured
Agile that works and the tools we love
rasmusluckow
331
21k
What’s in a name? Adding method to the madness
productmarketing
PRO
24
3.7k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Docker and Python
trallard
46
3.6k
A better future with KSS
kneath
239
18k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
The Cult of Friendly URLs
andyhume
79
6.6k
Producing Creativity
orderedlist
PRO
347
40k
[RailsConf 2023] Rails as a piece of cake
palkan
57
5.9k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
190
55k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
249
1.3M
Transcript
Visibility for Web Developers Bruno Renié Django Island 2014
Working with ops & support staff Large infrastructure, lots of
moving parts whoami: lead dev @ exoscale
People need to know what's going on, anytime Not everyone
is a developer SSH, grep and tail not acceptable
Goal: visibility on infrastructure & apps
Errors Events Metrics
Errors
Easy: use sentry
Great tool for developers, but… People also need to know
about things that are not broken
Events
Event == log call Timestamped information about something worth knowing
Easy to emit logs Searching on remote machines, not so
much
Centralized logging Agent forwards logs to an aggregator Aggregator stores
logs and provides a search interface
Aggregator Hosts logstash+elasticsearch kibana log shippers
Collecting logs rsyslog / syslog-ng logstash-forwarder (lumberjack) logstash
In Python logging.handlers.SyslogHandler
Centralizing isn't enough
A common way to log things import logging logger =
logging.getLogger(__name__) logger.info("user {} logged in".format(user.email))
Not much information No structure
Structured logging Record = set of key-value pairs event="user logged
in" user="
[email protected]
" ip="1.2.3.4"
In Python: pip install structlog Hynek Schlawack logger = structlog.get_logger()
logger.info("user logged in", user="
[email protected]
" ip="1.2.3.4") {"event": "user.login", "user": "
[email protected]
", "host": "1.2.3.4"}
Contextual loggers log = logger.bind(user=request.user.email) log.info("payment", amount=120, currency="CHF") {"event": "payment",
"user": "
[email protected]
", "amount": 120, "currency": "CHF"}
structlog JSON elasticsearch Rich data for building dashboards
Demo
Marketing: "Who signed up today?" Finance: "How much income in
the past week?" Support: "What did this user do in the past 24 hours?"
Events Centralize logs Add structure Profit logstash, elasticsearch structlog kibana
Metrics
Time series data Continuous, regular intervals Big player: graphite ecosystem
carbon whisper graphite
<identifier> <value> <timestamp> db1.load.midterm 4.6 1399842778 Measure at regular intervals,
send using the carbon line protocol echo " … " | nc host port
Traditional metrics stack agents collectd, apps, scripts carbon whisper graphite
ingests metrics stores metrics renders graphs
Counting and timing: statsd In-memory buffer Aggregates and flushes to
carbon at regular intervals
In Python pystatsd django-statsd-mozilla from stastd import StastClient statsd =
StatsClient() @statsd.timer("something") def expensive_computation(): # …
Graphing Graphite: rendering API (json / PNG / etc) Countless
dashboard apps Grafana Graph-explorer etc
Demo
What about alerts? Riemann: metrics hub/proxy App server overloaded, full
disk, etc Analyzes metrics computes aggregates notifies
Demo
we are hiring! Questions! @brutasse Image credits: http://en.wikipedia.org/wiki/Explosion
Resources Events Metrics structlog.org rsyslog logstash elasticsearch kibana carbon /
whisper graphite-api grafana collectd statsd riemann.io bernhard
Hosted services (not exhaustive) Logging Metrics Loggly Papertrail Hosted Graphite
Librato JSON / structured logging support?