Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tips and Tricks from Shopify's codebase
Search
Christian Joudrey
May 23, 2014
Programming
2
560
Tips and Tricks from Shopify's codebase
Talk given at Montreal.rb on May 20th, 2014
Christian Joudrey
May 23, 2014
Tweet
Share
More Decks by Christian Joudrey
See All by Christian Joudrey
Writing NES games! with assembly!!
cjoudrey
1
740
Developing at Scale
cjoudrey
3
490
Scaling Rails for Black Friday / Cyber Monday at Shopify
cjoudrey
6
5.9k
Scaling Shopify
cjoudrey
3
530
#pairwithme
cjoudrey
3
250
Two-factor authentication
cjoudrey
4
390
Automate your Infrastructure with Chef
cjoudrey
9
620
Other Decks in Programming
See All in Programming
20251016_Rails News ~Rails 8.1の足音を聴く~
morimorihoge
3
740
はじめてのDSPy - 言語モデルを『プロンプト』ではなく『プログラミング』するための仕組み
masahiro_nishimi
4
16k
TFLintカスタムプラグインで始める Terraformコード品質管理
bells17
2
450
Building, Deploying, and Monitoring Ruby Web Applications with Falcon (Kaigi on Rails 2025)
ioquatix
4
2.5k
CSC305 Lecture 10
javiergs
PRO
0
260
「ちょっと古いから」って避けてた技術書、今だからこそ読もう
mottyzzz
12
7.2k
コード生成なしでモック処理を実現!ovechkin-dm/mockioで学ぶメタプログラミング
qualiarts
0
270
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
270
『毎日の移動』を支えるGoバックエンド内製開発
yutautsugi
2
290
monorepo の Go テストをはやくした〜い!~最小の依存解決への道のり~ / faster-testing-of-monorepos
convto
2
560
SwiftDataを使って10万件のデータを読み書きする
akidon0000
0
240
登壇は dynamic! な営みである / speech is dynamic
da1chi
0
360
Featured
See All Featured
How to Think Like a Performance Engineer
csswizardry
27
2.1k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.6k
KATA
mclloyd
PRO
32
15k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
2.5k
A designer walks into a library…
pauljervisheath
209
24k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
116
20k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
The Language of Interfaces
destraynor
162
25k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
1k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
30
2.9k
Navigating Team Friction
lara
190
15k
GraphQLの誤解/rethinking-graphql
sonatard
73
11k
Transcript
tips and tricks from Shopify’s codebase
None
None
None
measure it! if it moves...
statsd
None
Liquid::Template.extend StatsD::Instrument ! Liquid::Template.statsd_measure :render, 'Liquid.Template.render'
PaymentProcessingJob.statsd_count :perform, 'PaymentProcessingJob.processed'
None
None
slow queries
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1
None
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay*/
determining root cause
https://github.com/snormore/nginx-x-rid-header nginx request_id header proxy_set_header X-Request-ID "$request_id"; log_format main '...
$request_id' step 1
https://gist.github.com/mnutt/566725 Complete 200 OK in 100ms (Views: 60ms | ActiveRecord:
40ms | request_id=bc12813bce...) log_process_action ActionController::Instrumentation step 2
https://github.com/basecamp/marginalia User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id`
= 1 LIMIT 1 /*application:Shopify, controller:users,action:show, request_id:bc12813bce...*/ basecamp/marginalia step 3
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay, request_id:bc12813bce...*/ profit!
access.log rails.log slow_query.log profit! (2)
bonus! background jobs
schema migration with zero downtime
None
orders 20140520_orders
insert/delete/update triggers
INSERT INTO ... SELECT ... insert/delete/update triggers
async caveat
tracking exceptions
None
None
None
None
thanks! :)