Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tips and Tricks from Shopify's codebase
Search
Christian Joudrey
May 23, 2014
Programming
2
560
Tips and Tricks from Shopify's codebase
Talk given at Montreal.rb on May 20th, 2014
Christian Joudrey
May 23, 2014
Tweet
Share
More Decks by Christian Joudrey
See All by Christian Joudrey
Writing NES games! with assembly!!
cjoudrey
1
680
Developing at Scale
cjoudrey
3
460
Scaling Rails for Black Friday / Cyber Monday at Shopify
cjoudrey
6
5.6k
Scaling Shopify
cjoudrey
3
520
#pairwithme
cjoudrey
3
240
Two-factor authentication
cjoudrey
4
380
Automate your Infrastructure with Chef
cjoudrey
9
570
Other Decks in Programming
See All in Programming
Modern Angular with the NGRX Signal Store New Rules for Your Architecture @BASTA! 2024 in Mainz
manfredsteyer
PRO
0
170
Ebitengineの1vs1ゲーム WebRTCの活用
ponyo877
0
380
The Sequel to a Dream of Ruby Parser's Grammar
ydah
1
220
Ruby Parser progress report 2024
yui_knk
2
230
Go1.23で入った errorsパッケージの小さなアプデ
kuro_kurorrr
2
400
Scala アプリケーションのビルドを改善してデプロイ時間を 1/4 にした話 | How I improved the build of my Scala application and reduced deployment time by 4x
nomadblacky
1
180
GraphQLとGigaViewer for Apps
numeroanddev
2
190
"型"のあるRailsアプリケーション開発 / Typed Rails application development
sinsoku
1
660
RAGの回答精度評価用のQAデータセットを生成AIに作らせた話
kurahara
0
260
Rechartsで楽にゴリゴリにカスタマイズする!
10tera
1
170
Google Sign-inの移行から始めるCredential Manager活用
clockvoid
0
430
React + TextAliveでカッコいいLyric Applicatioinを作ろう!!
tosuri13
0
400
Featured
See All Featured
Building Applications with DynamoDB
mza
90
6k
Automating Front-end Workflow
addyosmani
1365
200k
Fontdeck: Realign not Redesign
paulrobertlloyd
80
5.1k
The Cult of Friendly URLs
andyhume
76
6k
Building an army of robots
kneath
302
42k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
190
16k
Scaling GitHub
holman
458
140k
Being A Developer After 40
akosma
84
590k
From Idea to $5000 a Month in 5 Months
shpigford
379
46k
Unsuck your backbone
ammeep
667
57k
The Invisible Customer
myddelton
119
13k
Intergalactic Javascript Robots from Outer Space
tanoku
268
26k
Transcript
tips and tricks from Shopify’s codebase
None
None
None
measure it! if it moves...
statsd
None
Liquid::Template.extend StatsD::Instrument ! Liquid::Template.statsd_measure :render, 'Liquid.Template.render'
PaymentProcessingJob.statsd_count :perform, 'PaymentProcessingJob.processed'
None
None
slow queries
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1
None
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay*/
determining root cause
https://github.com/snormore/nginx-x-rid-header nginx request_id header proxy_set_header X-Request-ID "$request_id"; log_format main '...
$request_id' step 1
https://gist.github.com/mnutt/566725 Complete 200 OK in 100ms (Views: 60ms | ActiveRecord:
40ms | request_id=bc12813bce...) log_process_action ActionController::Instrumentation step 2
https://github.com/basecamp/marginalia User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id`
= 1 LIMIT 1 /*application:Shopify, controller:users,action:show, request_id:bc12813bce...*/ basecamp/marginalia step 3
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay, request_id:bc12813bce...*/ profit!
access.log rails.log slow_query.log profit! (2)
bonus! background jobs
schema migration with zero downtime
None
orders 20140520_orders
insert/delete/update triggers
INSERT INTO ... SELECT ... insert/delete/update triggers
async caveat
tracking exceptions
None
None
None
None
thanks! :)