Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
みんなのIMAPを可視化する / Visualize IMAP Everybody
Search
linyows
July 14, 2020
Programming
2
560
みんなのIMAPを可視化する / Visualize IMAP Everybody
Fukuoka.go#16 で発表した資料です。Junly 14, 2020
linyows
July 14, 2020
Tweet
Share
More Decks by linyows
See All by linyows
研究の再現性を高める 仕組みをGoでつくる / Creating a system to improve the reproducibility of research using go
linyows
1
110
奥が深いメールのシステム / The depth of Email system
linyows
3
410
IaCにおけるテスト考察 / Tests in IaC
linyows
2
420
リバースエンジニアリングとGoでSlackの認知負荷を下げる / Reducing cognitive load in Slack with Reverse-engineering and Go
linyows
2
270
透過型SMTPプロキシによる送信メールの可観測性向上 / Improved observability of outgoing emails with transparent smtp proxy
linyows
2
940
プロダクションで使うGo Pluginの利便性とパフォーマンス性 / Simplicity and Performance of Go plugin for Production
linyows
0
500
求められるソフトウェアエンジニア像とキャリア戦略 / Engineers and Career Strategies Required Now
linyows
2
470
CloudflareのCAPTCHAを使って ユーザ体験を下げず不正利用を排除する / Cloudflare's captcha case study that does not degrade ux
linyows
0
350
なぜNotionを使うのか2022 / Why use notion as our workspace in 2022
linyows
3
6k
Other Decks in Programming
See All in Programming
gopls を改造したら開発生産性が高まった
satorunooshie
8
240
Generative AI Use Cases JP (略称:GenU)奮闘記
hideg
0
150
ピラミッド、アイスクリームコーン、SMURF: 自動テストの最適バランスを求めて / Pyramid Ice-Cream-Cone and SMURF
twada
PRO
9
980
『ドメイン駆動設計をはじめよう』のモデリングアプローチ
masuda220
PRO
7
430
CSC305 Lecture 13
javiergs
PRO
0
130
Streams APIとTCPフロー制御 / Web Streams API and TCP flow control
tasshi
1
290
デプロイを任されたので、教わった通りにデプロイしたら障害になった件 ~俺のやらかしを越えてゆけ~
techouse
52
32k
讓數據說話:用 Python、Prometheus 和 Grafana 講故事
eddie
0
350
カラム追加で増えるActiveRecordのメモリサイズ イメージできますか?
asayamakk
4
1.5k
Server Driven Compose With Firebase
skydoves
0
390
offers_20241022_imakiire.pdf
imakurusu
2
360
From Subtype Polymorphism To Typeclass-based Ad hoc Polymorphism- An Example
philipschwarz
PRO
0
160
Featured
See All Featured
Rebuilding a faster, lazier Slack
samanthasiow
79
8.6k
We Have a Design System, Now What?
morganepeng
50
7.2k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
27
4.1k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
26
1.4k
Building Your Own Lightsaber
phodgson
102
6k
The Language of Interfaces
destraynor
154
24k
What's in a price? How to price your products and services
michaelherold
243
12k
Designing for humans not robots
tammielis
249
25k
How To Stay Up To Date on Web Technology
chriscoyier
788
250k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
3
370
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
355
29k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5k
Transcript
Fukuoka.go#16 Tomohisa Oda / July 14, 2020 VisualizeIMAP Everybody
Fukuoka.go#16 @linyows Blog: tomohisaoda.com Affiliation: GMO pepabo, inc.
Fukuoka.go#16 ๏ ϗεςΟϯάࣄۀ෦Senior Engineering Lead ࣄۀ෦ͷCTOVPoEͷΑ͏ͳͷ ๏ ϖύϘݚڀॴभେֶڞಉݚڀ 2020Պݚඅ࠾IUUQTLBLFOOJJBDKQHSBOU,",&/)*130+&$5, ܰྔίϯςφʹΑΔେنߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ
In Pepabo…
Fukuoka.go#16 Private Projects ΞϓϦέʔγϣϯ͕ࣗϦϙδτϦ͔Β࠷৽ͷίʔυΛݕ ͯࣗ͠ಈͰσϓϩΠͱσϦόϦʔΛߦ͏πʔϧ MJOVYͷVTFS໊લղܾΛ(JU)VCͷ5FBNϝϯόʔϦϙδτ ϦͷΞΫηεݖݶऀ͔Βߦ͏πʔϧ
Fukuoka.go#16 Pepaboϗετ͍ͯ͠·͢
Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,
FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷ ͍ΖΜͳٕज़Λѻ͍ͬͯ·͢
Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,
FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷ ຊίίͷ
Fukuoka.go#16 Mail ͍ͬͯ·͔͢ʁ
Fukuoka.go#16 ͍·Ͳ͖ϝʔϧʁ͔ͭͬͯͳ͍ʁ͍͍͍ʜ — Chat Type — Messaging Type
Fukuoka.go#16 ΄Βɺࢥ͍ग़͠·͔ͨ͠ʁͪΌΜͱ͍ͬͯ·͢Ͷʁ
Fukuoka.go#16 େมਃ͠༁͋Γ·ͤΜ
Fukuoka.go#16 େମ(NBJMPS(4VJUFͰ͢ΑͶʁ
Fukuoka.go#16 ͍ʹ͑͠ͷٕज़ͱ͍͑ɺMailʹมΘΔ ΦʔϓϯͳMessagingγεςϜͳ͍
Fukuoka.go#16 MailͷΈ
Fukuoka.go#16 Architecture RFC 5598 ʹ͋Γ·͕͢ʜ
Fukuoka.go#16 Architecture ͬͱγϯϓϧͳͭ ͜ΕͰ࠷খߏͳͷͰࠜຊతʹ ΣϒΑΓෳࡶ Mail Submission Agent :587 Mail
Transferring Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP
Fukuoka.go#16 ϩϦϙοϓϔςϜϧʹ͍ͭͯ
Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳنײ
Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳنײ ϝʔϧαʔόʹΔ
ϝʔϧΞΧϯτ Max ϝʔϧαʔόͷ ؒʹૹΔϝʔϧ ௨ʢDeliveredʣ
Fukuoka.go#16 Our Challenges ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ ΞΧϯτͬऔΓෆਖ਼ར༻
ѱҙͷͳ͍େྔϝʔϧ৴ ๏ ड৴ ๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ ๏ Disk I/Oෛՙ
Fukuoka.go#16 ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ ΞΧϯτͬऔΓෆਖ਼ར༻ ѱҙͷͳ͍େྔϝʔϧ৴ ๏
ड৴ ๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ ๏ Disk I/Oෛՙ Our Challenges
Fukuoka.go#16 ๏ ͍͍ετϨʔδΛങ͑ղܾ͢Δ ๏ Ձ֨ଳͷαʔϏεͳͷͰݫ͍͠ ๏ ݱঢ়ɺෛՙͷߴ͍ΞΧϯτΛҠઃͯ͠Ϧιʔε ফඅΛۉ͢ઓུ ๏ ࣌ʹSFTUBSUʜ
IMAP Disk I/O
Fukuoka.go#16 ๏ ओʹPrometheusΛ͍ͬͯΔ ๏ ֤Proxyϝʔϧαʔόʹ͍Εͨnode_exporterʹΑΔϦιʔεࢹ ๏ mtailͰϝʔϧΩϡʔେखϓϩόΠμʔͷόϯεϝʔϧͷࢹ ๏ blackbox_exporterͰsmtp, pop,
imapͷPortࢹ ݱঢ়ͷࢹेͰͳ͍ Monitoring
Fukuoka.go#16
Fukuoka.go#16
Fukuoka.go#16
Fukuoka.go#16 imap_exporterΛ࡞ͬͨ
Fukuoka.go#16 ๏ ϛυϧΣΞʹґଘ͠ͳ͍ΑΓநతͳϓϩτίϧϨϕϧͳϝτϦΫεΛऔಘ͢Δ ๏ Mail User Agent͕ඞͣߦ͏Ұ࿈ͷίϚϯυΛൃߦ͔͔ͯͬͨ࣌ؒ͠ΛͱΔ ๏ Connection /
Login / List / Select / Fetch / Logout ๏ IMAP ClientɺطଘϞδϡʔϧΛ͏ͷͰͳࣗ͘લͰ࣮͢ΔʢIMAPͷؾ࣋ͪʹͳ Δʣ imap_exporter
Fukuoka.go#16 ๏ RFC2060ΛΑʔ͠ಡΉͧͱ ࢥͬͨΒ్தͰ૬ͳϘϦϡʔ Ϝ͕͋Δ͜ͱʹؾͮ͘ ๏ SMTP͕ϖʔδ΄Ͳʹରͯ͠ ϖʔδ IMAPͷ༷RFC2060
Fukuoka.go#16 ๏ ͜Μͳ͜ͱ͋Ζ͏͔ͱɺຊ୨͔ΒऔΓग़͢ ిࢠϝʔϧϓϩτίϧʕجຊɾ࣮ɾӡ༻ 2000/11
Fukuoka.go#16 ๏ IMAPʹͭͷεςʔτ͕͋Γ ๏ ະೝূ ๏ ೝূࡁ ๏ બࡁ ๏
ϩάΞτ ๏ ͦΕͧΕͷεςʔτʹ͓͍ͯΫϥΠΞϯτ͕ൃߦ͢ΔλάΛͬͯαʔόʔͱΫϥΠΞϯ τͷίϛϡχέʔγϣϯΛߦ͏ IMAP ίϚϯυ λά
Fukuoka.go#16 IMAP ΫϥΠΞϯτͷ࣮
Fukuoka.go#16 Ϩεϙϯεʹλά͕͋Δ͔Ͳ͏͔Λ ఆ͍ͯ͠Δ͚ͩͷγϯϓϧͳ࣮
Fukuoka.go#16 telnetopensslΛ͏Α͏ͳΠϝʔδͰίϚϯυΛ࣮ߦ͢Δ
Fukuoka.go#16 ๏ ࠨͷάϥϑखݩ͔ΒαϯϓϧΞΧϯτʹ ଓͨ͠ྫ ๏ ͜ΕΛશͯͷϝʔϧαʔόʹಋೖ͢Δ༧ఆ ๏ Disk I/OෛՙʹରԠ͢Δࢳޱ͕Έ͔ͭΕͱ͍ ͏ఔ
๏ IMAPͷঢ়ଶΛՄࢹԽ͢Δ͜ͱʹؒҧ͍ͳ͍ ๏ Ұ࿈ͷಈ࡞ΛFirst Loading Latencyͱ͍͏໊ લΛ͚ͭͨ First Loading Latency
Fukuoka.go#16 ๏ imap_exporterΛ࡞ͬͯMail Delivery AgentΛՄࢹԽ͠Α͏ͱ͍ͯ͠Δ ๏ IMAPϓϩτίϧSMTPͱҧͬͯ૬ෳࡶ ๏ ϓϩτίϧ༷Λཧղ͢Δ͜ͱͰGoඪ४ϥΠϒϥϦͰͬ͘͞ͱΫϥΠΞϯτ࣮Ͱ͖ͨ ͜ͷΑ͏ʹɺEngineeringͰϗεςΟϯάͷϝʔϧΛࢧ͍͑ͯ·͢ɻ࠷ۙɺMREͱ͍͏৬छ
ΛͿ্͍ͪ͛ͯ·͢ʂʂʂʂ Conclusion
Fukuoka.go#16 Messaging Reliability EngineeringʢMREʣɺਓͱਓͱ͕ιϑτΣΞΛ௨ͯ͡ ίϛϡχέʔγϣϯΛ͢ΔγεςϜʹ͓͍ͯɺεέʔϥϒϧͰ৴པੑͷߴ͍ιϑτ ΣΞγεςϜΛ࡞͠·͢ɻMREɺΠϯϑϥετϥΫνϟͱӡ༻ΛదԠͤ͞Δ Site Reliability EngineeringͷจԽΛMessagingͰड͚ܧ͍ͩͷͰ͢ɻجຊ తͳ͜ͱSREͱಉ͡Ͱ͕͢ɺѻ͏ٕज़͕ओʹϝʔϧͰɺϝʔϧͷӡ༻ΛιϑτΣ
ΞͰΤϯδχΞϦϯά͢Δ͜ͱʹͳΓ·͢ɻʮMREϖύϘʯͰݕࡧʂ Messaging Reliability Engineeringͱ
Fukuoka.go#16 Thank youaʕ⊙౪⚆ʔ