Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
みんなのIMAPを可視化する / Visualize IMAP Everybody
Search
linyows
July 14, 2020
Programming
2
560
みんなのIMAPを可視化する / Visualize IMAP Everybody
Fukuoka.go#16 で発表した資料です。Junly 14, 2020
linyows
July 14, 2020
Tweet
Share
More Decks by linyows
See All by linyows
透過型SMTPプロキシによる送信メールの可観測性向上: Update Edition / Improved observability of outgoing emails with transparent smtp proxy: Update edition
linyows
2
210
研究の再現性を高める 仕組みをGoでつくる / Creating a system to improve the reproducibility of research using go
linyows
1
120
奥が深いメールのシステム / The depth of Email system
linyows
3
420
IaCにおけるテスト考察 / Tests in IaC
linyows
2
440
リバースエンジニアリングとGoでSlackの認知負荷を下げる / Reducing cognitive load in Slack with Reverse-engineering and Go
linyows
2
280
透過型SMTPプロキシによる送信メールの可観測性向上 / Improved observability of outgoing emails with transparent smtp proxy
linyows
2
960
プロダクションで使うGo Pluginの利便性とパフォーマンス性 / Simplicity and Performance of Go plugin for Production
linyows
0
510
求められるソフトウェアエンジニア像とキャリア戦略 / Engineers and Career Strategies Required Now
linyows
2
480
CloudflareのCAPTCHAを使って ユーザ体験を下げず不正利用を排除する / Cloudflare's captcha case study that does not degrade ux
linyows
0
360
Other Decks in Programming
See All in Programming
Quine, Polyglot, 良いコード
qnighy
4
640
3 Effective Rules for Using Signals in Angular
manfredsteyer
PRO
1
100
受け取る人から提供する人になるということ
little_rubyist
0
230
最新TCAキャッチアップ
0si43
0
140
CSC509 Lecture 12
javiergs
PRO
0
160
3 Effective Rules for Using Signals in Angular
manfredsteyer
PRO
0
100
3 Effective Rules for Using Signals in Angular
manfredsteyer
PRO
0
110
「今のプロジェクトいろいろ大変なんですよ、app/services とかもあって……」/After Kaigi on Rails 2024 LT Night
junk0612
5
2.1k
イベント駆動で成長して委員会
happymana
1
320
Arm移行タイムアタック
qnighy
0
310
Pinia Colada が実現するスマートな非同期処理
naokihaba
4
220
Flutterを言い訳にしない!アプリの使い心地改善テクニック5選🔥
kno3a87
1
160
Featured
See All Featured
GitHub's CSS Performance
jonrohan
1030
460k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.7k
Practical Orchestrator
shlominoach
186
10k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
250
21k
Adopting Sorbet at Scale
ufuk
73
9.1k
Optimising Largest Contentful Paint
csswizardry
33
2.9k
The Cost Of JavaScript in 2023
addyosmani
45
6.7k
The Language of Interfaces
destraynor
154
24k
Happy Clients
brianwarren
98
6.7k
Intergalactic Javascript Robots from Outer Space
tanoku
269
27k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
246
1.3M
Transcript
Fukuoka.go#16 Tomohisa Oda / July 14, 2020 VisualizeIMAP Everybody
Fukuoka.go#16 @linyows Blog: tomohisaoda.com Affiliation: GMO pepabo, inc.
Fukuoka.go#16 ๏ ϗεςΟϯάࣄۀ෦Senior Engineering Lead ࣄۀ෦ͷCTOVPoEͷΑ͏ͳͷ ๏ ϖύϘݚڀॴभେֶڞಉݚڀ 2020Պݚඅ࠾IUUQTLBLFOOJJBDKQHSBOU,",&/)*130+&$5, ܰྔίϯςφʹΑΔେنߴूੵϝʔϧϗεςΟϯάج൫ʹ͓͚Δૹ৴ػೳͷߴػೳԽ
In Pepabo…
Fukuoka.go#16 Private Projects ΞϓϦέʔγϣϯ͕ࣗϦϙδτϦ͔Β࠷৽ͷίʔυΛݕ ͯࣗ͠ಈͰσϓϩΠͱσϦόϦʔΛߦ͏πʔϧ MJOVYͷVTFS໊લղܾΛ(JU)VCͷ5FBNϝϯόʔϦϙδτ ϦͷΞΫηεݖݶऀ͔Βߦ͏πʔϧ
Fukuoka.go#16 Pepaboϗετ͍ͯ͠·͢
Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,
FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷ ͍ΖΜͳٕज़Λѻ͍ͬͯ·͢
Fukuoka.go#16 ๏ Domain — DNS ๏ Web — HTTP, SSH,
FTP and WebDav… ๏ Mail — SMTP, POP, IMAP… ๏ Database — (MySQL, SQLite…) ϩϦϙοϓϔςϜϧ͕ϗετ͍ͯ͠Δͷ ຊίίͷ
Fukuoka.go#16 Mail ͍ͬͯ·͔͢ʁ
Fukuoka.go#16 ͍·Ͳ͖ϝʔϧʁ͔ͭͬͯͳ͍ʁ͍͍͍ʜ — Chat Type — Messaging Type
Fukuoka.go#16 ΄Βɺࢥ͍ग़͠·͔ͨ͠ʁͪΌΜͱ͍ͬͯ·͢Ͷʁ
Fukuoka.go#16 େมਃ͠༁͋Γ·ͤΜ
Fukuoka.go#16 େମ(NBJMPS(4VJUFͰ͢ΑͶʁ
Fukuoka.go#16 ͍ʹ͑͠ͷٕज़ͱ͍͑ɺMailʹมΘΔ ΦʔϓϯͳMessagingγεςϜͳ͍
Fukuoka.go#16 MailͷΈ
Fukuoka.go#16 Architecture RFC 5598 ʹ͋Γ·͕͢ʜ
Fukuoka.go#16 Architecture ͬͱγϯϓϧͳͭ ͜ΕͰ࠷খߏͳͷͰࠜຊతʹ ΣϒΑΓෳࡶ Mail Submission Agent :587 Mail
Transferring Agent :25 Mail Delivery Agent :993 Postfix Postfix Dovecot ✉ ✉ SMTP SMTP LMTP IMAP
Fukuoka.go#16 ϩϦϙοϓϔςϜϧʹ͍ͭͯ
Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳنײ
Fukuoka.go#16 Our Status 120 100k 2.2k Over /min ͜ͷΑ͏ͳنײ ϝʔϧαʔόʹΔ
ϝʔϧΞΧϯτ Max ϝʔϧαʔόͷ ؒʹૹΔϝʔϧ ௨ʢDeliveredʣ
Fukuoka.go#16 Our Challenges ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ ΞΧϯτͬऔΓෆਖ਼ར༻
ѱҙͷͳ͍େྔϝʔϧ৴ ๏ ड৴ ๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ ๏ Disk I/Oෛՙ
Fukuoka.go#16 ๏ ૹ৴ ๏ IP BanɺBlack ListೖΓ ΞΧϯτͬऔΓෆਖ਼ར༻ ѱҙͷͳ͍େྔϝʔϧ৴ ๏
ड৴ ๏ ϑΟογϯάɺΟϧεɺεύϜϝʔϧഉআ ๏ Disk I/Oෛՙ Our Challenges
Fukuoka.go#16 ๏ ͍͍ετϨʔδΛങ͑ղܾ͢Δ ๏ Ձ֨ଳͷαʔϏεͳͷͰݫ͍͠ ๏ ݱঢ়ɺෛՙͷߴ͍ΞΧϯτΛҠઃͯ͠Ϧιʔε ফඅΛۉ͢ઓུ ๏ ࣌ʹSFTUBSUʜ
IMAP Disk I/O
Fukuoka.go#16 ๏ ओʹPrometheusΛ͍ͬͯΔ ๏ ֤Proxyϝʔϧαʔόʹ͍Εͨnode_exporterʹΑΔϦιʔεࢹ ๏ mtailͰϝʔϧΩϡʔେखϓϩόΠμʔͷόϯεϝʔϧͷࢹ ๏ blackbox_exporterͰsmtp, pop,
imapͷPortࢹ ݱঢ়ͷࢹेͰͳ͍ Monitoring
Fukuoka.go#16
Fukuoka.go#16
Fukuoka.go#16
Fukuoka.go#16 imap_exporterΛ࡞ͬͨ
Fukuoka.go#16 ๏ ϛυϧΣΞʹґଘ͠ͳ͍ΑΓநతͳϓϩτίϧϨϕϧͳϝτϦΫεΛऔಘ͢Δ ๏ Mail User Agent͕ඞͣߦ͏Ұ࿈ͷίϚϯυΛൃߦ͔͔ͯͬͨ࣌ؒ͠ΛͱΔ ๏ Connection /
Login / List / Select / Fetch / Logout ๏ IMAP ClientɺطଘϞδϡʔϧΛ͏ͷͰͳࣗ͘લͰ࣮͢ΔʢIMAPͷؾ࣋ͪʹͳ Δʣ imap_exporter
Fukuoka.go#16 ๏ RFC2060ΛΑʔ͠ಡΉͧͱ ࢥͬͨΒ్தͰ૬ͳϘϦϡʔ Ϝ͕͋Δ͜ͱʹؾͮ͘ ๏ SMTP͕ϖʔδ΄Ͳʹରͯ͠ ϖʔδ IMAPͷ༷RFC2060
Fukuoka.go#16 ๏ ͜Μͳ͜ͱ͋Ζ͏͔ͱɺຊ୨͔ΒऔΓग़͢ ిࢠϝʔϧϓϩτίϧʕجຊɾ࣮ɾӡ༻ 2000/11
Fukuoka.go#16 ๏ IMAPʹͭͷεςʔτ͕͋Γ ๏ ະೝূ ๏ ೝূࡁ ๏ બࡁ ๏
ϩάΞτ ๏ ͦΕͧΕͷεςʔτʹ͓͍ͯΫϥΠΞϯτ͕ൃߦ͢ΔλάΛͬͯαʔόʔͱΫϥΠΞϯ τͷίϛϡχέʔγϣϯΛߦ͏ IMAP ίϚϯυ λά
Fukuoka.go#16 IMAP ΫϥΠΞϯτͷ࣮
Fukuoka.go#16 Ϩεϙϯεʹλά͕͋Δ͔Ͳ͏͔Λ ఆ͍ͯ͠Δ͚ͩͷγϯϓϧͳ࣮
Fukuoka.go#16 telnetopensslΛ͏Α͏ͳΠϝʔδͰίϚϯυΛ࣮ߦ͢Δ
Fukuoka.go#16 ๏ ࠨͷάϥϑखݩ͔ΒαϯϓϧΞΧϯτʹ ଓͨ͠ྫ ๏ ͜ΕΛશͯͷϝʔϧαʔόʹಋೖ͢Δ༧ఆ ๏ Disk I/OෛՙʹରԠ͢Δࢳޱ͕Έ͔ͭΕͱ͍ ͏ఔ
๏ IMAPͷঢ়ଶΛՄࢹԽ͢Δ͜ͱʹؒҧ͍ͳ͍ ๏ Ұ࿈ͷಈ࡞ΛFirst Loading Latencyͱ͍͏໊ લΛ͚ͭͨ First Loading Latency
Fukuoka.go#16 ๏ imap_exporterΛ࡞ͬͯMail Delivery AgentΛՄࢹԽ͠Α͏ͱ͍ͯ͠Δ ๏ IMAPϓϩτίϧSMTPͱҧͬͯ૬ෳࡶ ๏ ϓϩτίϧ༷Λཧղ͢Δ͜ͱͰGoඪ४ϥΠϒϥϦͰͬ͘͞ͱΫϥΠΞϯτ࣮Ͱ͖ͨ ͜ͷΑ͏ʹɺEngineeringͰϗεςΟϯάͷϝʔϧΛࢧ͍͑ͯ·͢ɻ࠷ۙɺMREͱ͍͏৬छ
ΛͿ্͍ͪ͛ͯ·͢ʂʂʂʂ Conclusion
Fukuoka.go#16 Messaging Reliability EngineeringʢMREʣɺਓͱਓͱ͕ιϑτΣΞΛ௨ͯ͡ ίϛϡχέʔγϣϯΛ͢ΔγεςϜʹ͓͍ͯɺεέʔϥϒϧͰ৴པੑͷߴ͍ιϑτ ΣΞγεςϜΛ࡞͠·͢ɻMREɺΠϯϑϥετϥΫνϟͱӡ༻ΛదԠͤ͞Δ Site Reliability EngineeringͷจԽΛMessagingͰड͚ܧ͍ͩͷͰ͢ɻجຊ తͳ͜ͱSREͱಉ͡Ͱ͕͢ɺѻ͏ٕज़͕ओʹϝʔϧͰɺϝʔϧͷӡ༻ΛιϑτΣ
ΞͰΤϯδχΞϦϯά͢Δ͜ͱʹͳΓ·͢ɻʮMREϖύϘʯͰݕࡧʂ Messaging Reliability Engineeringͱ
Fukuoka.go#16 Thank youaʕ⊙౪⚆ʔ