Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ISUCON3 - ISUCON Makers Casual Talks
Search
FUJIWARA Shunichiro
December 10, 2014
Programming
3
810
ISUCON3 - ISUCON Makers Casual Talks
FUJIWARA Shunichiro
December 10, 2014
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
alecthomas/kong はいいぞ
fujiwara3
6
1.8k
ecspressoの設計思想に至る道 / sekkeinight2025
fujiwara3
12
2.9k
さくらのIaaS基盤のモニタリングとOpenTelemetry/OSC Hokkaido 2025
fujiwara3
3
1.2k
監視のこれまでとこれから/sakura monitoring seminar 2025
fujiwara3
11
5.3k
k6による負荷試験 入門から日常的な実践まで/Re:TechTalk #01
fujiwara3
2
150
困難を「一般解」で解く
fujiwara3
10
3.8k
「隙間家具OSS」に至る道/Fujiwara Tech Conference 2025
fujiwara3
7
13k
alecthomas/kong はいいぞ / kamakura.go#7
fujiwara3
1
1.2k
ISUCONに強くなるかもしれない日々の過ごしかた/Findy ISUCON 2024-11-14
fujiwara3
11
1.4k
Other Decks in Programming
See All in Programming
Model Pollution
hschwentner
1
180
Back to the Future: Let me tell you about the ACP protocol
terhechte
0
120
プログラミングどうやる? ~テスト駆動開発から学ぶ達人の型~
a_okui
0
190
Serena MCPのすすめ
wadakatu
4
860
実践AIチャットボットUI実装入門
syumai
7
2.4k
ててべんす独演会〜Flowの全てを語ります〜
tbsten
1
220
Pythonスレッドとは結局何なのか? CPython実装から見るNoGIL時代の変化
curekoshimizu
4
1.2k
Local Peer-to-Peer APIはどのように使われていくのか?
hal_spidernight
2
440
iOSエンジニア向けの英語学習アプリを作る!
yukawashouhei
0
170
タスクの特性や不確実性に応じた最適な作業スタイルの選択(ペアプロ・モブプロ・ソロプロ)と実践 / Optimal Work Style Selection: Pair, Mob, or Solo Programming.
honyanya
2
120
Let's Write a Train Tracking Algorithm
twocentstudios
0
220
Learn CPU architecture with Assembly
akkeylab
1
1.3k
Featured
See All Featured
KATA
mclloyd
32
14k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
A better future with KSS
kneath
239
17k
Building Better People: How to give real-time feedback that sticks.
wjessup
368
20k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
2.6k
Learning to Love Humans: Emotional Interface Design
aarron
274
40k
Making the Leap to Tech Lead
cromwellryan
135
9.5k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
127
53k
The Cult of Friendly URLs
andyhume
79
6.6k
Designing Experiences People Love
moore
142
24k
Transcript
ISUCON'Makers'Casual'Talks ISUCON'3 2014/12/10&@fujiwara
About&me @fujiwara ISUCON'1,'2'༏উ'(fujiwara) ISUCON'3'ग़ ໘ന๏ਓΧϠοΫ
ग़ͷܦҢ ISUCON'1,'2'fujiwara࿈ʂ 2ͷ࠙ձʮग़͍ͯͩ͘͠͞Αʯʮ͍ʂʯ CROSS%2013%@941͞Μʹ͓ئ͍͞ΕΔ
Timeline 2013/1&CROSS&2013 2013/4'ΧϠοΫٕज़෦৽ଔݚमʮࣾISUCONʯ 2013/6'@941͞ΜͱϛʔςΟϯά 2013/7/31'ग़࡞ઓձٞ 2013/9/21'YAPC::Asia'2013'ʮࣾISUCONͷͭ͘Γ͔ͨʯൃද 2013/10/5,6)ISUCON)3)ΦϯϥΠϯ༧બ 2013/11/9'ISUCON'3'ຊબ
ISUCON'1,2'ͱ'3'ͷҧ͍ ΦϯϥΠϯ༧બ 2ճɺผͷΛ࡞Δ
ࣾISUCON ৽ଔݚमͷҰ ओʹαʔόαΠυΛΔΤϯδχΞ͚ HTTPΛֶͿ$→$WebΞϓϦΛ࡞Δ ࡞ͬͨΞϓϦͰISUCON! github.com/kayac/newbie2training
ࣾISUCON ݚमͷҰͷͨΊૉͳग़ • ੩తϑΝΠϧΛΞϓϦαʔό͕৴ • ΠϯσοΫε͕ுΒΕ͍ͯͳ͍ςʔϒϧ • ϧʔϓதͰΫΤϦ͕ඈͿ ੩తϑΝΠϧΛwebαʔό৴$+$ΠϯσοΫεΛదʹுΔ$+$ϧ ʔϓΫΤϦΛJOINʹॻ͖͑
→$είΞ20ഒ
ࣾISUCONͰͷܦݧ • Ր༵"WebAppॻ͘ • ਫ༵"ςετॻ͘ • ༵"ϕϯνϚʔΫॻ͘ • @acidlemon"ʹղ͍ͯΒ͏ •
༵ۚே͔Β"ISUCON (னؒݚमͷߨࢣۀ)
ࣾISUCONͰͷল • ్தͰϕϯνϚʔΫϗετ"㱻"֤VMͷଳҬةͳ͍ • ଳҬ͕͢ΔͱείΞ͕͔ࠩͳ͘ͳΔ • ్தͰ"Accept-Encoding: gzip"Ճ
2013/7/31 ग़࡞ઓձٞ !@fujiwara! !@acidlemon !@typester! !@songmu
memo ը૾ϦαΠζ ಉ࣌ଓܥɺνϟοτɺϨΠςϯγ push͔ΒͷҰ੪ΞΫηεΛΤϥʔʹ͠ͳ͍Ͱ͑Δ ಉใ RedisͬͯΔ͚Ͳ͍ํѱ͍ mongo git href="" PostgreSQLೖΕ͓ͯ͘
ຊબʹ࠾༻ • ը૾ϦαΠζ • ಉ࣌ଓܥɺϨΠςϯγ • Ұ੪ΞΫηε ༧બʮຊબͷωλόϨʹͳΒͳ͍ʯͷ
10/5,6'ΦϯϥΠϯ༧બ
༧બग़༰ Github'GistͷΑ͏ͳͷ ϩάΠϯػೳɻCookieʹΑΔηογϣϯҡ͕࣋ඞཁ ϓϥΠϕʔτߘػೳɻϓϥΠϕʔτઃఆ͞ΕͨͷϩάΠϯ ͍ͯ͠ΔϢʔβࣗʹ͔͠ݟ͑ͳ͍ MarkdownܗࣜͰߘ ʴ!᠘తͳઃఆ
༧બͷҙਤ ໌Β͔ʹຊબग़ʹ࣮ྗͷΓͳ͍νʔϜΛ;Δ͍མͱ͢ ×"੩తϑΝΠϧΛΞϓϦαʔό͕৴ ×"ΠϯσοΫε͕ͳ͍ςʔϒϧ ×"ϧʔϓΫΤϦ ×"Markdownมʹ֎෦ίϚϯυىಈ ී௨ͷWebΞϓϦͱͯ͠μϝͳ࣮ΛͤΕߴείΞʹ ʴ$Ϙʔφεͱͯ͠ʮ᠘ʯΛΫϦΞͯ͠Ճ
༧બ४උ ʙ!9/17!ग़ͷϓϩτλΠϓGoͷϕϯνΫϥΠΞϯτԾ࣮ 9/25ʙ%@acidlemon%ʹղ͍ͯΒ͍͡ΊΔ 9/30%/recent/*%ΛՃ 10/2,3'Ruby,'Python,'Go,'Node,'PHP'Ҡ২ 10/5,6'ISUCON3'༧બ
༧બ ͔ࣗΒIRCͰαϙʔτ
༧બͷʮ᠘ʯ
MySQL&InnoDB&memcached&plugin MySQL&5.6ͷ৽ػೳ • memcached(protocolͰMySQLͱ௨৴Ͱ͖Δ • port(11211(Λ(Listen • ΞϓϦέʔγϣϯ͔Βηογϣϯอଘʹ༻ • port(11212(Ͱී௨ͷ(memcached(ىಈͯ͋͠Δ
• ॳظঢ়ଶͰ༻͍ͯ͠ͳ͍
ҙຯͷͳ͍!CSS!link <link rel="stylesheet" href="/"> • ϕϯνϚʔΧʔϦϯΫΛḷͬͯ"/"ʹϦΫΤετΛඈ͢ • ͔͠͠είΞ"0 • PageViewͰͳ͍ͱ͍͏ѻ͍
• HTML͔Βআ͢ΔͱϦΫΤετ͠ͳ͘ͳΔ
ҰൃΞτͰͳ͍ݮํࣜ fail=3·Ͱݮͳ͠ failͷࣗ%&(ͨͱ͑&fails=6&ͳΒ& )&Λݮ fail%>=%13%Ͱ% %ݮʹͳΔͷͰείΞ%0
༧બͷল • είΞϦϯάͷৄࡉ͕ެ։͞Ε͍ͯͳ͔ͬͨ • /recent/*"ʹର͢ΔϕϯνϚʔΧʔͷνΣοΫ͕͔ͬͨ • ͋ͱ͔ΒՃͨ͠෦ • ͜͜Λ"Cache"͢ΔͱΑ͘ճΔ
༧બϕϯνϚʔΧʔ • Go • ࢀՃऀ͕ࣗ༝ʹͳΔϗετͰಈ࡞͢ΔͨΊɺղੳ͞Εʹ͍͘ Α͏ʹ • --workload$Φϓγϣϯॳಋೖ • ISUCON$1,2ͰௐͰ͖ͳ͔ͬͨ
༧બϕϯνϚʔΧʔ • ੩తϑΝΠϧ͚ͩΞΫηε͢Δ"sta&c"worker • ϦΫΤετ͝ͱʹ"10ms"wait • /recent/*"͚ͩΞΫηε͢Δ"recent"worker • ϩάΠϯɺߘɺӾཡɺϩάΞτɺӾཡΛ܁Γฦ͢"main" worker
༧બϕϯνϚʔΧʔ !͕߹ܭ!worker! w=1$>$ $(sta*c=1,$recent=1,$main=6) w=2$>$ $(sta*c=2,$recent=2,$main=14) w=3$>$ $(sta*c=3,$recent=3,$main=28)
༧બϕϯνϚʔΧʔͷʮ݀ʯ recent&worker& • wait&ͳ͠ʹճΔ • දࣔཁૉ͕ߋ৽͞Ε͍ͯΔ͔Ͳ͏͔ͷνΣοΫΛͯ͠ͳ ͍ʂʂʂ /recent/*!ΛΩϟογϡͯ͠ߴʹฦ͢ͱ!είΞ↑↑
༧બͷল είΞࢉग़ͷৄࡉެ։͢͠ աͳʮ᠘ʯΑ͘ͳ͍ ϕϯνϚʔΧʔͷ݀Ͱ͖ΔݶΓ௵͢͠
11/9$ຊબ
ຊબग़ʹͨͬͯ ʮ᠘ʯΊΑ͏ ෳαʔόΛ༗ޮʹ͑Δग़ʹ͠Α͏
ຊબ୲ !@fujiwara!ɺϕϯνϚʔΫɺείΞूܭγεςϜ !@acidlemon!ࣄલղɺϕϯνϚʔΫνΣοΫɺΦʔϓχϯ άɺϨΪϡϨʔγϣϯ !@handlename:!nodeҠ২ɺಈ࡞֬ೝ !@mix3:!GoҠ২ɺಈ࡞֬ೝ !@nobu_ohta:!RubyҠ২ɺPythonҠ২ !@m0t0k1ch1:!PHPҠ২ɺ෧σβΠϯ
ຊબ४උ 10/17%ΩοΫΦϑ 10/29%ग़ͱϕϯνϚʔΧʔϓϩτλΠϓͰղ͖࢝ΊΔ%ଟݴޠҠ ২։࢝ 11/5%5ߏͰղ͘,%ຊ൪༻αʔόࠐΈ 11/7%ຊ൪αʔό্Ͱςετ։࢝ 11/8%݁Ռूܭը໘࡞ɺσόοά 11/9%ISUCON3%ຊબ
ຊબग़༰ ը૾ߘ͕Ͱ͖ΔSNS HTMLΛฦ͢෦ͳ͘ɺJSON)APIͷΈ ϢʔβΛϑΥϩʔͰ͖ΔɻϑΥϩʔ͍ͯ͠ΔϢʔβͷߘ͕ྲྀΕ ͯ͘Δ Long%polling%API%͋Γ ϕϯνϚʔΧʔը૾Λߘͨ͠ΓJSONΛऔಘͨ͠Γ͢Δ
είΞࢉग़ํ๏ Լهͷ!(a)+(b)+(c)!͕૯߹είΞ (a)$ߘεϐʔυ$=$ϨεϙϯελΠϜ͕͍΄ͲߴείΞ (b)$λΠϜϥΠϯө$=$ߘͨ͠ΤϯτϦͷใ͕ө͞Ε Δͷ͕͍΄ͲߴείΞ (c)$ͦͷଞϦΫΤετ$=$1
είΞࢉग़ํ๏ ϨεϙϯελΠϜʹґଘ͢ΔείΞҎԼͷܭࢉࣜʹΑΔ 5"sec"="1 1"sec"="3.3 100"msec"="6.64 10"msec"="9.9
ຊબͷϙΠϯτ 5ͷαʔόΛͲ͏͏͔ ॳظঢ়ଶͰ1ͷΈʹΞϓϦέʔγϣϯ͕ىಈ͍ͯ͠Δ 2ʙ4ʮۭʯ(OSɺϛυϧΣΞͷॳظઃఆͷΈ) ଳҬ1͋ͨΓ • eth0&:&100Mbps&(ϕϯνϚʔΧʔ͕ΞΫηε) • eth1&:&1Gbps&(ࣗ༝ʹ͑Δ)
ຊબͷϙΠϯτ ը૾ͷϦαΠζΛ͍ͭߦ͏͔ ॳظঢ়ଶϦΫΤετ͝ͱʹຖճಈతੜ 1. ߘ࣌ʹ#S,#M,#L#ͷ3αΠζ#+#ΦϦδφϧΛશͯੜ͠อଘ 2. ߘ࣌ΦϦδφϧͷΈอଘ͠ɺαϜωΠϧӾཡ࣌ʹੜ ※!શͯͷαϜωΠϧ͕ϦΫΤετ͞ΕΔΘ͚Ͱͳ͍
ຊબͷϙΠϯτ ը૾ͷϦαΠζΛԿͰߦ͏͔ ॳظ࣮!ImageMagick!convert!ίϚϯυΛ(֎෦)࣮ߦ • ը૾ϥΠϒϥϦʹΑͬͯม݁Ռʹࠩҟ͕ͰΔ • ը૾Λൺֱ͢ΔπʔϧΛఏڙɻϦϑΝϨϯε࣮ͱͷޡ͕ࠩҰ ఆҎԼͰͳ͍ͱfail
ຊબͷϙΠϯτ Long%poll%API%ΛͲ͏ߴԽ͢Δ͔ ॳظ࣮2ඵsleepͯ͠ฦͤΔͷ͕͋Δ͔νΣοΫ • ฏۉϨεϙϯελΠϜ͕1ඵʹͳΔ • ԿΒ͔ͷΠϕϯτ௨Λར༻ͯ͠ߴʹөͰ͖ΔͱߴείΞ
ຊબͷϙΠϯτ ଳҬΛ͍͔ʹຒΊΔ͔ ϕϯνϚʔΧʔ࠷େ!100Mbps! !5!=!500Mbps!Λ͑Δ Ͱ͖Δ͚ͩଟ͘ͷϗετ͔Β৴͢ΔͱଳҬΛ༗ޮʹ͑Δ
ຊબϕϯνϚʔΧʔ Perl%(Furl%+%AnyEvent) post:&ΞΠίϯͱΤϯτϦΛPOSTɺαϜωΠϧΛGETͯ֬͠ೝ view:&ը૾ΛGET&(0.5&sec&sleep) check:&ϢʔβݖݶʹΑΓݟ͑Δݟ͑ͳ͍ͷݖݶνΣοΫ crawl:&·ͣϩάΠϯ͔ͯ͠ΒλΠϜϥΠϯ(Long&poll)ʹுΓ͘ ?meline:&λΠϜϥΠϯʹுΓ͖ɺྲྀΕ͖ͯͨϢʔβΛfollowɺ ը૾ΛGETͳͲ 6"process(post,"view,"check,"crawl"֤×1"5meline"×"2)"/"workload
ຊબϕϯνϚʔΧʔͷଳҬ੍ޚ LVS$Ͱ$localhost:[νʔϜʹରԠ͢Δport]$͔Βϕϯν૬खʹ$NAT ϕϯνϚʔΫϦΫΤετ࣌ʹ!NAT!ઌͷෳϗετΛ!on!/!off QoS$github.com/matsumoto1r/qos1control tc#ͷ#wrapper.#100Mbps#ʹ੍ݶ
ຊબͷল ଟগͷτϥϒϧ͋ͬͨͷͷ͓͓ΉͶָ͠ΜͰ͑ͨ • ϕϯνϚʔΧʔͷෳΓସ͑όά • Pythonॳظ࣮Ͱ·Εʹ(Fail(͢Δ • tmpfile(ͷ(flush(λΠϛϯά ࠷ޙ!Fail!͢ΔνʔϜ͕ଟ͔ͬͨͷ͕೦
None