Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
10倍速いNode.js並列プログラミング
Search
shigeru. nakajima
June 29, 2018
Technology
12
4.4k
10倍速いNode.js並列プログラミング
Node.jsのプログラミングを並列化して10倍速く動かした話です。
Node学園 31時限目 のLT
https://nodejs.connpass.com/event/90936/
shigeru. nakajima
June 29, 2018
Tweet
Share
More Decks by shigeru. nakajima
See All by shigeru. nakajima
Introduce dRuby
ledsun
0
460
Watching Ruby in browsers
ledsun
0
150
Using Ruby in the browser is wonderful
ledsun
1
3.8k
Rubyで書いたテトリスをブラウザで動かしてみた
ledsun
0
2.5k
ruby.wasm に関する進捗報告
ledsun
0
1.2k
Hacking Guide of the ruby.wasm
ledsun
0
1.7k
私の作ったruby.wasm アプリケーション
ledsun
0
730
Load gem from browser
ledsun
2
1.9k
パラメタライズドテスト
ledsun
0
640
Other Decks in Technology
See All in Technology
PHP開発者のためのSOLID原則再入門 #phpcon / PHP Conference Japan 2025
shogogg
4
750
Windows 11 で AWS Documentation MCP Server 接続実践/practical-aws-documentation-mcp-server-connection-on-windows-11
emiki
0
970
2025-06-26_Lightning_Talk_for_Lightning_Talks
_hashimo2
2
100
Welcome to the LLM Club
koic
0
170
第9回情シス転職ミートアップ_テックタッチ株式会社
forester3003
0
240
AWS CDK 実践的アプローチ N選 / aws-cdk-practical-approaches
gotok365
6
750
Amazon ECS & AWS Fargate 運用アーキテクチャ2025 / Amazon ECS and AWS Fargate Ops Architecture 2025
iselegant
16
5.6k
Oracle Audit Vault and Database Firewall 20 概要
oracle4engineer
PRO
3
1.7k
HiMoR: Monocular Deformable Gaussian Reconstruction with Hierarchical Motion Representation
spatial_ai_network
0
110
AWS Summit Japan 2025 Community Stage - App workflow automation by AWS Step Functions
matsuihidetoshi
1
270
~宇宙最速~2025年AWS Summit レポート
satodesu
1
1.8k
M3 Expressiveの思想に迫る
chnotchy
0
110
Featured
See All Featured
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Navigating Team Friction
lara
187
15k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.9k
Rebuilding a faster, lazier Slack
samanthasiow
82
9.1k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
281
13k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
YesSQL, Process and Tooling at Scale
rocio
173
14k
The Straight Up "How To Draw Better" Workshop
denniskardys
234
140k
Why You Should Never Use an ORM
jnunemaker
PRO
57
9.4k
Adopting Sorbet at Scale
ufuk
77
9.4k
Agile that works and the tools we love
rasmusluckow
329
21k
Transcript
10ഒ͍Node.jsฒྻϓϩάϥϛϯά Shigeru Nakajima a.k.a. @ledsun Luxiar co., ltd NodeֶԂ LT
2018/06/29
Node εΫϨΠϐϯά͓͡͞Μ ʮnode εΫϨΠϐϯάʯͰάάͬͯ
PANQ panq http://www.panq.jp/ ͱ͍͏αʔϏεΛ࡞ͬ ͍ͯΔ QiitaͷهࣄΛࢀরͰධՁ Qiita APIͰࢀরऔΕͳ͍ εΫϨΠϐϯάͰऔಘ 8000هࣄΛूΊͨ
NodeͱεΫϨΠϐϯά εΫϨΠϐϯά = HTMLμϯϩʔυɹ + ɹHTMLύʔε = Network IOɹɹɹɹɹ+ɹ CPU
work
εΫϨΠϐϯάඇಉظϓϩ άϥϛϯάϞσϧͱΑ͘߹͏ request('http://example.com/', (e, res, body) => { const doc
= libxmljs.parseHtmlString(body) const data = doc.get('(//dl[@class="newsList"])[1]/dt[1]/text()') }) ;ͭ͏ʹॻ͘ͱޮతʹಈ͘
࠶ύʔε • ධՁ߲Λมߋ͍ͨ͠ • 8000htmlϑΝΠϧΛ࠶ύʔε
20min
ͬͱ͍ͨ͘͠
C++ Addon: libxmljs • ύʔεॲཧΛ͘͢Δ • parse5Λlibxmljsʹม͑Δ • 1200s(20min) =>
160s • 7.5ഒ͘ͳͬͨ • parse5: pure js • libxmljs: libxml2 (c) ͷϥούʔ
ͬͱ͍ͨ͘͠
ॲཧ༰ͷੳ • SSDڥͰFileಡΈࠐΈշ • IOͪຆͲແ͍ • CPUॲཧ͕େΛΊΔ • ୯ମͷCPUॲཧΛ͘͢Δͷ͍͠
ฒྻϓϩάϥϛϯά ݱͷύιίϯෳίΞ͕ࡌ͍ͬͯΔ ෳͷίΞͰCPUॲཧΛฒྻʹΒͤΔ => ฒྻϓϩάϥϛϯά
ຊͷ͓ Node.jsϓϩάϥϜΛฒྻԽͯ͠ੑೳΛ্͛Δ
ฒྻԽ • 1ͷϚγϯͷෳͷίΞΛͬͯɺಉ࣌ʹෳ ͷܭࢉΛ࣮ߦ • ෳϚγϯͷѻΘͳ͍
ϓϩηε • ฒྻϓϩάϥϛϯάͱ͍͑εϨου • Node.jsʹεϨουͳ͍ • ϓϩηεΛ͏
child_process.fork() ϫʔΧʔϓϩηεΛཱͯΔ const processes = [] for (var i =
0; i < number; i++) { processes.push(fork(program, [], { stdio: ['ignore', 'ignore', process.stderr, 'ipc'] })) }
֤ϓϩηεʹৼΓ͚Δ // ϑΥϧμͷϑΝΠϧҰཡΛऔಘ const dir = `${process.cwd()}/data/public/cache` const stream =
readdirp({ root: dir, fileFilter: '*.html' }) // ࢠϓϩηεͰύʔε let count = 0 stream.on('data', (data) => { // ϥϯυϩϏϯͰϑΝΠϧΛ count++; processes[count % processes.length].send(data) })
݁Ռɿ80sʂ͖Ε͍ʹ2ഒ
ͬͱ͍ͨ͘͠
Node.jsʹεϨουͳ͍ͱ ݴͬͨͳɺ͋Εӕͩ https://github.com/xk/node-threads-a-gogo • C++ AddonͰωΠςΟϒεϨουΛىಈ
ɿNode 6.x Ͱಈ͘ • Node 6.xͰ͔͠ಈ͔ͳ͍ • Node 6.x ͩͱͦΕ͚ͩͰ30%͙Β͍͍
• node-threads-a-gogo ͷ Node 10ରԠ • 4࣌ؒؤுͬͯఘΊͨ
Node.jsʹεϨουͳ͍ͱ ݴͬͨͳɺ͋Εӕͩʢ̎ʣ https://nodejs.org/api/worker_threads.html • v10.5.0@6/20 Ҏ߱ • ΤΫεϖϦϝϯλϧ • --experimental-worker
• ϑϥάΛ͚ͭΔͱಡΈࠐ·ΕΔϞδϡʔϧ
worker_threads.Worker child_process.fork()ͱେମҰॹ const workers = [] for (var i =
0; i < number; i++) { workers.push(new Worker(program)) }
threadʹৼΓ͚ // ϑΥϧμͷϑΝΠϧҰཡΛऔಘ const dir = `${process.cwd()}/data/public/cache` const stream =
readdirp({ root: dir, fileFilter: '*.html' }) // ϫʔΧʔͰύʔε let count = 0 stream.on('data', (data) => { // ϥϯυϩϏϯͰϑΝΠϧΛ count++; workers[count % workers.length].postMessage(data) })
ɿC++ Addonಈ͔ͳ ͍ • libxmljs͕͑ͳ͍ • 7.5ഒ͕ɺ͘ͳΔ • https://github.com/nodejs/node/issues/ 21481
ͬͱ͍ͨ͘͠
ίΞΛ૿ͤฒྻ • 2ίΞͰ2ഒ͕ݶքͳΒɺͨ͘͞ΜίΞͷ͋ ΔϚγϯΛ͍͍͑ • AWS EC2 c4.8xlarge • 36ίΞ
=> 80s͕2.2sʹ!? • 2.016USD/࣌ؒ
None
c4.8xlarge • ͦͦCPU͕͍͢͝ • 1ฒྻͰ90sʢx1.8ʣ • 12ฒྻͰ8s • ݶքʢ36ίΞΛ͍Εͳ͍ʣ
ϘτϧωοΫෆ໌ • ಡΈࠐΈIO͕ϘτϧωοΫʁ • SSDϚγϯʢi3.4xlargeʣΛ͕ͬͯ มΘΒͳ͍ • ϓϩηεͷݶքʁ • εϨουΛࢼ͔ͨͬͨ͠
ຊͷ·ͱΊ • libxmljsΛͬͯ 7.5ഒ • 2$/hͷϚγϯͰ 1.8ഒ • ฒྻϓϩάϥϛϯάͰ 11.3
ഒ • worker threadsͷҠߦ͕؆୯ ฒྻϓϩάϥϛϯά͓͍͍͠
ฒྻϓϩάϥϛϯά͠Α͏ worker threadsΛָ͠Έʹ͠·͠ΐʔ