Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
怖くないメモリ肥大化
Search
Tsukasa OISHI
October 31, 2024
Programming
1
91
怖くないメモリ肥大化
Nihonbashi.js #9のLTで発表しました。
Node.jsでのメモリ肥大化にどう対応するかというお話です。
Tsukasa OISHI
October 31, 2024
Tweet
Share
More Decks by Tsukasa OISHI
See All by Tsukasa OISHI
SLI/SLO をストリーム アラインドチームに導入した話
tsukasa_oishi
0
180
JITをたどるとそこはYARVの中
tsukasa_oishi
0
570
The Cacher in the Rye
tsukasa_oishi
1
5.9k
ISeqで遊ぼう
tsukasa_oishi
0
5.1k
Rubyを30倍速くした話
tsukasa_oishi
0
1.2k
はてブ砲をくらったときのお話
tsukasa_oishi
0
2.1k
食べログで動いている自作ライブラリのお話
tsukasa_oishi
0
310
奥さんとプログラミングを両立させる方法
tsukasa_oishi
0
79
MiyazakiResistanceを作ってみたよ
tsukasa_oishi
0
980
Other Decks in Programming
See All in Programming
AIでLINEスタンプを作ってみた
eycjur
1
230
JSONataを使ってみよう Step Functionsが楽しくなる実践テクニック #devio2025
dafujii
1
530
Performance for Conversion! 分散トレーシングでボトルネックを 特定せよ
inetand
0
890
スケールする組織の実現に向けた インナーソース育成術 - ISGT2025
teamlab
PRO
1
110
複雑なフォームに立ち向かう Next.js の技術選定
macchiitaka
2
160
「待たせ上手」なスケルトンスクリーン、 そのUXの裏側
teamlab
PRO
0
550
The Past, Present, and Future of Enterprise Java with ASF in the Middle
ivargrimstad
0
140
Namespace and Its Future
tagomoris
6
700
はじめてのMaterial3 Expressive
ym223
2
840
Azure SRE Agentで運用は楽になるのか?
kkamegawa
0
2.4k
Compose Multiplatform × AI で作る、次世代アプリ開発支援ツールの設計と実装
thagikura
0
160
テストカバレッジ100%を10年続けて得られた学びと品質
mottyzzz
2
600
Featured
See All Featured
How STYLIGHT went responsive
nonsquared
100
5.8k
It's Worth the Effort
3n
187
28k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.9k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Facilitating Awesome Meetings
lara
55
6.5k
Unsuck your backbone
ammeep
671
58k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
We Have a Design System, Now What?
morganepeng
53
7.8k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
840
Product Roadmaps are Hard
iamctodd
PRO
54
11k
How to train your dragon (web standard)
notwaldorf
96
6.2k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
Transcript
ා͘ͳ͍ ϝϞϦංେԽ
͓͓͍͔ͭ͠͞ !UTVLBTB@PJTIJ
'"5"-&33033FBDIFEIFBQMJNJU"MMPDBUJPOGBJMFE +BWB4DSJQUIFBQPVUPGNFNPSZ
ຊ൪͚ͩى͜ Δ
None
ຊ൪ͰϓϩϑΝ ΠϦϯά͢Δ͠ ͔ͳ͍
IUUQT[FOOEFWVCJF@EFWBSUJDMFTCGBFGF
͍͍ͩͨͷॴΛ ݟ͚ͭΔͷ͕ Ұ൪େมͳͷͰ ͜͜ΛΫϦΞͰ͖ͨΒ ͋ͱॗʑͱΔ͚ͩ
ᶃ
ςετ Λॻ͘
ᶄ
ίʔυ ΛಡΉ
($͕͔͔Βͳ͍ύλʔϯ
const cache: Map<string, any> = new Map(); άϩʔόϧมʹେ͖ͳσʔλঃʑʹ૿͑ΔσʔλΛ࣋ͨͤΔ
($͕͔͔ΔલʹώʔϓΛѹഭ͢Δύλʔϯ
ϧʔϓॲཧͰେྔͳΦϒδΣΫτΛੜ function generateLargeNumberOfObjects() { const objects: { id: number; data:
string }[] = []; for (let i = 0; i < 1000000; i++) { const obj = { id: i, data: ‘aaa….’ }; objects.push(obj); } return objects; }
ΫϩʔδϟʔͰ૿͑ΔσʔλΛอ࣋ function createDataAccumulator() { let data: number[] = []; return
function addData(newData: number): void { data.push(newData); }; } Const accumulateData = createDataAccumulator(); accumulateData(1); accumulateData(2); …
େ͖ͳσʔλΛҰʹѻ͏ const data = fs.readFileSync(filePath, 'utf8');
Կ͔ͷด͡Ε
const connection = mysql.createConnection({…}); connection.query('SELECT * FROM hoge_table’); // connection.end();
ࣗͨͪͷॻ͍ͨίʔυ͕ݪҼͳΒ ͍͍ͩͨίʔυಡΊݟ͔ͭΔ
ࣗͨͪͷॻ͍ͨίʔυͰͳ͍߹ ϓϩϑΝΠϥʔ͕ͳ͍ͱݫ͍͠
ྫ
None
νϟοτͷ ձ͕૿͑Δ ͱΤϥʔ
ɾຊ൪ڥͰͷΈൃੜ͢Δ ɹɾ,T.#ϝϞϦ ɾϩʔΧϧ։ൃڥͰϓϩηεͷϝϞϦݟͯ.#͘Β͍ ɹɾ.BD(#ϝϞϦ '"5"-&33033FBDIFEIFBQMJNJU"MMPDBUJPOGBJMFE +BWB4DSJQUIFBQPVUPGNFNPSZ
($
IFBQPVUPG NFNPSZ
NODE_OPTIONS=—max-old-space-size=250
const chatPrompt = ChatPromptTemplate.fromMessages([ [ "system", "You are a helpful
assistant that translates {input_language} to {output_language}.", ], ["human", "{text}"], ["human", "{text}"], ["human", "{text}"], ["human", "{text}"], ["human", "{text}"], ["human", "{text}"], ["human", "{text}"], ["human", “{text}"], … ["human", “{text}"], ]);
memoryCheck() { const heap = process.memoryUsage(); const msg = [];
for (const key in heap) { msg.push(`${key}: ${Math.round(heap[key] / 1024 / 1024)} MB`); } console.log(msg.join(', ')); }
start rss: 164 MB, heapTotal: 31 MB, heapUsed: 28 MB
end rss: 1137 MB, heapTotal: 999 MB, heapUsed: 959 MB
ίʔυେৎͦ͏
σόοΨʔ
ৄࡉهࣄʹ͋Γ·͢ IUUQT[FOOEFWVCJF@EFWBSUJDMFTGEE
export async function getEncoding( encoding: TiktokenEncoding, options?: { signal?: AbortSignal;
extendedSpecialTokens?: Record<string, number>; } ) { (লུ) return new Tiktoken( await cache[encoding], options?.extendedSpecialTokens ); }
start rss: 110 MB, heapTotal: 58 MB, heapUsed: 34 MB
end rss: 231 MB, heapTotal: 116 MB, heapUsed: 93 MB
None
·ͱΊ
ϓϩϑΝΠϥʔ Λ͑ΔΑ͏ʹ ͓ͯ͘͠
Α͋͘Δ ύλʔϯ Λ͓ͬͯ͘
࠶ݱςετ ͕ॻ͚Ε ҆৺
͓͠·͍