Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Goで作る大量プロセス管理機構
Search
Keita Mohri
July 13, 2019
Technology
2
3.6k
Goで作る大量プロセス管理機構
Go Conference'19 Summer in Fukuoka
Keita Mohri
July 13, 2019
Tweet
Share
More Decks by Keita Mohri
See All by Keita Mohri
Excelを扱うRubyGemまとめ 2022
ktam1219
0
610
モクえもんのお時間です
ktam1219
0
190
在宅ワーク中だけど会社にしかGPSマルチユニットがない?でも大丈夫!そう、mockmockがあればね。
ktam1219
0
410
IoTデバイスの疑似データ送信システムにおける サーバーレスなログ処理機構の試行錯誤
ktam1219
0
590
実写版モクえもん in Explorer ~愛・おぼえていますか~
ktam1219
0
330
エンジニアのおしごと
ktam1219
0
150
mockmockの大量のログをいい感じに捌きたい
ktam1219
0
1.1k
わりとゴツいKubernetesハンズオン そのあとに
ktam1219
0
650
明太子とEndorseと私
ktam1219
0
700
Other Decks in Technology
See All in Technology
AIエージェント就活入門 - MCPが履歴書になる未来
eltociear
0
530
開発と脆弱性と脆弱性診断についての話
su3158
1
1.1k
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
30k
「守る」から「進化させる」セキュリティへ ~AWS re:Inforce 2025参加報告~ / AWS re:Inforce 2025 Participation Report
yuj1osm
1
130
そのコンポーネント、サーバー?クライアント?App Router開発のモヤモヤを可視化する補助輪
makotot
4
570
[OCI Skill Mapping] AWSユーザーのためのOCI(2025年8月20日開催)
oracle4engineer
PRO
2
150
実践アプリケーション設計 ③ドメイン駆動設計
recruitengineers
PRO
3
210
退屈なことはDevinにやらせよう〜〜Devin APIを使ったVisual Regression Testの自動追加〜
kawamataryo
3
680
トヨタ生産方式(TPS)入門
recruitengineers
PRO
3
250
Amazon Bedrock AgentCore でプロモーション用動画生成エージェントを開発する
nasuvitz
6
430
KiroでGameDay開催してみよう(準備編)
yuuuuuuu168
1
130
Go で言うところのアレは TypeScript で言うとコレ / Kyoto.なんか #7
susisu
7
1.8k
Featured
See All Featured
Designing for Performance
lara
610
69k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Code Reviewing Like a Champion
maltzj
525
40k
What's in a price? How to price your products and services
michaelherold
246
12k
Music & Morning Musume
bryan
46
6.7k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.5k
Balancing Empowerment & Direction
lara
2
590
The World Runs on Bad Software
bkeepers
PRO
70
11k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Transcript
(PͰ࡞Δ େྔϓϩηεཧػߏ ໟརܒଠ
‣ 'VTJD$P -UE ‣ ൃҊऀϓϩμΫτΦʔφʔ ‣ *P5ܥͷडୗҊ݅ ‣
(P 3VCZ "84 403"$0. ໟརܒଠʢ͏Γ͚͍ͨʣ
ࣗࣾαʔϏε
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3
Ϋϥυ *P5 'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3 Ϋϥυ *P5
'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ ύʔτφʔ
ࠓ͓͍ͨ͜͠ͱ
ͷલʹʜ
ͬͯͳʹʁ
*P5։ൃऀ͚ ٙࣅσʔλੜαʔϏε
*P5ͷ
Ϟϊ͕ Thing
Πϯλʔωοτܦ༝Ͱ Thing
αʔόʹσʔλΛૹΓ Backend Thing D a t a
ղੳͨ͠Γͯ͠ Backend Thing D a t a
Կ͔͢Δ Backend Thing D a t a
*P5ͷ ͓ΘΓ
*P5 ։ൃ͠·͢
ςετ͍ͨ͠ Backend Thing D a t a
σʔλૹͬͯ΄͍͠ Backend Thing D a t a
Ͱ
Ϟϊʹͩͬͯ ߹͋Δ
͑͐ͬɺ ։ൃʹ͔͔Δͷʂʁ
͑͐ͬɺ ϗϯτʹյ͞ͳ͍ͱ Τϥʔσʔλग़ͤͳͷʂʁ
ෛՙࢼݧ༻ʹ ཉ͍͚͠Ͳʜ ݸສԁʂʁ
ෛՙࢼݧ༻ʹ ༻ҙ͚ͨ͠Ͳʜ ୭͕ૢ࡞͢Δͷ͜Εʂʁ
ςετʹࠔΔ
ͦ͏ͩʂ
γϛϡϨʔλʔΛ࡞Ζ͏ʂ Backend Simulator D a t a
͍ɺ ϦϦʔεʹؒʹ߹Θͳ͍ʂ
͗ͯ͢͠ ʓʓ͞Μ ࡞ͬͨਓ ͔͠ ಈ͔ͤͳ͍ʜ
ٱ͠ͿΓʹಈ͔ͨ͠Β શવಈ͔ͳ͍ʜ
ͦ͜Ͱ
ࢥ͍Ͳ͓ΓʹͳΔϞϊ Backend D a t a
͍ͭͰ ཉ͍͠σʔλΛ ཉ͍͠ྔ͚ͩ ૹͬͯ͘ΕΔʂ
͏Ε͍͠
ը໘Πϝʔδ ࢥ͍௨ΓͷϑΥʔϚοτͰ+40/σʔλΛੜ ԾσόΠεͷεςʔλεͱͦͷมԽΛҙʹઃఆ ͞·͟·ͳ࣌ܥྻσʔλੜػೳ
ͷͬ͘͟Γͨ͠Έ
PublicSubnet PrivateSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3
Bucket (Lot) S3 Bucket (Binary) Web Web Worker Worker Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ PrivateSubnet Mock Mock Mock Mock Mock
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker ίϯιʔϧͰ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock mockىಈ Mock Mock [ mock ] GoόΠφϦ͔Βཱͯͨ ԾσόΠεϓϩηε [ tower ] GoόΠφϦ͔Βཱͯͨ mockཧϓϩηε
ࠓ͓͍ͨ͜͠ͱ վΊͯʂ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ ઃఆϑΝΠϧ όΠφϦΛDL
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ όΠφϦͷ ߋ৽ Ͳ͏͢Δͷʁ
ͳΜͰϓϩηεʁ ͳΜͰ(P
‣ NPDLJT αʔόʔ߽՚͗͢Δʜ ίϯςφएׯ໘ͩ͠ɺίϯςφԽ͢Δඞཁͳ͍ ϓϩηεͪΐ͏Ͳ͍͍ʂ εϨουσόοάେมͦ͏ɺڞΕͦ͠͏
ͳΜͰϓϩηεʁ
‣ αʔόʔʹԿΠϯετʔϧͨ͘͠ͳ͍ ͨͱ͑3VCZͩͱ3VCZΛΠϯετʔϧ͢Δඞཁ͕ʜ ‣ γϯάϧόΠφϦʹ͢Εஔ͚ͩ͘Ͱىಈ͕Ͱ͖Δ ‣ γϯάϧόΠφϦ͕Ͱ͖ΔͭͰָͦ͠͏ͳͷ ‣ ͱ͍͑(Pʂ
ओ؍ ͳΜͰ(Pʁ
‣ ίʔυͷن ςετআ͘ NPDLߦ UPXFSߦ ͪͳΈʹ
େྔNPDLͷૢ࡞
‣ ϑΝΠϧϕʔεͰͷૢ࡞ ಛఆͷσΟϨΫτϦʹϑΝΠϧΛஔ͔ΕͨͷΛݕͯ͠ॲཧΛߦ͏ ૢ࡞͕ඞཁͳ͚ͩϑΝΠϧΛஔ͚͍͍ NPDLͷૢ࡞ NPDLىಈ
NPDLఀࢭ ىಈτϦΨʔ ϑΝΠϧઃஔ tower͕ݕ͠ mockΛىಈ ఀࢭτϦΨʔ ϑΝΠϧઃஔ ֤mock༻ ఀࢭτϦΨʔ ϑΝΠϧઃஔ mock͕ݕ͠ ࣗݾফ໓
‣ ݪ࢝త͚ͩͲγϯϓϧͰ৴པੑߴ͘Γ͍͢ ԿΠϯετʔϧ͠ͳ͍͍ͯ͘ ॲཧ͕ྃͨ͠ޙʹϑΝΠϧΛফ͢Α͏ʹ͓͚ͯ͠ɺ ͠UPXFS్͕தͰ࠶ىಈͯ͠ϑΝΠϧΛݟΕϨδϡʔϜͰ͖Δ ‣ ͦͷଞͷ͔ͬ͜Α͛͞ͳΓํΊΜͲ͍͘͞ؾ͕͢Δ
)551αʔόʔ 1VC4VC2VFVF NPDLͷૢ࡞ ૹ৴ࣦഊͨ͠ͱ͖ͷ࠶ૹʜ ૢ࡞ྃલʹUPXFS͕ࢮΜͩ߹ʜ
‣ https://github.com/fsnotify/fsnotify ‣ ϑΝΠϧมߋݕϥΠϒϥϦ GTOPUJGZ
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ } sleepೖΕͯͪΌΜͱશ෦௨ͯ͘͠ΕΔͷͰ QueueͬΆ͍ײ͡ʹͳΔ
NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid)
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ)
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PIDΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) ऴྃͨ͠ΒprocessExitChannelʹ௨ WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ case pid := <-processExitChannel: mock := mockProcessTable[pid] // ऴྃޙͷ͓ย͚ΖΖ delete(mockProcessTable, pid) } }
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ ఀࢭτϦΨʔϑΝΠϧͱ ಈ࡞தͷϓϩηεΛൺֱ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } }
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର ϓϩηεڧ੍ऴྃ
όΠφϦͷߋ৽
‣ NPDL Քಇதߋ৽ෆཁ ಉ͡ઃఆͰಈ͖ଓ͚ΔͷͰ ىಈ࣌ʹ࠷৽൛ͷόΠφϦΛ࣋ͬͯ͘Ε0, ‣ UPXFS
NPDL͕ՔಇதͰߋ৽͍ͨ͠ ࠶ىಈ چUPXFSΛఀࢭ৽UPXFSΛىಈ Ͱ͖ΔΑ͏ʹ͓ͯ͘͠ NPDLࢹঢ়گͷϨδϡʔϜΛ͕ΜΔʂ όΠφϦͷߋ৽
‣ ࠶ىಈखॱ <چ>UPXFSͷ࠶ىಈτϦΨʔϑΝΠϧͷݕ <چ>τϦΨʔϑΝΠϧʹॻ͔Ε͍ͯΔόʔδϣϯΛ4͔Β%-ͯ͠ىಈ <چ>ऴྃ <৽>NPDLͷࢹঢ়گΛϨδϡʔϜ
‣ ϨδϡʔϜ͠ͳ͍ͱ͍͚ͳ͍ͷ NPDL$PEFͱ1*%Λؔ࿈͚ΔNBQ NPDLϓϩηεΛࢹ͢ΔHPSPVUJOF UPXFSͷ࠶ىಈ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } }
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ ϝΠϯϧʔϓ͔Βൈ͚Δͱͦͷ··ऴྃ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } }
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ লུͯ͠Δ͚Ͳɺ ϓϩηε͕ݟ͔ͭΒͳ͔ͬͨΒPIDϑΝΠϧΛআ
·ͱΊ
‣ ɹɹʹ͓͍ͯɺ(PΛͬͯ େྔͷϓϩηεΛͲͷΑ͏ʹཧ͍ͯ͠Δ͔Λհ ͦͦͳΜͰϓϩηεʁͳΜͰ(P ϑΝΠϧϕʔεͷNPDLૢ࡞ NPDLͷࢮ׆ࢹ
όΠφϦͷߋ৽ ‣ ʮ໌͔Β͑ΔʯΑ͏ͳ༰Ͱͳ͍͚ͣͩͲɺ ָ͠ΜͰΒ͑ͨͳΒ͍Ͱ͢ ‣ (Pͨͷ͍͠ʂ ·ͱΊ
ΛҰॹʹ ։ൃͯ͘͠ΕΔؒΛืूதʂ ͝ڵຯ͋Δํໟར·Ͱʂ Go Ruby AWS IoT ͬͱ͏·͘ΕΔʂ ͓͠Ζͦ͏ʂ @Fukuoka