Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Goで作る大量プロセス管理機構
Search
Keita Mohri
July 13, 2019
Technology
2
3.6k
Goで作る大量プロセス管理機構
Go Conference'19 Summer in Fukuoka
Keita Mohri
July 13, 2019
Tweet
Share
More Decks by Keita Mohri
See All by Keita Mohri
Excelを扱うRubyGemまとめ 2022
ktam1219
0
610
モクえもんのお時間です
ktam1219
0
190
在宅ワーク中だけど会社にしかGPSマルチユニットがない?でも大丈夫!そう、mockmockがあればね。
ktam1219
0
410
IoTデバイスの疑似データ送信システムにおける サーバーレスなログ処理機構の試行錯誤
ktam1219
0
590
実写版モクえもん in Explorer ~愛・おぼえていますか~
ktam1219
0
330
エンジニアのおしごと
ktam1219
0
150
mockmockの大量のログをいい感じに捌きたい
ktam1219
0
1.1k
わりとゴツいKubernetesハンズオン そのあとに
ktam1219
0
650
明太子とEndorseと私
ktam1219
0
700
Other Decks in Technology
See All in Technology
Strands Agents & Bedrock AgentCoreを1分でおさらい
minorun365
PRO
6
250
風が吹けばWHOISが使えなくなる~なぜWHOIS・RDAPはサーバー証明書のメール認証に使えなくなったのか~
orangemorishita
15
5.5k
オブザーバビリティプラットフォーム開発におけるオブザーバビリティとの向き合い / Hatena Engineer Seminar #34 オブザーバビリティの実現と運用編
arthur1
0
350
いかにして命令の入れ替わりについて心配するのをやめ、メモリモデルを愛するようになったか(改)
nullpo_head
5
1.7k
AIエージェントを現場で使う / 2025.08.07 著者陣に聞く!現場で活用するためのAIエージェント実践入門(Findyランチセッション)
smiyawaki0820
6
750
Lambda management with ecspresso and Terraform
ijin
2
150
ロールが細分化された組織でSREと協働するインフラエンジニアは何をするか? / SRE Lounge #18
kossykinto
0
200
Mambaで物体検出 完全に理解した
shirarei24
2
220
相互運用可能な学修歴クレデンシャルに向けた標準技術と国際動向
fujie
0
210
LLM 機能を支える Langfuse / ClickHouse のサーバレス化
yuu26
6
630
✨敗北解法コレクション✨〜Expertだった頃に足りなかった知識と技術〜
nanachi
1
590
LTに影響を受けてテンプレリポジトリを作った話
hol1kgmg
0
320
Featured
See All Featured
Documentation Writing (for coders)
carmenintech
73
5k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
A better future with KSS
kneath
238
17k
Typedesign – Prime Four
hannesfritz
42
2.7k
The Invisible Side of Design
smashingmag
301
51k
Embracing the Ebb and Flow
colly
86
4.8k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.5k
Testing 201, or: Great Expectations
jmmastey
45
7.6k
Automating Front-end Workflow
addyosmani
1370
200k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
21
1.4k
Transcript
(PͰ࡞Δ େྔϓϩηεཧػߏ ໟརܒଠ
‣ 'VTJD$P -UE ‣ ൃҊऀϓϩμΫτΦʔφʔ ‣ *P5ܥͷडୗҊ݅ ‣
(P 3VCZ "84 403"$0. ໟརܒଠʢ͏Γ͚͍ͨʣ
ࣗࣾαʔϏε
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3
Ϋϥυ *P5 'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3 Ϋϥυ *P5
'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ ύʔτφʔ
ࠓ͓͍ͨ͜͠ͱ
ͷલʹʜ
ͬͯͳʹʁ
*P5։ൃऀ͚ ٙࣅσʔλੜαʔϏε
*P5ͷ
Ϟϊ͕ Thing
Πϯλʔωοτܦ༝Ͱ Thing
αʔόʹσʔλΛૹΓ Backend Thing D a t a
ղੳͨ͠Γͯ͠ Backend Thing D a t a
Կ͔͢Δ Backend Thing D a t a
*P5ͷ ͓ΘΓ
*P5 ։ൃ͠·͢
ςετ͍ͨ͠ Backend Thing D a t a
σʔλૹͬͯ΄͍͠ Backend Thing D a t a
Ͱ
Ϟϊʹͩͬͯ ߹͋Δ
͑͐ͬɺ ։ൃʹ͔͔Δͷʂʁ
͑͐ͬɺ ϗϯτʹյ͞ͳ͍ͱ Τϥʔσʔλग़ͤͳͷʂʁ
ෛՙࢼݧ༻ʹ ཉ͍͚͠Ͳʜ ݸສԁʂʁ
ෛՙࢼݧ༻ʹ ༻ҙ͚ͨ͠Ͳʜ ୭͕ૢ࡞͢Δͷ͜Εʂʁ
ςετʹࠔΔ
ͦ͏ͩʂ
γϛϡϨʔλʔΛ࡞Ζ͏ʂ Backend Simulator D a t a
͍ɺ ϦϦʔεʹؒʹ߹Θͳ͍ʂ
͗ͯ͢͠ ʓʓ͞Μ ࡞ͬͨਓ ͔͠ ಈ͔ͤͳ͍ʜ
ٱ͠ͿΓʹಈ͔ͨ͠Β શવಈ͔ͳ͍ʜ
ͦ͜Ͱ
ࢥ͍Ͳ͓ΓʹͳΔϞϊ Backend D a t a
͍ͭͰ ཉ͍͠σʔλΛ ཉ͍͠ྔ͚ͩ ૹͬͯ͘ΕΔʂ
͏Ε͍͠
ը໘Πϝʔδ ࢥ͍௨ΓͷϑΥʔϚοτͰ+40/σʔλΛੜ ԾσόΠεͷεςʔλεͱͦͷมԽΛҙʹઃఆ ͞·͟·ͳ࣌ܥྻσʔλੜػೳ
ͷͬ͘͟Γͨ͠Έ
PublicSubnet PrivateSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3
Bucket (Lot) S3 Bucket (Binary) Web Web Worker Worker Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ PrivateSubnet Mock Mock Mock Mock Mock
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker ίϯιʔϧͰ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock mockىಈ Mock Mock [ mock ] GoόΠφϦ͔Βཱͯͨ ԾσόΠεϓϩηε [ tower ] GoόΠφϦ͔Βཱͯͨ mockཧϓϩηε
ࠓ͓͍ͨ͜͠ͱ վΊͯʂ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ ઃఆϑΝΠϧ όΠφϦΛDL
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ όΠφϦͷ ߋ৽ Ͳ͏͢Δͷʁ
ͳΜͰϓϩηεʁ ͳΜͰ(P
‣ NPDLJT αʔόʔ߽՚͗͢Δʜ ίϯςφएׯ໘ͩ͠ɺίϯςφԽ͢Δඞཁͳ͍ ϓϩηεͪΐ͏Ͳ͍͍ʂ εϨουσόοάେมͦ͏ɺڞΕͦ͠͏
ͳΜͰϓϩηεʁ
‣ αʔόʔʹԿΠϯετʔϧͨ͘͠ͳ͍ ͨͱ͑3VCZͩͱ3VCZΛΠϯετʔϧ͢Δඞཁ͕ʜ ‣ γϯάϧόΠφϦʹ͢Εஔ͚ͩ͘Ͱىಈ͕Ͱ͖Δ ‣ γϯάϧόΠφϦ͕Ͱ͖ΔͭͰָͦ͠͏ͳͷ ‣ ͱ͍͑(Pʂ
ओ؍ ͳΜͰ(Pʁ
‣ ίʔυͷن ςετআ͘ NPDLߦ UPXFSߦ ͪͳΈʹ
େྔNPDLͷૢ࡞
‣ ϑΝΠϧϕʔεͰͷૢ࡞ ಛఆͷσΟϨΫτϦʹϑΝΠϧΛஔ͔ΕͨͷΛݕͯ͠ॲཧΛߦ͏ ૢ࡞͕ඞཁͳ͚ͩϑΝΠϧΛஔ͚͍͍ NPDLͷૢ࡞ NPDLىಈ
NPDLఀࢭ ىಈτϦΨʔ ϑΝΠϧઃஔ tower͕ݕ͠ mockΛىಈ ఀࢭτϦΨʔ ϑΝΠϧઃஔ ֤mock༻ ఀࢭτϦΨʔ ϑΝΠϧઃஔ mock͕ݕ͠ ࣗݾফ໓
‣ ݪ࢝త͚ͩͲγϯϓϧͰ৴པੑߴ͘Γ͍͢ ԿΠϯετʔϧ͠ͳ͍͍ͯ͘ ॲཧ͕ྃͨ͠ޙʹϑΝΠϧΛফ͢Α͏ʹ͓͚ͯ͠ɺ ͠UPXFS్͕தͰ࠶ىಈͯ͠ϑΝΠϧΛݟΕϨδϡʔϜͰ͖Δ ‣ ͦͷଞͷ͔ͬ͜Α͛͞ͳΓํΊΜͲ͍͘͞ؾ͕͢Δ
)551αʔόʔ 1VC4VC2VFVF NPDLͷૢ࡞ ૹ৴ࣦഊͨ͠ͱ͖ͷ࠶ૹʜ ૢ࡞ྃલʹUPXFS͕ࢮΜͩ߹ʜ
‣ https://github.com/fsnotify/fsnotify ‣ ϑΝΠϧมߋݕϥΠϒϥϦ GTOPUJGZ
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ } sleepೖΕͯͪΌΜͱશ෦௨ͯ͘͠ΕΔͷͰ QueueͬΆ͍ײ͡ʹͳΔ
NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid)
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ)
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PIDΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) ऴྃͨ͠ΒprocessExitChannelʹ௨ WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ case pid := <-processExitChannel: mock := mockProcessTable[pid] // ऴྃޙͷ͓ย͚ΖΖ delete(mockProcessTable, pid) } }
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ ఀࢭτϦΨʔϑΝΠϧͱ ಈ࡞தͷϓϩηεΛൺֱ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } }
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର ϓϩηεڧ੍ऴྃ
όΠφϦͷߋ৽
‣ NPDL Քಇதߋ৽ෆཁ ಉ͡ઃఆͰಈ͖ଓ͚ΔͷͰ ىಈ࣌ʹ࠷৽൛ͷόΠφϦΛ࣋ͬͯ͘Ε0, ‣ UPXFS
NPDL͕ՔಇதͰߋ৽͍ͨ͠ ࠶ىಈ چUPXFSΛఀࢭ৽UPXFSΛىಈ Ͱ͖ΔΑ͏ʹ͓ͯ͘͠ NPDLࢹঢ়گͷϨδϡʔϜΛ͕ΜΔʂ όΠφϦͷߋ৽
‣ ࠶ىಈखॱ <چ>UPXFSͷ࠶ىಈτϦΨʔϑΝΠϧͷݕ <چ>τϦΨʔϑΝΠϧʹॻ͔Ε͍ͯΔόʔδϣϯΛ4͔Β%-ͯ͠ىಈ <چ>ऴྃ <৽>NPDLͷࢹঢ়گΛϨδϡʔϜ
‣ ϨδϡʔϜ͠ͳ͍ͱ͍͚ͳ͍ͷ NPDL$PEFͱ1*%Λؔ࿈͚ΔNBQ NPDLϓϩηεΛࢹ͢ΔHPSPVUJOF UPXFSͷ࠶ىಈ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } }
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ ϝΠϯϧʔϓ͔Βൈ͚Δͱͦͷ··ऴྃ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } }
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ লུͯ͠Δ͚Ͳɺ ϓϩηε͕ݟ͔ͭΒͳ͔ͬͨΒPIDϑΝΠϧΛআ
·ͱΊ
‣ ɹɹʹ͓͍ͯɺ(PΛͬͯ େྔͷϓϩηεΛͲͷΑ͏ʹཧ͍ͯ͠Δ͔Λհ ͦͦͳΜͰϓϩηεʁͳΜͰ(P ϑΝΠϧϕʔεͷNPDLૢ࡞ NPDLͷࢮ׆ࢹ
όΠφϦͷߋ৽ ‣ ʮ໌͔Β͑ΔʯΑ͏ͳ༰Ͱͳ͍͚ͣͩͲɺ ָ͠ΜͰΒ͑ͨͳΒ͍Ͱ͢ ‣ (Pͨͷ͍͠ʂ ·ͱΊ
ΛҰॹʹ ։ൃͯ͘͠ΕΔؒΛืूதʂ ͝ڵຯ͋Δํໟར·Ͱʂ Go Ruby AWS IoT ͬͱ͏·͘ΕΔʂ ͓͠Ζͦ͏ʂ @Fukuoka