Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Goで作る大量プロセス管理機構
Search
Keita Mohri
July 13, 2019
Technology
2
3k
Goで作る大量プロセス管理機構
Go Conference'19 Summer in Fukuoka
Keita Mohri
July 13, 2019
Tweet
Share
More Decks by Keita Mohri
See All by Keita Mohri
Excelを扱うRubyGemまとめ 2022
ktam1219
0
520
モクえもんのお時間です
ktam1219
0
150
在宅ワーク中だけど会社にしかGPSマルチユニットがない?でも大丈夫!そう、mockmockがあればね。
ktam1219
0
360
IoTデバイスの疑似データ送信システムにおける サーバーレスなログ処理機構の試行錯誤
ktam1219
0
480
実写版モクえもん in Explorer ~愛・おぼえていますか~
ktam1219
0
250
エンジニアのおしごと
ktam1219
0
99
mockmockの大量のログをいい感じに捌きたい
ktam1219
0
870
わりとゴツいKubernetesハンズオン そのあとに
ktam1219
0
520
明太子とEndorseと私
ktam1219
0
580
Other Decks in Technology
See All in Technology
Azureの基本的な権限管理の勉強会
yhana
1
2.1k
Cypress or Playwright?
rainerhahnekamp
0
170
web-application-security
matsuihidetoshi
1
190
AWS学習者向けにAzureの解説スライドを作成した話
handy
3
190
ExaDB-D dbaascli で出来ること
oracle4engineer
PRO
0
2.1k
ワールドカフェI /チューターを改良する / World Café I and Improving the Tutors
ks91
PRO
0
150
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
5
37k
Max out Local LLM in Challenging Environments
sashimimochi
1
120
いいたいことちゃんという
tkengo
0
240
開発パフォーマンスを最大化するための開発体制
ham0215
7
1.1k
Building a RAG-poweredAI chat appwith Python and VS Code
pamelafox
0
160
家族アルバム みてねにおけるGrafana活用術 / Grafana Meetup Japan Vol.1 LT
isaoshimizu
1
1k
Featured
See All Featured
Ruby is Unlike a Banana
tanoku
96
10k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
21
1.4k
No one is an island. Learnings from fostering a developers community.
thoeni
16
2.1k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
660
120k
Building Adaptive Systems
keathley
32
1.9k
Stop Working from a Prison Cell
hatefulcrawdad
267
19k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
79
43k
For a Future-Friendly Web
brad_frost
172
9k
A Tale of Four Properties
chriscoyier
152
22k
Into the Great Unknown - MozCon
thekraken
14
1k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
123
39k
Faster Mobile Websites
deanohume
300
30k
Transcript
(PͰ࡞Δ େྔϓϩηεཧػߏ ໟརܒଠ
‣ 'VTJD$P -UE ‣ ൃҊऀϓϩμΫτΦʔφʔ ‣ *P5ܥͷडୗҊ݅ ‣
(P 3VCZ "84 403"$0. ໟརܒଠʢ͏Γ͚͍ͨʣ
ࣗࣾαʔϏε
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3
Ϋϥυ *P5 'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3 Ϋϥυ *P5
'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ ύʔτφʔ
ࠓ͓͍ͨ͜͠ͱ
ͷલʹʜ
ͬͯͳʹʁ
*P5։ൃऀ͚ ٙࣅσʔλੜαʔϏε
*P5ͷ
Ϟϊ͕ Thing
Πϯλʔωοτܦ༝Ͱ Thing
αʔόʹσʔλΛૹΓ Backend Thing D a t a
ղੳͨ͠Γͯ͠ Backend Thing D a t a
Կ͔͢Δ Backend Thing D a t a
*P5ͷ ͓ΘΓ
*P5 ։ൃ͠·͢
ςετ͍ͨ͠ Backend Thing D a t a
σʔλૹͬͯ΄͍͠ Backend Thing D a t a
Ͱ
Ϟϊʹͩͬͯ ߹͋Δ
͑͐ͬɺ ։ൃʹ͔͔Δͷʂʁ
͑͐ͬɺ ϗϯτʹյ͞ͳ͍ͱ Τϥʔσʔλग़ͤͳͷʂʁ
ෛՙࢼݧ༻ʹ ཉ͍͚͠Ͳʜ ݸສԁʂʁ
ෛՙࢼݧ༻ʹ ༻ҙ͚ͨ͠Ͳʜ ୭͕ૢ࡞͢Δͷ͜Εʂʁ
ςετʹࠔΔ
ͦ͏ͩʂ
γϛϡϨʔλʔΛ࡞Ζ͏ʂ Backend Simulator D a t a
͍ɺ ϦϦʔεʹؒʹ߹Θͳ͍ʂ
͗ͯ͢͠ ʓʓ͞Μ ࡞ͬͨਓ ͔͠ ಈ͔ͤͳ͍ʜ
ٱ͠ͿΓʹಈ͔ͨ͠Β શવಈ͔ͳ͍ʜ
ͦ͜Ͱ
ࢥ͍Ͳ͓ΓʹͳΔϞϊ Backend D a t a
͍ͭͰ ཉ͍͠σʔλΛ ཉ͍͠ྔ͚ͩ ૹͬͯ͘ΕΔʂ
͏Ε͍͠
ը໘Πϝʔδ ࢥ͍௨ΓͷϑΥʔϚοτͰ+40/σʔλΛੜ ԾσόΠεͷεςʔλεͱͦͷมԽΛҙʹઃఆ ͞·͟·ͳ࣌ܥྻσʔλੜػೳ
ͷͬ͘͟Γͨ͠Έ
PublicSubnet PrivateSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3
Bucket (Lot) S3 Bucket (Binary) Web Web Worker Worker Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ PrivateSubnet Mock Mock Mock Mock Mock
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker ίϯιʔϧͰ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock mockىಈ Mock Mock [ mock ] GoόΠφϦ͔Βཱͯͨ ԾσόΠεϓϩηε [ tower ] GoόΠφϦ͔Βཱͯͨ mockཧϓϩηε
ࠓ͓͍ͨ͜͠ͱ վΊͯʂ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ ઃఆϑΝΠϧ όΠφϦΛDL
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ όΠφϦͷ ߋ৽ Ͳ͏͢Δͷʁ
ͳΜͰϓϩηεʁ ͳΜͰ(P
‣ NPDLJT αʔόʔ߽՚͗͢Δʜ ίϯςφएׯ໘ͩ͠ɺίϯςφԽ͢Δඞཁͳ͍ ϓϩηεͪΐ͏Ͳ͍͍ʂ εϨουσόοάେมͦ͏ɺڞΕͦ͠͏
ͳΜͰϓϩηεʁ
‣ αʔόʔʹԿΠϯετʔϧͨ͘͠ͳ͍ ͨͱ͑3VCZͩͱ3VCZΛΠϯετʔϧ͢Δඞཁ͕ʜ ‣ γϯάϧόΠφϦʹ͢Εஔ͚ͩ͘Ͱىಈ͕Ͱ͖Δ ‣ γϯάϧόΠφϦ͕Ͱ͖ΔͭͰָͦ͠͏ͳͷ ‣ ͱ͍͑(Pʂ
ओ؍ ͳΜͰ(Pʁ
‣ ίʔυͷن ςετআ͘ NPDLߦ UPXFSߦ ͪͳΈʹ
େྔNPDLͷૢ࡞
‣ ϑΝΠϧϕʔεͰͷૢ࡞ ಛఆͷσΟϨΫτϦʹϑΝΠϧΛஔ͔ΕͨͷΛݕͯ͠ॲཧΛߦ͏ ૢ࡞͕ඞཁͳ͚ͩϑΝΠϧΛஔ͚͍͍ NPDLͷૢ࡞ NPDLىಈ
NPDLఀࢭ ىಈτϦΨʔ ϑΝΠϧઃஔ tower͕ݕ͠ mockΛىಈ ఀࢭτϦΨʔ ϑΝΠϧઃஔ ֤mock༻ ఀࢭτϦΨʔ ϑΝΠϧઃஔ mock͕ݕ͠ ࣗݾফ໓
‣ ݪ࢝త͚ͩͲγϯϓϧͰ৴པੑߴ͘Γ͍͢ ԿΠϯετʔϧ͠ͳ͍͍ͯ͘ ॲཧ͕ྃͨ͠ޙʹϑΝΠϧΛফ͢Α͏ʹ͓͚ͯ͠ɺ ͠UPXFS్͕தͰ࠶ىಈͯ͠ϑΝΠϧΛݟΕϨδϡʔϜͰ͖Δ ‣ ͦͷଞͷ͔ͬ͜Α͛͞ͳΓํΊΜͲ͍͘͞ؾ͕͢Δ
)551αʔόʔ 1VC4VC2VFVF NPDLͷૢ࡞ ૹ৴ࣦഊͨ͠ͱ͖ͷ࠶ૹʜ ૢ࡞ྃલʹUPXFS͕ࢮΜͩ߹ʜ
‣ https://github.com/fsnotify/fsnotify ‣ ϑΝΠϧมߋݕϥΠϒϥϦ GTOPUJGZ
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ } sleepೖΕͯͪΌΜͱશ෦௨ͯ͘͠ΕΔͷͰ QueueͬΆ͍ײ͡ʹͳΔ
NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid)
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ)
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PIDΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) ऴྃͨ͠ΒprocessExitChannelʹ௨ WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ case pid := <-processExitChannel: mock := mockProcessTable[pid] // ऴྃޙͷ͓ย͚ΖΖ delete(mockProcessTable, pid) } }
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ ఀࢭτϦΨʔϑΝΠϧͱ ಈ࡞தͷϓϩηεΛൺֱ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } }
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର ϓϩηεڧ੍ऴྃ
όΠφϦͷߋ৽
‣ NPDL Քಇதߋ৽ෆཁ ಉ͡ઃఆͰಈ͖ଓ͚ΔͷͰ ىಈ࣌ʹ࠷৽൛ͷόΠφϦΛ࣋ͬͯ͘Ε0, ‣ UPXFS
NPDL͕ՔಇதͰߋ৽͍ͨ͠ ࠶ىಈ چUPXFSΛఀࢭ৽UPXFSΛىಈ Ͱ͖ΔΑ͏ʹ͓ͯ͘͠ NPDLࢹঢ়گͷϨδϡʔϜΛ͕ΜΔʂ όΠφϦͷߋ৽
‣ ࠶ىಈखॱ <چ>UPXFSͷ࠶ىಈτϦΨʔϑΝΠϧͷݕ <چ>τϦΨʔϑΝΠϧʹॻ͔Ε͍ͯΔόʔδϣϯΛ4͔Β%-ͯ͠ىಈ <چ>ऴྃ <৽>NPDLͷࢹঢ়گΛϨδϡʔϜ
‣ ϨδϡʔϜ͠ͳ͍ͱ͍͚ͳ͍ͷ NPDL$PEFͱ1*%Λؔ࿈͚ΔNBQ NPDLϓϩηεΛࢹ͢ΔHPSPVUJOF UPXFSͷ࠶ىಈ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } }
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ ϝΠϯϧʔϓ͔Βൈ͚Δͱͦͷ··ऴྃ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } }
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ লུͯ͠Δ͚Ͳɺ ϓϩηε͕ݟ͔ͭΒͳ͔ͬͨΒPIDϑΝΠϧΛআ
·ͱΊ
‣ ɹɹʹ͓͍ͯɺ(PΛͬͯ େྔͷϓϩηεΛͲͷΑ͏ʹཧ͍ͯ͠Δ͔Λհ ͦͦͳΜͰϓϩηεʁͳΜͰ(P ϑΝΠϧϕʔεͷNPDLૢ࡞ NPDLͷࢮ׆ࢹ
όΠφϦͷߋ৽ ‣ ʮ໌͔Β͑ΔʯΑ͏ͳ༰Ͱͳ͍͚ͣͩͲɺ ָ͠ΜͰΒ͑ͨͳΒ͍Ͱ͢ ‣ (Pͨͷ͍͠ʂ ·ͱΊ
ΛҰॹʹ ։ൃͯ͘͠ΕΔؒΛืूதʂ ͝ڵຯ͋Δํໟར·Ͱʂ Go Ruby AWS IoT ͬͱ͏·͘ΕΔʂ ͓͠Ζͦ͏ʂ @Fukuoka