Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
組織的なインシデント対応を目指して〜成熟度評価と改善のステップ〜 / Towards an O...
Search
Narimichi Takamura
August 03, 2024
Technology
5
4.8k
組織的なインシデント対応を目指して〜成熟度評価と改善のステップ〜 / Towards an Organized Incident Response - Maturity Assessment and Improvement Steps -
SRE NEXT 2024の登壇資料です。
https://sre-next.dev/2024/schedule/#jp110
Narimichi Takamura
August 03, 2024
Tweet
Share
More Decks by Narimichi Takamura
See All by Narimichi Takamura
Waroomの開発モチベーションと今後のロードマップ / Waroom development motivation and roadmap
nari_ex
1
800
Engineering with Business Impact
nari_ex
2
230
How We Foster Reliability in Diversity
nari_ex
14
12k
SRE Practices in Organizations
nari_ex
16
8k
Hardening におけるトラブルシューティング / Troubleshooting in Hardening
nari_ex
1
250
私が Engineering Manager になるまでに経験してきたこと、大切にしてきたこと / Lecture materials for Introduction to Venture Business at UEC
nari_ex
0
170
運用技術者組織の設計と運用 / Design and operation of operational engineer organization
nari_ex
11
8.8k
エンジニアリング組織の基礎知識 / Basic knowledge of engineering organization
nari_ex
10
4.3k
エンジニアリング組織アーキテクチャの調査と設計要点に対する考察 / Investigation of engineering organization architecture and consideration of design points
nari_ex
7
2.7k
Other Decks in Technology
See All in Technology
[RSJ24] Task Success Prediction for Open-Vocabulary Manipulation Based on Multi-Level Aligned Representations
keio_smilab
PRO
0
240
技術ブログや登壇資料を秒で作るコツ伝授します
minorun365
PRO
23
5.2k
AIで変わるテスト自動化:最新ツールの多様なアプローチ/ 20240910 Takahiro Kaneyama
shift_evolve
0
120
不動産売買取引におけるAIの可能性とプロダクトでのAI活用
zabio3
0
170
Mocking in Rust Applications
taiki45
1
170
SAVEPOINT α版
savepoint
0
550
標準最高!標準はださくないぞ! at fukuoka.ts #1
yoiwamoto
0
160
セキュリティ監視の内製化 効率とリスク
mixi_engineers
PRO
7
870
[RSJ24] Object Retrieval in Large-Scale Indoor Environments Using Dense Text with a Multi-Modal Large Language Model
keio_smilab
PRO
0
240
デジタル化・DX推進あるある
y150saya
0
230
Autonomous Database Cloud 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
15
40k
LLM を現場で評価する
asei
4
700
Featured
See All Featured
Scaling GitHub
holman
458
140k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
22
1.7k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
29
2.6k
Automating Front-end Workflow
addyosmani
1365
200k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
28
1.6k
Building Better People: How to give real-time feedback that sticks.
wjessup
359
18k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
88
16k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
663
120k
The Invisible Side of Design
smashingmag
295
50k
VelocityConf: Rendering Performance Case Studies
addyosmani
321
23k
Done Done
chrislema
180
16k
What's new in Ruby 2.0
geeforr
340
31k
Transcript
None
2
גࣜձࣾTopotalʢͱΆͨΔʣ • h#ps:/ /topotal.com • SREΛओ࣠ʹϏδωεΛల։͢Δελ ʔτΞοϓ • 2ࣄۀΛӡӦ •
SRE as a Service • SaaS for SREʢWaroomʣ 3
SRE as a Service • SREʹಛԽٕͨ͠ज़ࢧԉαʔϏε • ࢧԉͷྫ • SLI/SLOͷಋೖɾӡ༻վળ
• CI/CDͷߏஙɾվળ • ΠϯγσϯτϚωδϝϯτͷվળ 4
WaroomʢϫϧʔϜʣ • h#ps:/ /waroom.com • ৫తʹΠϯγσϯτରԠΛߦ͏ͨΊ ͷSaaS • Slack AppϕʔεͰ࡞ΒΕ͓ͯΓɺීஈ
௨ΓରԠ͢Δ͚ͩͰࣗಈԽɾলྗԽ͕ Ͱ͖Δ 5
6
ΠϯγσϯτϨεϙϯεͷվળʹऔΓΉ͜ͱ͕ଟ͍ • ۩ମతʹɺҎԼͷ2ͭͷۀΛ௨ͯؔ͠ΘΓ͕͋Δ • SREaaS SRE: ސ٬ͷΠϯγσϯτϨεϙϯεڥΛվળ͢Δ • Waroom PdM:
ΠϯγσϯτϨεϙϯεSaaSͷػೳΛߟ͑Δ • ͍ͣΕͷ߹ଐਓԽΛղফ͠ɺ৫తʹରԠͰ͖Δମ੍ͮ͘ Γ͕ٻΊΒΕΔ 7
ຊߨԋͷϞνϕʔγϣϯͱ֓ཁ • ৫తͳΠϯγσϯτରԠͷ࣮ʹؔ৺͕͋Δ • ΠϯγσϯτϨεϙϯεΛվળ͢ΔࡍʹཱͭಓඪͷΑ͏ͳͷ Λͭ͘Γ͍ͨ • ͞·͟·ͳاۀͷվળ͕গ͠ͰḿΔ͖͔͚ͬʹͳΕخ͍͠ • →
ख़ϞσϧΛϕʔεʹاۀͷΠϯγσϯτϨεϙϯεڥΛ ධՁ͠ɺஈ֊తʹվળ͢Δख๏Λ͓͠·͢ 8
ΞδΣϯμ 1. ΠϯγσϯτϚωδϝϯτͷվળͷ 2. ΠϯγσϯτରԠख़Ϟσϧͱվળͷεςοϓ 3. ϑΣʔζϚΠάϨʔγϣϯͷϙΠϯτ 9
ΠϯγσϯτϨεϙϯεͷվળ͕ Ή͔͍ͣ͠ 10
՝1: اۀ͝ͱʹղܾࡦ͕ҟͳΔͨΊɺෆ࣮֬ੑ͕ߴ ͍ • اۀ͝ͱʹڥ͕ΘΓͱେ͖͘ҟͳΔ • ex. πʔϧɺϑϩʔɺϙϦγʔ...... • ͞·͟·ͳاۀSREࢧԉΛ͢ΔͨΊɺޮԽ͍͕ͨ͠௫Έॴ͕ͳ͍
• ex. AࣾͰ͏·͍ͬͨ͘ϓϥΫςΟε͕ɺBࣾͰ͏·͍͘͘ͱݶΒͳ͍ • ݁Ռͱͯ͠ɺํײͳ͘ঢ়گΛஅ͠ͳ͕Βվળ͢Δ͜ͱʹ → اۀͷঢ়گͱղܾࡦͷύλʔϯ͕େͰ͋ΓɺΞυϗοΫͳରԠʹͳͬͯ͠·͏ 11
՝2: ϕετϓϥΫςΟεͷಋೖ͕͏·͍͔͘ͳ͍έʔε͕͋Δ • ސ٬ͷ՝ײ • ϫʔΫϑϩʔ͕ఆ·͍ͬͯͳ͍ͷͰɺඋΛͯ͠৫తʹରԠ͍ͨ͠ • վળࡦ • ϕετϓϥΫςΟεʹج͍ͮͨϫʔΫϑϩʔͷಋೖ
• ex. ίϚϯμʔϩʔϧͷಋೖɺSEVͷఆٛͳͲ • ݁Ռ • ϫʔΫϑϩʔ͕ཧղ͞Εͣɺఆண͢Δ·ͰʹఆΑΓଟ͘ͷ͕͔͔࣌ؒͬͨ 12
ϕετϓϥΫςΟεͷྫ1 • ΠϯγσϯτίϚϯμʔ(IC) ɺ্ڃ ཧ৬ͷϝϯόʔͰ͋Δඞཁͳ͘ɺత ͱํੑΛ࣋ͬͯΠϯγσϯτରԠΛਐ ΊΒΕΕ୭ͰΑ͍ • ׂ୲Λ͢Δ͜ͱͰɺ৫త͔ͭޮ తʹରԠ͕Ͱ͖Δ
→ ͞·͟·ͳલఏ͕ͬͯ͡ΊͯޮՌ Λൃش͢ΔɻاۀʹΑͬͯ୯ͳΔΦʔό ʔϔουʹͳΔՄೳੑ͋Δ 1 Incident Management for Opera3ons 13
՝3: ʮ৫తͳରԠʯͷظ͕اۀʹΑͬͯҟͳΔ • ʮ৫తͳΠϯγσϯτରԠʯͱҰݴͰ͍ͬͯɺاۀ͝ͱʹ ཧঢ়ଶ͕ҟͳΔ • ΑΓख़ͨ͠৫Ͱɺ୯ʹෳਓ͕࿈ಈͯ͠ରԠ͢Δ͜ͱͰ ͳ͘ɺਓγεςϜ͕ΑΓޮతʹ࿈ಈ͠ͳ͕ΒରԠͰ͖Δ ମ੍ΛٻΊΔ͕͋Δ •
→ ख़ͨ͠৫Ͱ͋ͬͯཁٻʹݟ߹ͬͨվળΛ͍͖͍ͯͨ͠ 14
3ͭͷʹ͖߹͏ • 1: اۀͷঢ়گͱղܾࡦͷύλʔϯ͕େͰ͋ΓɺΞυϗο ΫͳରԠʹͳͬͯ͠·͏ • 2: पғΛר͖ࠐΉγʔϯͰɺվળ͕ࢥ͏Α͏ʹਐ·ͳ͍͜ ͱ͕͋Δ •
3: ʮ৫తͳରԠʯͷظ͕اۀʹΑͬͯҟͳΔ 15
3ͭͷʹ͖߹͏ • 1: اۀͷঢ়گͱղܾࡦͷύλʔϯ͕େͰ͋ΓɺΞυϗοΫͳରԠʹͳͬͯ͠·͏ • → ؇͔ʹྨ্ͨ͠ͰɺதظతͳվળͷํੑΛࣔͤΔΑ͏ʹͳΓ͍ͨ • ex. ʮࣗͨͪࠓʓʓͱ͍͏ঢ়گͳͷͰɺ□□ͷঢ়ଶΛࢦͯ͠ɺ△△✗✗ʹऔΓΈ·͠ΐ͏ʂʯ
• 2: पғΛר͖ࠐΉγʔϯͰɺվળ͕ࢥ͏Α͏ʹਐ·ͳ͍͜ͱ͕͋Δ • → ৫Λר͖ࠐΈ͘͢͢ΔͨΊʹɺஈ֊తͳվળͷεςοϓΛͭ͘Γ͍ͨ • 3: ʮ৫తͳରԠʯͷظ͕اۀʹΑͬͯҟͳΔ • → ख़ͨ͠اۀ͕ࢦ͢ཧঢ়ଶؚΊͯݴޠԽ͢Δ ্هͷ՝Λղܾ͢ΔͨΊʹɺख़ϞσϧͷߏஙΛ͢Δ͜ͱʹ 16
ख़Ϟσϧͷߏங 17
ख़Ϟσϧͱ2 ৫͕ϓϩηεΛఆΊચ࿅͢ΔͨΊͷख ஈɻҎԼΛఏڙ͢Δɻ • Կ͔Βணख͖͔͢ • ڞ௨ͷݴޠͱɺϏδϣϯͷڞ༗ • ࣮ߦͷ༏ઌॱҐ͚ͮͷΈ •
ࣗͨͪͷ৫ʹͱͬͯվળ͕ҙຯ͢ Δ͜ͱΛ໌֬ʹ͢Δํ๏ 2 ΟΩϖσΟΞ: ೳྗख़Ϟσϧ౷߹ 18
SREͷίϯςΩετΛख़ϞσϧʹऔΓࠐΉ • ΠϯγσϯτϨεϙϯεɺϞχλϦϯάσϓϩΠͳͲͷप ลྖҬͷӨڹΛड͚͍͢ • ৫ʹ͓͚ΔSREͷঢ়گΛͱʹஈ֊తʹఆ͍ٛͨ͠ • → ৴པੑͷϚΠϯυηοτͷਫ४Λ༻͍ͯख़ϨϕϧΛఆٛ ͢Δ
19
ิ: ৴པੑͷϚΠϯυηοτ 3 • ৫ͷ৴པੑΛ5ͭͷجຊతஈ֊ʹ͚ͨͷ • Absent: ৫ʹͱͬͯ৴པੑೋ࣍తͳߟྀࣄ߲ • Reac.ve:
৴པੑͷ / ϦεΫͷରԠ͕࠷ۙͷαʔϏεఀࢭʹ݁ͼ͚ ΒΕɺࢄൃతͳϑΥϩʔ͕ߦΘΕΔɻγεςϜͷͷमਖ਼ʹظతͳ ࢿ͕ߦΘΕΔ͜ͱ΄ͱΜͲͳ͍ɻ • Proac.ve: ఆظతͳ৫ϓϩηεΛ௨ͯ͡જࡏతͳ৴པੑϦεΫ͕ಛఆ͞ Εରॲ͞ΕΔ • Strategic: ͜ͷϨϕϧʹ͋Δ৫ɺΞʔΩςΫνϟɺϓϩμΫτɺϓϩη εΛମܥతʹมߋ͢Δ͜ͱͰϦεΫͷΫϥεΛཧ͢Δ • Visionary: ৴པੑͷ࠷ߴҐʹ౸ୡ͓ͯ͠Γɺ৴པੑͷ෯͍औΓΈΛ ϕετϓϥΫςΟε͓Αͼܦݧʹج͍ͮͯࣾ֎ͰਪਐͰ͖Δʢͨͱ͑ ॻྨͷ࡞ࣝͷڞ༗ͳͲʣ 3 ৫ͷ৴པੑͷϚΠϯυηοτ:Google SRE ͷݟ 20
ิ: ৴པੑͷϚΠϯυηοτ ͱϓϩμΫτͷঢ়ଶ • Absent: ։ൃதͷϓϩμΫτʹͯ·ΔՄೳੑ͕͋ Δ • Reac-ve: ϦϦʔεલ·ͨ҆ఆతظҡ࣋ϑΣʔζ
ͷϓϩμΫτʹͯ·Δ • Proac-ve: ΄ͱΜͲͷϓϩμΫτ͕͜ͷϨϕϧʹ͋Δ ͖ • Strategic: ϏδωεΫϦςΟΧϧͳχʔζΛຬͨͨ͢ Ίʹߴ͍Մ༻ੑΛඞཁͱ͢ΔϓϩμΫτʹͯ·Δ • Visionary: ͜ͷϨϕϧʹ౸ୡ͍ͯ͠ΔϓϩμΫτ΄ ͱΜͲͳ͍ 21
ࢀߟ: ϓϩμΫτͷϑΣʔζͱٻΊΒΕΔ৴པੑͷมԽ 22
ख़Ϩϕϧͷఆٛ ҎԼͷ4ஈ֊ͷఆٛΛߦͬͨ(Visionary֘͢Δέʔε͕গͳ͍ͨΊׂѪ)ɻ • Absent • ΠϯγσϯτϨεϙϯεڥ͕΄΅ະඋͰ͋ΓɺଐਓతͳରԠ͕ৗଶԽ͍ͯ͠Δঢ়ଶ • Reac*ve • ॏେͳোͷରԠํఆ·͍ͬͯΔͷͷɺΠϯγσϯτϨεϙϯεͷڥվળ΄ͱΜͲߦΘΕ͍ͯͳ͍ঢ়ଶ
• Proac*ve • ৫શମͰରԠΛߦ͏ମ੍͕͓ͬͯΓɺPre-IncidentPost-IncidentͷϑΣʔζͷऔΓΈʹΑͬͯࣄલʹϦεΫΛݮ ͍ͯ͠Δঢ়ଶ • Strategic • ͦΕͧΕͷϓϩηε͕ମܥԽɾΈԽ͞Ε͓ͯΓɺϑΟʔυόοΫϧʔϓΛճ͠ͳ͕ΒΠϯγσϯτରԠͷෛ୲Λ࠷খԽ ͠ଓ͚͍ͯΔঢ়ଶ 23
ධՁࢦඪͷࡉԽ • ΠϯγσϯτϨεϙϯεͷϓϩηεଟذʹΘͨΔͨΊɺ֤Ϩϕϧͷఆٛͩ ͚Ͱ࣮༻ੑ͕͍͠ • → ΠϯγσϯτରԠલɺରԠதɺରԠޙͷ3ϑΣʔζ͝ͱʹɺͦΕͧΕ3ͭ ͷϓϩηεΛධՁ͢Δ • Pre-Incident
ϑΣʔζ: ݕɺରԠϑϩʔɺτϨʔχϯά • Response ϑΣʔζ: ݖݶҕৡɺΈԽɺίϥϘϨʔγϣϯ • Post-Incident ϑΣʔζ: ֶशɺੳɺࣄޙλεΫ 24
ΠϯγσϯτϨεϙϯεख़Ϟσϧ 25
26
27
ΠϯγσϯτϨεϙϯεվળͷεςοϓ 1. ख़ϞσϧΛͱʹɺ9ͭͷϓϩηεʹରͯ͠ϨϕϧΛఆ͢Δ 2. 1ΛͱʹɺAbsentʙStrategicͷͲͷ͋ͨΓʹ͕ࣗͨͪҐஔ͍ͯ͠Δ ͔Λ֬ೝ͢Δ 3. ؔऀͱͱʹɺΠϯγσϯτϨεϙϯεͷ͋Δ͖ঢ়ଶΛσΟεΧο γϣϯ͢Δ 4.
վળͷํੑ͕ఆ·ͬͨΒɺ֤ϓϩηε͝ͱʹ۩ମతͳվળͷΞΫγ ϣϯΛఆΊΔ 28
վળͷεςοϓͷ۩ମྫ 1. ख़ϞσϧΛͱʹ9ͭͷϓϩηεʹରͯ͠ධՁΛߦ͏ • ex. Training: AbsentɺDetec5on: Reac5ve...... 2. 1ΛͱʹɺAbsentʙStrategicͷͲͷ͋ͨΓʹ͕ࣗͨͪҐஔ͍ͯ͠Δ͔Λ֬ೝ͢Δ
• ex. 9ͭதେΛΊ͍ͯΔϨϕϧ͋Δ͔Λ֬ೝ͢Δ 3. ؔऀͱͱʹɺΠϯγσϯτϨεϙϯεͷ͋Δ͖ঢ়ଶΛσΟεΧογϣϯ͢Δ • ex. Pre-IncidentϑΣʔζ͕શମతʹ͍͚Ͳվળͨ͠΄͏͕Α͍ͩΖ͏͔ 4. վળͷํੑ͕ఆ·ͬͨΒɺ֤ϓϩηε͝ͱʹ۩ମతͳվળͷΞΫγϣϯΛఆΊΔ • ex. ఆܕλεΫͷࣗಈԽʹऔΓ͏ 29
֘ՕॴΛ৭͚͢Δͱશମײ͕͔ͭΈ͍͢ 30
ϑΣʔζϚΠάϨʔγϣϯͷϙΠϯτ 31
Absent → Reac,ve • վળ֓ཁ • ΫϦςΟΧϧͳোͷϑΥϩʔ͕ਝʹͰ͖ ΔΑ͏ʹͳΓɺ৴པੑ্͕͢Δ • ΩʔϙΠϯτ
• ॏେͳΠϯγσϯτͷΈʹείʔϓΛߜ্ͬͨ ͰɺPre-IncidentϑΣʔζͱPost-IncidentϑΣ ʔζͷ׆ಈΛ෦తʹ͡ΊΔ͜ͱʹྗ͢Δ • ҙ • ݕͷΈ͚ͩΛඋͯ͠ɺରԠϑϩʔ ͕ະఆٛͰࣦഊʹऴΘΔࣄ͕ଟ͍ 32
Reac%ve → Proac%ve • վળ༰ • ΠϯγσϯτϨεϙϯεࣗମͷվળ͕ߦΘΕɺτΠ ϧղফ࠶ൃࢭ͕ਐΉͨΊɺ৫શମͷΠϯγσ ϯτରԠෛՙ͕ܰݮ͞Ε͡ΊΔ •
ΩʔϙΠϯτ • ֤ϓϩηεͷମܥԽͱΈԽΛओ؟ʹ্͓͍ͯ ͰɺιϑτΣΞΤϯδχΞϦϯάΛϕʔεʹվળ ׆ಈΛߦ͏ • ҙ • ৫શମΛר͖ࠐΉࢪࡦ͕૿͑ΔͨΊɺ͖ʹج ͍ͮͯҰؾʹਐΊͨΓͤͣɺ֤ϓϥΫςΟε͝ͱ ʹஈ֊తʹਐΊΔͱΑ͍ 33
Proac&ve → Strategic • վળ༰ • গͳ͍ϦιʔεͰ࠷େݶͷՁ͕ಘΔͨΊʹɺ ͜Ε·Ͱߏஙͨ͠ΈΛ͞ΒʹϒϥογϡΞ οϓ͠ɺΠϯγσϯτͷෛ୲Λ࠷খԽ͢Δ •
ΩʔϙΠϯτ • σʔλυϦϒϯͳվળ͕ϕʔεʹͳΔͨΊɺଞ ͷΩʔϝτϦΫεͱ࿈ܞ͠ͳ͕ΒɺΠϯύΫτ ͷେ͖͍ࢪࡦʹྗ͢Δ • ҙ • ߴͳઐࣝΛඞཁͱ͢Δࢪࡦ͕ଟ͍ͨΊɺ վળ׆ಈࣗମ͕ଐਓԽ͠ͳ͍Α͏ʹҙ͢Δ 34
ख़ϞσϧΛΑΓޮՌతʹ׆༻͢ΔͨΊʹ • ࠓճͷϞσϧΛ͖ͨͨͱͯ͠ɺࣗ৫͚ʹվมͯ͠ར༻͢ Δ • ex. ߲ΛݮΒ͢/૿͢ɺҰஈ֊ͣͭϨϕϧΛͣΒ͢ • ۩ମతͳΞΫγϣϯϓϥϯ͕ఆͰ͖Δ߹ه͢Δ •
৫ͷϚΠϯυηοτΛϑΣʔζϚΠάϨʔγϣϯ͢ΔͨΊʹ ɺΠϯγσϯτϨεϙϯεҎ֎ͷྖҬͷվળॏཁ 35
ҙ: దͳशख़Ϩϕϧͷݕ౼ • ͯ͢ͷ৫͕ Strategic Λࢦ͢ඞཁͳ͍ • ৴པੑͷϚΠϯυηοτಉ༷ɺϓϩμΫτͷεςʔδ৫ͷ ΧϧνϟʔʹΑͬͯɺదͳϨϕϧҟͳΔ •
ex. ϦϦʔεલͷϓϩμΫτ => ৴པੑͷ༏ઌ͕ஶ͍͘͠ ͨΊ Absent Ͱͳ͠ 36
ख़ϞσϧʹΑͬͯಘΒΕͨͷ • ؇͔ʹྨ্ͨ͠ͰɺதظతͳվળͷํੑΛࣔͤΔΑ͏ʹͳΓ͍ͨ • → ख़ϨϕϧΛϕʔεʹඪΛఆΊΔ͜ͱͰɺํੑΛڞ༗͠ͳ͕Βվળ͕ਐΊ ΒΕΔΑ͏ʹͳͬͨ • ৫Λר͖ࠐΈ͘͢͢ΔͨΊʹɺஈ֊తͳվળͷεςοϓΛͭ͘Γ͍ͨ •
→ 9ͭͷϓϩηε͝ͱʹஈ֊తʹਐΊΔ͜ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ • ख़ͨ͠اۀ͕ࢦ͢ཧঢ়ଶؚΊͯݴޠԽ͢Δ • → StrategicͷఆٛʹΑͬͯɺ(ࠓ·ͰΑΓ)ΑΓൃలతͳվળఏҊͰ͖ͦ͏(ະݕূ) 37
·ͱΊ • ΠϯγσϯτϨεϙϯεͷख़ϞσϧΛఏҊ͠·ͨ͠ • ख़ϞσϧΛ׆༻͢Δ͜ͱͰɺϓϩηε୯ҐͰͷվળͪ ΖΜɺํੑΛࣔ͠ͳ͕Βվળ͢Δํ๏Λࣔ͠·ͨ͠ • ख़ϞσϧΛΑΓ࣮ફతʹ͢ΔͨΊʹɺΑΓৄࡉͳυΩϡϝ ϯτͷඞཁੑʹݴٴ͠·ͨ͠ 38
͋Γ͕ͱ͏͍͟͝·ͨ͠ 39