Share
Embed
#33 “Destroying networks for fun (and profit)”
ACM HotNets 2015
https://conferences.sigcomm.org/hotnets/2015/program.html
paper: https://dl.acm.org/doi/10.1145/2834050.2834099
slide: https://tik-db.ee.ethz.ch/file/2a818c8733e21ab762ec0a92b7057c4e/hotnets.pdf
cafenero_777
June 22, 2023
Transcript
20 slides
-
Research Paper Introduction #33 “Destroying networks for fun (and pro
fi t)” ௨ࢉ#92 @cafenero_777 2022/02/10 1 -
Agenda •ରจ •֓ཁͱಡ͏ͱͨ͠ཧ༝ 1. INTRODUCTION 2. MOTIVATION & BACKGROUND 3.
ARCHITECTURE 4. COMPUTING SMART FAILURES 5. PRELIMINARY EVALUATION 6. DISCUSSIONS AND FUTURE WORK 7. CONCLUSION 2 -
ରจ •Destroying networks for fun (and pro fi t) •
Nick Shelly∗§, Brendan Tschaen†, Klaus-Tycho Förster∗, Michael Chang‡, Theophilus Benson†, Laurent Vanbever∗ • ∗ETH Zürich, ‡Princeton University, †Duke University, §Forward Networks • ACM HotNets 2015 • paper: https://dl.acm.org/doi/10.1145/2834050.2834099 • slide: https://tik-db.ee.ethz.ch/ fi le/2a818c8733e21ab762ec0a92b7057c4e/hotnets.pdf 3 -
֓ཁͱಡ͏ͱͨ͠ཧ༝ •֓ཁ • SDNʢNW੍ޚιϑτʣ͕NWରোੑΛอূ͢Δ͕͋Δɺ͚Ͳ࣮ࡍ͍͠ • Failure injection framework (Armageddon)ͷհ •
ޮతͳނোγφϦΦɾΞϧΰϦζϜΛ༻͍ͯɺ3ճͷγφϦΦ࣮ߦͰlinkোͷ80%ΛΧόʔ •ಡ͏ͱͨ͠ཧ༝ • աڈʹಡΜͩChaos EngineeringͳจͰ݁ߏҾ༻͞Ε͍ͯͨͷͰ • TitleʹҾ͔Εͯ (fun -> ?) • ͨ·ʹӡ༻ͬΆ͍ͷ (ࢥͬͨΑΓӡ༻ͬΆ͘ͳ͔ͬͨ) • ͪΐͬͱݹ͍ʁʢSDNίϯτϩʔϥͷઃܭͱʙɺSDNͷ৴པੑͱʙʣ 4 -
Net fl ixͱC.E.ͷྺ࢙ ͱɺ͜Ε·Ͱհͨ͠ͷ • 2008ʹAWSશ໘ҠߦʢCDNҎ֎ʣ • Chaos Monkey: VMࢭΊΔ
• 2012ΫϦεϚεΠϒʹ୯ҰϦʔδϣϯͰେো • Chaos Kong: RegionࢭΊΔ • ϥϯμϜʹյ͢͜ͱ=CE, ͱ͍͏ޡղ͕ൃੜ • 2015: Destroying networks for fun (and pro fi t) • 2016: ChAP (ChaosAutomation Platform), ϚΠΫϩαʔϏεʹಋೖ • 2017: PRINCIPLES OF CHAOS ENGINEERINGΛެ։ • 2019: Automating chaos experiments in production • 2022: ??? Chaos engineering: Building con fi dence in system behavior through experiments C.E.ͷํ๏ͳ͕த৺ ʢ۩ମతͳྫΛݟͳ͕Βʣ ࣮ࡍʹγεςϜΛ࡞ͬͯɺ C.E.ӡ༻ͨ͠ ࠓճͷ -
Chaos as a Service •AWS FIS (Fault Injection Simulator) 6
•Azure Chaos Studio https://aws.amazon.com/jp/ fi s/ 2021/03/15- https://azure.microsoft.com/ja-jp/services/chaos-studio/ 2021/11/13- (preview൛) -
1. INTRODUCTION •Net fl ixͷChaos Monkey • ຊ൪ڥͷVMαʔϏεΛແ࡞ҝʹࡴ͢ɻ2012͚ͩͰ65k VMΛఀࢭɻόάͷൃݟͱमਖ਼ɻোؾ͔ͮΕͣɻ •DC,
WAN, SDNނো͢Δ 7 •ϗϫΠτϘοΫεతͳΞϓϩʔν • ։ൃڥͰϞσϧݕࠪɺγϯϘϦοΫ࣮ߦɺxςετͳͲ • ຊ൪ڥɾHWͰͷΈൃੜ͢Δόά͋ΔʢϨʔείϯσΟγϣϯͳͲʣ •ϒϥοΫϘοΫεతͳΞϓϩʔν • ߹ཧతͳʢى͖ͦ͏ͳɺ֬ೝͨ͠΄͏͕ྑͦ͞͏ͳʣোΛ࣮ࡍʹࢼ͢ • ΧόʔʢશͯͷΈ߹ΘͤͰ࣮ߦɺ֤ϦιʔεͰ1ճ͝ͱʣ • োࢼߦճʢ෮ճɺΠςϨʔγϣϯʣ࠷ݶʹ͢Δ • SLAຬͨͤΔʁ->ී௨طʹԽ͞Ε͍ͯΔɻʢΤϯδχΞ͕͍Δ࣌ʹʣதଳ͕ྑ͍ •Armageddon • E2EͷଓੑΛอ࣋ͨ͠··ɺোγφϦΦΛΞϧΰϦζϜͰੜ • NWૄ௨ʹݟ͔ͭΔͱςετதࢭ͠ՕॴͷϨϙʔτੜ • ࢼߦճͱނোΧόʔͷධՁ -
2. MOTIVATION & BACKGROUND •NWͷϞσϧԽγϛϡϨʔγϣϯͷݶք • Ϟσϧݕࠪɺ*testͰݶքɻi.e. ಉ࣮࣌ߦόάɺOpenFlow༷४ڌʢnot࣮४ڌʣɺ·ͨͦͷΑ͏ͳςετϑϨʔϜϫʔΫ •ෆมྔͷνΣοΫ •
ϦΞΫςΟϒΞϓϩʔν: C-planeग़ྗD-plane࣮ߦ݁Ռ͔ΒόάΛݕ • ΞΫςΟϒʹόάΛݕ: ArmageddonͰ࣮ࡍʹfailͤ͞Δ •োͷݕͱ෮چ • ઌߦݚڀ: con fi gνΣοΫʢԾڥΛ͏ʣɺbugճආख๏ʢԾճઢΛ͏ʣɺDCࣗಈ෮چख๏ • Armageddon࣮ࡍͷNWશମΛͬͯݕɻʢԾԽ͕ෆཁɺD-planeͷ؆ૉԽʣ • Armageddon͖ͳλΠϛϯάͰӡ༻ɾ࣮ߦͰ͖ΔɻඞཁͳΒ෮چπʔϧΛ͏ 8 -
3. ARCHITECTURE •Inducing failures • failuresΛSDNܦ༝Ͱ (OpenFlowϝοηʔδͰ)հ࣮͠ߦ •Monitoring for correctness
• fl owঢ়ଶมԽΛݕޙɺinvariant (ෆมྔ)νΣοΧʔ͕ಈ࡞ • NetPlumberͰFlowͷঢ়ଶΛೝ •Generating reports and restoring correctness • োγφϦΦͱτϨʔε݁ՌΛใࠂ • ࣗಈͰ͠ɻΒͳ͍߹ରγεςϜվम͕ѱ͍ͱߟ͑Δ •Dealing with concurrent physical failures • োݕͨ͠Β͙͢ʹఀࢭɾ෮چϑϩʔʹೖΔɻ • ཧϦιʔεނোݕ࣌ͦͦ߈ܸ࣮ߦ͞Εͳ͍ 9 https://www.usenix.org/conference/ons2014/technical-sessions/presentation/al-shabibi ೖྗ τϙϩδͱෆมྔ ෆมྔΛҡ࣋͠ͳ͕ΒোςετΛ࣮ߦ fl owհ࣮ߦ (active/passive) -
4. COMPUTING SMART FAILURES (1/4) •Coverage • ϦϯΫোΛى͜͠ɺE2E reachability͕͋Δ͔ •
SDNίϯτϩʔϥ͕ਖ਼͘͠ಈ࡞͢Δ͔ʁ • શͯͷϦϯΫΛॱ൪ʹམͱ͢ʁ •Coverage + Speed • ࠷2ճࣗ໌ɺ࠷ߴnճ͔͔Δɻ͋ΔτϙϩδͰͷ࠷খճʁ 10 -
4. COMPUTING SMART FAILURES (2/4) ΞϧΰϦζϜ͓͞Β͍ •άϥϑϊʔυʂ •࠷খશҬʢMST: Minimum Spanning
TreeʣΛٻΊΔΞϧΰϦζϜ • ϓϦϜ๏ • ΫϥεΧϧ๏ • https://algo-logic.info/kruskal-mst/ 11 -
4. COMPUTING SMART FAILURES (3/4) ࠓճղ͖͍ͨ = “graph Jenga” •࠷Ұຊܨ͕͍ͬͯΔNWʢ=
Spanning TreeʣΛҡ࣋͠ɺͦΕҎ֎ΛΔ •Ͱ͖Δ͚ͩগͳ͍෮ճͰɺશͯͷϦϯΫΛগͳ͘ͱҰΓ͍ͨ 12 ײతͳྫ ࠓճͷΞϧΰϦζϜ (Greedy Killer Algorithm) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 vertex (node) edge (link) 1 1 1 1 1 1 1 1 1 1 1 1 1 -
4. COMPUTING SMART FAILURES (4/4) •ΞϧΰϦζϜ • MST + α
•ܭࢉྔ • O (ve log v) • e: edge, link • v: vertex, node 13 ͠T1, T2͕ඃΒͣʹղͰ͖Ε E1ো࣮ߦ͠ɺE2ো࣮ߦͯ͠ऴΘΓ ʢ࠷খճͷ߹ʣ શͯͷedge (link)ͷॏΈΛ1ͱ͢Δ MSTͳT(V, E’)Λ࡞Δ E’’ (= E not E’)ʹো࣮ߦ োରlinkΛΧϯτ E’’ͳlinkͷॏΈΛ0ʹ͢Δ োର͕ͳ͍ or શͯͷॏΈ͕0ͳΒऴྃ -
5. PRELIMINARY EVALUATION •σʔληοτ • Internet Topology Zoo͔Β261ݸͷτϙϩδɺRocketFuelτϙϩδ͔Β7ݸ • େنISP~தنΩϟϯύεNWΛఆ
• ߹ܭ265τϙϩδɻαΠζ36@50%ile, 962@Max •ΞϧΰϦζϜͷ࣮ߦ݁Ռ • 60%5ճҎԼͰ࣮ߦՄೳɺ91%8ճҎԼͰ࣮ߦՄೳ • 28ճඞཁͳέʔε -> ੑ͕͍ɻ͋ΔछͷϦϯάτϙϩδ •ΞϧΰϦζϜͷ࠷ద͞ • 52% (138/265)࠷దʢ࠷খͷ෮ճʣͩͬͨ 14 -
6. DISCUSSIONS AND FUTURE WORK •ͬͱޮతʹ࣮ߦ͍ͨ͠ •ΧόϨοδಛੑͷར༻ • ಛఆॱংͷlink downͷΈൃੜ͢Δোͷ߹ͳͲɻϞσϧͷ׆༻ɻ
• υϝΠϯࣝʢτϥϑΟοΫύλʔϯͳͲʣ͔ΒγφϦΦΛݮΒ͢ •ߴͳಛੑͷอ࣋ • bandwidthอ࣋ͳͲΛߟྀ • ΞϧΰϦζϜվྑʢෆมྔͷΈ߹ΘͤʣͰͰ͖ͦ͏ 15 -
4. COMPUTING SMART FAILURES (3/4) ࠓճղ͖͍ͨ = “graph Jenga” •࠷Ұຊܨ͕͍ͬͯΔNWʢ=
Spanning TreeʣΛҡ࣋͠ɺͦΕҎ֎ΛΔ •Ͱ͖Δ͚ͩগͳ͍෮ճͰɺશͯͷϦϯΫΛগͳ͘ͱҰΓ͍ͨ 20 ײతͳྫ ࠓճͷΞϧΰϦζϜ (Greedy Killer Algorithm) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 1 1 1 0 0 vertex (node) edge (link)