Scala and Akka apps on Kubernetes in ChatWork

2017-09-09 Scala関西Summit 2017

1. Scala and Akka app on Kubernetes In ChatWork Scalaؔ੢Summit 2017 2017/09/09
2. ࣗݾ঺հ 2017/09/09 © ChatWork All rights reserved. 2 • ྛɹେհ
 Twitter: @hayasshi_
 GitHub: hayasshi • ChatWorkגࣜձࣾ
 αʔόʔαΠυΤϯδχΞ
 ScalaϓϩμΫτ୲౰ • Scalaྺ4೥͘Β͍
 2013೥Ґ͔Β৮Γ࢝ΊΔ
 ۀ຿ͰΨοπϦ͸2015೥͔Β
3. νϟοτϫʔΫʹ͍ͭͯ © ChatWork All rights reserved. 3 2017/09/09 • ϏδωενϟοταʔϏε
 άϧʔϓνϟοτɾλεΫ؅ཧɾϑΝΠϧڞ༗ɾϏσΦ௨࿩ • ಋೖࣾ਺ 147,000ࣾҎ্ • ྦྷܭϝοηʔδ਺ 20ԯ Ҏ্
 
 (2017೥8݄຤࣌఺)
4. 'BMDPOʹ͍ͭͯ © ChatWork All rights reserved. 4 2017/09/09 • ScalaMatsuri 2017
 ChatWorkͷScala࠾༻ϓϩμΫτ “Falcon” ϦϦʔε·Ͱͷࣦഊͱ੒ޭͷྺ࢙
 https://speakerdeck.com/j5ik2o/history-of-falcon-the-way-to-production-release • AWS Dev Day Tokyo 2017
 ChatWorkͷ৽ϝοηʔδϯάγεςϜΛࢧ͑Δٕज़
 https://speakerdeck.com/j5ik2o/chatworkfalsexin-metusezingusisutemuwozhi-eruji-shu • DataWorks Summit
 https://dataworkssummit.com/san-jose-2017/sessions/worldwide-scalable-and- resilient-messaging-services-by-cqrs-and-event-sourcing-using-akka-kafka-streams- and-hbase/ • Kafka Summit
 https://kafka-summit.org/sessions/worldwide-scalable-resilient-messaging-services- kafka-kafka-streams/
5. ࠓ೔͓͸ͳ͢͠Δ͜ͱ © ChatWork All rights reserved. 5 2017/09/09 • ChatWorkͰͷKubernetesͷར༻ʹ͍ͭͯ
 ͳͥKubernetesΛಋೖͨ͠ͷ͔
 ͲͷΑ͏ʹScalaΞϓϦΛϏϧυͯ͠σϓϩΠ͍ͯ͠Δ͔
 ͲͷΑ͏ͳϝϦοτ͕͔͋ͬͨ • ChatWorkͰͷΞϓϦέʔγϣϯϝτϦΫε
 ͳͥϝτϦΫεΛͱ͍ͬͯΔͷ͔
 ͲͷΑ͏ʹScalaΞϓϦͷϝτϦΫεΛͱ͍ͬͯΔͷ͔
 ͲͷΑ͏ͳϝτϦΫεΛͱ͍ͬͯΔͷ͔
6. ,VCFSOFUFT © ChatWork All rights reserved. 6 2017/09/09
7. ,VCFSOFUFTͱ͸ © ChatWork All rights reserved. 7 2017/09/09 • ίϯςφΦʔέετϨʔγϣϯπʔϧ
 ࣮ߦ؀ڥͰ͋Δϗετ܈Λ·ͱΊ(ΫϥελԽ)
 ίϯςφͷӡ༻؅ཧΛߦͬͯ͘ΕΔ
8. ,VCFSOFUFTͱ͸ © ChatWork All rights reserved. 8 2017/09/09 • ΞϓϦέʔγϣϯ͸”Pod”ͱ͍ΘΕΔ୯ҐͰ؅ཧ͞ΕΔ
 ͍͔ͭ͘ͷΞϓϦͱϘϦϡʔϜΛ·ͱΊΔ • “Deployment”ͱ͍͏࢓૊ΈͰɺಉҰछྨͷ”Pod”ͷ഑උల ։ͷίϯτϩʔϧ΍৑௕ԽΛ؅ཧͰ͖Δ
 →ϫϯίϚϯυͰεέʔϧͰ͖ͯศར
9. ͳͥ,VCFSOFUFTΛಋೖͨ͠ͷ͔ © ChatWork All rights reserved. 9 2017/09/09 • Πϯϑϥ໘ͷ՝୊
 ؀ڥߏங΍σϓϩΠʹ͓͚ΔΠϯϑϥνʔϜͷίετ͕ߴ͍
 ੹຿͕ଟ͘ɺεϐʔσΟʔʹରԠ͕Ͱ͖ͳ͍ঢ়ଶͩͬͨ • ͜ΕΒΛղܾ͢ΔͨΊΞϓϦέʔγϣϯ։ൃऀ͕ΞϓϦͷӡ༻΋ߦ͍΍͘͢͢Δ
 DevOpsͷਪਐ ಋೖલ ಋೖޙ ؀ڥߏங ΠϯϑϥνʔϜ ΠϯϑϥʴΞϓϦ ϦϦʔεɾσϓϩΠ ΠϯϑϥνʔϜ ΞϓϦνʔϜ Πϯϑϥӡ༻ ΠϯϑϥνʔϜ ΠϯϑϥνʔϜ ΞϥʔτରԠ ΠϯϑϥνʔϜ ΠϯϑϥʴΞϓϦ
10. σϓϩΠΠϝʔδ © ChatWork All rights reserved. 10 2017/09/09 • ίϯςφΠϝʔδͷ࡞੒͸ sbt-native-packager • ConcourseCIͰҰ࿈ͷλεΫΛύΠϓϥΠϯԽ • ΞϓϦέʔγϣϯ։ൃऀ͸σϓϩΠͷఆٛΛ࡞੒͠ύΠϓϥΠϯʹઃఆ͢Δ͚ͩ
11. ,VCFSOFUFTಋೖʹΑΔϝϦοτ © ChatWork All rights reserved. 11 2017/09/09 • ΞϓϦέʔγϣϯ։ൃऀ͕ࣗ෼ͨͪͰ
 ΞϓϦέʔγϣϯͷӡ༻؅ཧΛߦ͑ΔΑ͏ʹͳͬͨ • DevOpsͷਪਐʹΑΓগͣͭ͠จԽ͕Ͱ͖͖ͯͨ • ΞϓϦέʔγϣϯͷΦϖϨʔγϣϯ
 ಛʹσϓϩΠ΍εέʔϧΠϯɾΞ΢τ͕ඇৗʹָʹͳͬͨ • KubernetesͷπʔϧͰίϚϯυΛྲྀ͚ͩ͢
12. ΞϓϦέʔγϣϯ
    ϝτϦΫε © ChatWork All rights reserved. 12 2017/09/09
13. ͳͥϝτϦΫεΛऔΔඞཁ͕͋Δͷ͔ © ChatWork All rights reserved. 13 2017/09/09 • ChatWorkͰ͸େ͖͘3ͭͷ໨తͰऔಘ • ҟৗݕ஌ • ো֐࣌ͷݪҼௐࠪ • ΞϓϦέʔγϣϯͷνϡʔχϯά
14. ͳͥϝτϦΫεΛऔΔඞཁ͕͋Δͷ͔ © ChatWork All rights reserved. 14 2017/09/09 • ChatWorkͰ͸େ͖͘3ͭͷ໨తͰऔಘ • ҟৗݕ஌ • ো֐࣌ͷݪҼௐࠪ • ΞϓϦέʔγϣϯͷνϡʔχϯά ֎͔Βݟͯ
 ҙຯͷ͋ΔϝτϦΫε ಺෦ͷϝτϦΫε
15. ͲͷΑ͏ʹϝτϦΫεΛऔ͍ͬͯΔ͔ © ChatWork All rights reserved. 15 2017/09/09 • Kamon
 ΞϓϦ಺෦ͷϝτϦΫεऩू • Datadog
 ϞχλϦϯάαʔϏε • PagerDuty
 Ξϥʔτ௨஌
16. ΞϓϦέʔγϣϯͷ֎͔Βݟͯҙຯͷ͋ΔϝτϦΫε © ChatWork All rights reserved. 16 2017/09/09 • ॲཧ࣌ؒ • ୯Ґ࣌ؒ͋ͨΓͷॲཧྔ(εϧʔϓοτ) • ΤϥʔϨʔτ • ͖͍͠஋Λઃ͚ɺͦΕΛ௒͑ͨ࣌ʹΞϥʔτ͢Δ͜ͱͰɺ
 ҟৗΛૉૣ͘ݕ஌͢Δ͜ͱ͕Ͱ͖Δ • جຊతʹΧελϜϝτϦΫεͱͯ͠ɺܭଌ༻ͷϩδοΫΛ ಺෦ʹ࢓ࠐΉඞཁ͕͋Δ
17. • ҟৗ࣌ͷεϧʔϓοτ • ҟৗ࣌ͷॲཧ࣌ؒ © ChatWork All rights reserved. 17 2017/09/09 ΞϓϦέʔγϣϯͷ֎͔Βݟͯҙຯͷ͋ΔϝτϦΫε
18. • ҰൠతͳઃఆΛ͍ͯ͠Δ • N෼ؒͷฏۉॲཧ͕࣌ؒ˓˓msΛ௒͍͑ͯͨ৔߹ • N෼ؒͣͬͱ࠷େॲཧ͕࣌ؒ˓˓msΛ௒͍͑ͯͨ৔߹ • N෼ؒͷΤϥʔϨʔτ(Τϥʔ਺ / ॲཧ਺)͕̋%Λ௒͑ ͨ৔߹ • ͜ΕҎ֎ʹ΋ϝοηʔδΩϡʔʹ͋Δॲཧ͞Ε͍ͯͳ͍ϝο ηʔδ਺ͳͲʹ΋͖͍͠஋Λ͚ͭͨΓ͍ͯ͠Δ ϝτϦΫεΛ༻͍ͨݕ஌ͷ͖͍͠஋ © ChatWork All rights reserved. 18 2017/09/09
19. ΞϓϦέʔγϣϯ಺෦ͷϝτϦΫε 4DBMB © ChatWork All rights reserved. 19 2017/09/09 • جຊతͳ΋ͷ͕த৺ • CPU࢖༻཰ • ώʔϓ࢖༻ྔ • GCλΠϜɾGCΧ΢ϯτɹͳͲ • ಛʹScala͸୹໋ΦϒδΣΫτ͕ଟ͘ͳΓ΍͍͢ͷͰɺ͔ͬ͠ΓGC Ͱճऩ͞Ε͍ͯΔ͔֬ೝ͢Δ • ্هʹϓϥεͯ͠ɺΞϓϦέʔγϣϯͷಛੑʹΑͬͯɺ
 ݟΔ΂͖ϝτϦΫεΛݕ౼͢Δ
20. ΞϓϦέʔγϣϯ಺෦ͷϝτϦΫε © ChatWork All rights reserved. 20 2017/09/09 • ͳΜ͔࠷ॳྑͦ͞͏͚ͩͲ • ؾ͕͍ͭͨΒ༨༟͕ͳ͘ͳ͍ͬͯͨ • খ͞ͳϝϞϦϦʔΫ͕͋Γ௕ظؒՔಇͰṧഭ
 →JVMͷΫϥεɾώετάϥϜΛग़ྗͯ͠ൃ֮
 →ϞχλϦϯά΋େ੾͕ͩैདྷͷJVM؅ཧπʔϧͰͷ֬ೝ΋ॏཁ
21. ΞϓϦέʔγϣϯ಺෦ͷϝτϦΫε "LLB © ChatWork All rights reserved. 21 2017/09/09 • Akka͸جຊతʹฒߦॲཧ • AkkaͷฒߦॲཧΛ͑͞͞Δ࢓૊ΈͰ͋ΔDispatcher·ΘΓͷ ϝτϦΫεΛ͓͑͞Δ • εϨουϓʔϧ
 →ϓʔϧ಺ͷશͯͷεϨου͕ৗʹΞΫςΟϒʹͳͬͯ ͍ͳ͍͔ • ϝʔϧϘοΫε
 →ಛఆͷDispatcherͷϝʔϧϘοΫεʹϝʔϧ͕଺ཹ͠ ͍ͯͳ͍͔
22. ΞϓϦέʔγϣϯ಺෦ͷϝτϦΫε "LLB © ChatWork All rights reserved. 22 2017/09/09 • Kamon͕ऩूͯ͘͠ΕΔ • εϨουΛ࢖͍੾Δঢ়ଶ͕ଓ͍ͨΓɺActorͷϝʔϧϘοΫεʹϝοηʔδ͕଺ཹ ͢Δ͜ͱ͕ଟ͘ͳΔͱɺνϡʔχϯάͷαΠϯ • blocking-ioΛߦ͏ॲཧ΍ܭࢉͷଟ͍ॲཧͷDispatcherΛผʹ͢ΔͳͲ
23. ·ͱΊ © ChatWork All rights reserved. 23 2017/09/09 • KubernetesΛಋೖͨ͜͠ͱͰ • DevOpsͷจԽ͕গͣͭ͠Ͱ͖͖ͯͨ • ΞϓϦέʔγϣϯͷӡ༻؅ཧָ͕ʹͳͬͨ • ϝτϦΫε͸ • ໨తʹΑͬͯऔಘ͢ΔϝτϦΫεΛݟۃΊΔ • ऩूπʔϧͱΫϥ΢υϞχλϦϯάαʔϏεͰ
 ӡ༻؅ཧ΍ΞϓϦνϡʔχϯάָ͕ʹͳͬͨ
24. 8F
    BSF
    )JSJOH © ChatWork All rights reserved. 24 2017/09/09 ҰॹʹνϟοτϫʔΫΛྑ͍͖͍ͯͨ͘͠ΤϯδχΞืूʂ IUUQDPSQDIBUXPSLDPNKBSFDSVJUFOHJOFFS
25. ͓͠·͍ © ChatWork All rights reserved. 25 2017/09/09