Event Sourcing+CQRS を活用するスケーラブルアプリケーション開発 / event-sourcing-cqrs-v2

AWS Dev Day 2018
マイクロサービスアーキテクチャの裏側の巻

Transcript

1. Event Sourcing+CQRSΛ׆༻͢Δ εέʔϥϒϧΞϓϦέʔγϣϯ։ൃ yuuki takezawa AWS Dev Day Tokyo 2018
2. Profile • ஛ᖒ ༗و / ytake • גࣜձࣾΞΠελΠϧ CTO • PHP, Hack, Go, Scala • Apache Hadoop, Apache Spark, Apache Kafka
 • twitter https://twitter.com/ex_takezawa • facebook https://www.facebook.com/yuuki.takezawa • github https://github.com/ytake
3. ΞϓϦέʔγϣϯͷ੒௕
4. ΞϓϦέʔγϣϯྫ • ϩάΠϯϢʔβʔ͸ίϝϯτ͕౤ߘͰ͖Δ • ະϩάΠϯϢʔβʔ͸ӾཡͷΈ͕Ͱ͖Δ • ͓ؾʹೖΓͨ͠ϢʔβʔͷίϝϯτΛӾཡͰ͖Δ • ࣌ܥྻͰίϝϯτΛӾཡͰ͖Δ • ίϝϯτͷ࡟আ͕Ͱ͖Δ • ίϝϯτͷฤू͸Ͱ͖ͳ͍ 
 ͳͲ
5. ΞϓϦέʔγϣϯ࣮૷ • ೝূػೳͷ࣮૷
 • ίϝϯτ౤ߘɺฤूɺӾཡػೳ࣮૷
 • ݕࡧػೳͷ࣮૷
6. γϯϓϧͳΞϓϦέʔγϣϯߏ੒
7. ෳࡶԽ͢ΔΞϓϦέʔγϣϯ • ػೳ௥Ճ
 • ίϝϯτ৘ใΛଞͷػೳͰར༻͍ͨ͠
 • ৚݅ʹΑΔҰཡ৘ใͷιʔτมߋ
 Ϣʔβʔ৘ใଐੑʹΑΔιʔτॱมߋ
8. ෳࡶԽ͢ΔΞϓϦέʔγϣϯ
9. ංେԽ͢ΔΞϓϦέʔγϣϯ • ऩӹԽͷͨΊͷػೳ
 • རศੑͷͨΊͷػೳ • طଘػೳͷ֦ு
10. ංେԽ͢ΔΞϓϦέʔγϣϯ
11. ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά • ݁߹ͨ͠ػೳΛίϯϙʔωϯτఏڙ
 • APIʹΑΔσʔλૢ࡞ͷந৅Խ
 • σʔλϕʔεύϑΥʔϚϯεվળ

12. ΞϓϦέʔγϣϯɾσʔλϕʔεϦϑΝΫλϦϯά
13. ڊେͳΞϓϦέʔγϣϯ΁
14. ਂࠁͳ໰୊ • ࢓༷มߋʹ൐͏ίϯϙʔωϯτमਖ਼ͱϦϦʔεෳࡶԽ
 ϥΠϒϥϦͷόʔδϣϯ͋͛ͳ͖Όɾɾʂ
 ͋ͷػೳΛऔΓࠐΉʹ͸͋Εͱ͜Εͱɾɾɾ • ΞϓϦέʔγϣϯؒ࿈ܞͷෳࡶԽ
 APIमਖ਼ʹ൐͏Өڹൣғௐࠪ • σʔλಉظͷෳࡶԽ
 ϦΞϧλΠϜͰσʔλ͕ཉ͍͠
 ΠϯσοΫεߋ৽͕ఆظ࣮ߦ
15. ੒௕͍ͯ͘͠ࣄۀʹ߹Θͤͯɺ ΞϓϦέʔγϣϯΛ Ͳ͏εέʔϧ͍͔ͤͯ͘͞
16. มΘΓΏ͘؀ڥ • σʔλΛऔಘ͢Δʹ͸ʁ
 APIίʔϧʹΑΔ౎౓֬ೝʁ
 Ωϟογϡ͸Կ෼ʁ
 • ͦΕͧΕͷαʔϏεͰͦΕͧΕͷݴޠ΁
 ͋ΔαʔϏε͸PHPɺ͋ΔαʔϏεͰ͸Go ScalaͳͲ

17. σʔλॲཧ Ϣʔβʔ৘ใ͸౎౓໰͍߹Θͤ ৘ใΛߋ৽ͨ͠৔߹Ͱ΋
 ґଘαʔϏε͕ͳ͍ͨΊ໰୊ͳ͠
18. σʔλॲཧͷෳࡶԽ Ϣʔβʔ৘ใมߋ Ϣʔβʔ৘ใ࡟আ Ϣʔβʔ৘ใ͕ݟ͔ͭΒͳ͍ͨΊ
 σʔλॲཧ͝ͱʹ"1*ΞΫηε ͋ͦ͜ͱͦ͜Ͱ৘ใ͕ҧ͏ʂ
 Ͳͷσʔλ͕ຊ෺ʁ શͯͷΞϓϦέʔγϣϯ͔Β
 Ωϟογϡͳ͠ͰΞΫηε
 ৗʹߴෛՙঢ়ଶʹ
19. Event / Application Side
20. Event? • Observerύλʔϯ • ʙͨ࣌͠ʹɺʙ͢Δ Λදݱ͢Δ • ༷ʑͳΞϓϦέʔγϣϯͰར༻͞ΕΔ
21. Event / Application • γεςϜཁ݅ɾػೳͱͯ͠औΓೖΕΔέʔε΋ଟ͍ • Request͕དྷͨΒɺRouteΛىಈͤ͞Δ • ʙͷॲཧ͕׬ྃͨ͠Βɺϩάʹॻ͖ग़͢
22. public function __invoke( FulltextRequest $request ): RedirectResponse { // ొ࿥ॲཧޙʹevent try { $entity = new Entity(1, ‘testing’); $this->dispatcher->dispatch( new SinkConnect($this->usecase->run($entity)) ); } catch(\RuntimeException $e) { // Կ͔errorॲཧ } return $this->redirector->route('fulltext.index'); }
23. ҰͭͷΞϓϦέʔγϣϯͷ৔߹ • ΞϓϦέʔγϣϯ಺ͷ༷ʑͳग़དྷࣄΛ
 ObserverΛ࢖࣮ͬͯ૷͢Δ • ΞϓϦέʔγϣϯͷΫϥε͕ංେԽͨ࣌͠
 খ͞ͳΫϥεʹ෼ׂ͢Δ৔߹΍ɺ
 ϦϑΝΫλϦϯάʹར༻͢Δ͜ͱ΋
 • ୯ҰͷΞϓϦέʔγϣϯ಺Ͱ࣮૷͢Δ৔߹ʹ༗༻
24. ߟྀ͢Δ͜ͱ • ΞϓϦέʔγϣϯؒͰ࿈ܞ͢Δ৔߹͸Ͳ͏͢΂͖͔ • ʙͨ࣌͠ʹɺʙ͢Δ
 ଞͷΞϓϦέʔγϣϯɾݴޠͰ࣮ߦ͞ΕΔ͔
 ੾Γग़͢ඞཁ͕͋Δ͔Ͳ͏͔
25. Event Sourcing
26. Event Sourcing • ΠϕϯτΛه࿥ͯ͠࠶ݱ͢Δύλʔϯ • ΠϕϯτΛΠϕϯτετΞʹอଘ • ه࿥͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩ • ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτ͸ෆม • ൃੜͨ͠ࣄ࣮Λ఻͑Δ
27. Event SourcingΛಋೖ͢΂͖Ͱ͸ͳ͍৔߹ • ΠϕϯτΛه࿥ͯ͠࠶ݱ͢Δඞཁ͕ͳ͍৔߹ • ه࿥͞ΕͨΠϕϯτ͔Β
 ΦϒδΣΫτΛ෮ݩ͢Δ͜ͱ͕ͳ͍৔߹ • খن໛ͳΞϓϦέʔγϣϯɺ
 γϯϓϧͳCRUD͔͠ͳ͍ΞϓϦέʔγϣϯ
28. Event Sourcing • Ͳ͏΍࣮ͬͯ૷͢Δ͔ʁ • RDBMSͱ૊Έ߹ΘͤͯશͯͷΠϕϯτ৘ใΛอ؅ʁ • େن໛ͳσʔλʹ͸ରԠͰ͖Δ͔ʁ • ר͖໭͠ͷॲཧ͸Ͳ͏͢Δ͔ʁ
29. Asynchronous / Synchronous Event
30. ಉظΠϕϯτ • Πϕϯτ͸ಉظॲཧͰ͋Δ΂͖͔Ͳ͏͔
 Πϕϯτൃੜ࣌ʹଈ࠲ʹଞΞϓϦέʔγϣϯʹ൓ө͢Δ ඞཁ͕͋Δ͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋ɺ
 αϒεΫϥΠόʹΑΔσʔλӬଓԽΛߦ͏
31. ඇಉظΠϕϯτ • Πϕϯτ͸ඇಉظʹͰ͖Δ͔Ͳ͏͔
 Πϕϯτ͕ൃੜͨ͜͠ͱΛ୲อ͠ɺ
 ൓ө͸஗Ԇͤͯ͞΋Α͍͔ • ΠϕϯτൃߦɺΠϕϯτཤྺอ࣋·Ͱ
 σʔλӬଓԽ͸஗ԆͰ࣮ߦ͞ΕΔ
32. CQRS
33. ΞϓϦέʔγϣϯͷ੒௕Ͱൃੜ͢Δ໰୊ • σʔλͱύϑΥʔϚϯε • αʔϏε੒௕ͱڞʹ૿͑ΔෳࡶͳΫΤϦͱ༰ྔ • σʔλͷਖ਼͠͞(ਖ਼نԽ)͔ޮ཰Խ͔(ඇਖ਼نԽ) • RDBMSͷෆಘҙ෼໺ + NoSQLʹΑΔΧόʔ
 ಉظํ๏ͷෳࡶԽ
34. σʔλϕʔεઃܭͷ೰Έ • ਖ਼نԽͱநग़͍ͨ͠σʔλͷΪϟοϓ • நग़͍ͨ͠σʔλʹ߹ΘͤͨઃܭΛ͢Δ͜ͱʹΑΔ
 ੍໿͕͔͚ΒΕͳ͍δϨϯϚ • ࣮૷ίʔυͰ୲อ͢ΔͨΊʹෳࡶԽ
35. ཁٻ࢓༷ʹରԠ͢ΔͨΊʹ
 σʔλϕʔεͰશͯରԠ͢Δͷ͸ਖ਼͍͔͠ ਖ਼نԽɺඇਖ਼نԽ͸
 ಉ࣠͡Ͱߟ͑Δ΂͖͔
36. CQRS "A few myths about CQRS". Ouarzy's Blog. 
 http://www.ouarzy.com/2016/10/02/a-few-myths-about-cqrs/ ࢀর
37. CQRSͱ͸Կ͔ • ॻ͖ࠐΈ(෭࡞༻͋Γ)ͱ
 ಡΈࠐΈ(෭࡞༻ͳ͠)Λ෼཭͢Δ • ෭࡞༻͋Γ͸σʔλͳͲʹԿΒ͔ͷมߋΛ༩͑Δ΋ͷ
 • ॻ͖ࠐΈ => Command
 • ಡΈࠐΈ => Query
38. CQRS • ͋ΒΏΔϝιου͸ɺΞΫγϣϯΛ࣮ߦ͢ΔίϚϯυ ͔ɺݺͼग़͠ݩʹσʔλΛฦ͢ΫΤϦ͔ͷ͍ͣΕ͔Ͱ ͋ͬͯɺ྆ํΛߦͬͯ͸ͳΒͳ͍ɻ͜Ε͸ɺ࣭໰Λ͢Δ ͜ͱͰճ౴ΛมԽͤͯ͞͸ͳΒͳ͍ͱ͍͏͜ͱͩɻ
 
 "Greg YoungྲྀCQRS - Mark Nijhof". Digital Romanticism. 
 http://d.hatena.ne.jp/digitalsoul/20100712/1278886009 ࢀর 

39. Command • ੔߹ੑ
 ੍໿Λ͔͚ͯਖ਼͘͠ɺݎ͘ • ॻ͖ࠐΈɺߋ৽ɺ࡟আॲཧ͸
 ಡΈࠐΈʹൺ΂ͯޮ཰ੑ΍଎౓ɺ
 εέʔϥϏϦςΟΛҙࣝ͢Δ͜ͱ͸͋·Γͳ͍
40. Query • ੔߹ੑΑΓ΋ݕࡧޮ཰ • ΄ͱΜͲͷΞϓϦέʔγϣϯʹ͓͍ͯɺ
 ॻ͖ࠐΈΑΓ΋ݕࡧͷൺॏ͕େ͖͍͜ͱ͕΄ͱΜͲ • εέʔϧ͕༰қͰ͋Δ͜ͱɺ
 ߴ଎ͳύϑΥʔϚϯεͰ͋Δ͜ͱ
41. Command, Query෼཭࣌ͷσʔλϕʔε
42. Command
43. Command
44. Query
45. CQRS Application Application
46. ཧ۶͸Θ͔Δ͚Ͳɺ Ͳ͏࣮૷͢Ε͹͍͍ͷ͔ʁ ಉظ͸Ͳ͏͢Δͷʁ
47. Event Sourcing + CQRS
48. Event Sourcing / Command ྫ
49. Event Sourcing / Command ྫ ΞϓϦέʔγϣϯ͔Β Πϕϯτૹ৴
50. Event Sourcing / Command ྫ ϝοηʔδΛड͚औΓɺ ஋ͷՃ޻ͳͲ
51. Event Sourcing / Command ྫ RDBMS΍ɺNoSQLͳͲ ༻్ʹ߹Θͤͯॻ͖ࠐΈ
52. Event Sourcing / Command ྫ ଞͷΞϓϦέʔγϣϯ͔Β σʔλϕʔεͷΈΛࢀর Ճ޻ʹ͍ͭͯͷ஌ࣝ͸஌Βͳ͍
53. Cache System σʔλϕʔεߋ৽
 $PNNBOE σʔλϕʔεߋ৽׬ྃΠϕϯτ
 &WFOU
    4PVSDJOH σʔλϕʔεߋ৽׬ྃݕ஌
 &WFOU
    -JTUFOFS Ωϟογϡ࡟আͳͲ
54. Event Sourcing • CommandɺQueryɺEvent Sourcing͸
 ผΞϓϦέʔγϣϯɾݴޠͰ࣮૷Ͱ͖Δ͘Β͍ͷ
 γϯϓϧ͞ • EventอଘͱΞϓϦέʔγϣϯʹରԠͨ͠
 QueryΛͲ͏΍ͬͯ
 ޮ཰తʹ࣮૷͢Δ͔
55. ΞϓϦέʔγϣϯ෼ׂྫ
56. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ
57. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH etc
58. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH etc ޿ࠂ഑৴ܥAPI ޿ࠂ഑৴։ൃνʔϜ
59. ΞϓϦέʔγϣϯ෼ׂྫ PHP WebΞϓϦέʔγϣϯ webΞϓϦ։ൃνʔϜ ूܭॲཧ push etc CLI։ൃνʔϜ / DWH etc ޿ࠂ഑৴ܥAPI ޿ࠂ഑৴։ൃνʔϜ ϢʔβʔͷӾཡ৘ใʹج͍ͮͨ ৘ใ͕ཉ͍͠Ͱ͢ ϦΞϧλΠϜͰ
60. Event Sourcing / Message Broker
61. Event SourcingͱMiddlewareબఆ • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ • ϝοηʔδอ؅͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷ Λ࠾༻͢Δͷ͕๬·͍͠ • εέʔϥϒϧͰ͋Δ͜ͱ
62. Event Sourcing / RDBMS • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ
 ػೳ͸ͳ͍ͷͰόονॲཧͳͲͰ
 ఆظతʹσʔλ௥Ճ͕͋Δ͔Ͳ͏͔Λݕࡧ͢Δ
 • ϝοηʔδอ؅͕ՄೳͰɺ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷΛ࠾༻͢Δͷ ͕๬·͍͠
 ಘҙ෼໺ • εέʔϥϒϧͰ͋Δ͜ͱ
 Slaveɺmulti masterͳͲͰରԠ͸Ͱ͖Δ
63. Event Sourcing / Redis • PubSubͰشൃੑͷͳ͍΋ͷΛબ୒͢Δ
 PUBLISH SUBSCRIBEͳͲ͕͋ΔͷͰ࣮ݱ͸Ͱ͖Δ͕ɺ
 شൃੑʹ͍ͭͯ͸ઃఆͰؤுΔ
 • ϝοηʔδอ؅͕ՄೳͰɺ
 ࠶ݱ͢Δ͜ͱ͕Մೳͳ΋ͷΛ࠾༻͢Δͷ͕๬·͍͠
 τϥϯβΫγϣϯ͸͋Δ • εέʔϥϒϧͰ͋Δ͜ͱ
 ༰қʹͰ͖Δ
66. Message Broker
68. Apache Kafka • ZookeeperΛར༻ͨ͠ΫϥελϦϯάʹΑΔߴՄ༻ੑ • ϝοηʔδͷӬଓԽɺϨϓϦέʔγϣϯɺ࠶औಘՄ • ϏοάσʔλରԠ • ετϦʔϜରԠͷϝοηʔδϯάϛυϧ΢ΣΞ • Kafka ConnectʹΑΔपลγεςϜͱͷߴ͍਌࿨ੑ
69. Event Sourcing • ΠϕϯτΛΠϕϯτετΞʹอଘ
 Kafkaࣗମʹอଘ͢Δ͜ͱ͕Ͱ͖ɺ
 σʔλϕʔεʹసૹՄೳ • ه࿥͞ΕͨΠϕϯτ͔ΒΦϒδΣΫτΛ෮ݩͰ͖Δ
 ॲཧ͞ΕͨΠϕϯτ͸࠶ൃߦՄೳ • ΠϕϯτʴϩάʴΦϒδΣΫτɺΠϕϯτ͸ෆม
 ൃߦޙͷΠϕϯτΛॻ͖׵͑Δ͜ͱ͸Ͱ͖ͳ͍ • ൃੜͨ͠ࣄ࣮Λ఻͑Δ
 τϥϯβΫγϣϯ͋Γ
70. Apache Kafka֓ཁ • Partition
 ෛՙ෼ࢄ༻్ʹར༻
 ෳ਺ͷConsumer͕ͦΕͧΕͷPartitionΛࢀর͠ɺ
 ͦΕͧΕ͕ॲཧΛߦ͏
 ॲཧϑϩʔͷσβΠϯʹΑͬͯଟ༷ͳར༻ํ๏ • librdkafka͕ඇৗʹߴ଎
71. Example Partition
72. Consumer
73. Kafka Connect • Kafka Connectͱ͸ɺ
 पลγεςϜ͔ΒͷσʔλΛऔΓࠐΈ(Source)ɺ
 σʔλૹ৴(Sink)ͷೋछྨΛαϙʔτ͢Δػೳ • Amazon SQS΍MongoDBͷσʔλΛKafkaͰऔࠐΉɺ
 ϝοηʔδΛͦͷ··Elasticsearch΍RDBMSʹ֨ೲɺ
 ͕ߦ͑Δ • Connect͸ࣗ༝ʹ֦ுͯ͠ಠࣗConnectΛ࣮૷Մೳ
74. Kafka Connect • γϯϓϧͳέʔεͰ͋Ε͹ɺ
 ಛʹ࣮૷͢Δ͜ͱͳ͘QueryΛαϙʔτ • ෳࡶͳσʔλΛ࡞੒͢ΔͷͰ͋Ε͹
 Kafka StreamsɺSpark StreamingͰσʔλՃ޻
76. Amazon Kinesis • εέʔϥϒϧ • ϝοηʔδ 7೔ؒอ؅(࠷େ) • ετϦʔϜରԠͷKinesis Stream • Kinesis FirehoseΛ࢖ͬͨσʔλసૹ
 Amazon S3ɺAmazon Redshiftɺ
 Amazon Elasticsearch Service etc
77. Event Sourcing+ CQRS / Kafka • Command QueryΛ୲౰͢ΔWebΞϓϦέʔγϣϯΛ
 γϯϓϧʹ • Event SourcingʹΑͬͯɺΞϓϦέʔγϣϯʹՁ஋Λ௥ Ճ͢Δ͜ͱ͕Ͱ͖Δ(෼ੳͳͲ) • Command QueryͷಉظΛαϙʔτ͠ɺ
 ΞϓϦέʔγϣϯʹ߹Θͤͯෛՙ෼ࢄ͕༰қ
78. ఆظߋ৽ͤ͞Δόονॲཧͷߟ͑ํ΋มΘΔ
79. ࣮ྫ
80. ྫ
81. ྫ ༷ʑͳΞϓϦέʔγϣϯ͔Β
 σʔλૹ৴
82. ྫ "QBDIF
    ,BGLB͕
 ϝοηʔδड৴
83. ྫ ,BGLB 4QBSL
    4USFBNJOH
    ΞϓϦέʔγϣϯ͔Βૹ৴͞Εͨσʔλͱɺ 3%#.4ʹ֨ೲ͞ΕͨσʔλΛ݁߹͠ɺ
    ूܭɾू໿Λߦ͏
84. ྫ ूܭɾू໿͞ΕͨσʔλΛɺ
 ಡΈࠐΈͰར༻͢ΔΞϓϦέʔγϣϯʹ
 ߹Θͤͯอ؅
    $BTTBOESBͱ4QBSL
    4USFBNJOHͷΈͰ
 ೖग़ྗΛߦ͏έʔε΋
85. ྫ ूܭɾू໿͞Εͨσʔλͷ͏ͪ
    ༷ʑͳՕॴͰར༻͞ΕΔ΋ͷ͸ɺ)%'4΁
    ࠶ܭࢉ΍ɺো֐ൃੜ࣌ʹ෮چͤ͞ΔͳͲ
86. ྫ ΞϓϦέʔγϣϯଆ͔Β͸
    $BTTBOESBͷΈʹ໰͍߹ΘͤΛߦ͏
87. ྫ ͦΕͧΕͷΞϓϦέʔγϣϯͷ
    $POTVNFSͰͦΕͧΕͷॲཧ
88. ྫ 3BCCJU.2΁ϝοηʔδૹ৴
 ଞαʔϏε͕ར༻
89. ྫ ֤αʔϏεͷίϯγϡʔϚʔ͕
    ଞͷαϒγεςϜͰར༻͢ΔσʔλΛੜ੒
90. ·ͱΊ • େن໛ΞϓϦέʔγϣϯʹͳΔ͜ͱ͕༧ଌͰ͖ͨΒ • ϚΠΫϩαʔϏεΞʔΩςΫνϟΛࢧ͑Δ • ϛυϧ΢ΣΞͷ஌ࣝ͸ඞཁʹ • ෼ࢄॲཧ΍ෳࡶͳσʔλॲཧͷ
 ϦϑΝΫλϦϯάʹ΋׆͖Δύλʔϯ