Full Stack Reactive

This talk is based on the learning series I’ve published with IBM Developer called Reactive in Practice. I’ve been collaborating with IBM for months to bring these learnings to anyone interested in applying the reactive, event sourcing, and CQRS patterns to real world enterprise systems. My background is mostly in banking and finance, so all of the event sourcing and CQRS examples are framed around IBMs classic Stock Trader. The series rethinks that classic application as a reactive system.

Learn more about Reactive in Practice here: https://kevinwebber.ca/blog/full-stack-reactive-june-19/

1. Full Stack Reactive Let's see some real-world CQRS and Event Sourcing! — Kevin Webber — Indie Consultant, ex Lightbend/Typesafe w: kevinwebber.ca m: medium.com/@kvnwbbr t: @kvnwbbr
2. Why? — React, Vue, and other front-end frameworks are enabling innovative user experiences
3. Why? — React, Vue, and other front-end frameworks are enabling innovative user experiences — Meanwhile, relational databases continue to dominate the server side
4. Why? — React, Vue, and other front-end frameworks are enabling innovative user experiences — Meanwhile, relational databases continue to dominate the server side — Reactive principles and frameworks help create an end-to-end event-driven, message-driven channel, all the way up and all the way down
5. Traditional architecture — Unbounded in complexity and side effects — Putting React or Vue on top won't help much — Batch-mode, high-latency, high-failure rates — Has an impact — Customer happiness — The ability to hire the best — Visit TheDailyWTF
6. What we will cover 1. Raw ingredients → Commands and Events 2. Structure and modelling → Domain Driven Design 3. Show me the code → Java, Lagom, Play, Akka
7. Events are — Interesting things, that have — happened in the past.
8. Systems have been event driven for centuries!
9. Event sourcing — The one thing missing from event storming models is a root entity — Commands are applied against an entity, the entity can accept or reject a command — If accepted, an entity will generate an event and then apply that event in order to change state
10. Defining aggregate boundaries — Structures implementation around related items — The yellow sticky in the middle represents state
11. Defining bounded contexts — Group together related business concepts — Contain domain complexity — A single person can master an aggregate — A single team can master a bounded context
12. Reactive Stock Trader — Reference application for Reactive in Practice — Vue, Lagom, Play, Akka, Cassandra, Kafka, Kubernetes — Bounded contexts — Portfolio — Broker — Wire transfers — https://github.com/RedElastic/reactive-stock-trader
14. Interface architecture — Backend for Frontend (BFF) is a pattern that involves creating a 1:1 relationship between backends and UIs — The BFF then handles all calls to various underlying microservices (Portfolio, Broker, Wires, etc) — Also performs authentication, authorization, etc
15. CQRS for services — Play framework serves as BFF — Adapts and routes all requests to Lagom microservices — Lagom is a framework built on top of Akka for developing reactive microservices — Based on the principles of CQRS (Command Query Responsibility Segregation)
16. Placing a wire transfer We'll cover the command channel first. 1. Create command 2. Send over REST 3. Direct command to entity 4. Create event 5. Change state 6. Publish event
18. Submit transfer request (Vue.js) onSubmit() { this.submitted = true; submitTransfer(this.form) .then(() => { this.submitted = false; this.onReset(); }); },
19. Play (command side) @Override public ServiceCall<Transfer, TransferId> transferFunds() { TransferId transferId = TransferId.newId(); return transfer -> transferRepository .get(transferId) // 1 (get reference to entity) .ask(TransferCommand.TransferFunds.builder() .source(transfer.getSourceAccount()) .destination(transfer.getDestinationAccount()) .amount(transfer.getFunds()) .build() ) // 2 (ask pattern) .thenApply(done -> transferId); }
20. Lagom (command side) — Based around persistent entities — Backed by Cassandra as the event log public class TransferEntity extends PersistentEntity<TransferCommand, TransferEvent, Optional<TransferState>> { // ... }
21. Lagom (event sourcing) — No mutable data, only immutable state — State is only changed on successful event application — Events are journaled — Entity can always recover in- memory by replaying events from the journal — Entities are not queried directly, read-side queries are backed by views
22. Lagom (event sourcing) private Behavior empty() { BehaviorBuilder builder = newBehaviorBuilder(Optional.empty()); builder.setCommandHandler(TransferCommand.TransferFunds.class, (cmd, ctx) -> { // 1 (handler is invoked from ask pattern) TransferDetails transferDetails = TransferDetails.builder() .source(cmd.getSource()) .destination(cmd.getDestination()) .amount(cmd.getAmount()) .build(); // 2 (build up details of xfer) ObjectMapper mapper = new ObjectMapper(); TransferUpdated tc = buildTransferUpdated( transferDetails, "Transfer Initiated"); publishedTopic.publish( mapper.valueToTree(tc).toString()); // 3 (publish to stream) return ctx.thenPersist( new TransferEvent .TransferInitiated( getTransferId(), transferDetails), evt -> ctx.reply(Done.getInstance()) ); // 4 (persist event & return ID) }); builder.setEventHandlerChangingBehavior( TransferEvent.TransferInitiated.class, this::fundsRequested); // 5 (change state) return builder.build(); }
23. Lagom architecture
24. Streaming 1. Render initial page with precomputed view over HTTP/ REST 2. Switch to unidirectional streaming for updates (events over WS) 3. Commands over REST will still cause full page refreshes (can change unidirectional stream to bidi stream in future)
25. Streaming (Lagom) This code exposes a Reactive Streams Source via Lagom, for Play to attach to. @Override public ServiceCall<NotUsed, Source<String, ?>> transferStream() { return request -> { // subscribe to events on a specific topic ("transfer") final PubSubRef<String> topic = pubSub.refFor( TopicId.of(String.class, "transfer")); // return the Source as a future (standard async Java 8) return CompletableFuture.completedFuture(topic.subscriber()); }; }
26. Streaming (architecture) Pub-sub works by broadcasting events to subscribers. — Publisher is TransferEntity — Subscriber is WireTransferServiceImpl — This will create a streaming Source
27. Play Streaming Create a materialized graph all the way down.
28. WireTransferController public WebSocket ws() { return WebSocket.Text.acceptOrResult(req -> { return wireTransferService .transferStream() .invoke() .thenApply(source -> { return F.Either.Right( Flow.fromSinkAndSource( Sink.ignore(), source ) ); }); }); }
29. WebSockets (Vue.js) connect() { this.socket = new WebSocket( "ws://localhost:9000/api/transfer/stream"); this.socket.onopen = () => { this.socket.onmessage = (e) => { let event = JSON.parse(e.data); var index = -1; // 1. determine if we're updating a row (initiated) // or adding a new row (completed) for (var i = 0; i < this.transfers.length; i++) { if (this.transfers[i].id === event.id) { index = i; break; } } if (index === -1) { // unshift is similar to push, but prepends this.transfers.unshift({ // ... 3. create object with id, status, etc }); } else { let t = { // ... 4. create object with id, status, etc }; this.transfers.splice(index, 1, t); this.updateCashOnHand(); } }; }; }
30. What about views? — Read side processors build a precomputed table — Queries are only against precomputed tables — Use this to populate data on initial page load
31. We need to build the data pump using Lagom read side processors.
32. Lagom ReadSide processor Update a precomputed query table (Cassandra) on every new event we subscribe to. @Override public ReadSideHandler<PortfolioEvent> buildHandler() { return readSide.<PortfolioEvent>builder("portfolio_offset") // 1 .setGlobalPrepare(this::prepareCreateTables) // 2 .setPrepare(tag -> prepareWritePortfolios()) // 3 .setEventHandler(Opened.class, this::processPortfolioChanged) // 4 .build(); } private CompletionStage<Done> prepareWritePortfolios() { return session .prepare("INSERT INTO portfolio_summary (portfolioId, name) VALUES (?, ?)") .thenApply(ps -> { this.writePortfolios = ps; return Done.getInstance(); }); }
33. Queries are then executed against the precomputed table @Override public ServiceCall<NotUsed, PSequence<PortfolioSummary>> getAllPortfolios() { return request -> { CompletionStage<PSequence<PortfolioSummary>> result = db.selectAll("SELECT portfolioId, name FROM portfolio_summary;") .thenApply(rows -> { List<PortfolioSummary> summary = rows.stream().map(row -> PortfolioSummary.builder() .portfolioId(new PortfolioId( row.getString("portfolioId"))) .name(row.getString("name")) .build()) .collect(Collectors.toList()); return TreePVector.from(summary); }); return result; }; }
34. Building the read-side
35. Read side queries
36. Conclusion — CQRS separates writes and reads for reliability and performance — Event sourcing eliminates the mutability of relational databases — Operationally Lagom is cloud- native and ready to deploy to AWS, Azure, GCP, etc, via Kubernetes
37. Reactive in Practice For a complete look at this material, visit IBM Developer at developer.ibm.com and check out Reactive in Practice, a 12 part series. — Twitter: @kvnwbbr — Web: kevinwebber.ca — Medium: medium.com/ @kvnwbbr