Lagom: Microservices "Just Right"

Opinionated Microservices to Crush the Monolith

Transcript

1. Lagom: Microservices "Just Right" Opinionated Microservices to Crush the Monolith Brendan McAdams Reactive Architecture Lead, Lightbend @rit JavaDay Kyiv 2016 1
2. “Lagom” /lah-gome/ — Swedish: “Just right; neither too much or too little.” — Build microservices that are just the right size. JavaDay Kyiv 2016 2
3. Lagom is Opinionated Lagom is designed around an opinionated view of how a good Microservice should be built. JavaDay Kyiv 2016 3
4. Sometimes Strong Opinions Can Be Bad... JavaDay Kyiv 2016 4
5. ... But Well Informed Opinions Lead To Good Software — These opinions are there to simplify our development cycle... — Strong patterns guide our code into the right shape. — Lock into the right path, and build great systems. — The core tenets and opinions of Lagom are outlined in “Reactive Microservices Architecture” – a short, free ebook by Lightbend's co-Founder & CTO, Jonas Bonér. JavaDay Kyiv 2016 5
6. “Microservices Come In Systems” JavaDay Kyiv 2016 6
7. “Microservices Come In Systems” — A typical application will be made up of more than one microservice–a system. — Many existing frameworks make individual microservices easy, but systems hard. — Lagom aims to let us build great microservices which compose into great systems. JavaDay Kyiv 2016 7
8. “Building Microservices Shouldn't Hurt” JavaDay Kyiv 2016 8
9. “Building Microservices Shouldn't Hurt” — It was very important, from the beginning, that Lagom provide a great development environment. — It shouldn't take a mountain of scripts to boot dev instances of our system. — A new developer should be able to jump right in, setup without any special scripts, and start coding. — Lagom is just that. Complete with hot reloading of services when code changes. JavaDay Kyiv 2016 9
10. There's one more important opinion... JavaDay Kyiv 2016 10
11. JavaDay Kyiv 2016 11
12. Reactive Principles Matter Reactive Systems Are: — Responsive: They respond in a timely manner. — Resilient: They stay responsive in the face of failure. — Elastic: They can remain responsive under varying workloads. — Message Driven: They have a strong message driven foundation... they react to events. JavaDay Kyiv 2016 12
13. Lagom As A Reactive Framework Lagom is built from the ground up to help you build true Reactive Systems. Lagom is Asynchronous by default, from the I/O layer up. - Akka Streams for async streaming. - Java 8 CompletionStage for async computation. - Interservice communication via streaming. - Websocket support builtin, supporting streaming & async messaging. JavaDay Kyiv 2016 13
14. Lagom As A Reactive Framework Lagom is built from the ground up to help you build true Reactive Systems. Lagom is distributed by default, using builtin clustering technology. JavaDay Kyiv 2016 14
15. The Best Tools Build The Best Framework Lagom is built with strong, proven technologies to make sure you succeed. — A Java 8+ API* leveraging Lambdas provides a powerful, fluid API for building Lagom services. — Cassandra acts as the default database for Lagom's Persistence module– the development environment even comes with an embedded server. — The popular Jackson library provides a default implementation for JSON Serialization & Deserialization. * A Scala API is forthcoming JavaDay Kyiv 2016 15
16. The Best Tools Build The Best Framework — Play Framework: Lightbend's Netty based web framework powers the HTTP Engine of Lagom. — Guice dependency injection works hand-in-hand with Play to simplify configuration and composition. JavaDay Kyiv 2016 16
17. The Best Tools Build The Best Framework — Akka: Lightbend's battle tested Actor framework, is the foundation of Lagom services. — Clustering, to easily scale across multiple servers.§ — Persistence, for Event Sourcing. — Streams, for strong asynchronous interactions. — Kafka acts as a message broker for interservice information sharing. Like Cassandra, it is built into the development environment. § Subscribers to the Lightbend Reactive Platform can access additional features with ConductR and the Split Brain Resolver JavaDay Kyiv 2016 17
18. lagom's Service API JavaDay Kyiv 2016 18
19. Service Descriptors In Lagom, we describe our services as interfaces; these are known as Service Descriptors. Service Descriptors define the invocation and implementation, as well as metadata on how to call and serve them. These descriptors are transport agnostic–they are equally usable with REST, Websockets... or any other transport layer. JavaDay Kyiv 2016 19
20. import com.lightbend.lagom.javadsl.api.*; import static com.lightbend.lagom.javadsl.api.Service.*; public interface HelloService extends Service { ServiceCall<String, String> sayHello(); default Descriptor descriptor() { return named("hello").withCalls( call(this::sayHello) ); } } — The name of the descriptor - HelloService - is used by services to for discovery. — We have defined a single service call: sayHello — With REST, Lagom maps this to a POST request on the path /sayHello, with a Content-Type: text/plain for both request and response bodies. JavaDay Kyiv 2016 20
21. ServiceCall<NotUsed, Order> getOrder(long id); default Descriptor descriptor() { return named("orders").withCalls( pathCall("/order/:id", this::getOrder) ); } It is also possible to accept path parameters; Lagom will attempt to deserialize the appropriate type from the path argument. JavaDay Kyiv 2016 21
22. ServiceCall<NotUsed, PSequence<Item>> getItems(long orderId, int pageNo, int pageSize); default Descriptor descriptor() { return named("orders").withCalls( pathCall("/order/:orderId/items?pageNo&pageSize", this::getItems) ); } There is querystring support, too! JavaDay Kyiv 2016 22
23. Service Calls interface ServiceCall<Request, Response> { CompletionStage<Response> invoke(Request request); } — Our ServiceCall is invoked when consuming a service. — Request: type of incoming request message (e.g., String). — Response: type of outgoing response message (e.g., String). — By default, JSON is used as the serialization format for request/ response messages. — There are two kinds of request/response messages–Strict, and Streamed. JavaDay Kyiv 2016 23
24. Strict Messages — Strict messages are represented by simple Java objects. — The messages are buffered into memory, and then parsed from the appropriate source, such as JSON. — If both Request and Response are strict, it is considered to be a synchronous call. JavaDay Kyiv 2016 24
25. Streamed Messages ServiceCall<String, Source<String, ?>> tick(int interval); default Descriptor descriptor() { return named("clock").withCalls( pathCall("/tick/:interval", this::tick) ); } — Streamed messages represent an Akka Stream, and are represented by the Source type; they allow truly asynchronous handling of messages. — In this example, we have a strict request, and a streamed response. In this case, we will stream out a String at the specified interval. — Streamed service calls automatically select a Websocket transport protocol. JavaDay Kyiv 2016 25
26. Implementing Services import static java.util.concurrent.CompletableFuture.completedFuture; public class HelloServiceImpl implements HelloService { public ServiceCall<String, String> sayHello() { return name -> completedFuture("Hello " + name); } } — To implement our service, we provide an implementation of the descriptor interface. — Note that this example is implemented using a Lambda, meaning the returned result is not immediately executed; this allows for easier call composition. JavaDay Kyiv 2016 26
27. Lagom's Persistence API JavaDay Kyiv 2016 27
28. Persistence in Lagom Microservices should own and isolate their data. - Only a service has direct access to its DB. - Other services must access the DB through the service's protocol. - Since we've already admitted to being opinionated: The best way to do this is Event Sourcing and CQRS... JavaDay Kyiv 2016 28
29. Event Sourcing With Event Sourcing, all state changes are captured as events– which are treated as immutable records. — Rather than having advanced Object-Relational mapping, events are captured directly to an event log (AKA a journal) — This event stream can be analyzed to derive useful information, building new read views without complicating writes. — Excellent write performance due to append-only writes. — Easy testing and debugging–simulation is often trivial. JavaDay Kyiv 2016 29
30. CQRS CQRS, or Command Query Responsibility Segregation, separates our domain, treating write operations distinctly from (most) read operations. Because of this, it pairs nicely with Event Sourcing. — Writes are always targed to a specifically identified entity; we can do reads on specific entities in the same code as well. — Reads that require more complex views, such as looking at multiple entities, are built as separate sections of code. JavaDay Kyiv 2016 30
31. Event Sourcing & CQRS Together We can derive a lot of value from this pairing, especially given the appending log of events. - Time traveling (playback of events, such as rebuilding state) & auditing become trivial. - Say goodbye to the tyranny of ORMs, and never suffer a database migration script. - Performance, Scalability, Testing, and Debugging all benefit from the event logged model. JavaDay Kyiv 2016 31
32. Writing with Event Sourcing We create our own Command and Event classes, and subclass PersistentEntity - Define Command and Event handlers. - Entities can be access from anywhere in a cluster. - Conceptually, this corresponds to an Aggregate Root in Domain Driven Design (DDD) JavaDay Kyiv 2016 32
33. PersistentEntity A PersistentEntity has a stable identifier, which is used to access it from service implementations (and other places). - Interaction with a PersistentEntity is accomplished by sending it Commands. - Commands are processed sequentially, one at a time (per entity instance). - Depending on our code a Command may result in a state change, which can be persisted as events (which represent the command's effect). - NOTE: The current state is not stored for every change; it can be derived from the replay of the events. - The event log is append only, with no mutation. This yields high throughput and write efficiency. JavaDay Kyiv 2016 33
34. Individual Entity Instances — Entities are automatically distributed (sharded) across cluster nodes. — Entities are kept alive, holding their state in memory for as long as it is used. It may eventually timeout and automatically be passivated. JavaDay Kyiv 2016 34
35. Using the Persistence API The Persistence API is not enabled by default in a Lagom project; we will need to add a dependency. SBT: libraryDependencies += lagomJavadslPersistence Maven: <dependency> <groupId>com.lightbend.lagom</groupId> <artifactId>lagom-javadsl-persistence_2.11</artifactId> <version>${lagom.version}</version> </dependency> JavaDay Kyiv 2016 35
36. Using the Persistence API The default, and recommended (remember that whole opinionated thing?) datastore is Cassandra. Being highly scalable and distributed, Cassandra is a perfect fit for Lagom's model. JavaDay Kyiv 2016 36
37. Implementing PersistentEntity If we start with a light stub, a PersistentEntity might look like such: import com.lightbend.lagom.javadsl.persistence.PersistentEntity; public class Post1 extends PersistentEntity<BlogCommand, BlogEvent, BlogState> { @Override public Behavior initialBehavior(Optional<BlogState> snapshotState) { BehaviorBuilder b = newBehaviorBuilder( snapshotState.orElse(BlogState.EMPTY)); // TODO define command and event handlers return b.build(); } } JavaDay Kyiv 2016 37
38. Implementing PersistentEntity There are three type parameters in our class. public class Post1 extends PersistentEntity<BlogCommand, BlogEvent, BlogState> { — Command - the base interface for a command. We implemented it as BlogCommand. — Event - the base interface for an event. We implemented it as BlogEvent — State - the base interface which represents the current state of the entity. We've implemented this as BlogState JavaDay Kyiv 2016 38
39. Implementing PersistentEntity initialBehavior is an abstract method which we must implement. It returns a Behavior, representing how our entity handles Commands. The Lagom API provides a mutable builder via newBehaviorBuilder to simplify this. @Override public Behavior initialBehavior(Optional<BlogState> snapshotState) { JavaDay Kyiv 2016 39
40. Handling Commands Using setCommandHandler (where b is an instance of BehaviorBuilder), we can register command handlers. b.setCommandHandler(AddPost.class, (AddPost cmd, CommandContext<AddPostDone> ctx) -> { final PostAdded postAdded = PostAdded.builder().content(cmd.getContent()).postId(entityId()).build(); return ctx.thenPersist(postAdded, (PostAdded evt) -> // After persist is done additional side effects can be performed ctx.reply(AddPostDone.of(entityId()))); }); — Command handlers are invoked for incoming messages, and must return events to be persisted (if any) JavaDay Kyiv 2016 40
41. Issuing Persist Events Our command handler must return an instance of a Persist directive. These define which events (if any) to persist to the event log. — thenPersist will persist a single event — thenPersistAll will persist several events atomically (i.e., either all events are stored, or none if an error occurs) — done indicates no events are to be persisted. — It is possible for us to perform side effects after successful persistence, using afterPersist JavaDay Kyiv 2016 41
42. Validating Commands Although optional, it is possible (and often advisable) to validate a command before persisting any state change. ctx.invalidCommand or ctx.commandFailed will reject a command. b.setCommandHandler(AddPost.class, (AddPost cmd, CommandContext<AddPostDone> ctx) -> { if (cmd.getContent().getTitle() == null || cmd.getContent().getTitle().equals("")) { ctx.invalidCommand("Title must be defined"); return ctx.done(); } JavaDay Kyiv 2016 42
43. You Do Not HAVE To Change State A command is not required to effect a state change. A common example of this is a query command, which would instead return the current state to the caller. To do so, we register a setReadOnlyCommandHandler, and invoke the reply method on the context. b.setReadOnlyCommandHandler(GetPost.class, (cmd, ctx) -> ctx.reply(state().getContent().get())); NOTE: Commands must be immutable to avoid the perils of concurrency. We don't want someone mutating our command instance from outside! JavaDay Kyiv 2016 43
44. What To Do When Persistence Succeeds Once an event has successfully persisted, we update the current state by applying the event. To do so, we register a handler with setEventHandler on the BehaviorBuilder b.setEventHandler(PostAdded.class, evt -> state().withContent(Optional.of(evt.getContent()))); — This handler returns the new state; our state must be immutable (that pesky concurrency thing again). — The current state is always accessible from within our event handler via PersistentEntity's state method. — We should define one event handler for each event class our entity persists. JavaDay Kyiv 2016 44
45. A Few More Things... — Every command handler must define what type of message to send as a reply - the class must implement the PersistentEntity.ReplyType interface. — It is possible to change the behavior of an Entity; useful to implement a Finite State Machine. — Snapshots allow you to "roll up" your events to a single point. JavaDay Kyiv 2016 45
46. Reading with Event Sourcing With a typical Lagom setup, our Read side is often tightly integrated with Cassandra. To start with, we will subclass CassandraReadSideProcessor, which consumes any events produced by our PersistentEntity, and updates tables in Cassandra which are optimized for our intended queries. — Retrieving data can then be easily accomplished through Cassandra's Query Language, e.g., SELECT id, title FROM postsummary JavaDay Kyiv 2016 46
47. Running Lagom in Production — The sbt-native-packager build plugin can be used to produce zip, MSI, RPM, DEB, or Docker files. — Lightbend ConductR[^†] (our container orchestration tool) can simplify & enhance coordination of cluster setups. — In addition to ConductR, the Lightbend Reactive Platform[^†] includes: — Split Brain Resolver (for Akka Clustering) — Lightbend Monitoring [^†] The Reactive Platform requires a subscription, but is free to use during development. JavaDay Kyiv 2016 47
48. What's In The Future — We've recently added integration for Maven (a more common tool for Java devs than sbt), and Kafka — In the near future, we will add a Scala API — Enhanced support for writing integration tests — Work is being done on support for other cluster orchestration tools — Lagom is open source, and we would love contributions to our Kubernetes integration project! https://github.com/ huntc/kubernetes-lib JavaDay Kyiv 2016 48
49. Thanks! Any questions? brendan.mcadams@lightbend.com @rit JavaDay Kyiv 2016 49