Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Revitalizing Aging Architectures (Gartner Catal...

Revitalizing Aging Architectures (Gartner Catalyst)

We take a look into the Walmart Canada case study and how Scala, Play, and Akka was leveraged to modernize a heritage architecture. We then dive into the latest in reactive microservices development and walk through a modern architecture on how a heritage architecture can be re-architected as an event-driven real-time system.

Kevin Webber

August 16, 2016
Tweet

More Decks by Kevin Webber

Other Decks in Programming

Transcript

  1. The world by 2020 » 4 billion connected people »

    25+ million apps » 25+ billion embedded systems » 40 zettabytes (40 trillion gigabytes) » 5,200 GB of data for every person on Earth
  2. Goals » ~100% availability ("nine nines") » Consistent responsiveness under

    varying load conditions » Predictable spikes of traffic, e.g, Black Friday » Less predictable spikes of traffic, e.g, marketing campaign driving traffic
  3. Goals » ~100% availability ("nine nines") » Consistent responsiveness under

    varying load conditions » Predictable spikes of traffic, e.g, Black Friday » Less predictable spikes of traffic, e.g, marketing campaign driving traffic » Enable LOBs to rapidly adapt to market conditions
  4. walmart.ca » Added Play in front of customer experience »

    API enabled heritage architecture for scale » Added Akka in front of back-office » Circuit breakers for resilience » Async message-passing for parallelism
  5. Walmart's Business Uplift » Conversions up 20% » Mobile orders

    up 98% » No downtime Black Friday or Boxing Day
  6. Walmart's Operational Savings » Moved off expensive hardware » On

    cheap virtual x86 servers » 20% - 50% cost savings » ~ 40% compute cycles
  7. Challenges of scale Heritage solution: » Replicate stateless servers in

    front of middlware » Push out state to the database tier Reality: » The database becomes a single point of failure
  8. Challenges of resilience Heritage solution: » DR, active/passive Reality: »

    No graceful way to handle cascading failure in a heritage system without downtime
  9. heritage applications » Single points of failure » Difficult to

    scale out (and back in) due to state » Different non-functional requirements between logical boundaries » Long, risky release cycles
  10. SOA Stateless: messaging, message exchange patterns (sync req/resp, async p2p

    messaging, pub/sub, streams), routing, service orchestration (service aggregation), security (encryption and signing), monitoring, protocol conversion, legacy adapters, payload transformation, schema validation, governance (the ability to apply business rules uniformly), service catalog
  11. SOA Stateful: process choreography (BPM), complex event processing (CEP), reliable

    delivery, transaction management, split and merge (conflating, windowing, ordering)
  12. heritage systems » Productivity depends on working within the system

    context » Creates difficult productivity barriers » E.g, systems-level exploratory development on a laptop?
  13. "Jack of all trades, master of none, often times better

    than the master of one." Give up control to gain flexibility. Embrace new patterns to achieve balance.
  14. We need to build these types of systems. asynchronous, non-blocking,

    real-time, highly- available, loosely coupled, scalable, fault-tolerant, concurrent, reactive, event-driven, push instead of pull, distributed, low latency, high throughput
  15. Strategy Microservices: » Domain Driven Design (DDD) » Bounded contexts

    » Process isolation » Distributed systems » Event sourcing » CQRS » Eventual consistency
  16. Overview » Lagom - Microservices framework » Play - RESTful

    API framework » Akka - Distributed computing toolkit » Spark - General purpose compute engine
  17. Lagom » Core abstract are Persistent Entities » They move

    around the cluster automatically » Can replay all state changes » One command to launch all services in a developer environment
  18. Play » Familiar MVC paradigm » Embraces flows of data

    » WebSockets, SSE » Reactive Streams » File uploads » Integration with Akka » Distribute work via messaging
  19. Actor model » Actors » Conceptually they are very lightweight

    threads (~300 bytes each) that communicate with each other by messaging » Inspired by Erlang
  20. Fast Data Architecture » Akka Streams » < 1 second

    latency per- event processing » Spark Streaming » 1-10+ second latency for mini-batch event processing » Kafka captures the complete event stream
  21. Microservices » Single responsibility » Independently deployable, scalable, resilient »

    Communicate via async messaging » Own their data » Don't expose a public API
  22. API gateway » Authentication, authorization » Quality of service »

    Protocol conversions » Well-documented (self- documented) API » Completely stateless
  23. Services & integration » Stateful services » Async messaging within

    service boundaries » Publish/subscribe rather than request/response between services » A system made up entirely of commands, queries, and events
  24. Kafka » Publish-subscribe messaging rethought as a distributed commit log

    » The event stream becomes the database » Distributed and replicated for resilience » Ordering and at-least once delivery
  25. Clustered microservices » Responsive to events, load, failure, users »

    Clusters are resilient » Across nodes, regions, data centers » Actors can fail on one node and come back on another node, with state intact
  26. booth 317 Reactive Microservices Architecture: » Go deeper into the

    theory behind microservices » Learn more about the new architectural patterns required for a successful microservices project » Come by the booth and pick up a copy!
  27. Thank you! Visit https://www.lightbend.com to get started Get in touch:

    Kevin Webber, Enterprise Architect, Lightbend Email: [email protected] Medium: medium.com/@kvnwbbr Twitter: @kvnwbbr Booth: 317