Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud-native Streaming and Event-Driven Microse...

Cloud-native Streaming and Event-Driven Microservices

Talk delivered at Spring IO in Barcelona (May 20, 2016)

The talk introduces Spring Cloud Stream as a framework for developing event-driven microservices, as well as Spring Cloud Data Flow as an orchestration and deployment layer.

Marius Bogoevici

May 20, 2016
Tweet

More Decks by Marius Bogoevici

Other Decks in Technology

Transcript

  1. CLOUD-NATIVE STREAMING AND EVENT- DRIVEN MICROSERVICES • Use cases: •

    predictive maintenance, fraud detection, QoS, IoT • enterprise integration meets ‘Big Data’ • Specific requirements • High throughput, low latency • Grouping, ordering, windowing • Middleware adaptations: beyond JMS, Rabbit MQ • Kafka, Amazon Kinesis, Google Pub/Sub, Azure Event Hubs • Brokerless: Aeron, ZeroMQ Spring I/O 2016
  2. CLOUD-NATIVE STREAMING AND EVENT- DRIVEN MICROSERVICES • Architectural style advantages:

    • bounded context, development agility • Process-specific resource tuning: • memory, CPU, instance count • Messaging vs. HTTP • Discoverability • Availability • Consistency across heterogenous resources Spring I/O 2016
  3. CLOUD-NATIVE STREAMING AND EVENT- DRIVEN MICROSERVICES • Modern platforms •

    Resource management • memory, CPU, instance count • Scaling up/down • Health monitoring and failover • Routing and load balancing Spring I/O 2016 Apache YARN Apache Mesos Kubernetes
  4. Spring Cloud Stream Spring Cloud Data Flow Spring I/O 2016

    Spring Cloud Data Flow Spring Cloud Stream Spring Cloud Task 2015
  5. Spring Cloud Stream • Event-driven microservice framework • Built on

    battle-tested components • Spring Boot: full-stack standalone applications • Spring Integration: Messaging, EIP patterns, connectors • Opinionated primitives • Pluggable middleware abstractions Spring I/O 2016
  6. Consumer Groups • Borrowed from Kafka, applied to all binders

    • Groups of competing consumers within the pub-sub destination • Used for scaling and partitioning Spring I/O 2016
  7. Partitioning • Required for stateful processing • Outputs specify a

    partitioning strategy • Inputs consuming from specific partition Spring I/O 2016
  8. Spring Cloud Data Flow • Orchestration: • DSL for Stream

    topologies • REST API • Shell • UI • Portable Deployment SPI • OOTB apps for common integration use- cases Spring I/O 2016
  9. Stream DSL: pipes and filters Stream definition Boot Apps build

    on top of Spring Ecosystem httpfile = http | file | Spring I/O 2016 ‘pipe’, or how to communicate Rabbit, Kafka, Gemfire
  10. Spring Cloud Deployer • SPI for deploying applications to modern

    runtimes • Local (for testing) • Cloud Foundry • YARN • Kubernetes • Mesos + Marathon • Different resources supported • Spring Boot Uberjars • Docker Images • Different locations • File system, HDFS, HTTP, Maven, Docker Hub Spring I/O 2016
  11. Timeline • Spring Cloud Stream • 1.0.0.GA May 10, 2016

    • 1.1.0.M1 July 2016 • 1.1.0.GA Q3 2016 • Spring Cloud Data Flow • 1.0.0.M3 May 7, 2016 • 1.0.0.RC1 End of May, 2016 Spring I/O 2016