Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud Native Streaming and Event Driven Microse...

Cloud Native Streaming and Event Driven Microservices

Slides from the talk delivered at Devoxx Belgium

Marius Bogoevici

November 10, 2016
Tweet

More Decks by Marius Bogoevici

Other Decks in Technology

Transcript

  1. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Streaming • Processing large quantities of

    data as fast as possible (near real-time) • Use cases: • predictive maintenance, fraud detection, QoS, IoT • enterprise integration meets ‘Big Data’ • High throughput / low latency • Grouping, ordering, windowing • Transition in messaging middleware design: • Traditional (JMS-based, RabbitMQ) • To log-style: Kafka, Amazon Kinesis, Google Pub/Sub, Azure Event Hubs
  2. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Microservices • Preferred model for distributed

    data processing applications • Failure isolation • Independent scaling • Process-specific resource tuning: • memory/CPU/Instance count/network • Typical benefits and drawbacks of microservices: • bounded context as cohesive units • development agility • added complexity
  3. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Event-driven • Messaging vs HTTP/“fire-and-forget” vs.

    request-reply • Decoupling • Discoverability • Availability • Eventual consistency vs distributed transactions • especially over heterogenous resources • Strong similarities between: • data streaming - ‘big data’: ingestion, analytics • event streaming - async interaction, event sourcing, etc
  4. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Cloud Native • Focus on modern

    platform capabilities • Scaling up/down • Resource management (memory, CPU, instance count) • Health monitoring and failover • Routing and load balancing Apache YARN Apache Mesos Kubernetes
  5. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream • Event-driven microservice

    framework • Built on battle-tested components • Spring Boot: full-stack standalone applications • Spring Integration: Messaging, EIP patterns, connectors • Focusing on scalable, structured distributed applications • Opinionated primitives • Pluggable middleware abstractions
  6. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream in a nutshell

    Application Core Spring Boot Spring Integration Spring Messaging Reactive APIs inputs outputs
  7. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Data Flow • Orchestration:

    • DSL for Stream topologies • REST API • Shell • UI • Portable Deployment SPI • OOTB apps for common integration use-cases
  8. @mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Deployer • SPI for

    deploying applications to modern runtimes • Local (for testing) • Cloud Foundry • YARN • Kubernetes • Mesos + Marathon • Different resources supported • Spring Boot Uberjars • Docker Images • Different resource location supported • File system, HDFS, HTTP, Maven, Docker Hub