Share
Embed
Cloud Native Streaming and Event Driven Microservices
Slides from the talk delivered at Devoxx Belgium
Marius Bogoevici
November 10, 2016
Transcript
24 slides
-
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Cloud Native Event-Driven and Streaming Microservices
Marius Bogoevici, Pivotal @mariusbogoevici -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Streaming • Processing large quantities of
data as fast as possible (near real-time) • Use cases: • predictive maintenance, fraud detection, QoS, IoT • enterprise integration meets ‘Big Data’ • High throughput / low latency • Grouping, ordering, windowing • Transition in messaging middleware design: • Traditional (JMS-based, RabbitMQ) • To log-style: Kafka, Amazon Kinesis, Google Pub/Sub, Azure Event Hubs -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Microservices • Preferred model for distributed
data processing applications • Failure isolation • Independent scaling • Process-specific resource tuning: • memory/CPU/Instance count/network • Typical benefits and drawbacks of microservices: • bounded context as cohesive units • development agility • added complexity -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Event-driven • Messaging vs HTTP/“fire-and-forget” vs.
request-reply • Decoupling • Discoverability • Availability • Eventual consistency vs distributed transactions • especially over heterogenous resources • Strong similarities between: • data streaming - ‘big data’: ingestion, analytics • event streaming - async interaction, event sourcing, etc -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Cloud Native • Focus on modern
platform capabilities • Scaling up/down • Resource management (memory, CPU, instance count) • Health monitoring and failover • Routing and load balancing Apache YARN Apache Mesos Kubernetes -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream In a single sentence … dataflow:>
stream create demo --definition “http | file” -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream 2015 Spring XD
Spring Cloud Data Flow Spring Cloud Task -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream • Event-driven microservice
framework • Built on battle-tested components • Spring Boot: full-stack standalone applications • Spring Integration: Messaging, EIP patterns, connectors • Focusing on scalable, structured distributed applications • Opinionated primitives • Pluggable middleware abstractions -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Stream in a nutshell
Application Core Spring Boot Spring Integration Spring Messaging Reactive APIs inputs outputs -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Programming model @EnableBinding + Binder Implementation
Apache Kafka JMS Google PubSub Production-ready: Experimental -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Data Flow • Orchestration:
• DSL for Stream topologies • REST API • Shell • UI • Portable Deployment SPI • OOTB apps for common integration use-cases -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Stream DSL Stream Definition Spring Boot
applications built on top of Spring ecosystem -
@mariusbogoevici @springcloud #Devoxx #SpringCloudStream Spring Cloud Deployer • SPI for
deploying applications to modern runtimes • Local (for testing) • Cloud Foundry • YARN • Kubernetes • Mesos + Marathon • Different resources supported • Spring Boot Uberjars • Docker Images • Different resource location supported • File system, HDFS, HTTP, Maven, Docker Hub