Streaming Data Analysis with Kubernetes

Dealing with real-time, in-memory, streaming data is a unique challenge and with the advent of the smartphone and IoT (trillions of internet connected devices), we are witnessing an exponential growth in data at scale.

To be able to handle potential data growth, you want to process data in a cloud environment that can easily scale. In this space, Kubernetes offers great container orchestration and auto-scaling capabilities that are perfectly suited for streaming data use cases. When combined with Infinispan, an in-memory data grid from Red Hat, it empowers you with state of the art distributed data processing capabilities to tackle these challenges.

In this session, we will identify critical patterns and principles that will help you achieve greater scale and response speed. On top of that, you will witness how Infinispan follows these patterns and principles to tackle a Big Data situation via a live coding demonstration on top of a container platform orchestrated by Kubernetes.

Transcript

1. #BaselOne17
2. STREAMING DATA ANALYSIS WITH KUBERNETES Basel One Galder Zamarreño Arrizabalaga @galderz 19th October 2017
3. INSERT DESIGNATOR, IF NEEDED ENGINEER 3 Since 2006 Community Lead and Core Developer INFINISPAN CO-FOUNDER (2009) Since 2013 Working from home LIVE A FEW MINUTES AWAY MOI
4. BUILD STREAMING DATA ANALYSIS APPLICATION ON TOP OF A KUBERNETES-BASED PLATFORM
5. INSERT DESIGNATOR, IF NEEDED THE PROBLEM 5 EXPONENTIAL DATA GROWTH YEAR ON YEAR Smartphones, IOT devices, trillions of internet connected devices... REAL-TIME STREAMING DATA PROCESSING IS CHALLENGING Delays can have a big impact
6. INSERT DESIGNATOR, IF NEEDED ZZZ… NO! 6
7. INSERT DESIGNATOR, IF NEEDED 7 Platform-as-a-Service Based on Kubernetes Public or Private! Multi-language THE PLATFORM
8. INSERT DESIGNATOR, IF NEEDED 8 Vert.x is a toolkit for building reactive apps On JVM, event-driven and non-blocking RxJava integrates with Vert.x Great at event transform and coordination Works best with many source of events (modern apps!) THE GLUE
9. INSERT DESIGNATOR, IF NEEDED 9 IN-MEMORY DATA GRIDS Analytical Framework Custom Applications Mobile Applications Web Apps & Websites JBoss Middleware Fuse "memory" across machines into a unified data store Read-through, write-through, write-behind • Polyglot • Extreme Performance • Linear Scalability • Fault Tolerant • Event driven Infinispan
10. INSERT DESIGNATOR, IF NEEDED 10 THE DATA http://transport.opendata.ch
11. INSERT DESIGNATOR, IF NEEDED 11 THE DEMO
12. DEMO
13. INSERT DESIGNATOR, IF NEEDED 13 VERSATILITY OF INFINISPAN Expiration Listeners Eviction Embedded | Remote Transactions Persistence Querying Code execution Security Management and monitoring Cloud Integrations Distributed Cache Shared Memory Event Broker Analytics
14. BUILD STREAMING DATA ANALYSIS APPLICATION ON TOP OF A KUBERNETES-BASED PLATFORM
15. github.com/infinispan-demos/streaming-data-kubernetes infinispan.org redhat.com/en/technologies/jboss-middleware/data-grid openshift.com vertx.io THANK YOU