Patterns d’utilisation de systèmes in-memory

Les systèmes in-memory comme Infinispan sont largement utilisés et deviennent le lubrifiant WD-40 de la donnée: accélérer l'accès aux données, tailler la donnée aux petits oignons pour le cas d'utilisation, etc. Ces outils sont versatiles et peuvent être utilisés d'un milliard de façons différentes. Mais contrairement au ruban adhésif et au WD-40, il n'est pas intuitif de savoir quand et comment les utiliser.

Dans cette session, nous allons explorer des architectures et des cas d'utilisation pratiques où les systèmes in-memory font la différence. Vous sortirez de cette présentation avec des exemples concrets pour dégripper vos données.

Transcript

1. #DevoxxFR #inmemory in-memory data systems what they can do for you Emmanuel Bernard Galder Zamarreño
2. Emmanuel Bernard Galder Zamarreño @emmanuelbernard @galderz
3. Data is coming Data is flowing
4. Insightful application Observe - collect data Orient - Put in context Decide - select course of action Act - Do it Continuous improvement loop Data services Event stream Data analysis Iterative storage Reasoning, decision models and optimization Execution engine
5. In-memory systems an case
6. Key foundation key/value store Distributed Elastic Optional schema ASL 2.0
8. in-memory systems in an architecture
9. Caching Caching service data Get faster App 1 App 2 Infinispan Datastore
10. (derived) Primary store Stateless apps in a stateful universe Microservice “local” storage (Fast) Search capabilities Analytics Distributed tasks App 1 App 2 Infinispan
11. Distributed stream Move computation to data map(λ) λ λ
12. In action!
13. Station board stop station train
14. What is the time of the day when there is the biggest ratio of delayed trains?
15. Mixing stores Mix traditional datastores and in-memory Temporary data Heavy computation App JPA OGM ORM App JPA OGM ORM App JPA OGM ORM Infinispan RDBMS
16. Data virtualisation duct taping Microservice 1 Big old Datasource Dedicated Schema 1 Dedicated Schema 2 Dedicated Schema 3 Data Virtualisation Microservice 2 Microservice 3 Old app
17. Analytics and Spark Computation out of Spark Source of data Source of events Immutable RDD Fast RDD caching Spark cluster Infinispan Cassandra Hive RDBMS App 1
18. Syncing grids Active / Active Data synced between sites One grid per usage OLTP Analytics App 1 App 2 Infinispan Infinispan Active / active cross site replication Datacenter 1 Datacenter 2 Load balancer
19. Pay attention, be reactive Reactive application Reactive microservice Also look at Debezium App 1 “Reactive app” 2 Infinispan Change event “Reactive app” 1 Change event
20. Continuous query Reactive to data changes Index the query, not the data from User u where u.age >18 and u.country = ‘France’ from User u where u.age >18 and u.hasSnapshat = true from User u where u.email contains ‘yahoo.com’ { “country”: “France”, “age”: 17, “email”: “e@gmail.com”, “hasSnapshat”: true } { “country”: “France”, “age”: 19, “email”: “e@gmail.com”, “hasSnapshat”: true } =>
21. In action!
22. Central Train Delays Dashboard
23. Debezium and CDC Change Data Capture Transforms datastores into data change events No change to existing apps and datastores Add features as separate apps Decorrelate event consumers (each at its own pace)
24. Data are like Gremlins but good Observe - collect data Orient - Put in context Decide - select course of action Act - Do it Continuous improvement loop
25. Thanks http://infinispan.org http://blog.infinispan.org @infinispan DEMOs: https://github.com/ galderz/datagrid-patterns http://debezium.io http://teiid.org Explore your data Take calculated risks Any use case?
26. Credits engineer by Wilson Joseph from the Noun Project panel by gira Park from the Noun Project Approve by Aha-Soft from the Noun Project Database sharing by YuguDesign from the Noun Project ram by Andrea Rizzato from the Noun Project Database Search by Nimal Raj from the Noun Project Cloud Analytics by Kevin Augustine LO from the Noun Project Broken Computer by Dan Hetteix from the Noun Project data search by Gregor Črešnar from the Noun Project Server by Creative Stall from the Noun Project Network by Creative Stall from the Noun Project transformation by Felipe Perucho from the Noun Project analytics by Roman Kovbasyuk from the Noun Project Server by Designify.me from the Noun Project