Literate Kafka with Jackdaw

Using Jackdaw to unleash the power of Kafka streams in Clojure.

Transcript

1. Literate Kafka with Jackdaw Francisco Viramontes
2. Funding Circle is a global small business loans platform 2 UK NL DE USA Investors Small Businesses
3. Kafka basics 3 "recipes" "stock tips" writes 12 8 6 5 "recipes" "stock tips" reads 4 2 2 5 subscribers publishers Kafka Recipes I Recipes! I Recipes! I Stocks! I Stocks! Stock Tips Advice Charles Reese, "Kafka and the REPL: Stream Processing, the Functional Way, Clojure/conj 2018
4. Kafka as a Streaming Platform
5. Kafka as a streaming platform 5 Kafka topics stream processors 1 3 2 Charles Reese, "Kafka and the REPL: Stream Processing, the Functional Way, Clojure/conj 2018
6. Kafka as a streaming platform 6 Kafka topics stream processors internal state 1 3 2 Charles Reese, "Kafka and the REPL: Stream Processing, the Functional Way, Clojure/conj 2018
7. Kafka Streams API 7 Interface KStream<K,V> Record stream of key-value pairs corresponding to an event. Interface KTable<K,V> Changelog stream from a primary-keyed table. Each record is an update. Class TopologyBuilder Used to construct topologies. Charles Reese, "Kafka and the REPL: Stream Processing, the Functional Way, Clojure/conj 2018
8. Interface KStream<K,V>
9. WordCountDemo.java
10. Why Clojure works 10
11. Funding Circle Marketplace 11 Kafka API Front End Metrics Data Lake Orders Payments Servicing Accounting Charles Reese, "Kafka and the REPL: Stream Processing, the Functional Way, Clojure/conj 2018
12. Funding Circle Marketplace 12 Kafka API Front End Metrics Data Lake Charles Reese, "Kafka and the REPL: Stream Processing, the Functional Way, Clojure/conj 2018
13. https://github.com/FundingCircle/jackdaw
14. 14 Jackdaw namespaces jackdaw.client Kafka Producer and Consumer APIs jackdaw.streams Kafka Streams API jackdaw.admin Kafka AdminClient API jackdaw.serdes Serdes (EDN, JSON, Avro) jackdaw.test Integration Testing Charles Reese, "Kafka and the REPL: Stream Processing, the Functional Way, Clojure/conj 2018
15. (Literate Examples) https://github.com/kidpollo/jackdaw-literate-examples/blob/master/streams.org
16. 16 https://www.youtube.com/watch?v=eJFBbwCB6v4&t=41s Want to learn more? Kafka and the REPL: Stream Processing, the Functional Way - Charles Reese