Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Everything You Wanted to Know About Apache Kafk...

Everything You Wanted to Know About Apache Kafka But Were Too Afraid to Ask!

Presented in the Data Science Meetup in San Antonio, TX.

Ricardo Ferreira

June 27, 2019
Tweet

More Decks by Ricardo Ferreira

Other Decks in Programming

Transcript

  1. Join the Confluent Community Slack Subscribe to the Confluent blog

    cnfl.io/slack cnfl.io/read Welcome to the San Antonio Apache Kafka® Meetup! 6:00pm Doors open 6:00pm - 6:30pm Pizza, Drinks and Networking 6:30pm - 7:30pm Ricardo Ferreira, Confluent 7:30pm - 7:45pm Additional Q&A & Networking Apache, Apache Kafka, Kafka and the Kafka logo are trademarks of the Apache Software Foundation. The Apache Software Foundation has no affiliation with and does not endorse the materials provided at this event.
  2. @riferrei | #kafkameetup | @CONFLUENTINC Everything you wanted to know

    about kafka But you were too afraid to ask! @riferrei | #kafkameetup | @CONFLUENTINC
  3. About Us: • Ricardo Ferreira ❑ Developer Advocate @ Confluent

    ❑ Ex-Oracle, Red Hat, IONA Tech ❑ [email protected] ❑ https://riferrei.net • Alexa (echo dot) ❑ The voice behind Amazon ❑ Ex-Raspberry Pi, Arduino ❑ She is a female in character! @riferrei @alexa99
  4. @riferrei | #kafkameetup | @CONFLUENTINC SQL DBs 25 years ago…

    SQL DBs Today Dude, you're embarrassing me in front of the wizards…
  5. @riferrei | #kafkameetup | @CONFLUENTINC Solution for "Combining" Processing and

    Data: NoSQL Solution for Large Amounts of Data: big data
  6. @riferrei | #kafkameetup | @CONFLUENTINC “The truth is the log.

    The database is a cache of a subset of the log.” — pat helland Immutability changes everything http://cidrdb.org/cidr2015/Papers/CIDR15_Paper16.pdf
  7. 23 ETL/Data Integration Messaging Batch Expensive Time Consuming Difficult to

    Scale No Persistence After Consumption No Replay Highly Scalable Durable Persistent Ordered Fast (Low Latency) What is happening in the world What happened in the world
  8. 24 ETL/Data Integration Messaging Batch Expensive Time Consuming Difficult to

    Scale No Persistence After Consumption No Replay Highly Scalable Durable Persistent Ordered Fast (Low Latency) What is happening in the world What happened in the world Highly Scalable Durable Persistent Ordered Fast (Low Latency) Event Streaming Thinking
  9. @riferrei | #kafkameetup | @CONFLUENTINC 01 Messaging done right 02

    Scalable data pipelines 03 Stream processing @riferrei | #kafkameetup | @CONFLUENTINC
  10. 27 Event-Driven App (Location Tracking) Only Real-Time Events Messaging Queues

    and pub/sub Platforms can do this Contextual Event-Driven App (ETA) Real-Time combined with stored data Only Streaming data Platforms can do this Where is my driver? When will my driver get here? Where is my driver? When will my driver arrive? Why Combine Real-time With Historical Context? 2 min