Upgrade to Pro — share decks privately, control downloads, hide ads and more …

End-to-End Encryption for Streaming Data Pipelines @ Berlin Buzzwords 2024

End-to-End Encryption for Streaming Data Pipelines @ Berlin Buzzwords 2024

This talk explains what it takes to bring end-to-end encryption to streaming data pipelines built on top of Apache Kafka and Flink. A live demo illustrates how to encrypt/decrypt sensitive payload fields by means of single message transformations and user-defined functions without any custom code.

* Abstract:
Security is a key concern for Apache Kafka: authentication, authorization, and over-the-wire encryption help to ensure the confidentiality of your data. This notwithstanding, workloads with very sensitive data might require end-to-end encryption between Kafka client applications.

This talk introduces Kryptonite for Kafka, written and open-sourced by the speaker. It's a community project that performs client-side field-level cryptography for payloads produced to and consumed from Kafka topics. A demo scenario built on top of Kafka Connect and Flink SQL illustrates how to encrypt and decrypt sensitive payload fields by means of applying turn-key ready message transformations as well as user-defined functions without the need for any custom code.

Join this session to learn how Kryptonite for Kafka makes your streaming data pipelines more secure, safeguarding your most sensitive payload fields against any form of uncontrolled or illegal access on the brokers.

* Recording: https://www.youtube.com/watch?v=0joqjio3mNg

Hans-Peter Grahsl

June 10, 2024
Tweet

More Decks by Hans-Peter Grahsl

Other Decks in Programming

Transcript

  1. ! whoami ! • Hans-Peter Grahsl • Developer ! Advocate

    @ Red Hat • Open-Source Enthusiast • Confluent Community Catalyst • MongoDB Champion since 2020 @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 2
  2. ? data in use by BROKERS @hpgrahsl | June 10th

    2024 | #BerlinBuzzwords | Berlin 13
  3. BROKERS see everything ... and so does any legitimate Kafka

    client @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 14
  4. BROKERS see everything ... and so does any legitimate Kafka

    client @hpgrahsl | June 10th 2024 | #BerlinBuzzwords | Berlin 15