Upgrade to Pro — share decks privately, control downloads, hide ads and more …

apidays Australia 2023 - The Playful Bond Betwe...

apidays
October 24, 2023

apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warren Vella, Confluent

apidays Australia 2023 - Platforms, Products, and People: The Power of APIs
October 11 & 12, 2023
https://www.apidays.global/australia/

The Playful Bond Between REST And Data Streams
Warren Vella, Solutions Engineer at Confluent

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

apidays

October 24, 2023
Tweet

More Decks by apidays

Other Decks in Programming

Transcript

  1. The Playful Bond Between REST And Data Streams Warren Vella

    | Solutions Engineer, Melbourne | Confluent
  2. Operational Databases Analytic Systems Azure Synapse How do we integrate

    these systems together? SaaS Applications Applications Microservices Monoliths LEGACY INTEGRATION REST APIs only work here
  3. REST APIs 8 1. …have proliferated as a way to

    integrate microservices. 2.…are often the default when exchanging data between organisations. 3.…don’t meet all our integration needs.
  4. 1. Physical 2. Data Link 3. Network 4. Transport 5.

    Session 6. Presentation 7. Application REST API POS DataWarehouse
  5. Synchronous request-response REST APIS are addictive. Like an addiction, they

    give us a false sense of comfort of being always-available. The more we build, the more we want but we often overlook the exponential costs associated with building failure mechanisms for never ending point- to-point integrations. We end up relapsing to REST APIs.
  6. REST APIs 26 1. …have proliferated as a way to

    integrate microservices. 2.…are often the default when exchanging data between organisations. 3.…don’t meet all our integration needs. But we have ended up with a death star. But are not suited for high-throughput. So we have lots of different integration tools.
  7. REST API • Great for Synchronous Responses • Tight-coupling •

    No delivery guarantees • Receiving systems can be overwhelmed • Emphasis on Requests/Responses • No Replay • True Decoupling • Stronger delivery guarantees. Kafka is a buffer. • Receiving systems process at a rate they can handle. • Emphasis on events. • Rebuild state by replay.
  8. Payment Gateway Data Topic Can REST and Kafka co-exist? DataWarehouse

    Realtime Analytics CRM Inside Boundary POS Microservices Outside Boundary Request / Response
  9. 1. Physical 2. Data Link 3. Network 4. Transport 5.

    Session 6. Presentation 7. Application POS Realtime Analytics
  10. Operational Databases SaaS Applications Analytic Systems Azure Synaps e |

    DATA STREAMING PLATFORM Today, a fourth estate is emerging that provides a new foundation for your data strategy
  11. Key Takeaways 41 1. ✅ Reduce the amount of lines

    between systems with Kafka. 2.✅ REST APIs are great for Request- Response but also consider Kafka as a pattern for exchanging data between organisations. 3.✅ A Data Streaming Platform integrates both application and data estates.