Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Data Gateways: Building “Data-as-a-Service” for...

Data Gateways: Building “Data-as-a-Service” for the Hybrid Cloud

Application architecture evolution has fragmented the backend implementation into independent microservices and functions. However there is still a gap on the way this evolution has dealt with data as it tends to avoid dealing with static environments. At the same time, microservices encourage developers to create new polyglot data persistence layers that then, need to be composite to deliver business value. How can we apply the knowledge from API gateways to these new data stores?

In this session we talked about:
- How data gateways act like API gateways to offer a secured abstraction layer on top of the physical data stores,
- the different data gateway types and architectures, and
- extended data-proxy for hybrid cloud deployments.

Hugo Guerrero

October 08, 2020
Tweet

More Decks by Hugo Guerrero

Other Decks in Programming

Transcript

  1. Legacy data for microservices Data Gateways Hugo Guerrero APIs &

    Messaging Developer Advocate, Red Hat DATA GATEWAYS 1
  2. Agenda DATA GATEWAYS 2 Architecture Evolution The inevitable transformation Microservices

    Data The rise of the gateways Microservices Data Data Gateways Types Hugo Guerrero Mexican @ MA, USA APIs & Event-Driven Specialist | Open Source Advocate | History, Travel & Food Enthusiast @hguerreroo
  3. @hguerreroo DATA GATEWAYS Modernize this • 10,000+ types of interconnected

    apps ◦ function focused … ◦ 300,000 system interfaces • 100,000+ databases ◦ 95% unused data • Siloed product/service orgs/processes
  4. @hguerreroo DATA GATEWAYS 6 Photo credit: JD Hancock on Visualhunt.com

    / CC BY MICROSERVICES ARCHITECTURE MONOLITHIC DATABASE
  5. @hguerreroo 9 DATA GATEWAYS The Rise of API Gateways Developer

    Apps API Backend 2 API Gateway API Request Authorized API Request API Backend 1 API Backend 3
  6. @hguerreroo 10 DATA GATEWAYS Microservices Data • Independent database per

    microservice ◦ Independent database instance ◦ Independent schema and logical grouping of tables • Only one microservice owns the data • Polyglot persistence layer
  7. @hguerreroo Source: https://www.infoq.com/articles/data-gateways-cloud-native/ “A data gateway is like an API

    gateway, but it understands and acts on the data layer implementation rather than the networking layer” DATA GATEWAYS 12
  8. @hguerreroo 13 DATA GATEWAYS Data Gateways Developer Apps API-based Data

    Store Data Gateway API REST / OData Authorized API Request Relational Data Store Non-relational Data Store BI Users Data Scientists ODBC JDBC
  9. @hguerreroo 14 DATA GATEWAYS Data Gateway Capabilities • Abstraction /

    Decoupling ◦ Hides implementation ◦ Abstracts physical source • Security ◦ Access control based on data model (row, column, etc.) • Scaling ◦ Caching and materialized views • Federation • Schema-first based on SQL standard
  10. @hguerreroo 15 DATA GATEWAYS Many Types of Data Gateways •

    Classic Data Virtualization • Database with Federation • GraphQL bridges • Cloud-hosted • Secure tunneling data-proxies • Open source data gateways
  11. @hguerreroo DATA GATEWAY Virtual Application Network • An abstraction layer

    between the network and the application services / processes • Applications don’t need to be modified ◦ HTTP/REST ◦ gRPC ◦ Messaging ◦ General TCP/UDP (JDBC, etc.) • A natural fit into container platforms: Kubernetes /Docker • Application topology independent from network topology
  12. “Data has gravity, requires granular access control, is hard to

    scale, and difficult to move on/off/between cloud-native infrastructures. Having a data gateway component as part of the cloud-native tooling arsenal, which is hybrid and works on multiple cloud providers, supports different use cases is becoming a necessity.” DATA GATEWAYS 21 Source: https://www.infoq.com/articles/data-gateways-cloud-native/
  13. @hguerreroo DATA GATEWAY Other use cases • Data modernization &

    migration • Data firewall • API composition