Data Gateways: Building “Data-as-a-Service” for the Hybrid Cloud

Application architecture evolution has fragmented the backend implementation into independent microservices and functions. However there is still a gap on the way this evolution has dealt with data as it tends to avoid dealing with static environments. At the same time, microservices encourage developers to create new polyglot data persistence layers that then, need to be composite to deliver business value. How can we apply the knowledge from API gateways to these new data stores?

In this session we talked about:
- How data gateways act like API gateways to offer a secured abstraction layer on top of the physical data stores,
- the different data gateway types and architectures, and
- extended data-proxy for hybrid cloud deployments.

Transcript

1. Legacy data for microservices Data Gateways Hugo Guerrero APIs & Messaging Developer Advocate, Red Hat DATA GATEWAYS 1
2. Agenda DATA GATEWAYS 2 Architecture Evolution The inevitable transformation Microservices Data The rise of the gateways Microservices Data Data Gateways Types Hugo Guerrero Mexican @ MA, USA APIs & Event-Driven Specialist | Open Source Advocate | History, Travel & Food Enthusiast @hguerreroo
3. @hguerreroo DATA GATEWAYS Modernize this • 10,000+ types of interconnected apps ◦ function focused … ◦ 300,000 system interfaces • 100,000+ databases ◦ 95% unused data • Siloed product/service orgs/processes
4. @hguerreroo McKinsey: Next-generation core banking platforms: A golden ticket? https://www.mckinsey.com/industries/financial-services/our-insights/banking-matters/next-generation-core-banking-platforms-a-golden-ticket DATA GATEWAYS
5. @hguerreroo Source: IDC Moving toward the future of banking infrastructure DATA GATEWAYS Cornerstone of Hybrid Cloud
6. @hguerreroo DATA GATEWAYS 6 Photo credit: JD Hancock on Visualhunt.com / CC BY MICROSERVICES ARCHITECTURE MONOLITHIC DATABASE
7. @hguerreroo DATA GATEWAYS 7 Source: https://www.infoq.com/articles/data-gateways-cloud-native/ Distributed Architecture
8. We don’t need ESBs anymore, but we still need integration
9. @hguerreroo 9 DATA GATEWAYS The Rise of API Gateways Developer Apps API Backend 2 API Gateway API Request Authorized API Request API Backend 1 API Backend 3
10. @hguerreroo 10 DATA GATEWAYS Microservices Data • Independent database per microservice ◦ Independent database instance ◦ Independent schema and logical grouping of tables • Only one microservice owns the data • Polyglot persistence layer
11. @hguerreroo DATA GATEWAYS 11 Source: https://www.infoq.com/articles/data-gateways-cloud-native/ Gateways
12. @hguerreroo Source: https://www.infoq.com/articles/data-gateways-cloud-native/ “A data gateway is like an API gateway, but it understands and acts on the data layer implementation rather than the networking layer” DATA GATEWAYS 12
13. @hguerreroo 13 DATA GATEWAYS Data Gateways Developer Apps API-based Data Store Data Gateway API REST / OData Authorized API Request Relational Data Store Non-relational Data Store BI Users Data Scientists ODBC JDBC
14. @hguerreroo 14 DATA GATEWAYS Data Gateway Capabilities • Abstraction / Decoupling ◦ Hides implementation ◦ Abstracts physical source • Security ◦ Access control based on data model (row, column, etc.) • Scaling ◦ Caching and materialized views • Federation • Schema-first based on SQL standard
15. @hguerreroo 15 DATA GATEWAYS Many Types of Data Gateways • Classic Data Virtualization • Database with Federation • GraphQL bridges • Cloud-hosted • Secure tunneling data-proxies • Open source data gateways
16. @hguerreroo DATA GATEWAY Virtual Application Network • An abstraction layer between the network and the application services / processes • Applications don’t need to be modified ◦ HTTP/REST ◦ gRPC ◦ Messaging ◦ General TCP/UDP (JDBC, etc.) • A natural fit into container platforms: Kubernetes /Docker • Application topology independent from network topology
17. @hguerreroo DATA GATEWAY skupper.io @skupperio
18. @hguerreroo DATA GATEWAY Secure data gateway with Governance SSO
19. @hguerreroo DATA GATEWAY Federation and abstraction
20. @hguerreroo DATA GATEWAY BIAN Open Banking
21. “Data has gravity, requires granular access control, is hard to scale, and difficult to move on/off/between cloud-native infrastructures. Having a data gateway component as part of the cloud-native tooling arsenal, which is hybrid and works on multiple cloud providers, supports different use cases is becoming a necessity.” DATA GATEWAYS 21 Source: https://www.infoq.com/articles/data-gateways-cloud-native/
22. THANK YOU! DATA GATEWAYS @hguerreroo youtube.com/hguerreroo
23. @hguerreroo DATA GATEWAY Other use cases • Data modernization & migration • Data firewall • API composition