Information Architecture Recommoning: How Standardization Enables Differentiation

This slide deck accompanies a talk presented at Data Mesh Live, centered around a key idea: data alone has no value. It only acquires value when it is used to support an organization in achieving its strategic goals.

To do this, treating data as a product is not enough. We also need to productize the management of metadata, which turns data into usable information, and domain knowledge, which allows that information to be contextualized and applied consistently and effectively.

The key is to manage the entire information architecture as a product, or better, as a collection of reusable and composable products

In this presentation, we explore how a platform strategy can move us beyond the traditional linear data value chain, creating a data value network that engages all actors in the organizational ecosystem in the shared management of the information architecture—a practice we refer to as data commoning.

Starting with data products and moving through data contracts, computational policies, domain ontologies, and the enterprise knowledge graph, we examine the essential components of a modern information architecture.

We then explore available techniques and operating models to design and implement an architecture that is modular, composable, and future-proof, built in a distributed and collaborative way.

Finally, we look at how a well-designed information architecture can become a strategic enabler, providing organizations with the agility needed to navigate increasing demand volatility and the uncertainty of the global landscape.

Transcript

1. Information Architecture Recommoning How Standardization Enables Differentiation
2. Andrea Gioia Hi there 👋 I'm Andrea Gioia, CTO at Quantyca, and Co-founder of blindata.io With 20+ years in the game, I have navigated the data universe up and down, one project at a time. LinkedIn: /andreagioia Github: /andrea-gioia
3. Andrea Gioia Hi there 👋 I'm Andrea Gioia, CTO at Quantyca, and Co-founder of blindata.io With 20+ years in the game, I have navigated the data universe up and down, one project at a time. LinkedIn: /andreagioia Github: /andrea-gioia https://amzn.eu/d/7WsNmFC
4. Beyond Managing Data Products
5. The need for speed Business as Usual Business as Usual Something changes Back to normal Project Business as Usual Change Rate Change Rate Time Time When change is an exception it’s important to be a good guesser When change is a constant it’s important to be a fast learner ECONOMY of SCALE ECONOMY of DIFFERENTIATION Volatility, Uncertainty, Complexity & Ambiguity
6. Economic theory of the firm 6 Sources of competitive advantages according to economic theories: 1. Five Forces (M.Porter): external industry forces and competitive dynamics. 2. Resource Based View (J. Barney): internal resources and capabilities. 3. Dynamic Capabilities (D. Teece): adaptive capacity to reconfigure resources and capabilities over time.
7. Bipolar organizations 7 Increase variety Decrease variations Economy of Differentiation Economy of Scale Generates value by increasing desired differentiation (i.e. variety) Promotes freedom and autonomy Manages customer facing services and applications Generates value by decreasing unnecessary differentiations (i.e. variations) Promotes control and governance Manage infrastructural resources and core systems From the ongoing interaction between these opposing poles should emerge an optimal balance between autonomy and governance, avoiding both anarchic drift or bureaucratic dictatorship
8. Schizophrenic data management 8 Increase variety (Engineer to Order) Decrease variations (Make to Stock) Economy of Differentiation Economy of Scale Monolithic data platform Siloed data applications
9. Data is a particular type of asset Reusability Data increases in value the more it is used that is, it exhibits increasing returns to use. The major cost of information is in its capture, storage and maintenance— the marginal costs of using it are almost negligible. Composability Data generally becomes more valuable when it can be compared and combined with other data. Connecting different data sources enables deeper insights, uncovers patterns, and supports better decision-making. 9
10. Recommoning data management 10 Increase variety (Engineer to Order) Decrease variations (Make to Stock) Standardize to differentiate (Compose to Order) Economy of Differentiation Economy of Scope Economy of Scale Monolithic data platform Siloed data applications Cross cutting Services Building blocks & Blueprints Governance Policies Data Products Shared pool of reusable and composable resources
11. Federated governance In theory good for efficiency; in practice: • far from serving local needs, • Ignores the power of accountability • generates bottlenecks The system can easily become … 👉 bureaucratic and inefficient. The governance of the shared portfolio of commons is managed by a federation of the distributed centers of power. The optimal balance between local autonomy and global governance is achieved by intentionally establishing shared, sustainable agreements. Distributed Federated Centralized In theory good for efficacy; in practice: • far from serving shared goals, • ignores network effects, • generates negative externalities The system can easily become … 👉 fragmented and ineffective. 11
12. Information architecture DATA ONTOLOGIES + Understand KNOWLEDGE INTELLIGENCE INSIGHT + Act METADATA Comprehend INFORMATION + Data management cannot be limited to just managing data Data management must manage the whole information architecture Collect Processing Cognizing Reasoning Sensing 12
13. Data Products
14. Pure Data Product A pure data product is a modular unit within the data architecture, tailored to the cognitive capacity of the responsible team and developed following product management principles to make a data asset accurate, relevant, combinable, and readily usable for future value creation. 14
15. Pure Data Product’s “ilyties” Data Product Data Asset Accuracy Relevance Reusability Composability VALUE Identify and maintain Share and multiply 15
16. Why care about pure data products? Engineer To Order Compose To Order 16
17. Beware of stock Pure data products that don’t support analytical applications (stock to order) don’t make sense, as unused data becomes a non-productive cost. 17
18. Pure data product anatomy Applications Data Asset +Metadata Infrastructure Interfaces Interfaces INTERNAL COMPONENTS Application components acquire, transform, and share data Infrastructural components provide storage and compute resources INTERFACE COMPONENTS Input Ports Output Ports Discovery Ports Observability ports Control Ports 18
19. 19 Data Contracts and Data Product Specifications
20. Unit of data and metadata management 20 DATA PRODUCT Discovery ports Output ports Data Plane Information Plane Data Product Team
21. Data Product Interoperability Interoperability Federated Governance Standardizes Self-Serve Platform Supports 21
22. Data Contract vs Data Product Specs A DPS can be used to describe a pure data product from multiple perspectives, including: 1. Types of exposed services (ports) 2. APIs of the exposed services 3. Applicable SLOs/SLAs for the services 4. Terms of use 5. Billing policies for service usage 6. Internal application components 7. Internal infrastructure components 8. Lifecycle management 9. Dependencies between products A DCS can be used to describe a data asset from multiple perspectives, including: 1. The data model 2. Constraints applied to different elements of the model 3. Quality metrics and corresponding tests 4. Quality indicators with associated SLOs/SALs 5. Data provenance and static lineage 6. Links to other data assets (syntactic links) 7. Connections to business terms or concepts (semantic links) OD M Data Product Descriptor Specification Open Data Contract Specification Data Contract vs. Data Product Specifications: Friends, Enemies, or Frenemies? @Medium
23. Data Contracts Syntactic & tech. interoperability Data Product Data Product Descriptor Internal Components API Populate Accepts & consume Shared Lifecycle Metadata Data 23 Data Contract Ports
24. 24 Knowledge Products
25. The semantic gap 25 Different data assets are composable when they are interoperable at the following levels: 1. Technological 2. Syntactic 3. Semantic Semantic interoperability is a major challenge.
26. Lost in translation 26 Modeling knowledge has a cost, but how much does it cost not to do it?
27. Implicit & explicit knowledge 27 I know the concept of tree because i know that … … a tree is not a mineral … a tree is a plant … a tree is not a brush … plus other facts that I have learned through experience or reasoning … a tree has a trunk Mineral Natural thing Plant Bush Tree Trunk :type of :type of :type of :type of :has Implicit & Personal Knowledge Model Explicit & Shared Knowledge Model Formalized into Grounded on
28. Ontology Mineral Natural thing Plant Bush Tree Trunk :type of :type of :type of :type of :has An ontology is … … an explicit specification of a conceptualization (Gruber 93) … a formal specification of a shared conceptualization (Borst 97) … a formal, explicit specification of a shared conceptualization (Studer 98) SHARED FORMALIZED
29. Knowledge Mesh 29 Self-serve Platform X as a Product X Domain Ownership Computational Governance Knowledge Mesh Data Mesh Socio Technical
30. Knowledge as a product 30 Enterprise Ontology Upper Ontology Domain Ontologies Subdomain Ontologies Ontology Lifecycle
31. 31 A federated modeling team composed of representatives from each business domain manage the definition of the enterprise ontology. This team can further organize by dividing responsibilities based on data domains. Marketing Sales EMEA Sales Nordics Operations Business Domains Knowledge Domains Ontology Factual Data Customer Product Order Knowledge Domain Owner Knowledge Domain Owner Knowledge Domain Owner Business Domain Owner Business Domain Owner Business Domain Owner Business Domain Owner Knowledge domain ownership
32. Federated Governance 32 Domain Ontologies Federated Governance Team Self-serve Platform GLOBAL POLICIES Defines Enforces Conform to
33. Self-serve Knowledge Platform 33 Utility Plane Control Plane Experience Plane IT Landscape SELF-SERVE PLATFORM Provides access to underlying infrastructural resources (e.g., storage and compute). Automates the design, development, deployment, and monitoring of knowledge products. Provides access to the enterprise knowledge graph
34. 34 Putting it all together
35. Semantic linking 35 infra API Data Contract Ports Exposes Internal Components Schema Links to Data Product Data Product Descriptor Enterprise Ontology Information Plane Knowledge Plane Data Plane Data Product Team Modelling Teams Dati Apps
36. Semantic linking (example) 36
37. From data products to EKG Upper ontology Semantic Interop. Data products Enterprise Ontology Data products 1. enable access to physical data asset 2. aggregate technical metadata related to exposed data 3. create the semantic link between physical data asset and business concepts modeled in the enterprise ontology Data products are a pivotal element in the incremental and distributed construction of a knowledge graph Domain ontology Physical Data Subdomain ontologies Syntactic Interop.
38. Knowledge graph architectures 38 Data Centric Architecture LOAD MAP Knowledge Warehouse Architecture Logical Knowledge Warehouse Architecture Materialized Knowledge Graph Virtualized Knowledge Graph
39. Federated conceptual modelling Federated Governance Federated Modelling Team Self-serve platform Schema Constraints API Enterprise Ontology Data Contracts Data Data Product Defines Populate Links to Semantic interoperability Syntactic & tech. interoperability Uses Enforces Promotes Data Product Team 39
40. Conceptual modeling process Ontology Data products Knowledge Graph Linking Knowledge Plane Concepts + Relationships Information Plane Data + Metadata Data Management Solution Deploy Deploy Iterate Business Cases/Questions Modeling Team Data Product Team Business Analysis Knowledge Modeling Data Product(s) Implementation
41. DATA INFORMATION KNOWLEDGE Data Product Developer Platform Data Product Catalog Knowledge Product Developer Platform XOps Platform 41
42. THANKS !