Datashift___Air_France_KLM-From_Data_Maze_to_Data_Marketplace__Air_France___KLM_s_Data_Marketplace_journey.pdf

Transcript

1. FROM DATA MAZE TO DATA MARKETPLACE
2. Martijn Severijns Manager Data Governance & Quality Datashift Marjolein Daeter Product owner Data Management KLM
3. Data is everywhere. Impact is not. Let’s change that. Datashift in 2025 120+ Clients in 10 years with 400+ Project realizations 150+ Datashifters 15+ industries in EMEA Focus on Belgium & The Netherlands 5 Domains To build data driven organizations from Strategy to Implementation Trusted Collibra partner for 9+ years 40+ certified Collibra experts, 150,000 hours of hands-on experience & recognized, with our clients, by 4 Collibra Excellence Awards “Why Data Governance?” “How much longer can you afford to grow without it?” -> Trust data: innovate, comply & act with Impact!
4. 4 AIR FRANCE KLM IN 2024 98 million passengers transported 564 aircraft in operation 78.000 people 320+ destinations In 100+ countries 911,000 freight ton to 120+ direct intercontinental connection 3000 aircraft maintained For 200+ customers 25+ lines of business and 120+ Data Ownership Domains
5. 5 REDUCING THE DISTANCE BETWEEN USERS AND DATA x The Situation Today With One Search Where is the data? Ask a friend Send E-mails Schedule Meeting(s) Ask Others Find the data owner Ask for permission Get approvals Connection String Firewall Access Connect It’s the Wrong Data! Search Download / Provision / Access It’s the Data I need! Explain the Need Am I allowed? Find the technical owner Fix the Connect Ask the friend of a friend Review the data Review the data
6. Source : The Business Impact of Data Intelligent Management: How Data Intelligence Strategies Help Organizations Drive Success and Mitigate Risk. Forrester 2020. Finding data, getting access, validating data, ensuring it meets GDPR requirements, determining if analysis has already been done, cleansing and aggregating data into usable datasets 76% Performance analysis 14% Sharing results with organization 10% BUSINESS IMPACT OF POOR METADATA MANAGEMENT 6
7. AFKL DATA CATALOG The Data Catalog uses metadata – data that describes data It is categorized and searchable For the whole organization
8. 8 METADATA AT AIR FRANCE KLM 8 How does this data relate to other data How long can you store and use the data What processes use, produce and change the data? How does data go from creation to reports? Key capabilities: Definitions Source systems Owners Compliance Sensitivity Data Models Customer Engine Shipment Flight Concepts : Price
9. 9 AFKL Data Catalog How people, technology and process work together Data Governance How to onboard business in the meta data management transformation Change management TO BE SUCCESSFUL A DATA CATALOG IMPLEMENTATION NEEDS TO ADDRESS CHANGE AND GOVERNANCE • Target Operating model with roles & responsibilities • Data & IT architecture • Shared data definitions AFKL • Communication • Adoption • Trainings AFKL
10. 10 FROM CATALOG TO MARKETPLACE Access Management Defining ownership A NEED FOR A SHARED STRUCTURE AND GOVERNANCE IN A CENTRAL, EASY TO USE TOOL Quality of data
11. 11 DATA MARKETPLACE, THE SOLUTION DESIGNED TO MEET THE CHALLENGES & ACCELERATE DATA-DRIVEN TRANSFORMATION DATA MARKETPLACE • One stop shop • High quality data & AI assets • Intuitive user experience for consuming • Integrated governance
12. 12 AFKL DEFINITION OF A DATA PRODUCT From data … Towards Data Product A Data product is a certified reusable dataset which provides valuable information to consumers through one or multiple technical applications.
13. 13 AFKL DEFINITION OF A DATA PRODUCT Self-describing Interoperable Secure Trustworthy Addressable Discoverable A Data Product
14. 14 Data product creation/certification process Step 1 Step 2 Step 3 Step 4 Data Product Owner Data User Data Marketplace Governor Data officer Data Owner(s) Propose a Data Product Assess the value of the Data product Prepare the Data Product Maintain, update and promote Asses the Data product Publish the Data Product Consume
15. 15 DATA MARKETPLACE JOURNEY Q3 2023 Q2 2024 Q4 2024 Q2 2025 Q3 2025 Q4 2025 Q1 2026 Take aways Integrate with the Data Catalog Start Marketplace project Start identifying the requirements for a Data Marketplace, start working on the first key Data Products Start Data Catalog project Start implementation of a new Data Catalog tool. Start development of Marketplace features Working on workflows for the creation of Data Products, certification and access requests Incremental steps Governance Certification process of first Data Products The first 3 Data Products participated in the certification process Rollout of the Marketplace The technical go-live of the marketplace platform Filling the Marketplace Adding Data Products to our marketplace Launch! The opening of our data marketplace
16. Let’s share a few typical pitfalls while introducing ‘Data Products’ and the Data Marketplace 6 typical pitfalls while setting up a Data Marketplace Governance, privacy & licensing bolted on late Slow manual approvals, unclear usage rights and compliance issues result in risks and blocked releases. → standardized licenses/usage terms, procedures & automated ABAC/RBAC access, audit logs. Low Data Product Quality & no SLAs Data Product listing looks useful, but there is no adoption due to loss of trust in data and faster alternatives. → Introduce SLAs/Data Product Contracts, Automate DQ monitoring & discontinue alternatives Mistaking a Platform Façade for a Marketplace Shiny front-end and tooling, but little adoption. No tangible business outcome for producers nor consumers. → 3 to 5 flagships, measure progress: product usage / amount of prod. & consumers AI Models & Agents without guardrails Marketplace will list AI models as “data products” without transparency. Output may hallucinate or misuse sensitive data. → AI Product contracts, human-in-the-loop workflows, Bias, explainability & accuracy. Lack of a ‘Playbook’ Stakeholders get lost / misinterpret policies. Resulting in ‘shadow data exchanges’ & nothing on marketplace. → collection of step-by-step processes; that guide teams on how to publish data Poor discoverability & weak metadata Consumers can’t find or trust the right dataset: 0 n → metadata (e.g. underlying data, ownership), quality score, lineage/sample data
17. There are a lot of important topics that impact your program like Data Maturity, Data Quality, Regulatory Compliance, Privacy & Data security,… Let’s focus on a few more concrete features that can help to embed your Data Marketplace. 6 typical pitfalls while setting up a Data Marketplace Lack of a ‘Playbook’ Stakeholders get lost / misinterpret policies. Resulting in ‘shadow data exchanges’ and no input for the marketplace. → collection of step-by-step processes; that guide teams on how to publish data Governance, privacy & licensing bolted on late Slow manual approvals, unclear usage rights and compliance issues result in risks and blocked releases. → standardized licenses/usage terms, procedures & automated ABAC/RBAC access, audit logs. Low Data Product Quality & no SLAs Data Product listing looks useful, but there is no adoption due to loss of trust in data and faster alternatives. → Introduce SLAs/Data Product Contracts, Automate DQ monitoring & discontinue alternatives Poor discoverability & weak metadata Consumers can’t find or trust the right dataset: 0 n → mandatory metadata (e.g. underlying data, ownership,…), quality score, lineage/sample data,… Mistaking a Platform Façade for a Marketplace Shiny front-end and tooling, but little adoption. No tangible business outcome for producers nor consumers. → 3 to 5 flagships, measure progress in usage: product usage / amount of producers & consumers AI Models & Agents without guardrails Marketplace will list AI models as “data products” without transparency. Output may hallucinate or use sensitive data incorrectly. → Introduce AI Product contracts, human-in-the-loop workflows for use cases. Bias, explainability and accuracy testing before publishing. There are a lot of important topics that impact your program like Data Maturity, Data Quality, Regulatory Compliance, Privacy & Data security,… Let’s focus on a few more concrete features that can help to embed your Data Marketplace. Other related topics • Translate into Business language • Tailor to Stakeholder’s Roles • Continuous, not one-off Communication Focus • 1 clear & consistent story • translated into concrete steps (playbook) • Ambitious & realistic Team • Smart & multidisciplinary team you trust • From vision to execution • Establish a Shared Vocabulary and Principle & 3 concrete tips
18. Martijn Severijns Manager Data Governance & Quality Datashift Marjolein Daeter Product owner Data Management KLM Let’s meet at… Questions?
19. 19 DATA PRODUCT: DEFINITION AND CHARACTERISTICS Discoverable “Can I find it on the Data Marketplace? Is there a risk of overlap with another Data Product” Ensures data visibility and democratization, and prevents duplication of effort Addressable “After I am approved to access the Data Product, do I know how I can access it? Ensures that Data Products are accessible via identified way(s) of consumption Trustworthy “To whom do /I escalate if there are problems? Is there an SLA or quality level I can count on ?” Ensures that consumers trust the Data Products’ reliability by upholding standards Secure “Do I see the data I need to see without risk of breaching data regulations or company policy?” Ensures that Data Product meet standards of security & authorization Self-Describing “Do I understand how to use it from the Marketplace ? Can I find documentation in the Marketplace ?” Ensures sufficient metadata is provided so that consumers can leverage self-service The Data Products are aligned to business strategies and serve concrete business needs (use cases) Value Interoperable “Is the data referenced in the Data Catalog and compliant with data management standards ? ” Ensures that Data Products can be easily integrated with other Data Products