Collibra-Unlocking_Business_Value_with_Agentic_AI_Governance.pdf

Transcript

1. Keynote Unlocking the business value of agentic AI Governance
2. Copyright © 2025, Collibra. All rights reserved. 2 Today’s Collibra speaker Wouter Mertens Field CTO AI Governance
3. Agenda 01 The AI Speed Trap: Why execution lags behind ambition 02 The cost of ignoring governance 03 Getting AI right from the start 04 What agents change (and why oversight must evolve) 05 Live demo: Governance as the backbone of trusted AI 06 Final takeaways & next steps 07 Audience Q&A
4. Copyright © 2025, Collibra. All rights reserved. The AI Speed Trap Why execution lags behind ambition 4 Wouter Mertens, AI Governance Field CTO
5. Copyright © 2025, Collibra. All rights reserved. The AI Speed Trap 5 Going fast without visibility leads to failure 52% of AI projects never make it into production *** 56% take 6–18 months to deploy GenAI projects 8 months From prototype to production***
6. Copyright © 2025, Collibra. All rights reserved. Agentic AI is no exception 6 Speed is essential, but without governance, it leads you off course. 82% of organizations already use AI agents, but only 44% of organizations report having policies in place to secure them* Over 40% of agentic AI projects will be canceled by end of 2027 * “Most agentic AI projects right now are early stage experiments or proof of concepts that are mostly driven by hype and are often misapplied,” said Anushree Verma, Senior Director Analyst, Gartner. “This can blind organizations to the real cost and complexity of deploying AI agents at scale.
7. Copyright © 2025, Collibra. All rights reserved. The cost of ignoring AI governance 7
8. Copyright © 2025, Collibra. All rights reserved. No clear accountability unclear ownership and roles Decisions made in isolation disconnected teams/priorities Limited visibility no traceability across lifecycle No policy guardrails risk of non-compliance and bias Governance seen as optional leads to inconsistent practices 8 When governance is missing, AI becomes disconnected, risky, and directionless
9. Copyright © 2025, Collibra. All rights reserved. Introduced late causes delays and rework Siloed in compliance lacks shared responsibility Manual and bureaucratic reduces agility Reactive posture damage control only focus Disconnected oversight disaligned data+ models+ agents 9 When governance is weak, AI progress breaks down
10. Copyright © 2025, Collibra. All rights reserved. [ 10 ] If nothing is done properly, AI project stall or inevitably fail Now In 3 months In 9 months x100 Governance Debt Becomes a Systemic Risk +10 Missteps are still manageable Complexity spreads faster than control x50
11. Copyright © 2025, Collibra. All rights reserved. Getting AI right from the start with day-one governance 11 From idea intake to governed model and agent related outcomes.
12. Copyright © 2025, Collibra. All rights reserved. 12 Frame your AI assets at intake Apply governance early, where AI use begins 01 Day-one governance
13. Copyright © 2025, Collibra. All rights reserved. 13 Frame the use cases and AI assets at intake Start with business value, feasibility, and risk in mind Define the objective and expected outcome of each AI initiative and underlying models Validate availability and sensitivity of training data from the start Apply oversight to existing and new models
14. Copyright © 2025, Collibra. All rights reserved. 14 Are you solving the right problem? Ask yourself Are we mistaking technical excitement for real business need? Could we validate feasibility faster with smaller, simpler steps in a single tool ? Are we aligned on the model’s purpose, not just its capability?
15. Copyright © 2025, Collibra. All rights reserved. AI model registry Quickly track and manage all AI models in one place boosting transparency, consistency, and control across teams Bring AI models into a governed lifecycle and track their evolution Connect models to data, use cases, and policies ensuring auditability and regulatory readiness from day one. Track, review and govern AI models for increased auditability and compliance New
16. Copyright © 2025, Collibra. All rights reserved. 16 Align roles and federate around AI Frame your AI assets at intake Start with business value, feasibility, and risk in mind 01 02 Day-one governance
17. Copyright © 2025, Collibra. All rights reserved. 17 Align roles and federate accountability across the lifecycle Ensure clarity and accountability from idea to deployment and beyond Assign clear model ownership with sign-off checkpoints for each phase Provide shared templates and handoffs between data, model, and risk teams Address lifecycle traceability — including prebuilt, third-party, or legacy models
18. Copyright © 2025, Collibra. All rights reserved. 18 Who owns what and who signs off? Ask yourself Are sign-offs meaningful — or just procedural? Is model traceability in place, for inherited or new models? Are we duplicating efforts because teams aren’t sharing learnings?
19. Copyright © 2025, Collibra. All rights reserved. Lifecycle sign-offs 19 Add structured sign-offs directly into the AI model lifecycle to ensure responsible oversight. Capture rationale, ownership, and outcomes for each sign-off, supporting compliance and ongoing model evaluation. Ensure accountability and AI model integrity at every stage of the lifecycle. Embed oversight and traceability into every AI model decision—with clear sign-offs and accountability
20. Copyright © 2025, Collibra. All rights reserved. 20 Align roles and federate around AI Ensure continuous accountability from idea to deployment Frame your AI assets at intake Start with business value, feasibility, and risk in mind 01 02 Activate, monitor, and learn 03 Day-one governance
21. Copyright © 2025, Collibra. All rights reserved. 21 Activate, monitor, and learn from impact Measure impact, monitor risks, and improve continuously Define success metrics tied to business KPIs and model and use case performance Monitor for drift, bias, and trust signals once the model is live Use feedback to improve future use case framing and team learning
22. Copyright © 2025, Collibra. All rights reserved. 22 Is it working and what are you learning? Ask yourself Are we measuring what matters or just what’s available? Do we catch drift or misuse before it creates impact? Are we building a knowledge base of what works and what doesn't?
23. Copyright © 2025, Collibra. All rights reserved. Govern AI data quality, accuracy, and accessibility Product Team Engineering Team GRC Team Data Office Security Team Legal Team DATA AI Establish, track, validate & enforce AI compliance controls Set security protocols & conduct stress testing Develop, test, and monitor AI solutions Design AI products with compliance and customer needs in mind Vet AI risks (legal, compliance, ethical, security and performance) AI Governance Policy AI Acceptable Use Policy AI Literacy & Training AI Governance Committee AI use cases Activate your AI lifecycle
24. Copyright © 2025, Collibra. All rights reserved. 24 Align roles and federate around AI Ensure continuous accountability from idea to deployment Activate, monitor, and learn Measure impact, monitor risks, and improve continuously 01 02 03 Day-one governance Frame your AI assets at intake Start with business value, feasibility, and risk in mind
25. Copyright © 2025, Collibra. All rights reserved. Agentic AI Governance Why agents can’t be an afterthought 25
26. Copyright © 2025, Collibra. All rights reserved. Model and agentic governance Lineage & Documentation Track model cards and datasets to maintain compliance and reproducibility Validation Metrics Measure accuracy, fairness, and robustness to ensure quality Drift Monitoring Track performance changes over time to maintain reliability Access Control Manage model usage rights to prevent misuse Core Model Governance Autonomy Boundaries Define explicit permissions and operational limits Tool & API Usage Approve and monitor external tool execution patterns Memory & State Control persistent information storage and retention Behavior Monitoring Log reasoning, steps, and real-time actions for oversight Agentic Governance (Additional Controls)
27. Copyright © 2025, Collibra. All rights reserved. 1. Hidden agent activity Challenges Agentic specific Governance challenges 2. Humans left out of decisions Set up regular audits and alerts to detect high-risk actions and escalate them for human review. 3. Uncontrolled third-party integration Require vendors to disclose agent usage and evaluate their safeguards in high-risk areas. 4. Poor agent transparency Audit agent decisions and activities to enable traceability and post-hoc reviews. 5. Diffuse Accountability Define ownership, liability, and response protocols across all stakeholders in contracts and policy. Create and maintain a central inventory of AI agents with clear ownership and accountability. Recommendations
28. Copyright © 2025, Collibra. All rights reserved. Start centralizing oversight of agents in one place 28 Start centralizing AI agents mapped to business context and policy alignment Ensure every AI agent has a clearly assigned owner and tracked status Proactively register and oversee AI agents before their deployment starting with Azure AI Foundry as the first supported environment Now in Preview: First Tool-Agnostic AI Agent Registry
29. Copyright © 2025, Collibra. All rights reserved. Collibra in action A practical use case 29
30. Copyright © 2025, Collibra. All rights reserved. 30 Govern models from new integrations such as Azure AI Foundry and MLflow with traceability and lifecycle control Embed oversight and traceability into every AI model decision with signoff Enhance visibility and oversight over AI models and agents Now you can
31. Copyright © 2025, Collibra. All rights reserved. Final takeaways & next steps 31
32. Final takeaways #1 Governance isn’t a blocker — it’s your AI accelerator. The fastest, safest AI programs are the ones governed from the start. #2 Start with business value, not just technical ambition. Anchor every AI initiative to measurable, strategic outcomes. #3 Engage the full ecosystem in a federated way. with multidisciplinary teams (data, risk, IT, AI) and align early with executives #4 Unify your AI landscape: use cases, models, and agents. Don’t govern in silos. Connect every layer for traceability and oversight. #5 Collibra makes it real. A platform built to help you scale AI — reliably, traceably, and safely.
33. Copyright © 2025, Collibra. All rights reserved. Going further : Collibra resources https://www.collibra.com/resources/how-to-govern-ai-agents-at-scale https://www.collibra.com/resources/ai-literacy-empowering-ai-stakeholders-in-the-gener ative-ai-era
34. New York, NY • October 8 London, U.K. • October 16 Walldorf, Germany • October 28 AI-powered governance, everywhere Register today
35. Thank you
36. Copyright © 2025, Collibra. All rights reserved. Q&A 36