Leading Effective Engineering Teams in the AI Era

In this talk, we’ll explore how leaders should approach the “70% Problem” – how AI excels at routine coding tasks but still requires significant human expertise for the critical remaining 30% that makes the difference between functional code and excellent software. How teams approach the “30% that matters” will be vital moving forward.

Drawing from real-world case studies at companies like Google, GitHub, and Microsoft, I’ll share practical strategies for engineering leaders navigating this new landscape. We’ll examine how the role of technical leadership is evolving from hands-on problem-solving to strategic guidance and ethical oversight.

You’ll learn:

- How to establish effective “trust but verify” processes for AI-generated code
- Strategies to prevent skill erosion and maintain code quality
- Approaches for upskilling both junior and senior developers to thrive alongside AI
- Techniques for measuring impact beyond speed to ensure long-term code quality
- Frameworks for responsible AI governance and usage in your organization

Whether you’re skeptical or enthusiastic about AI coding tools, this talk will provide a balanced, pragmatic guide to leading effective engineering teams in an era where the question isn’t whether to use AI, but how to use it to build better software while keeping humans at the center of the creative process.

To learn more about AI-Assisted Engineering, check out my book "Beyond Vibe Coding"

Transcript

1. LEADING EFFECTIVE ENGINEERING TEAMS IN THE AI ERA ADDY OSMANI
2. FOLLOW ALONG SLIDES https://leetai.addy.ie
3. Maybe joke at the start “I am happy to announce we have all been replaced by AI”
4. WILL EVERY ENGINEER BECOME AN ORCHESTRATOR OF MULTIPLE CODING AGENTS? THIS SHIFTS US FROM IMPLEMENTORS TO DIRECTORS OF HUMAN-AI COLLABORATION
5. ENGINEER AS CONDUCTOR: FOCUSES ON A SINGLE TASK, STEERING THE AGENT’S BEHAVIOR DYNAMICALLY - TWEAKING, INTERVENING, AND GUIDING WHILE IT RUNS. ENGINEER AS ORCHESTRATOR: MANAGES A FLEET OF AGENTS WORKING IN CONCERT ON MULTIPLE, INTERDEPENDENT TASKS.
7. Claude Code Gemini CLI Cursor Copilot Bolt / Lovable / v0
8. 90% OF DEVELOPERS USE AI TOOLS 84% PER 2025 STACK OVERFLOW DEV SURVEY
9. AI IS NOT ABOUT WRITING MORE CODE FASTER. IT'S ABOUT BUILDING BETTER SOFTWARE.
10. AI-ASSISTED ENGINEERING VIBE CODING USE HIGH-LEVEL PROMPTS TO RAPIDLY GENERATE CODE BY TRUSTING AN AI'S OUTPUT, PRIORITIZING SPEED OVER RIGOROUS REVIEW. USING AI AS A SUPERVISED TOOL WITHIN A STRUCTURED DEVELOPMENT PROCESS WHERE THE HUMAN MAINTAINS FULL CONTROL AND ACCOUNTABILITY.
11. WHERE CAN AI IMPROVE DX?
12. WHERE CAN AI IMPROVE DX?
13. WHERE CAN AI IMPROVE DX?
14. WHERE CAN AI IMPROVE DX?
15. A GOOGLE TEAM’S AI CODING JOURNEY
16. THE AI-NATIVE ENGINEER: ASKS FOR EVERY TASK: “COULD AI HELP ME DO THIS FASTER, BETTER, OR DIFFERENTLY?”.
17. THE PILOT IS THE FIRST CHAPTER OF A MUCH LARGER AI STORY Credit: henrythe9th
18. VIBE CODING PLAUSIBLE DENIABILITY FOR ALL YOUR TECHNICAL DEBT
19. SKEPTIC: I ASKED THE AI TO REFACTOR OUR CODEBASE. IT'S NOW ALL EQUALLY TERRIBLE.
20. Source: Infosys AI SKEPTICS TO EVANGELISTS
21. HACK WEEKS ARE A SAFE SPACE TO LEARN, PLAY & BUILD CONFIDENCE OR JUST BLAME AI FOR MISTAKES
22. AI TOOLS DON'T REPLACE EXPERTISE - THEY AMPLIFY IT. THE MORE SKILL YOU BRING AS AN ENGINEER, THE MORE POWERFUL THE RESULTS YOU'LL GET.
23. 10% 30% 21% 13% MEASURED VELOCITY GAIN FOR DEVELOPERS ATTRIBUTED TO AI GENERATED BY AI (AND THEN REVIEWED AND ACCEPTED BY ENGINEERS) FASTER CODING ATTRIBUTED TO AI, IN A CONTROLLED UX STUDY MONTHLY CODE CHANGES SUBMITTED FOR DEVELOPERS USING AI CODING TOOLS
24. PROMPT ENGINEER: A FANCY WAY TO SAY “AI WHISPERER”
25. PROMPT ENGINEERING FAILURES STEM FROM CONTEXT MISMANAGEMENT https://www.instagram.com/p/DMc4ujyTuk6/?img_index=6
26. CONTEXT IS KING. CONTEXT- ENGINEERING IS THE NEW ESSENTAL SKILL. . SPEC, FILES, EXAMPLES, CONSTRAINTS. IT’S MORE THAN JUST A PROMPT.
27. ADD CONTEXT CONTEXT WINDOW USE CONTEXT IN EDITORS & IDES
28. https://www.youtube.com/watch?v=ysPbXH0LpIE
29. https://context.addy.ie
30. PLAN BEFORE YOU ACT FOR BETTER AI OUTCOMES. DON’T JUMP INTO CODING RIGHT AWAY SPEC-DRIVEN DEVELOPMENT
31. CREATE A MEMORY ANCHOR - A COMPOUNDING LEARNING LOOP FOR AI AGENTS, HAVE THEM DISTILL INSIGHTS AFTER EACH TASK. THIS MAKES THE NEXT RUN FASTER AND HIGHER QUALITY. LEARNINGS.MD Lots of fl exibility here for data (context.md, learnings.md, decisions.md) that persists across sessions
32. NEVER COMMIT CODE YOU CAN’T EXPLAIN TO OTHERS OR EVEN YOURSELF
33. READING CODE BECOMES THE JOB. BUILD THIS MUSCLE. YOU WILL ALWAYS READ MORE CODE THAN YOU WRITE
34. VIBE-CODING IS A GIFT FOR PROTOTYPING. INTERACTIVE PROTOTYPES > STATIC MOCKS
35. SHOPIFY: AI AS CULTURAL CATALYST LEADERSHIP-LED, INFRASTRUCTURE-FIRST, DEMOCRATIZED ACCESS
36. VIBE CODE WITH YOUR VOICE SPEAKING PROMPTS ALOUD IS OFTEN FASTER THAN TYPING. HUMANS SPEAK 3-5X FASTER THAN THEY TYPE (150+ WPM VS. 40-80 WPM).
37. AI IS A FORCE MULTIPLIER. WHAT HELPS THE HUMAN HELPS THE AI. TESTS, LINTING, CI
38. TESTS ARE A SAFETY NET. THEY DE-RISK AI CODING. BTW…IF YOU PLAN TO USE AI FOR WRITING TESTS, IT'S CRUCIAL TO ADOPT A STRATEGY THAT EMPHASIZES HUMAN OVERSIGHT
39. AI REALITY CHECK WHAT THE DATA SAYS ABOUT PRODUCTIVITY, TRUST, AND TEAM VELOCITY Source 2025 DORA and Faros AI Productivity fi ndings
40. ADOPTION IS UP, TRUST IS DOWN FAVORABLE VIEWS FELL FROM 70 TO 60% IN TWO YEARS 46% ACTIVELY DISTRUST AI ACCURACY 30% REPORT LITTLE TO NO TRUST IN AI GENERATED CODE
41. VIBE CODING VIBE DEBUGGING
42. CHROME DEVTOOLS MCP LETS AGENTS "SEE" AND INTERACT WITH THE LIVE BROWSER ENVIRONMENT.
43. INDIVIDUAL OUTPUT IS SURGING WITH AI CODING, DEVELOPERS COMPLETE 21% MORE TASKS 2025 DORA/FAROS AI PRODUCTIVITY REPORT MERGE 98% MORE PULL REQUESTS OVER 80% SAY AI HELPS THEIR PERSONAL PRODUCTIVITY TASKS PRS SENTIMENT
44. “I FEEL LIKE AI SOMETIMES WRITES BETTER CODE THAN I DO FOR CERTAIN THINGS, MAINLY BECAUSE I FEEL LIKE IT’S BEEN TRAINED REALLY WELL”
45. WITH AI CODING PULL REQUEST REVIEW TIMES INCREASED BY 91% ALL PRODUCTIVITY GAINS ABSORBED BY HUMAN VERIFICATION 2025 DORA/FAROS AI PRODUCTIVITY REPORT
46. “AI SOLUTIONS THAT ARE ALMOST RIGHT, BUT NOT QUITE, ARE NOW MY BIGGEST TIME SINK. THE CODE LOOKS PLAUSIBLE BUT I END UP SPENDING MORE TIME FIXING THOSE ‘HELPFUL’ SUGGESTIONS.” TOP FRUSTRATION: ALMOST RIGHT CODE. 66% CITE IT AS THE MAIN TIME SINK.
47. WHERE AI HELPS MOST GREENFIELD AND PROTOTYPING BOILERPLATE AND REPETITIVE CODE UNIT TEST GENERATION DOCUMENTATION AND SIMPLE API INTEGRATIONS LOW-CONTEXT, HIGH-REPETITION WORK
48. WHERE AI STRUGGLES COMPLEX LEGACY CODEBASES SECURITY CRITICAL IMPLEMENTATIONS AMBIGUOUS REQUIREMENTS CROSS SYSTEM LOGIC HIGH CONTEXT, NOVEL PROBLEMS
49. KEEP VIBE-CODING. WE CAN ALWAYS FIX IT LATER.
50. THE LEADERSHIP PLAYBOOK FOR AI-ASSISTED ENGINEERING
51. STRATEGY 1: MITIGATE SKILL EROSION. GROW CRITICAL THINKING.
52. SOFTWARE ENGINEERS ARE PAID TO SOLVE PROBLEMS, NOT JUST WRITE CODE. BUILDING SOFTWARE IS 10% CODING AND 90% CRITICAL THINKING - A TASK YOU CAN'T COMPLETELY DELEGATE TO AN LLM.
54. SOMETIMES THAT "LAST 30%" IS WHAT THE SENIORS SEE AND THE JUNIORS DON'T. THE 70% PROBLEM
55. 54% OF LEADERS EXPECT TO REDUCE JUNIOR HIRING THE BAR IS RISING, THE ROLE IS EVOLVING THE LEADDEV AI IMPACT REPORT 2025
56. EVOLVE CODE REVIEWS INTO LEARNING MOMENTS SHIFT FOCUS FROM "DOES IT WORK?" TO "DO YOU UNDERSTAND WHY IT WORKS?"
57. SENIOR GUIDES STRATEGY, JUNIOR DRIVES IMPLEMENTATION, AI ASSISTS PAIR PROGRAMMING + AI = TRIO PROGRAMMING
58. REGULAR "NO-AI CHALLENGES" KEEP SKILLS SHARP LIKE PILOTS TRAINING WITHOUT AUTOPILOT—ESSENTIAL FOR WHEN YOU NEED IT MOST
59. AI’S BASELINE IS “HEY, IT WORKS”: ENGINEERS MUST ENSURE IT'S SECURE, MAINTAINABLE, AND PRODUCTION- READY MICROSOFT COPILOT STUDY: 26% FASTER TASK COMPLETION WITH NO QUALITY DEGRADATION
60. MOVE BEYOND VANITY METRICS TO MEANINGFUL INDICATORS STRATEGY 2: MEASURE WHAT TRULY MATTERS
61. 60% CITE LACK OF CLEAR METRICS AS BIGGEST AI CHALLENGE ONLY 18% MEASURE IMPACT SYSTEMATICALLY 2025 LEADDEV AI IMPACT REPORT
63. A THREE-PILLAR MEASUREMENT FRAMEWORK THROUGHPUT & VELOCITY QUALITY & STABILITY DEVELOPER EXPERIENCE PULL REQUEST CYCLE TIME IS A TRUTH TELLING METRIC MEASURE TOTAL TIME FROM FIRST COMMIT TO MERGE CHECK FAILURE RATES AND THE USER EXPERIENCE WHAT PERCENTAGE OF DEPLOYMENTS REQUIRE REMEDIATION? QUALITATIVE SURVEYS REVEAL THE HUMAN STORY "DID AI HELP YOU COMPLETE WORK FASTER THIS WEEK?" "HOW CONFIDENT ARE YOU IN THE QUALITY OF CODE YOU SHIPPED?" "RATE YOUR FRUSTRATION LEVEL WITH CURRENT TOOLS"
64. ENABLE SAFE EXPERIMENTATION WITH CLEAR BOUNDARIES STRATEGY 3: ESTABLISH HUMAN- CENTERED GUARDRAILS
65. SAFETY STANDARDS BACK IN THE DAY
66. MOST DON'T KNOW THEIR COMPANY'S AI POLICY
67. LEADERSHIP'S ROLE : RESPONSIBLE AI USAGE "SET CLEAR AI BOUNDARIES: NO PROPRIETARY CODE IN PUBLIC TOOLS, VERIFY LICENSING, MAINTAIN HUMAN ACCOUNTABILITY.” "ESTABLISH AI POLICIES BEFORE PROBLEMS ARISE. ETHICS, SECURITY, AND LEGAL COMPLIANCE CAN'T BE AFTERTHOUGHTS.” "YOUR JOB: CREATE GUARDRAILS THAT LET TEAMS INNOVATE SAFELY WITH AI TOOLS."
68. THE MOST VALUABLE SKILLS IN THE AI ERA CRITICAL THINKING AND PROBLEM ANALYSIS ARCHITECTURAL DESIGN AND SYSTEM THINKING COMPLEX PROBLEM-SOLVING CLEAR COMMUNICATION AND SPECIFICATION
69. CHANGE MANAGEMENT IS AS IMPORTANT AS TECHNOLOGY: PEOPLE, PROCESS, AND CULTURE.
70. You’re absolutely right
71. USE AI TO BUILD BETTER SOFTWARE FASTER BUT ALWAYS WITH HUMAN OVERSIGHT, CREATIVITY AND PURPOSE GUIDING THE WAY QUESTIONS & DISCUSSION @ADDYOSMANI
72. THANK YOU & LEARN MORE BEYOND VIBE CODING BUILDING WEB APPS WITH BOLT LEADING EFFECTIVE ENG TEAMS ADDYOSMANI.COM