Prioritizing technical debt as if time and money mattered

Many code bases contain code that is overly complicated, hard to understand, and, hence, expensive to change and evolve. Prioritizing the technical debt to pay it off is a hard problem, as modern systems might have millions of lines of code and multiple development teams—no one has a holistic overview. In addition, there’s always a trade-off between improving existing code and adding new features, so we need to use our time wisely.

What if we could mine the collective intelligence of all contributing programmers and start to make decisions based on data from how the organization actually works with the code?

Adam Tornhill offers an approach that lets you prioritize the parts of your system that benefit the most from improvements so that you can balance short- and long-term goals based on data from how your code evolves. This new perspective on software development will change how you view code.

Transcript

1. @AdamTornhill PRIORITIZING TECHNICAL DEBT AS IF TIME AND MONEY MATTERED empear.com
2. Lehman’s “Laws” of Software Evolution Continuing Change “a system must be continually adapted or it becomes progressively less satisfactory” @AdamTornhill Increasing Complexity “as a system evolves, its complexity increases unless work is done to maintain or reduce it” M. Lehman, “Programs, life cycles, and laws of software evolution”, 1980
3. Are We Treating Symptoms Instead of the Real Issues? @AdamTornhill Long Lead Times Lack of Predictability Roadmap What the business see What the users experience Product Technical Debt Technical Debt
4. QUANTIFYING TECHNICAL DEBT? @AdamTornhill == ?
5. 4000 Years Ago => The Start of Recorded History @AdamTornhill 4000 Years of Technical Debt?
6. THE PERILS QUANTIFYING TECHNICAL DEBT @AdamTornhill Q: What behaviour do we reinforce by quantifying technical debt? important!
7. Quantifying Technical Debt Isn’t Actionable. @AdamTornhill @AdamTornhill There’s always a trade off between improving existing code versus adding new features. So how do we prioritise?
8. Version-Control — A Behavioral Data Source Commit: b557ca5 Date: 2016-02-12 Author: Kevin Flynn Fix behavior of StartsWithPrefix 8 27 src/Mvc.Abstractions/ModelBinding/ModelStateDictionary.cs 1 10 src/Mvc.Core/ControllerBase.cs 1 1 src/Mvc.Core/Internal/ElementalValueProvider.cs 1 39 src/Mvc.Core/Internal/PrefixContainer.cs Commit: fd6d28d Date 2016-02-10 Author: Professor Falken Make AddController not overwrite existing IControllerTypeProvider 8 1 src/Core/Internal/ControllersAsServices.cs 48 0 test/Core.Test/Internal/ControllerAsServicesTest.cs 13 0 test/Mvc.FunctionalTests/ControllerFromServicesTests.cs Commit: 910f013 Date :2016-02-05 Author Lisbeth Salander Fixes #4050: Throw an exception when media types are empty. 20 1 src/Mvc.Core/Formatters/InputFormatter.cs Social Information A Time Dimension Progress on Tasks Co-changing Files @AdamTornhill
9. CASE STUDY: PRIORITIZING TECHNICAL DEBT @AdamTornhill
10. Case Study: Android The Platform Framework Base in Numbers 3 Million Lines of Code 2,1 Million Lines of Java 2,000 Unique Authors @AdamTornhill
11. Case Study: Android The Platform Framework Base in Numbers 3 Million Lines of Code 2,1 Million Lines of Java 2,000 Unique Authors @AdamTornhill
12. Code Complexity Code Change Frequency Hotspot @AdamTornhill Principal Interest Rate
13. Putting Costs on Hotspots @AdamTornhill 76% defects => 76% unplanned work Collect process info from: - Jira, - Commit message tokens, or - Bug trackers (e.g. GitHub issues)
14. A Growing Problem Code Findings Low Cohesion, many responsibilities Overall Complex Methods, many conditionals Deeply Nested Logic, if-statements inside if-statements Primitive Obsession, missing a domain language Excess Function Arguments, missing abstractions
15. Where Do We Start? @AdamTornhill Actionable Insights? 20,000 Lines of Code! 74 Developers over the past 3 Months
16. Function Level Hotspots Parse Recommended functions to improve. Hotspots: X-Ray ActivityManagerService.java From https://pragprog.com/book/atevol/software-design-x-rays
17. X-Ray of ActivityManagerService.java @AdamTornhill
18. Source Code: 
 https://github.com/adamtornhill/code-maat Tooling: Try it on your own Code Track functions with git log -L :<funcname>:<file> @AdamTornhill https://codescene.io/
19. Why Hotspots Work @AdamTornhill
20. All Code is Equal …but some Code is more equal than others* * Sorry, George Orwell @AdamTornhill
21. Why You Don’t Have To Fix All Technical Debt 1 Year in Roslyn (C#, VB) 6 Years of Erlang 12 Years of Ruby on Rails Each file in the system Change Frequency Reference: Your Code as a Crime Scene, ISBN:1680500384 Prioritize improvements here! Ignore the long tail
22. Hotspots: A Language Neutral Approach @AdamTornhill Kubernetes - Go Language Kubernetes in Numbers: • 3.5 Million Lines of Code • Hotspots: 0.7 %, but 10% of the development effort Core CLR - C++ and C# Core CLR in Numers: • 3.6 Million Lines of code • Hotspots: 1.5 %, but 22% of the development effort Prioritize Technical Debt with Hotspots: ~ 1 - 4% of the Total Code
23. @AdamTornhill Blog on Software Evolution, Technical Debt, and Behavioral Code Analysis https://www.empear.com/blog/ Software Design X-Rays: Fix Technical Debt with Behavioral Code Analysis https://pragprog.com/book/atevol/software-design-x-rays adam.tornhill@empear.com Further Reading Showcases with Interactive Analysis Results https://codescene.io/showcase