Code Health Score: Gamifying the Reduction of Technical Debt at Slack

Transcript

1. CODE HEALTH SCORE GAMIFYING REDUCTION OF TECH DEBT AT Valera Zakharov
2. TECH DEBT? NEVER HEARD OF IT… ~ No Engineer. EVER.
3. WE ARE HERE TO SHIP FEATURES TO USERS
4. A HEALTHY CODEBASE ALLOWS US TO SHIP FASTER AND SAFER
5. AT SCALE, AD-HOC CLEAN UP IS NOT ENOUGH
10. 3RD-PARTY SOLUTIONS
11. SIMPLE HEALTH SCORE RECIPE ▸ Count problems ▸ Multiply each by how bad they are ▸ The score = 0 – (the sum of weighted problems)
12. A NAIVE IMPLEMENTATION ▸ Examine each fi le in the codebase ▸ For each check, return weighted score ▸ Aggregate by fi le & upload to a backend ▸ Display in a pretty dashboard
13. NAIVE IMPLEMENTATION HEALTH_SCORE_CHECK INTERFACE
14. ADDING A NEW CHECK = DEPLOYING NEW CODE
15. INTRODUCING A NEW CHECK REQUIRES DEPLOYING NEW CODE DECLARATIVE CONFIGURATION + USAGE_CHECK
16. MY CODEBASE IS LARGE. EXAMINING EACH FILE IS COSTLY.
17. MY CODEBASE IS LARGE HELP CRAWLER DO LESS WORK WITH FILTERS TAKES 55 SEC TO CALCULATE FOR ~30K KOTLIN FILES WITH ~2M LOC
18. BUT IT WASN’T ME WHO MADE THIS MESS!
19. TECH DEBT BY FUNCTIONAL AREA CODE OWNERSHIP ▸ Every fi le in codebase is mapped (ensured by breaking PR check)
20. TECH DEBT BY FUNCTIONAL AREA
21. TECH DEBT BY FUNCTIONAL AREA
22. WHAT IF I’M INTERESTED IN A PARTICULAR TOPIC?
24. CAN WE DO BETTER THAN STRING REGEX MATCHING?
25. GOING FURTHER WITH STATIC ANALYSIS BUILD ON TOP OF STATIC ANALYSIS TOOLS
26. GOING FURTHER WITH STATIC ANALYSIS ANDROID LINT .sarif fi le https://github.com/slackhq/slack-lints
27. GOING FURTHER WITH STATIC ANALYSIS V1 ANDROID HEALTH SCORE: BUILT ENTIRELY ON LINT “SickFile.kt”: { “DeprecatedCall”: -100, }
28. GOING FURTHER WITH STATIC ANALYSIS V1 LINT-BASED ANDROID HEALTH SCORE ▸ Advantages ▸ Easy to wire up dependencies (lint must produce output for health score) ▸ Ecosystem around lint ▸ Lots of existing checks ▸ Violations show up in IDE ▸ Issues ▸ New check = new lint rule = friction. ▸ Platform/build-system speci fi c ▸ Lint is slow ▸ Warnings are suppressible
29. GOING FURTHER WITH STATIC ANALYSIS V2: INTEGRATE STATIC ANALYSIS INTO THE SIMPLE MODEL
30. V3?
32. GOING FURTHER WITH STATIC ANALYSIS LINT++: BASELINES lint-baseline.xml
33. GOING FURTHER WITH STATIC ANALYSIS LINT++: BASELINES NEED TO BE MERGED https://github.com/slackhq/kotlin-cli- util/blob/main/src/main/kotlin/slack/ cli/lint/LintBaselineMergerCli.kt LINT-BASELINES-MERGED.SARIF
35. GOING FURTHER WITH STATIC ANALYSIS LINT++ RUNS WITHOUT APPLYING DETEKT BASELINE CONTINUES THE RUN AND LET’S IT PASS EVEN WITH ERRORS MERGES ALL THE SERIFS INTO 1 FILE
36. GOING FURTHER WITH STATIC ANALYSIS THE ONE SARIF TO RULE THEM ALL
37. 1 + 1 = 7
38. CAN WE PREVENT ADDING TECH DEBT IN THE FIRST PLACE?
39. PREVENTING NEW TECH DEBT FROM ENTERING THE SYSTEM PULL REQUEST IMPACT ON HEALTH SCORE: V1 ▸ On dev branch build ▸ Get a list of [modi fi ed, added, deleted] ▸ Calculate local score for only these fi les ▸ Check out main branch and do the same ▸ Diff main score vs dev score ▸ Report to GitHub via PR comment
40. PREVENTING NEW TECH DEBT FROM ENTERING THE SYSTEM PROBLEMS WITH V1 ▸ Doesn’t work well with lint (sarif output fi les are not in source control) ▸ Doesn’t show diffs if health score itself changes ▸ Mutates git state on CI nodes
41. PREVENTING NEW TECH DEBT FROM ENTERING THE SYSTEM PULL REQUEST IMPACT ON HEALTH SCORE: V2 ▸ Store health score output from main branch build in a json payload ▸ On dev build ▸ Download main branch health score from merge_base_commit ▸ Run health score on dev branch ▸ Diff main vs dev ▸ Post to GitHub
42. PREVENTING NEW TECH DEBT FROM ENTERING THE SYSTEM ` with skippy https://slackhq.github.io/slack-gradle-plugin/changelog/#project-skippy w/o skippy
43. IT’S HARD TO SEE WHAT EXACTLY REGRESSED IN MY PR
44. HELP WITH UNDERSTANDING REGRESSIONS TAKE 1: ANNOTATE LINT WARNINGS ON CHANGED FILES IN GITHUB Too noisy!
45. HELP WITH UNDERSTANDING REGRESSIONS TAKE 2: ANNOTATE LINT WARNINGS ONLY ON CHANGED LINES
46. WE ADDRESS TECH DEBT, BUT THE TARGET KEEPS MOVING!
47. PREVENT MOVING TARGET FREEZE HEALTH SCORE RECIPE FOR A PERIOD (E.G. QUARTER) ▸ Maintain 2 versions at the same time ▸ [Current]: used to track progress for this quarter ▸ [Next]: used to experiment with new recipe for next quarter
48. ALL OF TECH DEBT ALL OF TECH DEBT WHAT OUR CONFIG CAPTURES ▸ New tool versions ▸ New code deprecations ▸ Newly concluded feature fl ags ▸ New lint checks
49. ALL OF TECH DEBT ALL OF TECH DEBT WHAT OUR CONFIG CAPTURES
50. PREVENT MOVING TARGET FREEZE LIST OF WARNINGS ▸ message_ fi lters ▸ include_list
51. PREVENT MOVING TARGET FREEZE LIST OF WARNINGS ▸ message_ fi lters ▸ exclude_list
53. HEALTH SCORE IS JUST A TOOL
54. HOW DO WE ALIGN HEALTH SCORE WITH CURRENT PRIORITIES?
55. ALIGNING HEALTH SCORE WITH CURRENT PRIORITIES LIST OF CHECKS ▸ Best with buy in from codebase contributors ▸ May take time to align ▸ Start the conversation and optimize ▸ Some ideas ▸ Disabled tests/lints ▸ Deprecated API usage ▸ A11y violations ▸ Number of fi les in app module (build time bottleneck) ▸ Large Files ▸ .java Files
56. ALIGNING HEALTH SCORE WITH CURRENT PRIORITIES THOUGHTS ON WEIGHING ▸ How bad is this thing? ▸ How hard is it to fi x? ▸ How much do you want people to fi x it?
57. ALIGNING HEALTH SCORE WITH CURRENT PRIORITIES MODEL IT IN A SPREADSHEET
58. ALIGNING HEALTH SCORE WITH CURRENT PRIORITIES LITMUS TEST
59. HOW DO WE SUSTAIN ENGAGEMENT?
60. ENGAGEMENT WEEKLY REPORTS
61. ENGAGEMENT ENGINEERING LEADERSHIP ENGAGEMENT
62. ENGAGEMENT USE HEALTH SCORE TO TRACK LARGE MIGRATIONS
63. USING HEALTH SCORE IT’S A SCORE. THERE ARE POINTS. MAKE IT A GAME?
64. GAMIFICATION QUARTERLY LEADERBOARD
65. GAMIFICATION BI-WEEKLY UPDATES IN ANDROID ALL-HANDS
66. GAMIFICATION
67. GAMIFICATION PEOPLE PAY ATTENTION
68. GAMIFICATION SUCCESS ▸ 4 quarters in a row with positive health score trend ▸ Removed all accessibility lint violations ▸ Large reduction in the use of mocks in favor of fakes ▸ Consistent removal of stale feature fl ags and disabled tests
69. GENERALLY THE CODEBASE IS AMONG THE HIGHEST QUALITY IN TERMS OF QUALITY. I WOULD RECOMMEND IT TO PEOPLE OF ALL SKILL LEVEL TO WORK ON IT. Quote from latest devxp survey
70. USING HEALTH SCORE FUTURE ▸ Migrate back into the build system ▸ Shift left ▸ Faster lint, please
71. DEVELOPERS CARE ABOUT CODE HEALTH