Improving Gradle Builds

As the number of modules in your code base increases, speed and consistency of your Gradle builds becomes very important. However, improving your Gradle builds can be a tricky endeavor. In this talk, I’ll address these common questions in improving Gradle builds.

- How do we benchmark & profile our builds?
- How do you read build scans to identify problems?
- How do we maintain a consistent modular structure?
- What tools can you use to speed up builds?

I’ll take a deep dive into these questions with examples and insights from my experiences.

Transcript

1. Mohit Sarveiya Improving Gradle Builds @heyitsmohit
2. Improving Gradle Builds • Measure & Benchmark • Modularization • Optimizations • Tools
3. Measure Gradle Enterprise, Build Analyzer
4. https: // github.com/uber-common/android-build-eval Github
5. Uber Build Eval • Similar to Uber Production mobile apps • 1292 Modules • 21,235 Build Graph Edges
6. How do we analyze our builds?
7. How do we analyze our builds? • Build Analyzer in Android Studio
8. Team 1 Team 2 Team 3 Team 4 Build Data Collect
9. Team 1 Team 2 Team 3 Team 4 Build Data Analyze
10. How do we analyze our builds? • Build Analyzer in Android Studio • Gradle Enterprise
11. https: // gradle.com/ Gradle
12. Build Scan $ ./gradlew rootModule:assembleDebug —scan
13. Build Scan $ ./gradlew rootModule:assembleDebug —scan Publishing build scan ... https: // gradle.com/s/46dna3y4xauze
14. https: // gradle.com/ Gradle
15. https: // gradle.com/ Gradle
16. Build Times • Configuration time • Task execution time • Cache Hits/Misses
17. https: // gradle.com/ Gradle
18. https: // gradle.com/ Gradle
19. https: // gradle.com/ Gradle
20. Exploring Config Time • Plugin config • Script config
21. https: // gradle.com/ Gradle
22. https: // gradle.com/ Gradle
23. https: // gradle.com/ Gradle
24. https: // gradle.com/ Gradle
25. https: // gradle.com/ Gradle
26. https: // gradle.com/ Gradle
27. https: // gradle.com/ Gradle
28. https: // gradle.com/ Gradle
29. https: // gradle.com/ Gradle
30. Build Tags • Local • CI
31. https: // gradle.com/ Gradle
32. https: // gradle.com/ Gradle
33. How do we integrate for Local and CI?
34. Build Tags plugins { id "com.gradle.enterprise" version "3.8.1" }
35. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
36. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
37. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
38. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
39. https: // gradle.com/ Gradle
40. Summary • Gradle Enterprise • How do we view build times across teams
41. Optimizations
42. Optimizations • Config time • Dependencies • Build cache
43. Optimizations • Turn off unused Android Gradle Plugin features • Only apply AGP plugin where needed
44. https: // gradle.com/ Gradle
45. Optimizations android.defaults.buildfeatures.renderscript = false
46. Optimizations android.defaults.buildfeatures.shaders = false
47. AGP Features • Build Config • Data binding • View binding
48. Optimizations • Config time • Dependencies • Build cache
49. How do we remove unused plugins?
50. Dependency Analysis Plugin • Removed unused plugins & dependencies • Provides suggestions on using api vs impl
51. Dependency Analysis Plugin plugins { id 'com.autonomousapps.dependency-analysis' }
52. Dependency Analysis Plugin ./gradlew projectHealth
53. Dependency Analysis Plugin > Task :LibraryB:projectHealth Unused dependencies which should be removed: implementation("androidx.appcompat:appcompat:1.4.1") implementation("androidx.core:core-ktx:1.7.0") implementation("com.google.android.material:material:1.5.0")
54. Dependency Analysis Plugin Plugin advice: kotlin-kapt: no used annotation processors
55. Optimizations • Config time • Dependencies • Build cache
56. Build Cache Task Output Cache (id) Result
57. Build Cache Task Output Cache (id) Get
58. Gradle Doctor Plugin • Slower tasks from cache • Build scan tags
59. Gradle Doctor Plugin plugins { id "com.osacky.doctor" version "0.7.3" }
60. Gradle Doctor Plugin plugins { id "com.osacky.doctor" version "0.7.3" }
61. https: // gradle.com/ Gradle
62. Gradle Doctor Plugin • doctor-negative-savings • doctor-slow-build-cache-connection
63. How do you disable caching for a task?
64. Disable Local Caching tasks.named("<taskname>").configure { outputs.cacheIf { false } }
65. Disable Remote Caching tasks.named("<taskname>").configure { outputs.cacheIf { !isCI } }
66. Gradle Doctor Plugin • Set threshold on Dagger processing
67. Disable Remote Caching doctor { /** * Print a warning to the console if we spend more than this * amount of time with Dagger annotation processors. */ daggerThreshold = 5000 }
68. Android Build Cache Plugin • Plugin by Gradle • Gradle plugin to fix Android build problems • Workarounds for Room
69. Summary • Dependency Analysis Plugin • Gradle Doctor Plugin • Android Build Cache Fix Plugin
70. Modularization Graph Assertions, Version Catalogs
71. Modularization Module A Module B Module C
72. How do we enforce best practices?
73. Module Graph Assert Plugin • Check graph depth • Enforce conventions
74. Module Graph Assert Plugin id "com.jraska.module.graph.assertion"
75. Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 }
76. Module Graph Assert Plugin Max height the number of edges on the longest path from the node to a leaf.
77. Modularization Max height - 4
78. Module Graph Assert Plugin ./gradlew assertModuleGraph
79. Module Graph Assert Plugin > Task :app:assertMaxHeight FAILED 
 Module :app is allowed to have maximum height of 3, but has 4.
80. Modularization Max height - 3
81. Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 allowed = [‘:app’ -> ‘:feature-[a-z]’, ‘:feature.* -> :lib’] }
82. Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 allowed = [‘:app’ -> ‘:feature-[a-z]’, ‘:feature.* -> :lib’] restricted = [‘:feature-[a-z]*’ :forbidden-module,] }
83. How do we visualize all Gradle modules?
84. Module Graph Assert Plugin • Generate full graph of all modules • Visualize a subgraph
85. GraphViz • Tool to create graphs • Color nodes based on config
86. Module Graph Assert Plugin ./gradlew generateModulesGraphvizText
87. Module Graph Assert Plugin ./gradlew generateModulesGraphvizText 
 -Pmodules.graph.of.module=all_modules
88. Module Graph Assert Plugin Generate Graph GraphViz File Visualize with GraphViz
89. Module Graph Assert Plugin digraph G { ":leafModuleAvg" -> ":module785" ":leafModuleAvg" -> ":module574" ... }
90. Full Graph
91. Full Graph
92. Module Graph Assert Plugin ./gradlew generateModulesGraphvizText 
 -Pmodules.graph.of.module=:feature_a
93. How do we gets stats on all modules?
94. Module Graph Assert Plugin ./gradlew generateModulesGraphStatistics 

95. Module Graph Stats • Number of modules • Number of edges • Longest path
96. How do we setup dependencies?
97. Version Catalogs • Define all deps in TOML file • Gradle feature to share deps between modules
98. Version Catalogs [versions] androidx-core = “1.6.0-beta01”
99. Version Catalogs [versions] androidx-core = “1.6.0-beta01” [libraries] androidx-core-ktx = { module = "androidx.core:core-ktx", version.ref = “androidx-core" }
100. Version Catalogs dependencies { implementation(libs.androidx.core.ktx) }
101. Version Catalogs [bundles] compose = [ “androidx-composeUi", “androidx-composeMaterial”, “androidx-composeUiTooling" ]
102. Version Catalogs dependencies { implementation(libs.bundles.compose) }
103. Version Catalogs enableFeaturePreview("VERSION_CATALOGS")
104. Summary • Module Assert Graph Plugin • Version Catalogs
105. Benchmark Gradle Profiler
106. Gradle Profiler • Automates benchmarking builds • Detect regressions
107. Gradle Profiler
108. Gradle Profiler Change
109. How do we find regressions?
110. Gradle Profiler 1. Write a performance scenario 2. Specify number of iterations
111. Gradle Profiler brew install gradle-profiler
112. Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } assemble { }
113. Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } assemble { tasks = ["assemble"] }
114. Gradle Profiler gradle-profiler —benchmark
115. Gradle Profiler gradle-profiler —benchmark —iterations=
116. Gradle Profiler gradle-profiler —benchmark —iterations= Profiling - 1 Benchmarking - 10
117. Gradle Profiler gradle-profiler —benchmark —iterations=10
118. Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups= Profiling - 2 Benchmarking - 6
119. Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups=6
120. Gradle Profiler
121. Gradle Profiler Mean: 348 ms Min: 319 ms P25: 330 ms Median: 341 ms P75: 368 ms Std dev: 22.71 ms
122. How do we test incremental builds?
123. Gradle Profiler Change
124. Gradle Profiler Iteration 1 Iteration 2 Iteration 3
125. Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Abi Change
126. Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Result
127. Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” }
128. Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” apply-android-resource-change-to = “strings.xml” tasks = ["assemble"] }
129. Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups=6
130. Gradle Profiler
131. How do we test Gradle sync?
132. Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } androidStudioSync { android-studio-sync { } }
133. Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } androidStudioSync { android-studio-sync { studio-jvm-args = ["-Xms256m", "-Xmx4096m"] } }
134. Gradle Profiler 1. Write a performance scenario 2. Specify number of iterations
135. How do we automate profiling on CI?
136. CI Pipeline 1. Create pipeline runs nightly 2. Collect metrics overtime
137. CI Pipeline Docker image with 
 Gradle Profiler
138. CI Pipeline Docker image with 
 Gradle Profiler Performance 
 Scenarios
139. CI Pipeline Docker image with 
 Gradle Profiler Performance 
 Scenarios Run 
 nightly
140. Use Cases 1. Anvil 2. KSP
141. Anvil • Kotlin Compiler Plugin • Generate factories • Improve build times
142. Anvil Dagger Module A Module B Module C
143. How do we use Anvil to generate factories?
144. Anvil anvil { generateDaggerFactories = true }
145. Benchmarking Build • Benchmark with Anvil change • Benchmark without Anvil change (Baseline) • Compare results
146. Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” tasks = ["assemble"] }
147. Results Module A Module B Module C Baseline Anvil
148. Summary • How to setup & run Gradle Profiler • Performance scenarios • Anvil
149. Thank You! www.codingwithmohit.com @heyitsmohit