Gradle under the microscope

Talk by Aurimas Liutikas (Google) at Droidcon SF 2023

Transcript

1. under the microscope Aurimas Liutikas / Google @aurimas@androiddev.social
2. Who’s Aurimas? 11 years at Google 7.5 years on AndroidX Work closely with Android Studio, Gradle, and Jetbrains
4. In last 30 days, AndroidX did 16.5K presubmit builds 9K postsubmit builds
5. Initialization and Configuration Phase buildSrc build
6. Execution phase
7. Optimizing Execution Phase
8. Toggle bad defaults org.gradle.parallel=true org.gradle.caching=true (& remote cache) org.gradle.jvmargs=-Xmx8g
9. Execution phase optimizations Build only what you need Task local/remote cacheability Parallelization within project
10. Gradle Enterprise Helpful for experiments, debugging, monitoring Free version --scan ge.androidx.dev OSS instance Pricing not transparent, closed source
11. Lots of Great Talks / Posts available Chasing
13. Optimizing Configuration Phase
14. Configuration Caching (CC) Serializing task graph into disk, I/O instead of recalculating org.gradle.configuration-cache=true orders of magnitude faster Stable with Gradle 8.1 (works in 8.0 too)
15. CC Impact https://developer.squareup.com/blog/5-400-hours-a-year-saving-developers-ti me-and-sanity-with-gradles/
16. Why Invest In Configuration Optimization? CC is not portable, container based CI always runs it Once execution is great, configuration phase can become >50% of time Bottleneck for Studio Gradle Sync, never uses CC
17. Why run with configuration cache in CI Matching dev flow Catch bugs in build logic, plugins Up to date task validation savings
18. What happens during Configuration?
20. Stages of Gradle Configuration settings.gradle Settings plugins Remote Cache Initialization Parsing version catalogs
21. Stages of Gradle Configuration settings.gradle Settings plugins Remote Cache Initialization Parsing version catalogs buildSrc (~ included build) settings.gradle build buildSrc/build.gradle and dependencies
22. Stages of Gradle Configuration settings.gradle Settings plugins Remote Cache Initialization Parsing version catalogs buildSrc (~ included build) settings.gradle build buildSrc/build.gradle and dependencies every build.gradle :
23. Stages of Gradle Configuration settings.gradle Settings plugins Remote Cache Initialization Parsing version catalogs buildSrc (~ included build) settings.gradle build buildSrc/build.gradle and dependencies every build.gradle : :a :b
24. Stages of Gradle Configuration settings.gradle Settings plugins Remote Cache Initialization Parsing version catalogs buildSrc (~ included build) settings.gradle build buildSrc/build.gradle and dependencies every build.gradle : :a :b :a:a1 :a:a2
25. Stages of Gradle Configuration settings.gradle Settings plugins Remote Cache Initialization Parsing version catalogs buildSrc (~ included build) settings.gradle build buildSrc/build.gradle and dependencies every build.gradle : :a :b :a:a1 :a:a2 … :a:a1:aa1
26. Stages of Gradle Configuration settings.gradle Settings plugins Remote Cache Initialization Parsing version catalogs buildSrc (~ included build) settings.gradle build buildSrc/build.gradle and dependencies every build.gradle : :a :b :a:a1 :a:a2 … :a:a1:aa1 CC serialization single threaded walk through projects serializing tasks includes serializing all configurations (can be really slow)
27. Basic optimizations Tasks register vs create Load all plugins, apply when needed Write your own instead of loading giant plugins org.gradle.configureondemand=true (potentially a foot gun)
28. Build scans timeline is not helpful
29. Summary pages are also not great
30. Some Details
31. Not Quite Enough to Know What Happened
32. Gradle Profiler https://github.com/gradle/gradle-profiler Easier benchmarking and profiling Works with async profiler, JProfiler, YourKit
33. Creating a scenario to optimize clean CI before we get to executing tasks
34. Creating a scenario to optimize rerunDryRun { tasks = ["bOS"] gradle-args = ["--dry-run", "--rerun-tasks"] daemon = cold warm-ups = 1 clear-configuration-cache-state-before = BUILD }
35. Benchmarking ./gradle-profiler --benchmark --project-dir /path/to/project --gradle-user-home my-gradle-user-home --scenario-file ./perf.scenarios --iterations 4 rerunDryRun
36. Profiling YOURKIT_HOME=~/yourkit/ ./gradle-profiler --profile yourkit --project-dir /path/to/project --gradle-user-home my-gradle-user-home --scenario-file ./perf.scenarios rerunDryRun
37. Where Do I Look First? Your own build logic first More familiar, easier to fix Look for chunks >100ms
38. Filters are your friends
39. Except when they are not
43. Might be invoked from other plugins
44. Real-life Examples
45. Avoid calling external processes
46. Solution: remove or no-op when not needed In androidx it was to replace it with NoOpGitClient for local builds
47. Look at I/O
48. Be careful with I/O Avoid if you can Use BufferedInputStream if you cannot https://github.com/gradle/gradle/pull/24105
49. Look at Exceptions
50. An inexpensive work repeated is expensive
52. String.format() in a log statement
53. Simple fix
54. Cross project repeated work
55. Solution: build service / static state Compute once, share results We do that for some toml parsing
56. Avoid costly Gradle APIs
57. Kotlin Reflection
58. Kotlin reflection
59. “Dripping water hollows out stone” - Ovid If everyone in the audience today went and removed wasted 50ms… The more core the plugin, greater the impact
60. Prevent regressions Monitor
62. Glimpse of Hope - Project Isolation org.gradle.unsafe.isolated-projects=true Upcoming feature Will take a couple of years to be ready, get started now
63. Any questions?