Chasing the Speed of Gradle Builds: a history of Androidx build

AndroidX is a large collection of OSS Kotlin and Java libraries by Google built to make Android app developers’ lives easier. This talk will take you on a journey exploring how Gradle Build has evolved and grown from 1 to over 650 projects. Learn how Google kept CI times and quality at bay.

Transcript

1. Chasing the Speed of Gradle Builds A history of AndroidX build
2. About Google A mom-and-pop search engine that also builds AndroidX libraries for Android App Developers
3. About Me Aurimas Liutikas Software Engineer
4. Basic concepts • Project size ◦ # of projects ◦ # of lines of code • Capabilities • Optimizations
5. Where did it begin? 1 Project
6. Jump to 2013 H2 5 Projects +4 Projects added
7. Fast forward to 2015 H2 • make is the main build system • Gradle 2.2.1 / AGP 1.3.1 Capabilities • Gradle used for creating a maven zip • No presubmit and only make postsubmit 18 Projects +13 Java LoC 171k Lines of code at the time
8. 2016 H1 • make is the main build system • Gradle 2.10 / AGP 1.5.0 Capabilities • Gradle can build device tests and run them 33 Projects +15 Java LoC 196k
9. 2016 H2 • Gradle 3.2 / AGP 2.2.1 is the main build system • Still support make Capabilities • Using Doclava to build docs and run API checks • Android Lint and Android Studio support • Presubmit and postsubmit use Gradle and Make • Automated device tests Optimizations • -Xmx3g 36 Projects +3 Java LoC 283k
10. 2017 H1 • Gradle 3.3 / AGP 2.2.4 • Still support make Optimizations • Root build.gradle broken down into groovy scripts • Enabled configuration on demand • buildSrc with custom plugin in Groovy 41 Projects +5 Java LoC 298k
13. 2017 H2 • Gradle 4.3 / AGP 2.2.4 • Still support make Capabilities • First use of Kotlin • Enhanced publishing (maven gradle plugin, source jars) • Error prone compiler Optimizations • Start using Kotlin/Java in buildSrc code • Library versions and dependencies to buildSrc • -Xmx4g 76 Projects +35 Java LoC 379k Kotlin LoC 20k
14. 2018 H1 • Gradle 4.5 / AGP 3.0.0 • Still support make Capabilities • Adding jetifier to our builds Optimizations • JVM tests as separate builder • All buildSrc code in Kotlin • Our plugins move to one per project type 113 Projects +37 Java LoC 404k Kotlin LoC 30k
15. 2018 H2 • Gradle 4.10.1 / AGP 3.3.0 (make removed!) Capabilities • Kotlin docs using a Dokka fork • Kotlin API tracking using metalava Optimizations • Enabled local build cache • Build test APKs as separate builder • AffectedModuleDetector for device tests • -Xmx8g 185 Projects +72 Java LoC 557k Kotlin LoC 60k
16. 2019 H1 • Gradle 5.1.1 / AGP 3.4.0 Capabilities • Custom Android Lint checks • Android Resource API tracking • Max deps version testing Optimizations • Start migration to tasks.register • AffectedModuleDetector for other tasks 198 Projects +13 Java LoC 573k Kotlin LoC 83k
17. 2019 H2 • Gradle 6.0 / AGP 3.6.0 Capabilities • ktlint • Deeper integration with release tool • Library metrics tracking Optimizations • -Xms8g -Xmx8g for CI • buildSrc plugin restructured to one that reacts to plugins • buildSrc tests moved out • Task up-to-date / overlapping outputs enforcement • First Gradle testkit test • Gradle home and tmp to out directory • Start using Gradle workers for Metalava tasks 309 Projects +111 Java LoC 658k Kotlin LoC 274k
20. F E A B C D
21. 2020 H1 • Gradle 6.2.2 / AGP 4.0.0 (compose uses 4.1.0) Capabilities • Inspection integration with Android Studio • Build against HEAD Android Gradle Plugin • Support for Robolectric Optimizations • Remote build cache for local builds • Task cacheability fixes • JDK 11 upgrade • Disable unused AGP features (aidl, BuildConfig) 365 Projects +56 Java LoC 677k Kotlin LoC 339k
22. 2020 H2 • Gradle 6.7 / AGP 4.2.0 Capabilities • Playground (GitHub mirror) Optimizations • Learning about input normalizers (fix compose invalidations) • Rewrite docs integration with an explicit list of projects • Project subgroups for Studio • -XX:+UseParallelGC / split CI to -Xmx16g • Build log output validation • set no-parent OWNERS 427 Projects +62 Java LoC 752k Kotlin LoC 526k
23. 2021 H1 • Gradle 6.8.1 / AGP 4.2.0 Capabilities • Bundling JarJared inside shipped artifacts Optimizations • Adopt libs.versions.toml • CI -Xmx24g 473 Projects +46 Java LoC 786k Kotlin LoC 577k
24. 2021 H2 • Gradle 7.3 / AGP 7.1.0 Capabilities • Dependency signature verification • Add 3rd docs tool Dackka Optimizations • Enabled configuration caching in warn mode / fix lots of issues • Split buildSrc into public / private • Tweak kotlin.daemon.jvm.options / JVM tests get 2g heap • Gradle Enterprise for GitHub mirror / futher remote cache task restore fixes 575 Projects +102 Java LoC 853k Kotlin LoC 866k
25. 2022 H1 • Gradle 7.4 / AGP 7.4 Capabilities • Mac ARM support Optimizations • JVM test heap 3G • Enable remote Gradle cache in CI • Remove AffectedModuleDetector for everything except device tests • Strong push for remote cache for local builds • More remote cache restoration fixes 597 Projects +22 Java LoC 864k Kotlin LoC 975k
26. Today • Gradle 7.5 / AGP 8.0.0 Capabilities • JVM layoutlib based screenshot testing Optimizations • JDK 17 • A single documentation tools • Profiling builds and filing upstream bugs 657 Projects +60 Java LoC 990k Kotlin LoC 1.25M
29. A B D C
31. Future • Gradle Project isolation • Shadow builds • JDK upgrades
32. Takeaways • Productive engineers → better quality products • Monitor monitor monitor • Keep your tools up to date • Early feature adoption can be rough, but potentially worth it • Project growth does not have to mean linear CI/build times
33. Elephant in the room
34. THANKS Have any questions? Twitter: @_aurimas careers.google.com https://cs.android.com/androidx