Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ultimate Iteration Speeds with Gradle Configura...

Ultimate Iteration Speeds with Gradle Configuration Cache

Talk by Aurimas Liutikas (Google) at Droidcon SF 2024

Aurimas Liutikas

June 10, 2024
Tweet

More Decks by Aurimas Liutikas

Other Decks in Programming

Transcript

  1. Who’s Aurimas? 12 years at Google 8.5 years on AndroidX

    Work closely with Android Studio, Gradle, and Jetbrains
  2. Iteration repetition of a [...] procedure applied to the result

    of a previous application, [...] successively closer approximations to the solution [...]. - Oxford Languages
  3. Faster Iteration (Feedback) Loop Countless research - it leads to

    faster end-to-end and/or higher quality software
  4. Faster Iteration (Feedback) Loop Countless research - it leads to

    faster end-to-end and/or higher quality software xkcd.com/303/
  5. Gradle Configuration Cache The configuration cache is a feature that

    significantly improves build performance by caching the result of the configuration phase and reusing this for subsequent builds. org.gradle.configuration-cache=true
  6. Configuration Cache Miss (e.g. First Run) Initialization phase Configuration phase

    Configuration serialization / deserialization Execution phase
  7. Motivating Example 2: Parallel Tasks Within a Project Execute tasks

    belonging to different subprojects in parallel with org.gradle.parallel=true Gradle worker API allows parallel within project tasks, but non-trivial to migrate to org.gradle.configuration-cache=true all tasks parallel by default
  8. Example Task abstract class MyTask : DefaultTask() { @get:Input abstract

    val inputString: Property<String> @get:InputFile abstract val inputFile: RegularFileProperty @get:OutputFile abstract val outputFile: RegularFileProperty @TaskAction fun doIt() { outputFile.get().asFile.writeText( inputFile.get().asFile.readText() + inputString.get()) } }
  9. Custom Types as Inputs class CustomType(val text: String, val number:

    Int) abstract class MyTask : DefaultTask() { @get:Input abstract val inputCustomType: Property<CustomType> } tasks.register<MyTask>("myTask") { inputCustomType.set(CustomType("someText", 100)) }
  10. Custom Types as Inputs $ ./gradlew myTask > Task :myTask

    FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':myTask'. > Cannot fingerprint input property 'inputCustomType': value 'Build_gradle$CustomType@d00c052' cannot be serialized.
  11. file.exists() in Configuration Phase tasks.register<MyTask>("myTask") { val primary = File(projectDir,

    "primary.txt") if (primary.exists()) { inputFile.set(primary) } else { inputFile.set(File(projectDir, "secondary.txt")) } }
  12. If primary.txt Gets Created $ ./gradlew myTask Calculating task graph

    as configuration cache cannot be reused because the file system entry 'primary.txt' has been created.
  13. Where Did We See This https://issuetracker.google.com/issues/291331139 package.xml in SDK https://issuetracker.google.com/issues/289098893

    AndroidManifest.xml https://issuetracker.google.com/issues/278767328 analytics.settings https://issuetracker.google.com/issues/295395616 lint common.jar in fatjar task https://youtrack.jetbrains.com/issue/KT-61154/ konan native jar
  14. Alternatives - Set up proper task dependencies if input created

    by a different task - Add both files as @InputFile, mark one with @Optional - Perform exists(), isDirectory() during task execution
  15. Reading File Contents During Configuration tasks.register<MyTask>("myTask") { val primary =

    File(projectDir, "input.txt") inputString.set(primary.readText()) }
  16. If input.txt Contents Change $ ./gradlew myTask Calculating task graph

    as configuration cache cannot be reused because file 'input.txt' has changed.
  17. Init Scripts $ ./gradlew --init-script init.gradle myTask Calculating task graph

    as configuration cache cannot be reused because init script 'init.gradle' has been added.
  18. Environment Variable Change $ FOO=bar ./gradlew myTask Calculating task graph

    as configuration cache cannot be reused because environment variable 'FOO' has changed.
  19. System Property Change $ ./gradlew myTask3 -DFOO=bar Calculating task graph

    as configuration cache cannot be reused because system property 'FOO' has changed.
  20. Gradle Properties $ ./gradlew myTask -Pfoo=bar Calculating task graph as

    configuration cache cannot be reused because the set of Gradle properties has changed.
  21. Contents of FileTree Queried During Configuration abstract class MyTask :

    DefaultTask() { @get:InputFiles abstract val inputFiles: ListProperty<File> ... } tasks.register<MyTask>("myTask") { inputFiles.set(fileTree("mydir").files.sortedBy { it.name }) ... }
  22. New File Added to mydir $ ./gradlew myTask Calculating task

    graph as configuration cache cannot be reused because an input to build file 'build.gradle.kts' has changed.
  23. Where Did We See This? $ ./gradlew room:room-benchmark:assembleAndroidTest Calculating task

    graph as configuration cache cannot be reused because an input to plugin 'com.android.internal.library' has changed. https://issuetracker.google.com/issues/285320724
  24. Alternatives / Notes - Traverse files during task execution or

    use FileCollection with filters - Better reporting from Gradle https://github.com/gradle/gradle/issues/25469
  25. - Prioritized most common tasks - It took many months

    to go from WARN to ERROR - Reported and helped fix dozens of bugs in Gradle, Android Gradle Plugin, Kotlin Gradle Plugin, Protobuf Gradle Plugin, SPDX Gradle Plugin - Fixed dozens of issues within our own build logic - Upgrades to plugins still can come with regressions (e.g. KGP 2.0.0) → test, report and nudge AndroidX Adoption of Configuration Cache
  26. Preventing Regressions - Allow enumerating configuration cache inputs to prevent

    new additions https://github.com/gradle/gradle/issues/26019 - Allow confirming configuration cache was reused https://github.com/gradle/gradle/issues/27964
  27. Takeaways - Use progressive adoption, especially for compile/test iteration loops

    - Once you get it working, you will not want to go back - If you enable CC in CI, store configuration cache reports for debugging - Bring down configuration cache report entry count to a minimum - Once CC is enabled you’ll notice task performance more (👋 CompileKotlin) - It will not help Gradle Sync in Android Studio / Intellij