$30 off During Our Annual Pro Sale. View Details »

Gradle: The Build System That Loves To Hate You

Gradle: The Build System That Loves To Hate You

Talk by Aurimas Liutikas (Google) at DroidCon London 2024

Aurimas Liutikas

November 01, 2024
Tweet

More Decks by Aurimas Liutikas

Other Decks in Technology

Transcript

  1. 🐘 Gradle: The Build System That Loves To Hate You

    The Survival Tips Aurimas Liutikas / AndroidX @ Google @[email protected]
  2. 🐘 Who’s Aurimas? 12 years at Google 8.5 years on

    AndroidX Work closely with Android Studio, Gradle, and Jetbrains
  3. 🐘 tasks.create("myTask") { long start = System.currentTimeMillis() Thread.sleep(5000) long end

    = System.currentTimeMillis() println( "Spent ${end - start} ms" ) }
  4. 🐘 class MyBetterTask extends DefaultTask { @TaskAction void doTheWork() {

    long start = System.currentTimeMillis() Thread.sleep(5000) long end = System.currentTimeMillis() println("Spent ${end - start} ms") } } tasks.create("myBetterTask", MyBetterTask)
  5. 🐘 class MyBetterTask extends DefaultTask { @TaskAction void doTheWork() {

    long start = System.currentTimeMillis() Thread.sleep(5000) long end = System.currentTimeMillis() println("Spent ${end - start} ms") } } tasks.create("myBetterTask", MyBetterTask)
  6. 🐘 class MyBetterTask extends DefaultTask { @TaskAction void doTheWork() {

    long start = System.currentTimeMillis() Thread.sleep(5000) long end = System.currentTimeMillis() println("Spent ${end - start} ms") } } tasks.create("myBetterTask", MyBetterTask)
  7. 🐘 - Gradle should deprecate TaskContainer.create and friends https://github.com/gradle/gradle/issues/17705 (open

    since 2021) - Build engineers need to understand configuration vs execution phase - avoid doing expensive work in configuration phase Lessons
  8. 🐘 - Gradle should deprecate TaskContainer.create and friends https://github.com/gradle/gradle/issues/17705 (open

    since 2021) - Build engineers need to understand configuration vs execution phase - avoid doing expensive work in configuration phase - Gradle plugin authors should use androidx.lint:lint-gradle checks Lessons
  9. 🐘 tasks.create tasks.getByPath tasks.findByPath tasks.findByName tasks.withType tasks.getByName tasks.all tasks.matching Eager

    APIs to Avoid tasks.whenTaskAdded tasks.whenObjectAdded tasks.findAll tasks.iterator, thus all of the kotlin-stdlib collection extension functions, e.g. tasks.any { }
  10. 🐘 tasks.create tasks.getByPath tasks.findByPath tasks.findByName tasks.withType tasks.getByName tasks.all tasks.matching Eager

    APIs to Avoid tasks.whenTaskAdded tasks.whenObjectAdded tasks.findAll tasks.iterator, thus all of the kotlin-stdlib collection extension functions, e.g. tasks.any { }
  11. 🐘 - Gradle should deprecate all TaskContainer, TaskCollection, DomainObjectCollection methods

    that are not register, named, or configureEach - Build engineers should pay attention to eagerly created tasks tasks.register("eagerCanary") { throw Exception("Eagerly configured tasks!") } Lessons
  12. 🐘 - Gradle should deprecate all TaskContainer, TaskCollection, DomainObjectCollection methods

    that are not register, named, or configureEach - Build engineers should pay attention to eagerly created tasks tasks.register("eagerCanary") { throw Exception("Eagerly configured tasks!") } - Gradle plugin authors should use androidx.lint:lint-gradle checks Lessons
  13. 🐘 abstract class FooWriter : DefaultTask() { @get:OutputFile abstract val

    outputFile : RegularFileProperty @TaskAction fun doTheWork() { outputFile.get().asFile.writeText("Hello") } }
  14. 🐘 abstract class FooReader : DefaultTask() { @get:InputFile abstract val

    inputFile : RegularFileProperty @TaskAction fun doTheWork() { println(inputFile.get().asFile.readText()) } }
  15. 🐘 val foo = layout.buildDirectory.file("foo.txt") val writer = tasks.register<FooWriter>("fooWriter") {

    outputFile.set(foo) } tasks.register<FooReader>("fooReader") { inputFile.set(foo) }
  16. 🐘 $ ./gradlew fooReader > Task fooReader FAILED FAILURE: Build

    failed with an exception. * What went wrong: A problem was found with the configuration of task 'fooReader' (type 'FooReader'). - Type 'FooReader' property 'inputFile' specifies file 'foo.txt' which doesn't exist.
  17. 🐘 val foo = layout.buildDirectory.file("foo.txt") val writer = tasks.register<FooWriter>("fooWriter") {

    outputFile.set(foo) } tasks.register<FooReader>("fooReader") { inputFile.set(foo) dependsOn(writer) }
  18. 🐘 val foo = layout.buildDirectory.file("foo.txt") val writer = tasks.register<FooWriter>("fooWriter") {

    outputFile.set(foo) } tasks.register<FooReader>("betterFooReader") { inputFile.set(writer.flatMap { it.outputFile }) }
  19. 🐘 - Gradle should deprecate Task.dependsOn - Build engineers should

    use Provider<Task>.flatmap to establish dependencies Lessons
  20. 🐘 - Gradle should force task authors to mark tasks

    as @AlwaysRerunTask or declare an output Lessons
  21. 🐘 - Gradle should force task authors to mark tasks

    as @AlwaysRerunTask or declare an output - Build engineers should always add outputs to tasks tasks.register<FooReader>("betterFooReader") { inputFile.set(writer.flatMap { it.outputFile }) cacheEvenIfNoOutputs() } Lessons
  22. 🐘 abstract class FooEnhancer : DefaultTask() { @get:InputFile abstract val

    inputFile : RegularFileProperty @get:OutputFile abstract val outputFile : RegularFileProperty @TaskAction fun doTheWork() { outputFile.get().asFile.writeText( inputFile.get().asFile.readText() + "!" ) } }
  23. 🐘 $ ./gradlew fooEnhancer --info > Task :lib2:fooEnhancer Caching disabled

    for task ':lib2:fooEnhancer' because: Build cache is disabled Caching has not been enabled for the task Task ':lib2:fooEnhancer' is not up-to-date because: Output property 'outputFile' file build/enhancedFoo.txt has been removed.
  24. 🐘 UP-TO-DATE (default on) Task’s inputs and outputs did not

    change. FROM-CACHE (default off) Task’s outputs could be found from a previous execution.
  25. 🐘 $ ./gradlew fooEnhancer --info > Task :lib2:fooEnhancer Caching disabled

    for task ':lib2:fooEnhancer' because: Caching has not been enabled for the task Task ':lib2:fooEnhancer' is not up-to-date because: Output property 'outputFile' file build/enhancedFoo.txt has been removed.
  26. 🐘 @CacheableTask abstract class FooEnhancer : DefaultTask() { @get:[InputFile PathSensitive(PathSensitivity.NONE)]

    abstract val inputFile : RegularFileProperty @get:OutputFile abstract val outputFile : RegularFileProperty @TaskAction fun doTheWork() { outputFile.get().asFile.writeText(inputFile.get().asFile.readText() + "!") } }
  27. 🐘 $ ./gradlew fooEnhancer --info > Task :lib2:fooEnhancer FROM-CACHE Build

    cache key for task ':lib2:fooEnhancer' is eb57efc9cb2dd2b6587672bac94200b4 Task ':lib2:fooEnhancer' is not up-to-date because: Output property 'outputFile' file build/enhancedFoo.txt has been removed. Loaded cache entry for task ':lib2:fooEnhancer' with cache key eb57efc9cb2dd2b6587672bac94200b4
  28. 🐘 - Gradle should enable build caching by default and

    force task authors to mark tasks as @DisableCachingByDefault or @CacheableTask Lessons
  29. 🐘 - Gradle should enable build caching by default and

    force task authors to mark tasks as @DisableCachingByDefault or @CacheableTask - Build engineers should enable build cache and always mark their tasks @CacheableTask, unless it is an I/O operation such as copy. Lessons
  30. 🐘 - Gradle should enable build caching by default and

    force task authors to mark tasks as @DisableCachingByDefault or @CacheableTask - Build engineers should enable build cache and always mark their tasks @CacheableTask, unless it is an I/O operation such as copy. - Gradle Plugin authors should enable: tasks.withType<ValidatePlugins>().configureEach { enableStricterValidation.set(true) } Lessons
  31. 🐘 // projectA tasks.register("zip", Zip) { it.from(layout.projectDirectory.file("build.gradle.kts")) destinationDirectory.set(layout.buildDirectory.dir("zip")) archiveFileName.set("my.zip") }

    // projectB val zipTask = project(":projectA").tasks.named<Zip>("zip") tasks.register<ZipConsumer>("zipConsumer") { inputFile.set(zipTask.flatMap { it.archiveFile }) }
  32. 🐘 // projectA tasks.register("zip", Zip) { it.from(layout.projectDirectory.file("build.gradle")) destinationDirectory.set(layout.buildDirectory.dir("zip")) archiveFileName.set("my.zip") }

    // projectB val zipTask = project(":projectA").tasks.named<Zip>("zip") tasks.register<ZipConsumer>("zipConsumer") { inputFile.set(zipTask.flatMap { it.archiveFile }) }
  33. 🐘 // projectA val zipTask = tasks.register<Zip>("zip") { … }

    configurations.register("zipPublish") { isCanBeConsumed = true isCanBeResolved = false attributes { attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip"))} } artifacts { add("zipPublish", zipTask) }
  34. 🐘 // projectB val zipConfiguration = configurations.create("zipConsumer") { attributes {

    attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip")) } } zipConfiguration.dependencies.add(dependencies.create(project(":projectA"))) val zipFile : FileCollection = zipConfiguration.incoming.artifactView { }.files tasks.register<ZipConsumer>("zipConsumer") { inputFile.from(zipFile) }
  35. 🐘 // projectB val zipConfiguration = configurations.create("zipConsumer") { attributes {

    attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip")) } } zipConfiguration.dependencies.add(dependencies.create(project(":projectA"))) val zipFile : FileCollection = zipConfiguration.incoming.artifactView { }.files tasks.register<ZipConsumer>("zipConsumer") { inputFile.from(zipFile) }
  36. 🐘 // projectB val zipConfiguration = configurations.create("zipConsumer") { attributes {

    attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip")) } } zipConfiguration.dependencies.add(dependencies.create(project(":projectA"))) val zipFile : FileCollection = zipConfiguration.incoming.artifactView { }.files tasks.register<ZipConsumer>("zipConsumer") { inputFile.from(zipFile) }
  37. 🐘 - By default Gradle should flag cross-project task access

    and make cross project artifact sharing easier Lessons
  38. 🐘 - By default Gradle should flag cross-project task access

    and make cross project artifact sharing easier - Build engineers should test their builds with org.gradle.unsafe.isolated-projects=true to catch such violations Lessons
  39. 🐘 abstract class MyBuildService : BuildService<MyBuildServiceParameters> { val expensiveValue: String

    by lazy { Thread.sleep(5000) "hello" } } abstract class MyBuildServiceParameters : BuildServiceParameters
  40. 🐘 class MyPlugin : Plugin<Project> { override fun apply(target: Project)

    { val serviceProvider = target.gradle.sharedServices.registerIfAbsent( "myService", MyBuildService::class.java) target.tasks.register("myTask", MyTask::class.java) { it.myService.set(serviceProvider) it.usesService(serviceProvider) } } }
  41. 🐘 $ ./gradlew myTask FAILURE: Build failed with an exception.

    * What went wrong: A problem occurred configuring project ':projectB. > Could not create task ':projectB:myTask'. > Cannot set the value of task ':projectB:myTask' property 'myService' of type org.example.MyBuildService using a provider of type org.example.MyBuildService.
  42. 🐘 // projectA plugins { id("myPlugin") } // projectB plugins

    { id("myPlugin") alias(libs.plugins.ktfmt) } • myPlugin + its dependencies • myPlugin + its dependencies • ktmft + its dependencies build classloaders
  43. 🐘 - By default Gradle should forbid build classpath difference

    between projects and if they don’t, catch this failure with a better message https://github.com/gradle/gradle/issues/17559 Lessons
  44. 🐘 - By default Gradle should forbid build classpath difference

    between projects and if they don’t, catch this failure with a better message https://github.com/gradle/gradle/issues/17559 - Build engineers should add all their plugins in root build.gradle.kts while setting apply false Lessons
  45. 🐘 Takeaways - If you work at Gradle - deprecate

    APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation
  46. 🐘 Takeaways - If you work at Gradle - deprecate

    APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation
  47. 🐘 Takeaways - If you work at Gradle - deprecate

    APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation
  48. 🐘 Takeaways - If you work at Gradle - deprecate

    APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation