Share
Embed
Gradle: The Build System That Loves To Hateย You
Talk by Aurimas Liutikas (Google) at DroidCon London 2024
Aurimas Liutikas
November 01, 2024
Transcript
95 slides
-
๐ Gradle: The Build System That Loves To Hate You
The Survival Tips Aurimas Liutikas / AndroidX @ Google @[email protected] -
๐ Whoโs Aurimas? 12 years at Google 8.5 years on
AndroidX Work closely with Android Studio, Gradle, and Jetbrains -
๐ Sadly, the path to becoming a Gradle build engineer
is treacherous -
๐ tasks.create("myTask") { long start = System.currentTimeMillis() Thread.sleep(5000) long end
= System.currentTimeMillis() println( "Spent ${end - start} ms" ) } -
๐ $ ./gradlew lib:myTask > Configure project :lib Spent 5000
ms BUILD SUCCESSFUL in 6s -
๐ $ ./gradlew lib:jar > Configure project :lib Spent 5000
ms BUILD SUCCESSFUL in 6s -
๐ class MyBetterTask extends DefaultTask { @TaskAction void doTheWork() {
long start = System.currentTimeMillis() Thread.sleep(5000) long end = System.currentTimeMillis() println("Spent ${end - start} ms") } } tasks.create("myBetterTask", MyBetterTask) -
๐ class MyBetterTask extends DefaultTask { @TaskAction void doTheWork() {
long start = System.currentTimeMillis() Thread.sleep(5000) long end = System.currentTimeMillis() println("Spent ${end - start} ms") } } tasks.create("myBetterTask", MyBetterTask) -
๐ class MyBetterTask extends DefaultTask { @TaskAction void doTheWork() {
long start = System.currentTimeMillis() Thread.sleep(5000) long end = System.currentTimeMillis() println("Spent ${end - start} ms") } } tasks.create("myBetterTask", MyBetterTask) -
๐ tasks.register("myBetterTask", MyBetterTask) { // some expensive configuration // only
runs if task will be executed } -
๐ - Gradle should deprecate TaskContainer.create and friends https://github.com/gradle/gradle/issues/17705 (open
since 2021) - Build engineers need to understand configuration vs execution phase - avoid doing expensive work in configuration phase Lessons -
๐ - Gradle should deprecate TaskContainer.create and friends https://github.com/gradle/gradle/issues/17705 (open
since 2021) - Build engineers need to understand configuration vs execution phase - avoid doing expensive work in configuration phase - Gradle plugin authors should use androidx.lint:lint-gradle checks Lessons -
๐ tasks.create tasks.getByPath tasks.findByPath tasks.findByName tasks.withType tasks.getByName tasks.all tasks.matching Eager
APIs to Avoid tasks.whenTaskAdded tasks.whenObjectAdded tasks.findAll tasks.iterator, thus all of the kotlin-stdlib collection extension functions, e.g. tasks.any { } -
๐ tasks.create tasks.getByPath tasks.findByPath tasks.findByName tasks.withType tasks.getByName tasks.all tasks.matching Eager
APIs to Avoid tasks.whenTaskAdded tasks.whenObjectAdded tasks.findAll tasks.iterator, thus all of the kotlin-stdlib collection extension functions, e.g. tasks.any { } -
๐ - Gradle should deprecate all TaskContainer, TaskCollection, DomainObjectCollection methods
that are not register, named, or configureEach Lessons -
๐ - Gradle should deprecate all TaskContainer, TaskCollection, DomainObjectCollection methods
that are not register, named, or configureEach - Build engineers should pay attention to eagerly created tasks tasks.register("eagerCanary") { throw Exception("Eagerly configured tasks!") } Lessons -
๐ - Gradle should deprecate all TaskContainer, TaskCollection, DomainObjectCollection methods
that are not register, named, or configureEach - Build engineers should pay attention to eagerly created tasks tasks.register("eagerCanary") { throw Exception("Eagerly configured tasks!") } - Gradle plugin authors should use androidx.lint:lint-gradle checks Lessons -
๐ abstract class FooWriter : DefaultTask() { @get:OutputFile abstract val
outputFile : RegularFileProperty @TaskAction fun doTheWork() { outputFile.get().asFile.writeText("Hello") } } -
๐ abstract class FooReader : DefaultTask() { @get:InputFile abstract val
inputFile : RegularFileProperty @TaskAction fun doTheWork() { println(inputFile.get().asFile.readText()) } } -
๐ val foo = layout.buildDirectory.file("foo.txt") val writer = tasks.register<FooWriter>("fooWriter") {
outputFile.set(foo) } tasks.register<FooReader>("fooReader") { inputFile.set(foo) } -
๐ $ ./gradlew fooReader > Task fooReader FAILED FAILURE: Build
failed with an exception. * What went wrong: A problem was found with the configuration of task 'fooReader' (type 'FooReader'). - Type 'FooReader' property 'inputFile' specifies file 'foo.txt' which doesn't exist. -
๐ val foo = layout.buildDirectory.file("foo.txt") val writer = tasks.register<FooWriter>("fooWriter") {
outputFile.set(foo) } tasks.register<FooReader>("fooReader") { inputFile.set(foo) dependsOn(writer) } -
๐ val foo = layout.buildDirectory.file("foo.txt") val writer = tasks.register<FooWriter>("fooWriter") {
outputFile.set(foo) } tasks.register<FooReader>("betterFooReader") { inputFile.set(writer.flatMap { it.outputFile }) } -
๐ - Gradle should deprecate Task.dependsOn - Build engineers should
use Provider<Task>.flatmap to establish dependencies Lessons -
๐ $ ./gradlew lib2:betterFooReader > Task :lib2:betterFooReader Hello BUILD SUCCESSFUL
in 1s 2 actionable tasks: 1 executed, 1 up-to-date -
๐ - Gradle should force task authors to mark tasks
as @AlwaysRerunTask or declare an output Lessons -
๐ - Gradle should force task authors to mark tasks
as @AlwaysRerunTask or declare an output - Build engineers should always add outputs to tasks tasks.register<FooReader>("betterFooReader") { inputFile.set(writer.flatMap { it.outputFile }) cacheEvenIfNoOutputs() } Lessons -
๐ fun Task.cacheEvenIfNoOutputs() { this.outputs.file(getPlaceholderOutput()) } private fun Task.getPlaceholderOutput(): Provider<RegularFile>
{ return project.layout.buildDirectory.file( "placeholder/" + this.name.replace(":", "-") ) } -
๐ abstract class FooEnhancer : DefaultTask() { @get:InputFile abstract val
inputFile : RegularFileProperty @get:OutputFile abstract val outputFile : RegularFileProperty @TaskAction fun doTheWork() { outputFile.get().asFile.writeText( inputFile.get().asFile.readText() + "!" ) } } -
๐ $ ./gradlew fooEnhancer BUILD SUCCESSFUL in 965ms 2 actionable
tasks: 2 executed -
๐ $ ./gradlew fooEnhancer --info > Task :lib2:fooEnhancer Caching disabled
for task ':lib2:fooEnhancer' because: Build cache is disabled Caching has not been enabled for the task Task ':lib2:fooEnhancer' is not up-to-date because: Output property 'outputFile' file build/enhancedFoo.txt has been removed. -
๐ UP-TO-DATE (default on) Taskโs inputs and outputs did not
change. FROM-CACHE (default off) Taskโs outputs could be found from a previous execution. -
๐ $ ./gradlew fooEnhancer --info > Task :lib2:fooEnhancer Caching disabled
for task ':lib2:fooEnhancer' because: Caching has not been enabled for the task Task ':lib2:fooEnhancer' is not up-to-date because: Output property 'outputFile' file build/enhancedFoo.txt has been removed. -
๐ @CacheableTask abstract class FooEnhancer : DefaultTask() { @get:[InputFile PathSensitive(PathSensitivity.NONE)]
abstract val inputFile : RegularFileProperty @get:OutputFile abstract val outputFile : RegularFileProperty @TaskAction fun doTheWork() { outputFile.get().asFile.writeText(inputFile.get().asFile.readText() + "!") } } -
๐ $ ./gradlew fooEnhancer --info > Task :lib2:fooEnhancer FROM-CACHE Build
cache key for task ':lib2:fooEnhancer' is eb57efc9cb2dd2b6587672bac94200b4 Task ':lib2:fooEnhancer' is not up-to-date because: Output property 'outputFile' file build/enhancedFoo.txt has been removed. Loaded cache entry for task ':lib2:fooEnhancer' with cache key eb57efc9cb2dd2b6587672bac94200b4 -
๐ - Gradle should enable build caching by default and
force task authors to mark tasks as @DisableCachingByDefault or @CacheableTask Lessons -
๐ - Gradle should enable build caching by default and
force task authors to mark tasks as @DisableCachingByDefault or @CacheableTask - Build engineers should enable build cache and always mark their tasks @CacheableTask, unless it is an I/O operation such as copy. Lessons -
๐ - Gradle should enable build caching by default and
force task authors to mark tasks as @DisableCachingByDefault or @CacheableTask - Build engineers should enable build cache and always mark their tasks @CacheableTask, unless it is an I/O operation such as copy. - Gradle Plugin authors should enable: tasks.withType<ValidatePlugins>().configureEach { enableStricterValidation.set(true) } Lessons -
๐ // projectA tasks.register("zip", Zip) { it.from(layout.projectDirectory.file("build.gradle.kts")) destinationDirectory.set(layout.buildDirectory.dir("zip")) archiveFileName.set("my.zip") }
// projectB val zipTask = project(":projectA").tasks.named<Zip>("zip") tasks.register<ZipConsumer>("zipConsumer") { inputFile.set(zipTask.flatMap { it.archiveFile }) } -
๐ // projectA tasks.register("zip", Zip) { it.from(layout.projectDirectory.file("build.gradle")) destinationDirectory.set(layout.buildDirectory.dir("zip")) archiveFileName.set("my.zip") }
// projectB val zipTask = project(":projectA").tasks.named<Zip>("zip") tasks.register<ZipConsumer>("zipConsumer") { inputFile.set(zipTask.flatMap { it.archiveFile }) } -
๐ // projectA val zipTask = tasks.register<Zip>("zip") { โฆ }
configurations.register("zipPublish") { isCanBeConsumed = true isCanBeResolved = false attributes { attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip"))} } artifacts { add("zipPublish", zipTask) } -
๐ // projectB val zipConfiguration = configurations.create("zipConsumer") { attributes {
attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip")) } } zipConfiguration.dependencies.add(dependencies.create(project(":projectA"))) val zipFile : FileCollection = zipConfiguration.incoming.artifactView { }.files tasks.register<ZipConsumer>("zipConsumer") { inputFile.from(zipFile) } -
๐ // projectB val zipConfiguration = configurations.create("zipConsumer") { attributes {
attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip")) } } zipConfiguration.dependencies.add(dependencies.create(project(":projectA"))) val zipFile : FileCollection = zipConfiguration.incoming.artifactView { }.files tasks.register<ZipConsumer>("zipConsumer") { inputFile.from(zipFile) } -
๐ // projectB val zipConfiguration = configurations.create("zipConsumer") { attributes {
attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("special-zip")) } } zipConfiguration.dependencies.add(dependencies.create(project(":projectA"))) val zipFile : FileCollection = zipConfiguration.incoming.artifactView { }.files tasks.register<ZipConsumer>("zipConsumer") { inputFile.from(zipFile) } -
๐ - By default Gradle should flag cross-project task access
and make cross project artifact sharing easier Lessons -
๐ - By default Gradle should flag cross-project task access
and make cross project artifact sharing easier - Build engineers should test their builds with org.gradle.unsafe.isolated-projects=true to catch such violations Lessons -
๐ abstract class MyBuildService : BuildService<MyBuildServiceParameters> { val expensiveValue: String
by lazy { Thread.sleep(5000) "hello" } } abstract class MyBuildServiceParameters : BuildServiceParameters -
๐ class MyPlugin : Plugin<Project> { override fun apply(target: Project)
{ val serviceProvider = target.gradle.sharedServices.registerIfAbsent( "myService", MyBuildService::class.java) target.tasks.register("myTask", MyTask::class.java) { it.myService.set(serviceProvider) it.usesService(serviceProvider) } } } -
๐ $ ./gradlew myTask FAILURE: Build failed with an exception.
* What went wrong: A problem occurred configuring project ':projectB. > Could not create task ':projectB:myTask'. > Cannot set the value of task ':projectB:myTask' property 'myService' of type org.example.MyBuildService using a provider of type org.example.MyBuildService. -
๐ // projectA plugins { id("myPlugin") } // projectB plugins
{ id("myPlugin") alias(libs.plugins.ktfmt) } โข myPlugin + its dependencies โข myPlugin + its dependencies โข ktmft + its dependencies build classloaders -
๐ - By default Gradle should forbid build classpath difference
between projects and if they donโt, catch this failure with a better message https://github.com/gradle/gradle/issues/17559 Lessons -
๐ - By default Gradle should forbid build classpath difference
between projects and if they donโt, catch this failure with a better message https://github.com/gradle/gradle/issues/17559 - Build engineers should add all their plugins in root build.gradle.kts while setting apply false Lessons -
๐ Takeaways - If you work at Gradle - deprecate
APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation -
๐ Takeaways - If you work at Gradle - deprecate
APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation -
๐ Takeaways - If you work at Gradle - deprecate
APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation -
๐ Takeaways - If you work at Gradle - deprecate
APIs that have better alternatives - enable helpful features by default (build cache & configuration cache) - enforce best build practices by default - If you own a build - enable build cache, configuration cache, and test with project isolation - use well known Gradle builds as inspiration - https://github.com/android/gradle-recipes - https://github.com/slackhq/foundry - https://github.com/androidx/androidx - If you own a Gradle plugin - use androidx.lint:lint-gradle on it and enable stricter validation