Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Dive into the Android Gradle Plugin (Droid...

Deep Dive into the Android Gradle Plugin (Droidcon UK 2018)

Android developers use it every day, but who knows what actually goes on under the hood of the Android Gradle Plugin?

In this talk, you'll learn about AGP internals and answer questions like:

* What's D8 and R8?
* What is mergeResources and why does it take so long?
* Why are there so many tasks?
* How can I measure and speed up my builds?

You'll come away with a better understanding of the toolchain which will make you a better equipped Android developer!

Video: http://uk.droidcon.com/skillscasts/12718-deep-dive-into-the-android-gradle-plugin

John Rodriguez

October 26, 2018
Tweet

More Decks by John Rodriguez

Other Decks in Technology

Transcript

  1. • Initialization • Configuration • Execution Build phases rootProject.name =

    'cash' include ':analytics' include ':app' include ':db' include ':keypad' include ':presenters' include ':protos' include ':screens' include ':viewmodels'
  2. • Initialization • Configuration • Execution Build phases Settings rootProject.name

    = 'cash' include ':analytics' include ':app' include ':db' include ':keypad' include ':presenters' include ':protos' include ':screens' include ':viewmodels' project project project project project project project project project
  3. • Initialization • Configuration • Execution Build phases project('cash') project(':analytics')

    project(':db') project(':app') project(':keypad') project(':presenters') project(':protos') project(':screens') project(':viewmodels')
  4. • Initialization • Configuration • Execution Build phases project('cash') project(':analytics')

    project(':db') project(':app') project(':keypad') project(':presenters') project(':protos') project(':screens') project(':viewmodels') > Configure project : > Configure project :analytics > Configure project :app > Configure project :db > Configure project :keypad > Configure project :presenters > Configure project :protos > Configure project :screens > Configure project :viewmodels
  5. • Initialization • Configuration • Execution Build phases project('cash') project(':analytics')

    project(':db') project(':app') project(':keypad') project(':presenters') project(':protos') project(':screens') project(':viewmodels') > Configure project : > Configure project :db > Configure project :protos ./gradlew :db:assembleDebug —configure-on-demand
  6. Configuration buildscript { repositories { google() jcenter() }d dependencies {

    classpath 'com.android.tools.build:gradle:3.2.1' }f }e
  7. Configuration buildscript { repositories { google() jcenter() }d dependencies {

    classpath 'com.android.tools.build:gradle:3.2.1' classpath 'org.jetbrains.kotlin:kotlin-gradle-plugin:1.2.71' classpath 'net.ltgt.gradle:gradle-errorprone-plugin:0.0.16' classpath 'com.squareup.sqldelight:android-gradle-plugin:1.0.0' }f }e
  8. class GreetingPlugin implements Plugin<Project> { @Override void apply(Project project) {

    project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e
  9. class GreetingPlugin implements Plugin<Project> { @Override void apply(Project project) {

    project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e project.apply plugin: GreetingPlugin
  10. class GreetingPlugin implements Plugin<Project> { @Override void apply(Project project) {

    project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e apply plugin: GreetingPlugin
  11. class GreetingPluginExtension { String message String greeter }a class GreetingPlugin

    implements Plugin<Project> { @Override void apply(Project project) { project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e apply plugin: GreetingPlugin
  12. class GreetingPluginExtension {1 String message String greeter }a class GreetingPlugin

    implements Plugin<Project> {2 @Override void apply(Project project) {3 def extension = project.extensions.create('greeting', GreetingPluginExtension) project.tasks.register('Greeting') {4 it.doLast {0 println 'Hello from the Greeting Plugin' }b }a }f }e apply plugin: GreetingPlugin
  13. class GreetingPluginExtension {1 String message String greeter }a class GreetingPlugin

    implements Plugin<Project> {2 @Override void apply(Project project) {3 def extension = project.extensions.create('greeting', GreetingPluginExtension) project.tasks.register('Greeting') {4 it.doLast {0 println "${extension.message} from ${extension.greeter}" }b }a }f }e apply plugin: GreetingPlugin
  14. apply plugin: GreetingPlugin greeting { message 'Hi' greeter 'Gradle' }

    $ ./gradlew Greeting > Task :Greeting Hi from Gradle
  15. interface Project extends ExtensionAware, PluginAware { } interface ExtensionAware {

    ExtensionContainer getExtensions(); } interface PluginAware { void apply(…); PluginManager getPluginManager(); }
  16. class SomeTask extends DefaultTask { @OutputDirectory File getOutputDir() {…} @InputFiles

    List<File> getInputFiles() {…} @TaskAction void doStuff() {…}
  17. $ ./gradlew --no-daemon --no-build-cache —rerun-tasks / -Dorg.gradle.debug=true task To honour

    the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.10.2/ userguide/gradle_daemon.html. > Starting Daemon
  18. ?

  19. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') gradle.taskGraph.allTasks.each {

    task -> task.taskDependencies.getDependencies(task).each { dep -> // add task -> dep to graph }a }b 'dot -Tpng -O project.dot'.execute(…) }c
  20. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') gradle.taskGraph.allTasks.each {

    task -> task.taskDependencies.getDependencies(task).each { dep -> // add task -> dep to graph }a }b 'dot -Tpng -O project.dot'.execute(…) }c
  21. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') gradle.taskGraph.allTasks.each {

    task -> task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  22. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') def startTasks

    = gradle.startParameter.getTaskNames().join(" ") println "command: ./gradlew " + startTasks gradle.taskGraph.allTasks.each { task -> println "task: " + task.name + ", class: " + task.class.name task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  23. command: ./gradlew app:assembleDebug compileJava -> JavaCompile processResources -> ProcessResources classes

    -> DefaultTask jar -> Jar checkDebugClasspath -> AppClasspathCheckTask preBuild -> DefaultTask preDebugBuild -> AppPreBuildTask compileDebugAidl -> AidlCompile compileDebugRenderscript -> RenderscriptCompile checkDebugManifest -> CheckManifest generateDebugBuildConfig -> GenerateBuildConfig prepareLintJar -> PrepareLintJar mainApkListPersistenceDebug -> MainApkListPersistence generateDebugResValues -> GenerateResValues generateDebugResources -> DefaultTask mergeDebugResources -> MergeResources …e
  24. command: ./gradlew app:assembleDebug compileJava -> JavaCompile processResources -> ProcessResources classes

    -> DefaultTask jar -> Jar checkDebugClasspath -> AppClasspathCheckTask preBuild -> DefaultTask preDebugBuild -> AppPreBuildTask compileDebugAidl -> AidlCompile compileDebugRenderscript -> RenderscriptCompile checkDebugManifest -> CheckManifest generateDebugBuildConfig -> GenerateBuildConfig prepareLintJar -> PrepareLintJar mainApkListPersistenceDebug -> MainApkListPersistence generateDebugResValues -> GenerateResValues generateDebugResources -> DefaultTask mergeDebugResources -> MergeResources …e
  25. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') def startTasks

    = gradle.startParameter.getTaskNames().join(" ") println "command: ./gradlew " + startTasks gradle.taskGraph.allTasks.each { task -> println "task: " + task.name + ", class: " + task.class.name task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  26. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') def startTasks

    = gradle.startParameter.getTaskNames().join(" ") println "command: ./gradlew " + startTasks gradle.taskGraph.allTasks.each { task -> println "task: " + task.name + ", class: " + task.class.name println " inputs: " task.inputs.each { it.files.each { println " " + it } } println " outputs: " task.outputs.each { it.files.each { println " " + it } } task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  27. gradle.taskGraph.afterTask { task -> println " inputs: " task.inputs.files.each {

    println " " + it } println " outputs: " task.outputs.files.each { println " " + it } }
  28. buildTypes { debug { applicationIdSuffix '.debug' minifyEnabled false proguardFile file('proguard-cash.pro')

    proguardFile file('proguard-cash-debug.pro') } release { minifyEnabled true shrinkResources true proguardFile file('proguard-cash.pro') } }
  29. productFlavors { flavorDimensions 'environment' internal { dimension 'environment' applicationId 'com.squareup.cash.beta'

    } production { dimension 'environment' applicationId 'com.squareup.cash' } } debug release
  30. :app dependencies { implementation project(':api') }a :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 }
  31. :app dependencies { implementation project(':api') } :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 }
  32. :app dependencies { implementation project(':api') }a :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 }a
  33. :app dependencies { implementation project(':api') } :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 } :protos dependencies { api deps.wire.runtime }
  34. :app dependencies { implementation project(':api') } :api dependencies { api

    project(':protos') implementation deps.retrofit implementation deps.rx2 } :protos dependencies { api deps.wire.runtime }
  35. :app dependencies { implementation project(':api') } :api dependencies { api

    project(':protos') implementation deps.retrofit implementation deps.rx2 } :protos dependencies { api deps.wire.runtime }
  36. :app dependencies { implementation project(':api') }a :api dependencies { api

    project(':protos') implementation deps.retrofit api deps.rx2 }b :protos dependencies { api deps.wire.runtime }c
  37. :app dependencies { implementation project(':api') } :api dependencies { api

    project(':protos') api deps.retrofit api deps.rx2 } :protos dependencies { api deps.wire.runtime }
  38. class AppClasspathCheckTask extends ClasspathComparisionTask { @Override void onDifferentVersionsFound(…)a{ String message

    = String.format( "Conflict with dependency '%1$s:%2$s' in project '%3$s'. Resolved versions for runtime classpath (%4$s) and compile classpath (%5$s) differ. This can lead to runtime crashes. To resolve this issue …\n”, …); reporter.reportWarning(EvalIssueReporter.Type.GENERIC, message); } @TaskAction void run() { compareClasspaths(); } }e
  39. class AppClasspathCheckTask extends ClasspathComparisionTask { a{…} static class ConfigAction TaskConfigAction<AppClasspathCheckTask>

    { String getName() { return variantScope.getTaskName("check", "Classpath"); } @Override void execute(@NonNull AppClasspathCheckTask task) { task.setVariantName(variantScope.getFullVariantName()); task.runtimeClasspath = variantScope.getArtifactCollection(RUNTIME_CLASSPATH, …); task.compileClasspath = variantScope.getArtifactCollection(COMPILE_CLASSPATH, …); } } }e
  40. :moduleA dependencies { api libs.retrofit implementation libs.okhttp } :moduleB dependencies

    { implementation libs.retrofit } :app dependencies { implementation project(':module-a') implementation project(':module-b') }
  41. $ ./gradlew app:dependencies debugCompileClasspath: debug +--- project :module-a | \---

    com.squareup.retrofit2:retrofit:2.3.0 | \--- com.squareup.okhttp3:okhttp:3.8.0 | \--- com.squareup.okio:okio:1.13.0 \--- project :module-b … debugRuntimeClasspath: debug +--- project :module-a | +--- com.squareup.retrofit2:retrofit:2.3.0 | | \--- com.squareup.okhttp3:okhttp:3.8.0 -> 3.9.0 | | \--- com.squareup.okio:okio:1.13.0 | \--- com.squareup.okhttp3:okhttp:3.9.0 (*) \--- project :module-b \--- com.squareup.retrofit2:retrofit:2.3.0 (*)
  42. class JavaPreCompileTask extends AndroidBuilderTask { @OutputFile File getProcessorListFile() { return

    processorListFile; } @TaskAction void preCompile() { Set<String> classNames = Sets.newHashSet(); classNames.addAll( convertArtifactsToNames( collectAnnotationProcessors(annotationProcessorConfiguration) )); Gson gson = new GsonBuilder().create(); try (FileWriter writer = new FileWriter(processorListFile)) { gson.toJson(classNames, writer); } } /** * Returns a List of packages in the configuration believed to
  43. } } /** * Returns a List of packages in

    the configuration believed to * contain an annotation processor. * * We assume a package has an annotation processor if it contains the * META-INF/services/javax.annotation.processing.Processor file. */ List<…> collectAnnotationProcessors(ArtifactCollection config) {}
  44. > Task :app:mergeDebugResources inputs: ~/.gradle/caches/…/aapt2-3.2.1-4818971-osx.jar/… …/app/build/generated/res/resValues/debug …/app/build/generated/res/rs/debug …/app/src/main/res …/app/src/debug/res outputs:

    …/app/build/intermediates/blame/res/debug …/app/build/generated/res/pngs/debug …/app/build/intermediates/incremental/mergeDebugResources …/app/build/intermediates/res/merged/debug
  45. > Task :app:processDebugResources inputs: ~/.gradle/caches/…/aapt2-3.2.1-4818971-osx.jar/… …/app/build/intermediates/apk_list/…/apk-list.gson …/app/build/intermediates/res/merged/debug …/app/build/intermediates/split_list/…/split-list.gson …/app/build/intermediates/merged_manifests/…/merged outputs:

    …/app/build/intermediates/incremental/processDebugResources …/app/build/intermediates/processed_res/…/out …/app/build/generated/not_namespaced_r_class_sources/…/r …/app/build/intermediates/res/symbol-table…/…/package-aware-r.txt …/app/build/intermediates/symbols/debug/R.txt
  46. Resource processing • Before in aapt: take all resources and

    outputs one binary • Now in aapt2: compile + link • converts individual resources into binary flat files • merge at the end • uses Gradle incremental check feature for compile avoidance • Improving mergeReleaseResources for libraries • avoids full merge of resources for all the transitive dependencies
  47. ArtifactTransform API Request • aar -> android-manifest Internally • aar

    -> exploded-aar • exploded-aar -> android-manifest • Runs once, cached globally • Downloaded, transformed on demand
  48. android.libraryVariants.all { variant -> def runtimeConfiguration = variant.getRuntimeConfiguration() runtimeConfiguration.getIncoming() .artifactView(new

    Action<ArtifactView.ViewConfiguration>() { void execute(ArtifactView.ViewConfiguration config2) { config2.attributes(new Action<AttributeContainer>() { void execute(AttributeContainer c) { c.attribute(Attribute.of("artifactType", String), “android-assets“) } }) } }) .artifacts.artifacts.each { result -> // This file is an asset def file = result.file }
  49. android.libraryVariants.all { variant -> def runtimeConfiguration = variant.getRuntimeConfiguration() runtimeConfiguration.getIncoming() .artifactView(new

    Action<ArtifactView.ViewConfiguration>() { void execute(ArtifactView.ViewConfiguration config2) { config2.attributes(new Action<AttributeContainer>() { void execute(AttributeContainer c) { c.attribute(Attribute.of("artifactType", String), “android-assets“) } }) } }) .artifacts.artifacts.each { result -> // This file is an asset def file = result.file }
  50. ArtifactTransform API “give me all the assets” Before: view =

    configuration.getFiles() // check if AAR (could have JARS), then find assets… After: view = configuration.getIncoming().artifactView(config -> {...}) Downstream tasks can set up their inputs as a lazy FileCollection • allowing resolution on demand during task execution, as opposed to configuration. FileCollection will only contain the assets and tasks generating them • helpful for variant-awareness by not generating extra tasks
  51. • the process of transforming .class bytecode into .dex bytecode

    for Android
 • D8 - new dex compiler, replaces DX
 • DX takes a class,zip,jar,apk and converts to a single dex
 • When comparing with the current DX compiler, D8 compiles faster and outputs smaller .dex files, while having the same or better app runtime performance.
 • To test in 3.0, android.enableD8=true
 • Set to default in 3.1.
 Dexing
  52. DX $ $ANDROID_HOME/build-tools/LATEST/dx --dex —output=out input.jar D8 Debug mode build:

    $ java -jar build/libs/d8.jar --output out input.jar Release mode build: $ java -jar build/libs/d8.jar --release --output out input.jar Dexing
  53. • the process of transforming .class bytecode into .dex bytecode

    for Android
 • D8 - new dex compiler, replaces DX
 • DX takes a class,zip,jar,apk and converts to a single dex
 • When comparing with the current DX compiler, D8 compiles faster and outputs smaller .dex files, while having the same or better app runtime performance.
 • To test in 3.0, android.enableD8=true
 • Set to default in 3.1.
 Desugaring
  54. • Migration from PSI to UAST as of AGP 3.0

    • Jetbrains + Google • Kotlin UAST support as of AGP 3.1 • Quick fixes • Lots more detail here: • Lint to the Finish Line: https://www.youtube.com/watch?v=wFe0WZm_xm8 • Kotlin Static Analysis with Android Lint: https://www.youtube.com/watch?v=p8yX5-lPS6o Lint
  55. enum class BooleanOption( override val propertyName: String, override val defaultValue:

    Boolean = false, override val status: Option.Status = Option.Status.EXPERIMENTAL, override val additionalInfo: String = "" ) : Option<Boolean> { ENABLE_AAPT2("android.enableAapt2", true, DeprecationReporter.DeprecationTarget.AAPT), ENABLE_BUILD_CACHE("android.enableBuildCache", true), ENABLE_PROFILE_JSON("android.enableProfileJson", false), // Used by Studio as workaround for b/71054106, b/75955471 ENABLE_SDK_DOWNLOAD("android.builder.sdkDownload", true, status = Option.Status.STABLE), ENABLE_TEST_SHARDING("android.androidTest.shardBetweenDevices"), ENABLE_DEX_ARCHIVE( "android.useDexArchive", true, aDeprecationReporter.DeprecationTarget.LEGACY_DEXER ),a
  56. VERSION_CHECK_OVERRIDE_PROPERTY("android.overrideVersionCheck"), OVERRIDE_PATH_CHECK_PROPERTY("android.overridePathCheck"), ENABLE_DESUGAR( "android.enableDesugar", true, DeprecationReporter.DeprecationTarget.DESUGAR_TOOL), ENABLE_INCREMENTAL_DESUGARING( “android.enableIncrementalDesugaring", true, DeprecationReporter.DeprecationTarget.INCREMENTAL_DESUGARING),

    ENABLE_GRADLE_WORKERS("android.enableGradleWorkers", false), ENABLE_AAPT2_WORKER_ACTIONS( “android.enableAapt2WorkerActions", true), ENABLE_CORE_LAMBDA_STUBS( "android.enableCoreLambdaStubs", true, DeprecationReporter.DeprecationTarget.CORE_LAMBDA_STUBS), ENABLE_D8("android.enableD8", true, DeprecationReporter.DeprecationTarget.LEGACY_DEXER),
  57. AndroidProject.PROPERTY_REFRESH_EXTERNAL_NATIVE_MODEL, status = Option.Status.STABLE), IDE_GENERATE_SOURCES_ONLY( AndroidProject.PROPERTY_GENERATE_SOURCES_ONLY, status = Option.Status.STABLE), ENABLE_SEPARATE_APK_RESOURCES("android.enableSeparateApkRes",

    true), ENABLE_EXPERIMENTAL_FEATURE_DATABINDING( "android.enableExperimentalFeatureDatabinding", false), ENABLE_SEPARATE_R_CLASS_COMPILATION( "android.enableSeparateRClassCompilation"), ENABLE_JETIFIER("android.enableJetifier", false, status = Option.Status.STABLE), USE_ANDROID_X("android.useAndroidX", false, status = Option.Status.STABLE), ENABLE_UNIT_TEST_BINARY_RESOURCES( "android.enableUnitTestBinaryResources", false), DISABLE_EARLY_MANIFEST_PARSING(