Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automated migration of Android apps to Bazel bu...

Automated migration of Android apps to Bazel build system - Droidcon SF 2022

Migrating large projects that consist of hundreds or thousands of modules and being maintained by a large team, from Gradle to Bazel might be challenging. The talk describes the process of automation of the migration that includes the development and usage of a Gradle plugin as well as a type-safe and declarative Kotlin DSL for Bazel code generation.
An automated migration can save a lot of time and effort for organizations by making the transition to Bazel build system smoothly and allow to enjoy the benefits Bazel provides much sooner.
The ideas discussed in this talk were used in the project for automated migration to Bazel, called Airin: https://github.com/Morfly/airin
More information about it could be found in the talk from BazelCon 2021: https://youtu.be/dz-CFEwJuko
Or in this blog post: https://proandroiddev.com/304fa8b3680c
Source code with the example project: https://github.com/Morfly/android-bazel-migration-sample

Pavlo Stavytskyi

June 04, 2022
Tweet

More Decks by Pavlo Stavytskyi

Other Decks in Programming

Transcript

  1. About me • Mobile infra at Lyft • Grab, Singapore

    • Google Developer Expert for Android • 6 years of Android development 2
  2. Agenda • What is Bazel? • Creating a simple Gradle

    plugin for migration of Android app to Bazel 4
  3. Agenda • What is Bazel? • Creating a simple Gradle

    plugin for migration of Android app to Bazel • See a type-safe Kotlin DSL for generating Bazel scripts 5
  4. What is Bazel? • Build tool (similarly to Gradle) •

    Polyglot. Extensible • Granularity. Incremental builds 9
  5. What is Bazel? • Build tool (similarly to Gradle) •

    Polyglot. Extensible • Granularity. Incremental builds • Remote build cache/execution 10
  6. What is Bazel? • Build tool (similarly to Gradle) •

    Polyglot. Extensible • Granularity. Incremental builds • Remote build cache/execution • Query language 11
  7. plugins { id 'com.android.application' id 'kotlin-android' } android { compileSdkVersion

    30 defaultConfig { applicationId "com.morfly.sample" minSdkVersion 21 targetSdkVersion 30 } buildTypes { release { minifyEnabled false proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' } } compileOptions { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 } kotlinOptions { jvmTarget = '1.8' } } dependencies { implementation "org.jetbrains.kotlin:kotlin-stdlib:1.5.10" implementation 'androidx.core:core-ktx:1.6.0' implementation 'androidx.appcompat:appcompat:1.3.1' implementation 'com.google.android.material:material:1.4.0' implementation 'androidx.constraintlayout:constraintlayout:2.1.0' } Gradle script example 13
  8. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Gradle script example 14
  9. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 15
  10. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 16
  11. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 17
  12. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 18
  13. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 19
  14. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 20
  15. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 21
  16. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 22
  17. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 23
  18. load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "my_lib", srcs =

    glob([ "src/main/java/**/*.kt", "src/main/kotlin/**/*.kt", ]), custom_package = "com.morfly.sample", manifest = "src/main/AndroidManifest.xml", resource_files = glob(["src/main/res/**"]), deps = [ artifact("androidx.core:core-ktx"), artifact("androidx.appcompat:appcompat"), artifact("com.google.android.material:material"), artifact("androidx.constraintlayout:constraintlayout"), ], ) Bazel script components 24
  19. Gradle task purpose 35 For each project module… Get module

    dependencies Determine Bazel module type
  20. Bazel module types • Kotlin Android module • Pure Java

    Android module — android_library 37
  21. Bazel module types • Kotlin Android module • Pure Java

    Android module • Kotlin library module — kt_jvm_library 38
  22. Bazel module types • Kotlin Android module • Pure Java

    Android module • Kotlin library module • Pure Java library module — java_library ... 39
  23. Gradle task purpose 40 For each project module… Get module

    dependencies Determine Bazel module type Generate Bazel file from a template
  24. import org.gradle.api.Plugin import org.gradle.api.Project import org.gradle.kotlin.dsl.register class MigrationPlugin : Plugin<Project>

    { override fun apply(target: Project) { target.tasks.register<MigrateToBazelTask>("migrateToBazel") } } Creating Gradle plugin 43
  25. import org.gradle.api.Plugin import org.gradle.api.Project import org.gradle.kotlin.dsl.register class MigrationPlugin : Plugin<Project>

    { override fun apply(target: Project) { target.tasks.register<MigrateToBazelTask>("migrateToBazel") } } Creating Gradle plugin 44
  26. import org.gradle.api.Plugin import org.gradle.api.Project import org.gradle.kotlin.dsl.register class MigrationPlugin : Plugin<Project>

    { override fun apply(target: Project) { target.tasks.register<MigrateToBazelTask>("migrateToBazel") } } Registering Gradle task 45
  27. import org.gradle.api.Plugin import org.gradle.api.Project import org.gradle.kotlin.dsl.register class MigrationPlugin : Plugin<Project>

    { override fun apply(target: Project) { target.tasks.register<MigrateToBazelTask>("migrateToBazel") } } Registering Gradle task 46
  28. import org.gradle.api.Plugin import org.gradle.api.Project import org.gradle.kotlin.dsl.register class MigrationPlugin : Plugin<Project>

    { override fun apply(target: Project) { target.tasks.register<MigrateToBazelTask>("migrateToBazel") } } Registering Gradle task 47
  29. fun android_library_template( name: String, packageName: String, artifactDeps: String, moduleDeps: String

    ) = """ load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = "$name", srcs = glob(["src/main/java/**/*.kt"]), resource_files = glob(["src/main/res/**"]), custom_package = "$packageName", manifest = "src/main/AndroidManifest.xml", visibility = ["./visibility:public"], deps = [ "./:dagger", $moduleDeps, $artifactDeps ], ) """ Bazel file template 56
  30. fun android_library_build( name: String, packageName: String, artifactDeps: List<String>, moduleDeps: List<String>

    ) = BUILD.bazel { load("@io_bazel_rules_kotlin./kotlin:kotlin.bzl", "kt_android_library") load("@rules_jvm_external./:defs.bzl", "artifact") kt_android_library( name = name, srcs = glob("src/main/java/**/*.kt"), resource_files = glob("src/main/res/**"), custom_package = packageName, manifest = "src/main/AndroidManifest.xml", visibility = list["./visibility:public"], deps = moduleDeps + artifactDeps.map { artifact(it) } ) } Type-safe Bazel file template 57
  31. BUILD.bazel { val NAME by "app" ... } Starlark DSL

    components Variable assignment 65
  32. BUILD.bazel { val NAME by "app" val SRCS by list["MainActivity.java",

    "MainViewModel.java"] ... } Starlark DSL components Variable assignment 66
  33. BUILD.bazel { val NAME by "app" val SRCS by list["MainActivity.java",

    "MainViewModel.java"] val MANIFEST_VALUES by dict { "minSdkVersion" to "21" } ... } Starlark DSL components Variable assignment 67
  34. BUILD.bazel { val SRCS: List<String> by list["MainActivity.java"] `+` "MainViewModel.java" Compilation

    error: Required: List Found: String ... } Starlark DSL components Concatenation 72
  35. BUILD.bazel { "i" `in` LIST take { it `+` it

    } ... } Starlark DSL components List comprehensions 75
  36. BUILD.bazel { "i" `in` LIST take { it `+` it

    } ... } Starlark DSL components List comprehensions <item> 76
  37. BUILD.bazel { "i" `in` LIST take { it `+` it

    } ... } Starlark DSL components List comprehensions <list> 77
  38. BUILD.bazel { "i" `in` LIST take { it `+` it

    } ... } Starlark DSL components List comprehensions <expression> 78
  39. SRCS = ["MainActivity.java"] android_binary( name = "app", srcs = SRCS

    + ["utils/StringUtils.java"] ) Starlark DSL components Build rules and functions 80
  40. BUILD.bazel { val SRCS by list["MainActivity.java"] android_binary( name = "app",

    srcs = SRCS `+` list["utils/StringUtils.java"] ) ... } Starlark DSL components Build rules and functions 81
  41. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 82
  42. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 83
  43. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 84
  44. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 85
  45. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 86
  46. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 87
  47. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 88
  48. VIEW_MODELS_WITH_RES_IMPORTS = ["MainViewModel.kt", ...] [ genrule( name = "modify_imports_in_" +

    file[0:-3], srcs = [file], outs = [file[0:-3] + "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import com.morfly.R/import com.morfly.viewmodels.R/g' > $(OUTS) """, ) for file in VIEW_MODELS_WITH_RES_IMPORTS ] APP_FILES = [ "modify_imports_in_" + file[0:-3] for file in VIEW_MODELS_WITH_RES_IMPORTS ] Starlark DSL components Putting it all together… 89
  49. fun android_library_build( packageName: String, ... ) = BUILD { val

    VIEW_MODELS_WITH_RES_IMPORTS by list["MainViewModel.kt", ...] "file" `in` VIEW_MODELS_WITH_RES_IMPORTS take { file .> genrule( name = "modify_imports_in_" `+` file[0..-3], srcs = list[file], outs = list[file[0..-3] `+` "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import $packageName.R/import $packageName.viewmodels.R/g' > $(OUTS) """.trimIndent() ) } val APP_FILES by ("file" `in` VIEW_MODELS_WITH_RES_IMPORTS take { "modify_imports_in_" `+` it[0..-3] }) } Starlark DSL components Putting it all together… 90
  50. fun android_library_build( packageName: String, ... ) = BUILD { val

    VIEW_MODELS_WITH_RES_IMPORTS by list["MainViewModel.kt", ...] "file" `in` VIEW_MODELS_WITH_RES_IMPORTS take { file .> genrule( name = "modify_imports_in_" `+` file[0..-3], srcs = list[file], outs = list[file[0..-3] `+` "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import $packageName.R/import $packageName.viewmodels.R/g' > $(OUTS) """.trimIndent() ) } val APP_FILES by ("file" `in` VIEW_MODELS_WITH_RES_IMPORTS take { "modify_imports_in_" `+` it[0..-3] }) } Starlark DSL components Putting it all together… 91
  51. fun android_library_build( packageName: String, ... ) = BUILD { val

    VIEW_MODELS_WITH_RES_IMPORTS by list["MainViewModel.kt", ...] "file" `in` VIEW_MODELS_WITH_RES_IMPORTS take { file .> genrule( name = "modify_imports_in_" `+` file[0..-3], srcs = list[file], outs = list[file[0..-3] `+` "_synthetic.kt"], cmd = """ cat $(SRCS) | sed 's/import $packageName.R/import $packageName.viewmodels.R/g' > $(OUTS) """.trimIndent() ) } val APP_FILES by ("file" `in` VIEW_MODELS_WITH_RES_IMPORTS take { "modify_imports_in_" `+` it[0..-3] }) } Starlark DSL components Putting it all together… 92
  52. Source code 97 A tool for auto-migration to Bazel github.com/morfly/airin

    Sample project source code github.com/morfly/ android-bazel-migration-sample