Develop your CI tools

Nowadays, setting up a basic unit and syntax testing pipeline for your mobile app has never been easier! That’s a great starting point, but what comes next? We can leverage the CI pipeline to do so much more! Things like automatic code generation and advanced static analysis—all with an aim to ensuring the reliability and performance of your applications.

In this talk, we’ll take a look at several tools and techniques that you can use to improve your quality of life as a developer, largely by letting the CI do the hard work for you.

Transcript

1. Develop your CI Tools
2. Xavier F. Gouchet Senior Software Engineer at Datadog @xgouchet / @datadoghq
4. 0. Introduction
5. Small team ▫ 3 engineers ▫ Lots of features ▫ Lots of bug fixes ▫ Lots of improvements ▫ Small bandwidth Unsplash photo by @marvelous
6. CI setup ▫ Gitlab ▫ Bitrise * photo by @xgouchet
7. Delegating to the CI ▫ Runs automatically ▫ Strict and thorough ▫ Great way to offload the PR Review
8. Code Review Pyramid Credit: Gunnar Morling (CC-BY-SA 4.0) CODE STYLE TESTS IMPLEMENTATION API SEMANTICS DOCUMENTATION
9. 1. Detekt Your code, your rules…
10. What is Detekt? ▫ Kotlin Static Analysis ▫ Code smells ▫ Performance Issues ▫ Code complexity ▫ Documentation Missing ▫ …
11. Detekt Report
12. How does Detekt work ▫ Visitor pattern over the Syntax Tree (PSI) ▫ With Type Resolution ▫ Report issues on specific patterns
13. Custom Rule class MyCustomRule : Rule() { override val issue: Issue = Issue( javaClass.simpleName, Severity.CodeSmell, "A description of the issue…", Debt.TWENTY_MINS ) }
14. Custom Rule class MyCustomRule : Rule() { override val issue: Issue = Issue( javaClass.simpleName, Severity.CodeSmell, "A description of the issue…", Debt.TWENTY_MINS ) }
15. Custom Rule class MyCustomRule : Rule() { override val issue: Issue = Issue( javaClass.simpleName, Severity.CodeSmell, "A description of the issue…", Debt.TWENTY_MINS ) }
16. Custom Rule override fun visitExpression(expression: KtExpression) { if (invalidExpression(expression)) report( CodeSmell( issue, Entity.from(expression), "A message" ) ) }
17. Custom Rule override fun visitExpression(expression: KtExpression) { if (invalidExpression(expression)) report( CodeSmell( issue, Entity.from(expression), "A message" ) ) }
18. Custom Rule override fun visitExpression(expression: KtExpression) { if (invalidExpression(expression)) report( CodeSmell( issue, Entity.from(expression), "A message" ) ) }
19. A few examples of our custom rules ▫ Forbid throwing exceptions ▫ Forbid using check or require ▫ Forbid usage of !! ▫ Flag unsafe third party methods ▫ Flag TODO comments without a Jira ticket
20. Going Further ▫ Make the rule configurable from detekt.yml
21. Custom Rule class MyCustomRule( config: Config ) : Rule(config) { private val enableFoo: Boolean by config(defaultValue = true) //… }
22. Custom Rule class MyCustomRule( config: Config ) : Rule(config) { private val enableFoo: Boolean by config(defaultValue = true) //… }
23. Custom Rule class MyCustomRule( config: Config ) : Rule(config) { private val enableFoo: Boolean by config(defaultValue = true) //… }
24. More info… ▫ https:/ /detekt.dev/ ▫ https:/ /github.com/detekt/detekt ▫ https:/ /thebakery.dev/30/
25. 2. Kotlin Symbol Processor
26. What is KSP? ▫ Lightweight Compiler Plugin ▫ Similar to (K)APT but faster ▫ Source code is read-only ▫ Code generation
27. How does KSP work ▫ Called at compile time ▫ Processor can query files and/or declarations ▫ Can generate new code (but not modify existing one)
28. Custom Processor class MyCustomSymbolProcessor( val codeGenerator: CodeGenerator, val logger: KSPLogger ) : SymbolProcessor { override fun process(resolver: Resolver): List<KSAnnotated> { return emptyList() } }
29. Custom Processor class MyCustomSymbolProcessor( val codeGenerator: CodeGenerator, val logger: KSPLogger ) : SymbolProcessor { override fun process(resolver: Resolver): List<KSAnnotated> { return emptyList() } }
30. Custom Processor class MyCustomSymbolProcessor( val codeGenerator: CodeGenerator, val logger: KSPLogger ) : SymbolProcessor { override fun process(resolver: Resolver): List<KSAnnotated> { return emptyList() } }
31. Custom Processor override fun process(resolver: Resolver): List<KSAnnotated> { val deferred = mutableListOf<KSAnnotated>() resolver.getSymbolsWithAnnotation("com.example.MyAnnotation") .forEach { if (!generateSomething(it)) deferred.add(it) } return deferred }
32. Custom Processor override fun process(resolver: Resolver): List<KSAnnotated> { val deferred = mutableListOf<KSAnnotated>() resolver.getSymbolsWithAnnotation("com.example.MyAnnotation") .forEach { if (!generateSomething(it)) deferred.add(it) } return deferred }
33. Custom Processor override fun process(resolver: Resolver): List<KSAnnotated> { val deferred = mutableListOf<KSAnnotated>() resolver.getSymbolsWithAnnotation("com.example.MyAnnotation") .forEach { if (!generateSomething(it)) deferred.add(it) } return deferred }
34. Custom Processor override fun process(resolver: Resolver): List<KSAnnotated> { val deferred = mutableListOf<KSAnnotated>() resolver.getSymbolsWithAnnotation("com.example.MyAnnotation") .forEach { if (!generateSomething(it)) deferred.add(it) } return deferred }
35. Custom Processor override fun process(resolver: Resolver): List<KSAnnotated> { val deferred = mutableListOf<KSAnnotated>() resolver.getSymbolsWithAnnotation("com.example.MyAnnotation") .forEach { if (!generateSomething(it)) deferred.add(it) } return deferred }
36. An example of custom processor ▫ Generate no-op implementation of annotated interfaces
37. DataWriter.kt @NoOpImplementation internal interface DataWriter<T : Any> { fun write(element: T) fun write(data: List<T>) }
38. NoOpDataWriter.kt internal class NoOpDataWriter<T : Any> : DataWriter<T> { public override fun write(element: T) { } public override fun write(data: List<T>) { } }
39. DataReader.kt @NoOpImplementation internal interface DataReader<T : Any> { fun readNext(): T? }
40. NoOpDataReader.kt internal class NoOpDataReader<T : Any> : DataReader<T> { public override fun readNext(): T? { return null } }
41. Going Further ▫ KotlinPoet library
42. KotlinPoet val file = FileSpec.builder("", "Greeter") .addType( TypeSpec.classBuilder("Greeter") .addFunction( FunSpec.builder("greet") .addParameter("name", String::class) .addStatement("println(%P)", "Hello, \$name").build() ).build() ).build()
43. KotlinPoet val file = FileSpec.builder("", "Greeter") .addType( TypeSpec.classBuilder("Greeter") .addFunction( FunSpec.builder("greet") .addParameter("name", String::class) .addStatement("println(%P)", "Hello, \$name").build() ).build() ).build()
44. KotlinPoet val file = FileSpec.builder("", "Greeter") .addType( TypeSpec.classBuilder("Greeter") .addFunction( FunSpec.builder("greet") .addParameter("name", String::class) .addStatement("println(%P)", "Hello, \$name").build() ).build() ).build()
45. KotlinPoet val file = FileSpec.builder("", "Greeter") .addType( TypeSpec.classBuilder("Greeter") .addFunction( FunSpec.builder("greet") .addParameter("name", String::class) .addStatement("println(%P)", "Hello, \$name").build() ).build() ).build()
46. KotlinPoet class Greeter() { fun greet(name: String) { println("""Hello, $name""") } }
47. More info… ▫ https:/ /kotlinlang.org/docs/ksp-overview.html ▫ https:/ /github.com/google/ksp ▫ https:/ /square.github.io/kotlinpoet/
48. 3. Gradle Plugin
49. What is Gradle? ▫ Custom tasks ▫ Compile time checks ▫ Helper classes
50. How does Gradle work ▫ Project object created by Gradle ▫ Plugin adds extension/tasks to it
51. Custom Task open class MyCustomTask : DefaultTask() { init { group = "MyCompany" description = "Description of the task" } @TaskAction fun applyTask() { } }
52. Custom Task open class MyCustomTask : DefaultTask() { init { group = "MyCompany" description = "Description of the task" } @TaskAction fun applyTask() { } }
53. Custom Task open class MyCustomTask : DefaultTask() { init { group = "MyCompany" description = "Description of the task" } @TaskAction fun applyTask() { } }
54. Custom Extension open class MyCustomExtension( var outputFile: File = File(DEFAULT_OUTPUT_FILENAME), var enableSomething: Boolean = false )
55. Custom Plugin class MyCustomPlugin : Plugin<Project> { override fun apply(target: Project) { val myExtension = target.extensions .create("myExtension", MyCustomExtension::class.java) val myTask = target.tasks .create("myTask", MyCustomTask::class.java) myTask.extension = myExtension } }
56. Custom Plugin class MyCustomPlugin : Plugin<Project> { override fun apply(target: Project) { val myExtension = target.extensions .create("myExtension", MyCustomExtension::class.java) val myTask = target.tasks .create("myTask", MyCustomTask::class.java) myTask.extension = myExtension } }
57. Custom Plugin class MyCustomPlugin : Plugin<Project> { override fun apply(target: Project) { val myExtension = target.extensions .create("myExtension", MyCustomExtension::class.java) val myTask = target.tasks .create("myTask", MyCustomTask::class.java) myTask.extension = myExtension } }
58. Custom Plugin class MyCustomPlugin : Plugin<Project> { override fun apply(target: Project) { val myExtension = target.extensions .create("myExtension", MyCustomExtension::class.java) val myTask = target.tasks .create("myTask", MyCustomTask::class.java) myTask.extension = myExtension } }
59. Custom Plugin class MyCustomPlugin : Plugin<Project> { override fun apply(target: Project) { val myExtension = target.extensions .create("myExtension", MyCustomExtension::class.java) val myTask = target.tasks .create("myTask", MyCustomTask::class.java) myTask.extension = myExtension } }
60. An example of custom Gradle Tasks ▫ Compute dependencies size and OSS Licenses ▫ Keep track of API Surface ▫ Generate Data Class from JSON Schema ▫ From a remote git repository
61. Going Further ▫ Link tasks together
62. Custom Plugin override fun apply(target: Project) { // ... target.afterEvaluate { p -> p.tasks.withType(KotlinCompile::class.java) { dependsOn(myTask) } } }
63. More info… ▫ https:/ /docs.gradle.org/current/userguide/custom_plugins.html ▫ https:/ /www.youtube.com/watch?v=ww0D8WMWPXc ▫ https:/ /speakerdeck.com/xgouchet/rock-the-gradle-mobileera
64. 4. Kotlin Compiler Plugin Let’s start with the first set of slides
65. What is it? ▫ Plugin getting access to Kotlin source code / IR ▫ Can manipulate the code before compilation into JVM/JS/… ▫ Similar to Aspect Oriented Programming
66. Cross-cutting concerns
67. How does it work Parsing Resolution Analysis Code Generation Source Code PSI (AST) IR JVM JS Native
68. Compiler Plugin ▫ Kotlin Compiler Plugin ▫ Gradle Plugin / Maven Plugin ▫ IDE Plugin (icons / syntax)
69. Examples of Kotlin Compiler Plugins ▫ Generate Compose boilerplate code ▫ All-open makes all class open by default
70. More info… ▫ https:/ /www.youtube.com/watch?v=w-GMlaziIyo ▫ https:/ /arrow-kt.io/docs/meta/ ▫ https:/ /medium.com/@heyitsmohit/writing-kotlin-compiler-pl ugin-with-arrow-meta-cf7b3689aa3e
71. *. A little bit more Let’s start with the first set of slides
72. PSI Viewer ▫ IDE Plugin for Intellij Idea and Android Studio ▫ Displays the actual PSI for the open tab
74. *. Conclusion Let’s start with the first set of slides
75. Detekt Gradle KSP KCP
76. “ Your Detekt rules, KSP, Gradle scripts are still code. Keep them as clean, maintainable and trustworthy as your production code
77. Useful links ▫ https:/ /github.com/datadog/dd-sdk-android
78. Thank You! Any questions? @xgouchet @datadoghq Unsplash photo by @marcosjluiz
79. Credits Special thanks to all the people who made and released these awesome resources for free: ▫ Presentation template by SlidesCarnival ▫ Photographs by Unsplash 79