Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A New Era of Testing

A New Era of Testing

Marcel Schnelle

August 27, 2024
Tweet

More Decks by Marcel Schnelle

Other Decks in Programming

Transcript

  1. JUnit • Successor to JUnit 4, the standard for testing

    on Android • First public release: September 11, 2016 • Rebuilt from scratch and divided into three components • Jupiter (for writing tests) • Platform (for running tests) • Vintage (for backwards compatibility)
  2. A simple JUnit 4 test case import org.junit.Assert.assertEquals import org.junit.Test

    class ExampleUnitTest { @Test fun test() { assertEquals(4, 2 + 2) } }
  3. android-junit • mannodermaus/android-junit5 • Developer tools for using JUnit 5

    with Android • Gradle plugin + libraries for instrumentation tests • First public release: September 11, 2017
  4. Setting up Unit Tests // libs.versions.toml [versions] androidJUnit5Plugin = “1.11.0.0”

    junit5 = “5.11.0” junit4 = “4.13.2” [plugins] android-junit5 = { id = "de.mannodermaus.android-junit5", version.ref = "androidJUnit5Plugin" } [libraries] junit-jupiter-api = { module = "org.junit.jupiter:junit-jupiter-api", version.ref = "junit5" } junit-jupiter-engine = { module = "org.junit.jupiter:junit-jupiter-engine", version.ref = "junit5" } junit-jupiter-params = { module = “org.junit.jupiter:junit-jupiter-params", version.ref = "junit5" } junit4 = { module = “junit:junit", version.ref = "junit4" } junit-vintage-engine = { module = “org.junit.vintage:junit-vintage-engine”, version.ref = “junit5" } 1. Add dependencies to project // libs.versions.toml [versions] androidJUnit5Plugin = “1.11.0.0” junit5 = “5.11.0” junit4 = “4.13.2” [plugins] android-junit5 = { id = "de.mannodermaus.android-junit5", version.ref = "androidJUnit5Plugin" } [libraries] junit-jupiter-api = { module = "org.junit.jupiter:junit-jupiter-api", version.ref = "junit5" } junit-jupiter-engine = { module = "org.junit.jupiter:junit-jupiter-engine", version.ref = "junit5" } junit-jupiter-params = { module = “org.junit.jupiter:junit-jupiter-params", version.ref = "junit5" } junit4 = { module = “junit:junit", version.ref = "junit4" } junit-vintage-engine = { module = “org.junit.vintage:junit-vintage-engine”, version.ref = “junit5" }
  5. Setting up Unit Tests // libs.versions.toml [versions] androidJUnit5Plugin = “1.11.0.0”

    junit5 = “5.11.0” junit4 = “4.13.2” [plugins] android-junit5 = { id = "de.mannodermaus.android-junit5", version.ref = "androidJUnit5Plugin" } [libraries] junit-jupiter-api = { module = "org.junit.jupiter:junit-jupiter-api", version.ref = "junit5" } junit-jupiter-engine = { module = "org.junit.jupiter:junit-jupiter-engine", version.ref = "junit5" } junit-jupiter-params = { module = “org.junit.jupiter:junit-jupiter-params", version.ref = "junit5" } junit4 = { module = “junit:junit", version.ref = "junit4" } junit-vintage-engine = { module = “org.junit.vintage:junit-vintage-engine”, version.ref = “junit5" } 1. Add dependencies to project // libs.versions.toml [versions] junit4 = “4.13.2” [libraries] junit4 = { module = “junit:junit", version.ref = "junit4" } junit-vintage-engine = { module = “org.junit.vintage:junit-vintage-engine”, version.ref = “junit5" } For JUnit 4
  6. Setting up Unit Tests // app/build.gradle.kts plugins { alias(libs.plugins.android.junit5) }

    dependencies { testImplementation(libs.junit.jupiter.api) testImplementation(libs.junit.jupiter.params) testRuntimeOnly(libs.junit.jupiter.engine) testImplementation(libs.junit4) testRuntimeOnly(libs.junit.vintage.engine) } 2. Prepare module build script file // app/build.gradle.kts plugins { alias(libs.plugins.android.junit5) } dependencies { testImplementation(libs.junit.jupiter.api) testImplementation(libs.junit.jupiter.params) testRuntimeOnly(libs.junit.jupiter.engine) testImplementation(libs.junit4) testRuntimeOnly(libs.junit.vintage.engine) }
  7. Setting up Unit Tests // app/build.gradle.kts dependencies { testImplementation(libs.junit4) testRuntimeOnly(libs.junit.vintage.engine)

    } 2. Prepare module build script file // app/build.gradle.kts plugins { alias(libs.plugins.android.junit5) } testImplementation(libs.junit.jupiter.api) testImplementation(libs.junit.jupiter.params) testRuntimeOnly(libs.junit.jupiter.engine) For JUnit 4
  8. Setting up Unit Tests import org.junit.Assert.assertEquals import org.junit.Test class JUnit4Test

    { @Test fun `test running with junit4`() { assertEquals(4, 2 + 2) } } import org.junit.jupiter.api.Assertions.assertEquals import org.junit.jupiter.api.RepeatedTest import org.junit.jupiter.api.RepetitionInfo import org.junit.jupiter.api.Test class JUnit5Test { @Test fun `test running with junit5`() { assertEquals(4, 2 + 2) } @RepeatedTest(2) fun `repeated test`(info: RepetitionInfo) { val i = info.currentRepetition assertEquals(i * 2, i + i) } }
  9. About the minSdk requirement • JUnit 5 uses several Java

    8 APIs (SimpleFileVisitor, java.nio.file.*) • Only available on API 26 (Android 8.0) and higher • Desugaring does not include these features • Result: Instrumentation tests are skipped on API < 26
  10. Setting up Instrumentation Tests 2. There is no step 2

    • Plugin automatically provides new APIs to you • Let's write some JUnit 5 tests!
  11. Simple: Test with ActivityScenario // ... import de.mannodermaus.junit5.ActivityScenarioExtension import org.junit.jupiter.api.Test

    import org.junit.jupiter.api.extension.RegisterExtension class JUnit5ActivityScenarioTest { @RegisterExtension val scenarioExtension = ActivityScenarioExtension.launch<MyActivity>() @Test fun test() { onView(withId(R.id.textView)).check(matches(isDisplayed())) onView(withId(R.id.button)).perform(click()) onView(withText("Updated")).check(matches(isDisplayed())) } }
  12. Fancy: Test with ActivityScenario class JUnit5ActivityScenarioTest { @RegisterExtension val scenarioExtension

    = ActivityScenarioExtension.launch<MyActivity>() @ValueSource(strings = ["Login", "Register"]) @ParameterizedTest fun test(buttonLabel: String) { onView(withId(R.id.textView)).check(matches(isDisplayed())) onView(withText(buttonLabel)).perform(click()) onView(withText("Updated")).check(matches(isDisplayed())) } }
  13. Fancy: Test with ActivityScenario class JUnit5ActivityScenarioTest { @RegisterExtension val scenarioExtension

    = ActivityScenarioExtension.launch<MyActivity>() @ValueSource(strings = ["Login", "Register"]) @ParameterizedTest fun test(buttonLabel: String) { onView(withId(R.id.textView)).check(matches(isDisplayed())) onView(withText(buttonLabel)).perform(click()) onView(withText("Updated")).check(matches(isDisplayed())) } }
  14. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource @MethodSource @EnumSource @ValueSource @ValueSource(ints = [1, 2, 3]) @ParameterizedTest fun testInts(value: Int) { // ... } @ValueSource(classes = [Foo::class, Bar::class, Baz::class]) @ParameterizedTest fun testClasses(value: Class<*>) { // ... }
  15. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource @MethodSource @EnumSource @ValueSource @ValueSource(ints = [1, 2, 3]) @ParameterizedTest fun testInts(value: Int) { // ... } @ValueSource(classes = [Foo::class, Bar::class, Baz::class]) @ParameterizedTest fun testClasses(value: Class<*>) { // ... }
  16. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource @MethodSource @EnumSource enum class Direction { North, East, South, West } @EnumSource(Direction::class) @ParameterizedTest fun testEnum(direction: Direction) { // ... }
  17. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource @MethodSource @EnumSource enum class Direction { North, East, South, West } @EnumSource(Direction::class) @ParameterizedTest fun testEnum(direction: Direction) { // ... }
  18. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource @MethodSource companion object { @JvmStatic fun arguments() = listOf( Arguments.of("First", Foo()), Arguments.of("Second", Bar()), Arguments.of("Third", Baz()), ) } @MethodSource("arguments") @ParameterizedTest fun testMethod(label: String, model: Base) { // ... }
  19. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource @MethodSource companion object { @JvmStatic fun arguments() = listOf( Arguments.of("First", Foo()), Arguments.of("Second", Bar()), Arguments.of("Third", Baz()), ) } @MethodSource("arguments") @ParameterizedTest fun testMethod(label: String, model: Base) { // ... }
  20. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource companion object { @JvmStatic val argumentsField = listOf(1234, 777, 573) } @FieldSource("argumentsField") @ParameterizedTest fun testField(number: Int) { // ... }
  21. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @FieldSource companion object { @JvmStatic val argumentsField = listOf(1234, 777, 573) } @FieldSource("argumentsField") @ParameterizedTest fun testField(number: Int) { // ... }
  22. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @CsvSource( "Alice, 34, f, true", "Bob, 16, m, false", "Charlie, 45, m, true", ) @ParameterizedTest fun testCsv(name: String, age: Int, gender: Char, expectedAdult: Boolean) { val person = Person(name, age, gender) assertEquals(expectedAdult, person.isAdult) }
  23. @ArgumentsSource @NullAndEmpty Source Parameterized Argument Sources @NullSource @EmptySource @CsvFileSource @CsvSource

    @CsvSource( "Alice, 34, f, true", "Bob, 16, m, false", "Charlie, 45, m, true", ) @ParameterizedTest fun testCsv(name: String, age: Int, gender: Char, expectedAdult: Boolean) { val person = Person(name, age, gender) assertEquals(expectedAdult, person.isAdult) }
  24. Compose UI Test with JUnit 5 @RegisterExtension val composeExtension =

    createComposeExtension() @CsvSource( // Dark mode | Font scale " false, 1.0", " false, 2.0", " true, 1.0", " true, 2.0", ) @ParameterizedTest(name = "dark={0}, font={1}") fun test(darkTheme: Boolean, fontScale: Float) = composeExtension.use { setContent { DK24ShowcaseTheme( darkTheme = darkTheme, fontScale = fontScale, ) { Greeting("DroidKaigi", onNameUpdated = {}) } } onNodeWithText("Hello DroidKaigi").assertExists() } @RegisterExtension val composeExtension = createComposeExtension() @CsvSource( // Dark mode | Font scale " false, 1.0", " false, 2.0", " true, 1.0", " true, 2.0", ) @ParameterizedTest(name = "dark={0}, font={1}") fun test(darkTheme: Boolean, fontScale: Float) = composeExtension.use { setContent { DK24ShowcaseTheme( darkTheme = darkTheme, fontScale = fontScale, ) { Greeting("DroidKaigi", onNameUpdated = {}) } } onNodeWithText("Hello DroidKaigi").assertExists() }
  25. @RegisterExtension val composeExtension = createComposeExtension() @CsvSource( // Dark mode |

    Font scale " false, 1.0", " false, 2.0", " true, 1.0", " true, 2.0", ) @ParameterizedTest(name = "dark={0}, font={1}") fun test(darkTheme: Boolean, fontScale: Float) = composeExtension.use { setContent { DK24ShowcaseTheme( darkTheme = darkTheme, fontScale = fontScale, ) { Greeting("DroidKaigi", onNameUpdated = {}) } } onNodeWithText("Hello DroidKaigi").assertExists() } Compose UI Test with JUnit 5
  26. Conditional Test Execution • Tests can be skipped when certain

    conditions apply, e.g.: • Environment variables • System properties • Operating system • android-junit5 adds more conditions, e.g.: • Build manufacturer • BuildConfig value • API level
  27. Conditional Annotations @Test @DisabledIfEnvironmentVariable(named = "CI", matches = "true") fun

    testOnlyOnLocal() { // ... } @Test @EnabledIfEnvironmentVariable(named = "CI", matches = "true") fun testOnlyOnCI() { // ... }
  28. Conditional Annotations @Test @EnabledOnManufacturer(["samsung"]) fun testOnlyOnSamsungDevices() { // ... }

    @Test @DisabledOnManufacturer(["google"]) fun testNotOnGoogleDevices() { // ... }
  29. Conditional Annotations @Test @EnabledIfBuildConfigValue(named = "DEBUG", matches = "true") fun

    testInDebugBuilds() { // ... } @Test @DisabledIfBuildConfigValue(named = "FLAVOR", matches = "prod") fun testNotInProd() { // ... }
  30. Conditional Annotations @Test @EnabledOnSdkVersion(from = 26, until = 30) fun

    testFromAndroid8UntilAndroid11() { // ... } @Test @DisabledOnSdkVersion(from = 33) fun testOnlyBelowAndroid13() { // ... }
  31. Parallel Test Execution • Can greatly reduce test execution time

    • All unit tests can (theoretically) be parallelized • For instrumentation tests • UI tests (Espresso, Compose) are not supported • Non-UI tests (e.g. Realm database) are supported
  32. Slow: Without Parallel Tests class ParallelTest { @RepeatedTest(5) fun test(info:

    RepetitionInfo) { val i = info.currentRepetition Thread.sleep(i * 1_000L) } }
  33. Setting up Parallel Test Execution // app/build.gradle.kts junitPlatform { configurationParameter(

    "junit.jupiter.execution.parallel.enabled", "true" ) } // app/build.gradle.kts junitPlatform { configurationParameter( "junit.jupiter.execution.parallel.enabled", "true" ) }
  34. Fast(er): With Parallel Tests @Execution(ExecutionMode.CONCURRENT) class ParallelTest { @RepeatedTest(5) fun

    test(info: RepetitionInfo) { val i = info.currentRepetition Thread.sleep(i * 1_000L) } }
  35. Robolectric: An overview • Framework for running unit tests with

    Android APIs • Physical device not necessary • Simulation of hardware events for better tests • Other libraries (Roborazzi, Robospec) are built on top
  36. Robolectric: An overview For JUnit 4 import android.content.Intent import android.net.Uri

    import org.junit.Assert.assertNotNull import org.junit.Test class HogeTest { @Test fun checkIntent() { val intent = Intent().setData(Uri.parse("app://main?campaign=1")) assertNotNull(intent) } }
  37. Robolectric: An overview For JUnit 4 import android.content.Intent import android.net.Uri

    import org.junit.Assert.assertNotNull import org.junit.Test class HogeTest { @Test fun checkIntent() { val intent = Intent().setData(Uri.parse("app://main?campaign=1")) assertNotNull(intent) } }
  38. Robolectric: An overview For JUnit 4 import android.content.Intent import android.net.Uri

    import org.junit.Assert.assertNotNull import org.junit.Test import org.junit.runner.RunWith import org.robolectric.RobolectricTestRunner @RunWith(RobolectricTestRunner::class) class HogeTest { @Test fun checkIntent() { val intent = Intent().setData(Uri.parse("app://main?campaign=1")) assertNotNull(intent) } }
  39. Robolectric: An overview For JUnit 4 import android.content.Intent import android.net.Uri

    import org.junit.Assert.assertNotNull import org.junit.Test import org.junit.runner.RunWith import org.robolectric.RobolectricTestRunner @RunWith(RobolectricTestRunner::class) class HogeTest { @Test fun checkIntent() { val intent = Intent().setData(Uri.parse("app://main?campaign=1")) assertNotNull(intent) } }
  40. How does Robolectric work? • Robolectric injects a modified version

    of android.jar into the runtime classpath • When the ClassLoader loads an Android API, it uses Robolectric's version instead of the default stub • RobolectricTestRunner takes care of this injection
  41. uT

  42. junit. -robolectric-extension • apter-tech/junit5-robolectric-extension • Class loader replacement logic for

    Robolectric & JUnit 5 • Still highly experimental, but works for basic cases
  43. Robolectric with JUnit 5 import android.content.Intent import android.net.Uri import org.junit.jupiter.api.Assertions.assertNotNull

    import org.junit.jupiter.api.Test import org.junit.jupiter.api.extension.ExtendWith import tech.apter.junit.jupiter.robolectric.RobolectricExtension @ExtendWith(RobolectricExtension::class) class HogeTest { @Test fun checkIntent() { val intent = Intent().setData(Uri.parse("app://main?campaign=1")) assertNotNull(intent) } }
  44. Robolectric with JUnit 5 import android.content.Intent import android.net.Uri import org.junit.jupiter.api.Assertions.assertNotNull

    import org.junit.jupiter.api.Test import org.junit.jupiter.api.extension.ExtendWith import tech.apter.junit.jupiter.robolectric.RobolectricExtension @ExtendWith(RobolectricExtension::class) class HogeTest { @Test fun checkIntent() { val intent = Intent().setData(Uri.parse("app://main?campaign=1")) assertNotNull(intent) } }
  45. What doesn't work (1) • junit5-robolectric-extension issue with Compose unit

    tests @ExtendWith(RobolectricExtension::class) class RobolectricComposeJUnit5Test { @RegisterExtension val composeExtension = createComposeExtension() @Test fun `robolectric compose test running with junit5`() { composeExtension.use { setContent { Text("Test") } onNodeWithText("Test").assertIsDisplayed() } } }
  46. What doesn't work (1) • junit5-robolectric-extension issue with Compose unit

    tests @ExtendWith(RobolectricExtension::class) class RobolectricComposeJUnit5Test { @RegisterExtension val composeExtension = createComposeExtension() @Test fun `robolectric compose test running with junit5`() { composeExtension.use { setContent { Text("Test") } onNodeWithText("Test").assertIsDisplayed() } } }
  47. What doesn't work (2) • Using Espresso in multiple unit

    tests causes a deadlock @ExtendWith(RobolectricExtension::class) class RobolectricEspressoJUnit5Test { @Test fun `robolectric espresso test #1 running with junit5`() { val scenario = ActivityScenario.launch(ViewActivity::class.java) onView(withId(R.id.textView)).check(matches(isDisplayed())) onView(withId(R.id.button)).perform(click()) onView(withText("Updated")).check(matches(isDisplayed())) scenario.close() } @Test fun `robolectric espresso test #2 running with junit5`() { // ... } }
  48. What doesn't work (2) • Using Espresso in multiple unit

    tests causes a deadlock @ExtendWith(RobolectricExtension::class) class RobolectricEspressoJUnit5Test { @Test fun `robolectric espresso test #1 running with junit5`() { val scenario = ActivityScenario.launch(ViewActivity::class.java) onView(withId(R.id.textView)).check(matches(isDisplayed())) onView(withId(R.id.button)).perform(click()) onView(withText("Updated")).check(matches(isDisplayed())) scenario.close() } @Test fun `robolectric espresso test #2 running with junit5`() { // ... } } will run forever...
  49. My testing recommendation in 2024 Unit Tests Instrumentation Tests Robolectric,

    Espresso, Compose Other Tests minSdk < 26 minSdk >= 26 (w/ Vintage Engine) + (slow adoption) 4 4