Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Effective Android Testing - A value driven appr...

Effective Android Testing - A value driven approach

Companion slides from my talk with same name.

Delivered in the following events

- PicPay Special Webinar (Online) - October/2021

Ubiratan Soares

October 27, 2021
Tweet

More Decks by Ubiratan Soares

Other Decks in Technology

Transcript

  1. Dimensions of testing • Which type of code a test

    touches • Which boundaries of the system it touches • Which real flows a test exercises • Which level of isolation are in place (if any) • Etc
  2. Entry point Exit point SUT Boundary Owned by Others Owned

    by Others Abstractions you own within the app boundaries Boundary
  3. Unit tests • Well-defined single entry point • Well-defined single

    exit point • Usually a small subject • Exercise abstractions you own • Not necessary 1:1 relationship to classes / functions • Never touches infrastructure ⚠
  4. Integration tests • Exercise entry points and/or exit points beyond

    your application boundaries • Always enforce some degree of isolation (ideally fully isolated) • Categorized according the number of integrations under test • Categorized according user-value under test
  5. Integration Level Rationale Single Exercises only one well-defined system boundary

    Multiple Exercises several well-defined system boundaries Component Exercises one well-defined component (user perspective) Acceptance Exercises several components with real user flows
  6. E2E tests • Black-box instrumentation over a deliverable artefact •

    Exercise user flows over a pre-production environment • The most expensive ones to write + maintain
  7. app Product onboarding login search … Features Platform core-persistance core-rest

    core-location core-translation common-android common-di common-assets …
  8. “If the vast majority of a codebase from a Mobile

    product deals with coordination over middleware and/or infrastructure, how do we do unit testing ?”
  9. Product Features Platform Unit tests (stable core) Integration tests Unit

    tests (on-demand) Contract tests Component tests Screenshot tests E2E tests Acceptance tests
  10. Unit Tests that bring value Tests over non-trivial validations over

    data provided by users via your App (eg, regexes)
  11. Unit Tests that bring value Tests over non-trivial data parsing

    that happens at Mobile level (eg URI / DeepLinks)
  12. Unit Tests that don’t bring value Tests over trivial data

    mappers used within your abstractions
  13. Integrated tests that bring value Comprehenstive tests over data sources,

    exercising thirdy-party transformation / serialization engines alongside with your own error handlers and/or data transformers
  14. class FactsDataSourceTests { @get:Rule val restInfrastructure = RestInfrastructureRule() private lateinit

    var dataSource: FactsDataSource @Before fun `before each test`() { val api = restInfrastructure.server.wireRestApi() dataSource = FactsDataSource(api) } @Test fun `should handle no results properly`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_no_results.json") ) val noFacts = emptyList<ChuckNorrisFact>() assertThat(simpleSearch()).isEqualTo(noFacts) } https://github.com/dotanuki-labs/norris
  15. class FactsDataSourceTests { @get:Rule val restInfrastructure = RestInfrastructureRule() private lateinit

    var dataSource: FactsDataSource @Before fun `before each test`() { val api = restInfrastructure.server.wireRestApi() dataSource = FactsDataSource(api) } @Test fun `should handle no results properly`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_no_results.json") ) val noFacts = emptyList<ChuckNorrisFact>() assertThat(simpleSearch()).isEqualTo(noFacts) } https://github.com/dotanuki-labs/norris
  16. class FactsDataSourceTests { @get:Rule val restInfrastructure = RestInfrastructureRule() private lateinit

    var dataSource: FactsDataSource @Before fun `before each test`() { val api = restInfrastructure.server.wireRestApi() dataSource = FactsDataSource(api) } @Test fun `should handle no results properly`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_no_results.json") ) val noFacts = emptyList<ChuckNorrisFact>() assertThat(simpleSearch()).isEqualTo(noFacts) } https://github.com/dotanuki-labs/norris
  17. @Test fun `should handle downstream error`() { restInfrastructure.restScenario(status = 500)

    assertErrorTransformed( whenRunning = this::simpleSearch, expected = RemoteServiceIntegrationError.RemoteSystem ) } https://github.com/dotanuki-labs/norris
  18. @Test fun `should fetch facts with valid query term`() {

    restInfrastructure.restScenario( status = 200, response = loadFile("200_search_with_results.json") ) val facts = listOf( ChuckNorrisFact( id = "2wzginmks8azrbaxnamxdw", shareableUrl = "https://api.chucknorris.io/jokes/2wzginmks", textual = "Chuck Norris divides by zero”, ) ) assertThat(simpleSearch()).isEqualTo(facts) } } https://github.com/dotanuki-labs/norris
  19. Integrated tests that bring value Component tests over entire screens,

    capturing the asynchonicity of data operations without coupling with implementation details (eg, Coroutines)
  20. @RunWith(AndroidJUnit4::class) class FactsActivityTests { private lateinit var screen: FakeFactsScreen @get:Rule

    val restInfrastructure = RestInfrastructureRule() @Before fun `before each test`() { // Switching over fake screen wrapper with DI support val testApp = TestApplication.setupWith( factsModule, factsTestModule, RestInfrastructureTestModule(restInfrastructure.server) ) screen = testApp.factsScreen() } https://github.com/dotanuki-labs/norris
  21. @Test fun `at first lunch, should start on empty state`()

    { whenActivityResumed<FactsActivity> { assertThat(screen.isLinked).isTrue() val expectedStates = listOf(Idle, Loading, FactsScreenState.Empty) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } // More tests @Test fun `when remote service fails, should display the error`() { restInfrastructure.restScenario(status = 503) PersistanceHelper.registerNewSearch("code") whenActivityResumed<FactsActivity> { val error = Failed(RemoteServiceIntegrationError.RemoteSystem) val expectedStates = listOf(Idle, Loading, error) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } } https://github.com/dotanuki-labs/norris
  22. @Test fun `at first lunch, should start on empty state`()

    { whenActivityResumed<FactsActivity> { assertThat(screen.isLinked).isTrue() val expectedStates = listOf(Idle, Loading, FactsScreenState.Empty) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } // More tests @Test fun `when remote service fails, should display the error`() { restInfrastructure.restScenario(status = 503) PersistanceHelper.registerNewSearch("code") whenActivityResumed<FactsActivity> { val error = Failed(RemoteServiceIntegrationError.RemoteSystem) val expectedStates = listOf(Idle, Loading, error) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } } https://github.com/dotanuki-labs/norris
  23. Screenshots tests over your UI delivery, decoupled from any data

    operations Integrated tests that bring value
  24. @LargeTest class FactsScreenshotTests : ScreenshotTest { private fun checkScreenshot(targetState: FactsScreenState)

    { val testActivity = prepareToCaptureScreenshot<FactsTestActivity> { target -> listOf(FactsScreenState.Idle, targetState).forEach { target.screen.updateWith(it) } } compareScreenshot(testActivity) } https://github.com/dotanuki-labs/norris
  25. @Test fun emptyState() { checkScreenshot(Empty) } @Test fun errorState() {

    val error = RemoteServiceIntegrationError.RemoteSystem checkScreenshot(Failed(error)) } @Test fun successState() { val facts = listOf( FactDisplayRow( url = "https://chucknorris.io/jokes/998877", fact = "Chuck Norris merges before commit anything on Git", displayWithSmallerFontSize = false ) ) val presentation = FactsPresentation("humor", facts) checkScreenshot(Success(presentation)) } } https://github.com/dotanuki-labs/norris
  26. @Test fun emptyState() { checkScreenshot(Empty) } @Test fun errorState() {

    val error = RemoteServiceIntegrationError.RemoteSystem checkScreenshot(Failed(error)) } @Test fun successState() { val facts = listOf( FactDisplayRow( url = "https://chucknorris.io/jokes/998877", fact = "Chuck Norris merges before commit anything on Git", displayWithSmallerFontSize = false ) ) val presentation = FactsPresentation("humor", facts) checkScreenshot(Success(presentation)) } } https://github.com/dotanuki-labs/norris
  27. Integrated tests that bring value Acceptance tests over a fake

    ready-to-deliver artefact, exercising real user flows in isolation
  28. @RunWith(AndroidJUnit4!"class) class NorrisAcceptanceTests { private val restInfrastructure = RestInfrastructureRule(customPort =

    4242) private val stressedExecution = FlakyTestRule() @get:Rule val rules: RuleChain = RuleChain.outerRule(stressedExecution).around(restInfrastructure) @Before fun beforeEachTest() { PersistanceHelper.clearStorage() } https://github.com/dotanuki-labs/norris
  29. @Repeat @Test fun shouldPerformFirstSearch_AfterFirstLunch_ByTypingATerm() { val searchTerm = "math" val

    suggestions = listOf("career", "dev", "humor") val fact = "Chuck Norris can divide by zero" val suggestionsPayload = RestDataBuilder.suggestionsPayload(suggestions) val factsPayload = RestDataBuilder.factsPayload(searchTerm, fact) restInfrastructure.run { restScenario(200, suggestionsPayload) restScenario(200, factsPayload) } https://github.com/dotanuki-labs/norris
  30. fun Project.applyAndroidApplicationConventions() { applyAndroidStandardConventions() val android = extensions.findByName("android") as ApplicationExtension

    android.apply { testBuildType = when { isTestMode() -> "release" else -> "debug" } defaultConfig { testInstrumentationRunner = AndroidDefinitions.instrumentationTestRunner } // More plugin-driven conventions } https://github.com/dotanuki-labs/norris
  31. fun Project.applyAndroidApplicationConventions() { applyAndroidStandardConventions() val android = extensions.findByName("android") as ApplicationExtension

    android.apply { testBuildType = when { isTestMode() -> "release" else -> "debug" } defaultConfig { testInstrumentationRunner = AndroidDefinitions.instrumentationTestRunner } // More plugin-driven conventions } https://github.com/dotanuki-labs/norris
  32. Integration Test Proposed Value Single integration over data sources Ensures

    data transformations + error handling over the target boundary Component Tests over entire screens Enables refactoring over target screen Screenshot Tests over entire screens Ensures association between state and Views without any assertions Acceptance Exercises several components with real user flows over a production-like artefact, emulating E2E at integration level
  33. Standards help large teams since they reduce both accidental complexity

    and also cognitive load when switching between contexts inside the same codebase
  34. Purpose Recommendation Reason Regular assertions Google Truth Max compatibility for

    JVM + Roboletric + Instrumentation Test Runner jUnit4 Junit5 is not supported for Android/Instrumentation Screenshot Tests Karumi/Shot Nice wrapper over Facebook tooling + useful add-ons Espresso Tests Barista Nice wrapper over Espresso tooling + useful add-ons Compose Tests Standard tooling Provided since day-zero Contract Tests Pact/JVM Most adopted solution E2E 👀 Up to you, as long it runs for as part of your pipeline
  35. What really matters is focusing on writting good tests that

    allow Engineers to move faster by maximizing the correlation between test code and value proposition (ie, probability of catching bugs)
  36. TL;DR • Be aware of unit tests that don’t bring

    any value • If the case for you, invest energy on integration tests specially over Data Sources and Screen Components • There is no such thing like effective Espresso tests over non-release builds • Let Mocks fade away from your codebase (and your life!) • Mind the conciliation between your Test Strategy and your Quality Strategy
  37. UBIRATAN SOARES Brazilian Computer Scientist Senior Software Engineer @ N26

    GDE for Android and Kotlin @ubiratanfsoares ubiratansoares.dev