Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Screenshot Testing for Android: Which Library i...

Screenshot Testing for Android: Which Library is Right for You?

Slides of my lighting talk "Screenshot Testing for Android: Which Library is Right for You?" at Droidcon Lisbon 2023
-------------------------------------------------------------------------------
In recent years, screenshot testing has become an increasingly popular way to automate UI testing in Android applications, with new libraries emerging regularly. That makes hard to determine which one is the best fit for your project.

In this tech-talk, we'll explore the top screenshot testing libraries, from on-device to JVM-based libraries, including the newest player in the game: Roborazzi, based on Google's Robolectric.

We'll dive deep into each library's unique features and compare their pros and cons to help you make an informed decision about which one is right for your project.

Whether you're new to screenshot testing or have experience with it, this tech-talk will take your UI testing skills to the next level. You'll leave this talk with the knowledge to confidently choose the best screenshot testing library for your projects. Don't miss out on this opportunity to become a screenshot testing Ninja in Android!
---------------------------------------------------------------
Repos of the lighting talk:
https://github.com/sergio-sastre/Android-screenshot-testing-playground
https://github.com/sergio-sastre/AndroidUiTestingUtils

Sergio Sastre Flórez

October 03, 2023
Tweet

More Decks by Sergio Sastre Flórez

Other Decks in Programming

Transcript

  1. Special type of UI tests take a screenshot of it,

    compare it to its reference we inflate a UI component, where and @Gio_Sastre
  2. jvm layoutlib robolectric robolectric * Needs to be con f

    igured * Screenshot testing libraries @Gio_Sastre
  3. Dropshots Shot tolerance record & verify gradle managed devices test

    reports general value for all tests only from CLI @Gio_Sastre standard xml configuration problematic Windows support Kotlin gradle scripts ⚠ ⚠ super easy per test from CLI & AS very nice HTML with diffs not supported not supported + diffs android-testify easy per test CLI or AS Plugin supported YAML + diffs (experimental) max 1 snapshot per test
  4. Dropshots Shot tolerance record & verify gradle managed devices test

    reports general value for all tests only from CLI @Gio_Sastre standard xml configuration problematic Windows support Kotlin gradle scripts ⚠ ⚠ super easy per test from CLI & AS very nice HTML with diffs not supported not supported + diffs android-testify easy per test CLI or AS Plugin supported YAML + diffs (experimental) max 1 snapshot per test
  5. Paparazzi Windows OS not supported Roborazzi Compose Multiplatform (as desktop)

    Dynamic Feature Modules not supported supported @Gio_Sastre supported 2 1 Due to Robolectric Native Graphics 2 Due to Robolectric supported 2 Bazel record & verify from CLI & AS only from CLI unofficial AS plugin supported 1 Via Skiko or not supported 3 3 not supported
  6. Paparazzi Windows OS not supported Roborazzi Compose Multiplatform (as desktop)

    Dynamic Feature Modules not supported supported @Gio_Sastre supported 2 1 Due to Robolectric Native Graphics 2 Due to Robolectric supported 2 Bazel record & verify from CLI & AS only from CLI unofficial AS plugin supported 1 Via Skiko or not supported 3 3 not supported
  7. Paparazzi complex animations (views) Side effects (compose) render elevation not

    supported Roborazzi different 
 API levels supported not supported not supported not fully supported not supported supported @Gio_Sastre supported2 1 2 1 Due to layoutlib 2 Due to Robolectric 2 supported supported2 2 Activities 
 & Fragments
  8. Paparazzi complex animations (views) Side effects (compose) render elevation not

    supported Roborazzi different 
 API levels supported not supported not supported not fully supported not supported supported @Gio_Sastre supported2 1 2 1 Due to layoutlib 2 Due to Robolectric 2 supported supported2 2 Activities 
 & Fragments
  9. @Gio_Sastre Configuration matters 1 3 Locale UI Mode 2 Orientation

    4 6 Custom themes Display Size 5 Font Size
  10. Android Ui Testing Utils @Gio_Sastre 1 Screenshot Testing under different

    configurations https://github.com/sergio-sastre/AndroidUiTestingUtils 2 Cross-library screenshot testing
  11. @Gio_Sastre possible Display size Font size + Locale UI Mode

    Orientation out of the box out of the box out of the box 1 via updateConfiguration() possible switch height & width 1 Activities under test must be open, otherwise still possible with AndroidUiTestingUtils out of the box Custom theme out of the box possible via updateConfiguration() out of the box out of the box out of the box robolectric based
  12. complex animations (views) Side effects (compose) render elevation not supported

    different 
 API levels Activity & Fragment not supported not supported supported not fully supported not supported supported @Gio_Sastre supported2 1 2 1 Due to layoutlib 2 Due to Robolectric supported supported 2 robolectric based 2 2
  13. supported supported supported supported supported complex animations (views) Side effects

    (compose) render elevation not supported different 
 API levels Activity & Fragment not supported not supported supported not fully supported not supported supported @Gio_Sastre supported2 1 2 1 Due to layoutlib 2 Due to Robolectric supported supported 2 robolectric based 2 2
  14. @Gio_Sastre * Only possible with Paparazzi and Roborazzi On-device on-device

    vs. jvm Best fidelity Foldable support Special configuration (adb) Jvm Very fast Less flaky Accessibility services * (no issues with emulators)