A Sneak Peak at the Screenshot Testing Landscape in 2024

Lightning talk at Droidcon Berlin 2024
-----------------------------------------------------------------------------------
In recent years, screenshot testing has become an increasingly popular way to automate UI testing in Android applications, with new libraries and tools emerging regularly. This makes it hard to determine which ones are the best fit for your project.

In this lightning talk, we'll explore the most popular screenshot testing libraries and tools, including Google's brand new Compose Preview Screenshot Testing tool. We will highlight their unique features and analyse their pros and cons.

By the end of this session, you'll have the knowledge to confidently choose the optimal screenshot testing tools for your projects. Don't miss out on this opportunity to become a screenshot testing Ninja!

Transcript

1. BY SERGIO SASTRE LANDSCAPE IN 2024 SCREENSHOT TESTING A SNEAK PEAK AT THE 📸
2. How screenshot testing @Gio_Sastre works ?
3. record AND verify @Gio_Sastre
4. CI <code> write test + @Gio_Sastre
5. CI = record @Gio_Sastre
6. CI merge @Gio_Sastre
7. CI verify @Gio_Sastre <code> new PR refactoring
8. CI verify @Gio_Sastre <code> new PR refactoring
9. CI verify @Gio_Sastre <code> new PR refactoring
10. CI @Gio_Sastre <code> new PR refactoring
11. exist ? Which screenshot testing libraries @Gio_Sastre
12. Screenshot testing libraries Paparazzi Roborazzi Compose Preview jvm device emulator or android-testify Shot Dropshots @Gio_Sastre
13. Screenshot testing libraries Paparazzi Roborazzi Compose Preview jvm Layoutlib Robolectric based on @Gio_Sastre
14. @Gio_Sastre Auto-generate tests from @Previews @Previews source location Fully automated screenshotTest only + + main only main only
15. Composable Preview Scanner 1 Screenshot Tests from @Previews with ANY library https://github.com/sergio-sastre/ComposablePreviewScanner 2 Preview parameters (e.g. locale, fontScale, etc.) available @Gio_Sastre
16. ). apply { CardViewHolder ( val layout = LayoutIn fl ater.from (context).in fl ate (R. card_layout, null) bind ( CardItem( ) ) }. itemView … container = layout @Gio_Sastre “Android View” Previews
17. ). apply { CardViewHolder ( val layout = LayoutIn fl ater.from (context).in fl ate (R. card_layout, null) bind ( CardItem( ) ) }. itemView … container = layout @Gio_Sastre AndroidView ( modifier = Modifier.fillMaxSize() factory = { context -> ) } private fun CardViewHolderPreview() { @Composable @Preview CardViewHolderPreview CardViewHolderPreview - Dark @Preview (name = “Dark”, uiMode = UI_MODE_NIGHT_YES) “Android View” Previews
18. many configurations ? Screenshot tests under @Gio_Sastre
19. Configuration matters 1 3 Locale UI Mode 2 Orientation 4 Custom themes (Views) 5 Font Size @Gio_Sastre i.e. Light/Dark Mode
20. Font size UI Mode Orientation Custom theme (views) Locale out of the box out of the box out of the box out of the box out of the box possible via preview device orientation out of the box out of the box out of the box possible via ContextThemeWrapper @Gio_Sastre out of the box out of the box out of the box out of the box hard out of the box 1 out of the box 1 Activities under test must be open, otherwise still possible with AndroidUiTestingUtils out of the box 1 - - - - - - -
21. Cross-library screenshot testing for Views & Composables Android Ui Testing Utils 1 Screenshot Testing under different configurations https://github.com/sergio-sastre/AndroidUiTestingUtils 2 @Gio_Sastre
22. Instrumentation-based screenshot testing @Gio_Sastre
23. standard xml very nice HTML with diffs + diffs YAML + diffs JUnit reports + not supported not supported supported only from CLI from CLI & AS CLI or AS Plugin max 1 snapshot per test general value for all tests per test per test problematic Kotlin gradle scripts ⚠ super easy easy Dropshots Shot tolerance record & verify gradle managed devices test reports configuration android-testify @Gio_Sastre
24. Jvm-based screenshot testing @Gio_Sastre
25. Auto-generated from previews in “screenshotTest” source Use JUnit Use JUnit not supported not supported supported for - Android - iOS - Desktop independent + Bazel support new releases bundled to AGP upgrades new releases bundled to AGP upgrades Plenty - Accessibility - Tolerance - Animations (gif) - Screenshot file name Plenty - Accessibility - Tolerance - Animations (gif) - Screenshot file name none but coming supported supported not supported Roborazzi Compose Multiplatform Writing Tests AGP dependency Paparazzi compose-preview Dynamic Feature Modules Library options @Gio_Sastre
26. supported not supported not supported might render wrong might render wrong supported API 28+ Shadows & Elevation API 31+ RNG not supported supported supported not supported not supported from CLI & AS from CLI & AS only from CLI Roborazzi Activities 
 & Fragments Paparazzi compose-preview different 
 API levels record & verify side effects (compose) complex animations (views) @Gio_Sastre
27. Jvm ? Instrumentation or @Gio_Sastre
28. Instrumentation Jvm Very fast Less flaky no issues with emulators @Gio_Sastre What’s better? Best fidelity
29. @Gio_Sastre Still undecided.. @get:Rule class CrossLibraryTest { @RunWith (CrossLibraryScreenshotTestRunner :: class) val screenshotRule = CrossLibraryScreenshotTestRule ( con fi g = ScreenshotCongifForComposable ( locale = “en”, uiMode = UiMode.DAY, @CrossLibraryScreenshot @Test fun snapComposable { screenshotRule.snapshot( name = “screenshotName”) { MyComposable() ) ) ) ) } . /gradlew :module:recordPaparazziDebug -Plib=paparazzi . /gradlew :module:recordRoborazziDebug -Plib=roborazzi . /gradlew :module:executeScreenshotTest -Plib=shot . /gradlew :module:connectedAndroidTest -Plib=dropshots . /gradlew :module:screenshotRecord -Plib=android-testify
30. Android Ui Testing Utils 1 Screenshot Testing under different configurations https://github.com/sergio-sastre/AndroidUiTestingUtils 2 Cross-library screenshot testing for Views & Composables @Gio_Sastre
31. Android Screenshot Testing Playground 1 Ready to execute screenshot tests for https://github.com/sergio-sastre/Android-screenshot-testing-playground 2 Tests for Android Views, Jetpack Compose & Compose Previews @Gio_Sastre 🛝 + Cross-library
32. Android Screenshot Testing Playground 1 Ready to execute screenshot tests for https://github.com/sergio-sastre/Android-screenshot-testing-playground 2 Tests for Android Views, Jetpack Compose & Compose Previews @Gio_Sastre 🛝 + Cross-library @Gio_Sastre
33. screenshot testing ? Common problems with @Gio_Sastre
34. @Gio_Sastre Most scalable solution External cloud service Common screenshot testing issues Best Solution Record & Verify on the same machine File Management Problem Storage space Platform differences OS Problem Generated screenshots differ Firebase integration Problem Hard & not documented Solution Trial & error…
35. Screenshotbot 2 Records & verifies on the same machine https://github.com/screenshotbot/screenshotbot-gradle-plugin 3 Simple Firebase integration @Gio_Sastre 1 External cloud service
36. Screenshot testing tools Composable Preview Scanner https://github.com/sergio-sastre/ComposablePreviewScanner Android Ui Testing Utils https://github.com/sergio-sastre/AndroidUiTestingUtils @Gio_Sastre https://github.com/sergio-sastre/Android-screenshot-testing-playground 🛝 Android Screenshot Testing Playground Screenshotbot https://github.com/screenshotbot/screenshotbot-gradle-plugin
37. attention for your Thanks
38. Linkedin @SergioSastre sergio-sastre @Gio_Sastre Hashnode blogs Sergio Sastre Flórez GDE for Android Lead Android developer appdev.de Github sergio-sastre-florez