Efficient Android UI Testing, Droidcon Italy 2021

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate fast and stable automated tests into your project.

In this talk, we will discuss the following topics:
- How to create fast and stable UI tests
- How to avoid flaky tests
- How to cover applications which include traditional and Jetpack Compose views and screens
- How to share UI tests between local and instrumentation tests
- How DSL can speed up adding stable UI tests to the project

Transcript

1. Ef fi cient Android UI Testing @alex_zhukovich https://alexzh.com/
3. Multiple languages Light and Dark themes Accessibility options RTL and LTR support
4. Let's Run The Tests
5. ANDROID TESTS LOCAL INSTRUMENTATION
6. ANDROID TESTS LOCAL INSTRUMENTATION UI* Non-UI
7. ANDROID TESTS LOCAL INSTRUMENTATION UI* Non-UI UI Non-UI
8. ANDROID TESTS LOCAL INSTRUMENTATION UI* Non-UI UI Non-UI SHARED CODE
9. androidTest Requires device or emulator test Executes in local JVM
10. androidTest test commonTest android { ... sourceSets { androidTest { java.srcDirs += "src/commonTest/java" } test { java.srcDirs += "src/commonTest/java" } } }
11. User Interface tests Snapshot tests
12. app.apk tests.apk BUILD APKs
13. app.apk tests.apk >_ adb shell am instrument
14. app.apk tests.apk >_ adb shell am instrument
15. ACTIVITY VIEW / COMPOSABLE FRAGMENT
16. launchActivity launchFragmentInContainer ComposeContentTestRule ActivityScenario.launch( MainActivity::class.java )
17. launchActivity launchFragmentInContainer ComposeContentTestRule val activityOptions = bundleOf( Pair("param-1", 1), Pair("param-2", 2), Pair("param-3", 3), ) val scenario = ActivityScenario.launch( MainActivity::class.java, activityOptions ) scenario.moveToState(Lifecycle.State.STARTED) ActivityScenario.launch(
18. launchActivity launchFragmentInContainer ComposeContentTestRule val scenario = launchFragmentInContainer( fragmentArgs: Bundle? = null, @StyleRes themeResId: Int = 
 R.style.FragmentScenarioEmptyFragmentActivityTheme, initialState: Lifecycle.State = 
 Lifecycle.State.RESUMED, factory: FragmentFactory? = null ) scenario.moveToState(Lifecycle.State.STARTED) debugImplementation "androidx.fragment:fragment-testing:1.4.0-rc01"
19. launchActivity launchFragmentInContainer ComposeContentTestRule Activity Fragment
20. launchActivity launchFragmentInContainer ComposeContentTestRule androidx.fragment.app.testing.FragmentScenario$EmptyFragmentActivity Activity Fragment Compilation Added to AndroidManifest
21. launchActivity launchFragmentInContainer ComposeContentTestRule @get:Rule val composeTestRule = createComposeRule() composeTestRule.apply { setContent { LoginScreen() } onNodeWithTag("email") .performTextInput("test@test.com") onNodeWithTag("password") .performTextInput("password") onNodeWithTag("login") .performClick() ... } debugImplementation "androidx.compose.ui:ui-test-manifest:1:0:5"
22. launchActivity launchFragmentInContainer ComposeContentTestRule Component Activity @Composable
23. launchActivity launchFragmentInContainer ComposeContentTestRule androidx.activity.ComponentActivity Component Activity @Composable Compilation Added to AndroidManifest
25. What Is a Good Test Case?
26. Readable Stable Extendable Fast
27. Readable Stable Extendable Fast
28. onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
29. // Login screen onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) // Home screen val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
30. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .perform(click())
31. onView(withId(R.id.email)) login(EMAIL, PASSWORD) }
32. onView(withId(R.id.email)) login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } }
33. onView(withId(R.id.email)) login(EMAIL, PASSWORD) } open class BaseTestRobot { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } }
34. onView(withId(R.id.email)) login(EMAIL, PASSWORD) } open class BaseTestRobot { LoginScreenRobot().apply { func() }
35. BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = 
 enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } 
 ... } ESPRESSO
36. BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = 
 enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { val view = device.findObject(By.res(resId(viewId))) view.text = text } fun clickOnView(viewId: Int) { device.findObject(By.res(resIf(viewId))) 
 .click() } 
 ... } UI AUTOMATOR
37. open class BaseTestRobot { @get:Rule val composeTestRule = createComposeRule() fun enterText(tag: String, text: String) { composeTestRule.onNodeWithTag(tag) .performTextInput(text) } fun clickOnView(tag: String) { composeTestRule.onNodeWithTag(tag) .performClick() } } @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } BASIC OPERATIONS SCREEN ROBOTS TEST CASES COMPOSE UI TESTS class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText("email", email) fun enterPassword(email: String) = 
 enterText("password", password) ... }
38. Challenges 
 In Mobile Testing
39. Light and Dark themes
40. YES NO RTL support
41. Accessibility support Accessibility checks Hello … FONT SIZE Small, Default, Large, Largest DISPLAY SCALING Small, Default, Large, Largest
42. Testing permissions
43. Testing permissions UI interaction GrantPermissionRule ADB commands
44. Testing permissions UI interaction GrantPermissionRule ADB commands Android Test Orchestrator
45. Test Case Test Case Test Case Clean up data after test execution
46. /data/data/PACKAGE/ databases fi les shared_prefs Clean up data after test execution
47. databases fi les shared_prefs @get:Rul e val clearDatabaseRule = ClearDatabaseRule( ) @get:Rul e val clearFileRule = ClearFilesRule( ) @get:Rul e val clearPreferencesRule = ClearPreferencesRule( ) https://github.com/AdevintaSpain/Barista Clean up data after test execution
48. Server Database Pre-populating the database
49. Test Case Database DATA LAYER Query Dependencies UI interaction Pre-populating the database
50. Real data vs Fake data Prod server Dev server Mock/Fake
51. Real data vs Fake data Prod server Dev server DATA SYNCHRONIZATION We need to synchronize data between the production and dev servers. INTERACTION WITH THE SERVER We make requests to the production server. The connection can require certificates, VPN, etc. PRODUCTION BACK-END We always use the latest available production environment.
52. Real data vs Fake data Mock/Fake DATA SYNCHRONIZATION We need to synchronize predefined responses with responses from the production server. NO INTERACTION WITH THE SERVER Responses from the production server can differ from the predefined data. We can use predefined fake responses instead of calling the production server. MAKE TESTS MORE STABLE
53. End-To-End vs Functional tests
54. INTERACTION WITH THE SERVER Interaction with the production server. ENTRY POINT Similar to the entry point of an app. END-TO-END TESTS
55. FUNCTIONAL TESTS INTERACTION WITH THE SERVER Interaction with the non-production server. NAVIGATION Usually, navigation is not needed. ENTRY POINT We can start the test from a required Activity/Fragment.
56. END-TO-END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar to the entry point of an app. APP VERIFICATION Slow veri fi cation of the app. NAVIGATION Navigate to the required screen. SERVER INTERACTION Interaction with the production server. ENTRY POINT Start the required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast veri fi cation of components or screens.
57. END-TO-END TESTS FUNCTIONAL TESTS
58. Flaky Tests
59. External dependencies Framework Test case execution Device and emulator
60. External dependencies - Network connection (VPN) - Network speed - Back-end
61. Framework - Framework issues - Toast, Snackbar, etc - Custom Views
62. Device and emulator - Performance - Noti fi cations - Device memory
63. Test case execution - Simulate User actions - Incorrect state before/after a test case - Toast, Snackbar, etc
64. - Network connection (VPN) - Network speed - Back-end - Framework issues - Toast, Snackbar, etc - Custom Views - Simulate User actions - Incorrect state before/after a test case - Toast, Snackbar, etc - Performance - Noti fi cations - Device memory External dependencies Framework Test case execution Device and emulator
65. Best Practices
66. PAY ATTENTION TO NAMING
67. NO "SLEEP" IN TESTS
68. ALL TEST CASES SHOULD BE INDEPENDENT
69. USE MULTIPLE SMALL TESTS INSTEAD OF ONE BIG TEST
70. DO NOT SPEND TIME NAVIGATING TO THE REQUIRED SCREEN
71. FOLLOW THE 
 "NO FLAKY TESTS" POLICY
72. DO NOT RELY ONLY ON 
 UI TEST AUTOMATION
73. LEARN YOUR TOOLS
74. @alex_zhukovich https://alexzh.com/ THANK YOU FOR LISTENING!