Testing, how hard can it be? (Droidcon Lisbon 2022)

When people start looking into the testing domain, very similar questions arise:
What to test? And more important, what not? What should I mock? What should I test with unit tests and what with Instrumentation? Do I need Robolectric? When? Do we need an in-memory database?
Let’s look at some examples.

1. 1 We lead what's next in music Testing how hard can it be? Danny Preussler, SoundCloud ¯\_(ツ)_/¯
2. 2 We lead what's next in music
3. 3 We lead what's next in music
4. 4 We lead what's next in music
5. 5 We lead what's next in music
6. 6 We lead what's next in music
7. “Code without tests is bad code.” (Michael C. Feathers)
8. “any code without test is a legacy code.” (Michael C. Feathers)
9. “how do you know something works when you don’t have test for it?” (Robert ‘Uncle Bob’ Martin)
10. “Refactoring without good test coverage is changing shit” (Martin Fowler)
11. Android testing strategy Theory Unit tests Manual tests UI tests
12. Android testing strategy Reality Unit tests Manual tests UI tests Unit tests ¯\_(ツ)_/¯
13. Today’s task - Write the Login feature for a Wear OS client - Authentication via phone OAuth
14. @Composable fun LoginScreen() { val viewModel = hiltViewModel<LoginViewModel>() Box(modifier = Modifier.fillMaxSize()) { Button(onClick = { viewModel.login() }) { Text(viewModel.loginStatus.name) } } }
15. @HiltViewModel class LoginViewModel @Inject constructor( private val operations: AuthOperations ) : ViewModel() { enum class LoginStatus { NOT_LOGGED_IN, LOGGED_IN, LOGGING_IN } var loginStatus: LoginStatus by mutableStateOf(NOT_LOGGED_IN)
16. A first test
17. @Test fun `initially its not logged in`() { tested.loginStatus `should be` NOT_LOGGED_IN }
18. fun login(scope: CoroutineScope = viewModelScope) { loginStatus = LOGGING_IN scope.launch { try { operations.login() loginStatus = LOGGED_IN } catch(e: Exception) { e.printStackTrace() loginStatus = NOT_LOGGED_IN } } }
19. Easy to test
20. @Nested inner class `when logging in` { @Test fun `should set loading status`() = runTest { tested.login(this) tested.loginStatus `should be` LOGGING_IN }
21. @Nested inner class `when logging in successful` { @Test fun `should set status to success`{ runTest { tested.login(this) } tested.loginStatus `should be` LOGGED_IN }
22. @Nested inner class `when logging in successful` { init { runTest { tested.login(this) } } @Test fun `should set status to success`{ tested.loginStatus `should be` LOGGED_IN }
23. authentication
24. class AuthOperations @Inject constructor( private val authClient: AuthClient, private val apiClient: ApiClient ) { suspend fun login(): AuthToken { val code = authClient.authenticate() return apiClient.registerCodeForToken(code) } }
25. val authClient = mock<AuthClient> { onBlocking { authenticate(any()) } doReturn OAuthCode("123") }
26. scope.launch { try { val token = operations.login() settings.authToken = token loginStatus = LOGGED_IN } catch(e: Exception) { e.printStackTrace() loginStatus = NOT_LOGGED_IN } }
27. @Nested inner class `when logging in successful` { init { runTest { tested.login(this) } } @Test fun `should authenticate`() { verifyBlocking(authClient) { authenticate(any()) } } }
28. Lets write the ApiClient
29. interface ApiService { @FormUrlEncoded @POST("/oauth2/token") suspend fun getToken( @Field("code") code: String, @Field("grant_type") grantType: String, ): TokenResponse }
30. class ApiClient @Inject constructor( private val apiService: ApiService ) { suspend fun registerCodeForToken(code: OAuthCode): AuthToken { return apiService.getToken( grantType = "authorization_code", code = code.value ).toToken() } }
31. val authClient = mock<AuthClient> { onBlocking { authenticate(any()) } doReturn OAuthCode("123") }
32. val authClient = mock<AuthClient> { onBlocking { authenticate(any()) } doReturn OAuthCode("123") } val service = mock<ApiService>{ onBlocking { getToken(any()) } doReturn TokenResponse("456") } val tested = LoginViewModel( AuthOperations(authClient, ApiClient(service)) ) )
33. val authClient = mock<AuthClient> { onBlocking { authenticate(any()) } doReturn OAuthCode("123") } val service = mock<ApiService>{ onBlocking { getToken(any()) } doReturn TokenResponse("456") } val tested = LoginViewModel( AuthOperations(authClient, ApiClient(service)) ) )
34. @Test fun `should authenticate`() { verifyBlocking(service) { token( code = "123", grantType = "authorization_code" ) )
35. @Nested inner class `when logging in successful` { @Test fun `should authenticate`() { runTest { tested.login(this) } verifyBlocking(authClient) { authenticate(any()) } } }
36. val authClient = mock<AuthClient> { onBlocking { authenticate(any()) } doReturn OAuthCode("123") } val service = mock<ApiService>{ onBlocking { getToken(any()) } doReturn TokenResponse("456") } val tested = LoginViewModel( AuthOperations(authClient, ApiClient(service)) ) )
37. val authClient = object : OAuthClient { override suspend fun authenticate(clientId: ClientId) = OAuthCode("123") } val service = mock<ApiService>{ onBlocking { getToken(any()) } doReturn TokenResponse("456") } val tested = LoginViewModel( AuthOperations(authClient, ApiClient(service)) ) )
38. Use “the real thing” where possible
39. What about the token?
40. scope.launch { try { operations.login() loginStatus = LOGGED_IN } catch(e: Exception) { e.printStackTrace() loginStatus = NOT_LOGGED_IN } }
41. interface Settings { var authToken: AuthToken? }
42. private class InMemorySettings: Settings { override var authToken: AuthToken? = null } val tested = LoginViewModel( InMemorySettings(), AuthOperations(authClient, ApiClient(service)) )
43. @Nested inner class `when logging in successful` { init { runTest { tested.login(this) } } @Test fun `should save value`{ settings.authToken `should be equal to` AuthToken("456") }
44. I don’t like this
45. class PersistingAuthOperations @Inject constructor( private val settings: Settings, private val authOperation: AuthOperations ){ suspend fun login(): AuthToken { return authOperation.login().also { settings.authToken = it } } }
46. val tested = LoginViewModel( PersistingAuthOperations( settings, AuthOperations(authClient, ApiClient(service)) ) ) ¯\_(ツ)_/¯
47. Refactoring should not lead to test changes
48. Let’s save the token
49. interface TokenStorage { var authToken: AuthToken? } interface Settings : TokenStorage
50. class SharedPreferenceTokenStorage( context: Application, val authPrefs: Lazy<SharedPreferences> = encrypted(context) ) : TokenStorage { override var authToken: AuthToken? set(value) { authPrefs.value.edit { putString(KEY_TOKEN, value?.token) } }
51. @RestrictTo(RestrictTo.Scope.TESTS) class SharedPreferencesStub : SharedPreferences { private val editor: EditorStub = EditorStub(this) // .. } private class EditorStub( val parent: SharedPreferences, val items: HashMap<String?, Any?> = hashMapOf()) : Editor
52. Prefer Stubs over mocks, reuse them!
53. The auth client
54. internal class WearOAuthClient( private val context: Application, private val authClient: RemoteAuthClient = RemoteAuthClient.create(context) ) : OAuthClient {
55. override suspend fun authenticate(): OAuthCode = suspendCoroutine { continuation -> authClient.sendAuthorizationRequest( // .. object : RemoteAuthClient.Callback() { override fun onAuthorizationResponse(response: OAuthResponse) { continuation.resumeWith(success(OAuthCode(response.code))) } override fun onAuthorizationError(errorCode: Int) { continuation.resumeWith( failure(IOException("failed: $errorCode"))) } } }
56. override suspend fun authenticate(): OAuthCode = suspendCoroutine { continuation -> authClient.sendAuthorizationRequest( // .. object : RemoteAuthClient.Callback() { override fun onAuthorizationResponse(response: OAuthResponse) { continuation.resumeWith(success(OAuthCode(response.code))) } override fun onAuthorizationError(errorCode: Int) { continuation.resumeWith( failure(IOException("failed: $errorCode"))) } } }
57. @RunWith(RobolectricTestRunner::class) @RobolectricNeededAsOf(RemoteAuthClient::class) class WearOAuthClientTest { val code = OAuthCode("789") val redirect = "http://test?code=$code" val successRequest = mock<RemoteAuthClient> { on { sendAuthorizationRequest(any(), any(), any()) } doAnswer { val request = it.arguments[0] as OAuthRequest val response = Builder().setResponseUrl(redirect).build() (it.arguments[2] as Callback).onAuthorizationResponse(response) } } o_o
58. @Test fun `should extract the code when successful`() = runTest { tested = WearOAuthClient( application = RuntimeEnvironment.getApplication(), authClient = successRequest ) tested.authenticate() `should be equal to` code }
59. For external dependencies Use Mocks and Robolectric
60. Back to api
61. Refactorings: • AuthOperations -> AuthRepository • ApiClient used in new ApiDataSource No test change ¯\_(ツ)_/¯
62. internal class ApiAuthDataSource @Inject constructor( val api: Lazy<AuthService> ) : AuthDataSource {
63. val service = JsonReadingAuthService( "/api/public_api_token.json", AppModule.providesMoshi() ) val tested = ApiAuthDataSource { service }
64. @Test fun `should map token`() = runTest { response = tested.registerCodeForToken(Code(123)) response.token `should be equal to` "TH1S-1S-JUST-A-T3ST-T0KEN" }
65. Test the JSON from your backend
66. Let’s test the UI
67. class LoginViewModelStub : LoginViewModel() { override fun login() { loginStatus = LOGGED_IN } }
68. abstract class LoginViewModel : ViewModel() { enum class LoginStatus { LOGGED_IN, LOGGING_IN, NOT_LOGGED_IN } var loginStatus: LoginStatus by mutableStateOf(NOT_LOGGED_IN) abstract fun login() // .. viewModel: LoginViewModel = hiltViewModel<HiltLoginViewModel>()
69. @RunWith(RobolectricTestRunner::class) @Config(instrumentedPackages = ["androidx.loader.content"]) class LoginScreenTest { @get:Rule val composeTestRule = createComposeRule() val viewModel = LoginViewModelStub()
70. @Test fun `should login and update status`() = runComposeTest { setContent { ScTheme { LoginScreen(viewModel) } }
71. onNodeWithText(LOGGED_IN.name).assertDoesNotExist() onNodeWithText(NOT_LOGGED_IN.name).performClick() onNodeWithText(LOGGED_IN.name).assertExists()
72. Prefer Robolectric over Device for UI tests
73. What did we write? LoginScreen LoginViewModel LoginScreenTest LoginViewModelTest LoginRepository ApiDataSourceTest ApiDataSource WearOAuthClientTest SharedPreferenceTokenStorageTest
74. What did we write? LoginScreen LoginViewModel LoginScreenTest LoginViewModelTest LoginRepository WearOAuthClientTest SharedPreferenceTokenStorageTest ApiDataSource
75. This is not a unit test! (╯°□°)╯︵ ┻━┻
76. This is not a unit test!
77. The way most developers write their tests Test1 Test2 Class1 Class2 Class3 Test3
78. The way some developers write their tests Test1 Class1 Class2 Class3
79. The way TDD developers write their tests Test1 Class1
80. The way TDD developers write their tests Test1 Class1 Class2 Class3
81. Solid tests: A true story Test1 ViewModel Repository API
82. Solid tests: A true story Test1 ViewModel Repository API Store Reducer SideEffects
83. WHAT your code does is stable HOW your code does it is unstable
84. Solid tests: A true story Test1 ViewModel Repository API Store Reducer SideEffects
85. Tests should be coupled to the behavior of code and decoupled from the structure of code (Kent Beck)
86. True facts Fast and stable Flaky and slow
87. Can’t avoid flakyness at top Fast and stable Flaky and slow Your decision
88. Android testing strategy The real deal Junit (5) End to end Mocked endpoints Robolectric
89. The good pyramid • Prefer screen tests over end to end tests • Prefer mocked endpoint over api calls • Prefer mocks/stubs over mock web servers • Prefer Robolectric over Device tests • Prefer JVM tests over Robolectric • Do you really need that in memory database? Performance and flakyness
90. "Test only if you would want it to work.” Kent Beck
91. If it's worth building, it's worth testing If it's not worth testing, why are you wasting your time working on it?
92. If it's worth building, it's worth testing If it's not worth testing, why are you wasting your time working on it?
93. There is some ugliness in the test
94. val code = OAuthCode("789")
95. private var counter = 1 object AuthTokenFixture { operator fun invoke(token: String = "${counter++}"): AuthToken { return AuthToken(token) } } object OAuthCodeFixture { operator fun invoke(code: String = "${counter++}"): OAuthCode { return OAuthCode(code) } }
96. val code = OAuthCodeFixture()
97. /module /src /main /testFixtures /<package> Fixtures.java
98. Another Tip Avoiding Dispatchers.setMain
99. class CoroutineTaskExecutor : BeforeEachCallback, AfterEachCallback { override fun beforeEach(context: ExtensionContext?) { Dispatchers.setMain(TestCoroutineDispatcher()) } override fun afterEach(context: ExtensionContext?) { Dispatchers.resetMain() } }
100. @HiltViewModel internal class HiltLoginViewModel( private val authRepository: PersistingAuthRepository, private val scope: ViewModel.() -> CoroutineScope ) : LoginViewModel() { @Inject constructor( authRepository: PersistingAuthRepository ) : this(settings, authRepository, { viewModelScope })
101. 101 We lead what's next in music Thank You @PreusslerBerlin