Put Your Tests on a Diet: Testing the Behavior and Not the Implementation (droidcon London 2023)

How do you write tests? How much time do you spend writing tests? And how much time do you spend fixing them when refactoring?

A few short years ago, we would test a class in JUnit by stacking one test after the other, mocking each one of its dependencies. But for large apps, writing tests in this way without any consideration of architecture can easily become a bottleneck.

For a task that takes so much of our time every day as developers, there is surprisingly little discussion online about optimal test design. At Perry Street Software, publisher of two of the world's most popular LGBTQ+ dating apps, after struggling writing and maintaining our unit tests in the past, we have spent much time developing what we like to call the Unit Testing Diet.

Our Unit Testing Diet provides a healthy way to test and refactor an app at scale, by testing the behavior that a user performs and the outcome that they will perceive, without testing implementation details.

Join this talk to learn how to:

1. Test your ViewModels, UseCases, and Repositories in a Given-When-Then style, without using mocks
2. Test the behavior and not the implementation details
3. Refactor your code without breaking your tests

Video: https://www.droidcon.com/2023/11/15/put-your-tests-on-a-diettesting-the-behavior-and-not-the-implementation/

Transcript

1. Put Your Tests on a Diet Testing the Behavior and Not the Implementation Stelios Frantzeskakis - Perry Street Software droidcon London 2023
2. Do we still believe in tests?
3. We’ve been doing it wrong
4. Hi, I’m Stelios Frantzeskakis Staff Engineer at Perry Street Software, publisher of SCRUFF & Jack’d, serving more than 30M members Working with Android since the release of Android Gingerbread Passionate about Testing & Architecture @SteliosFran
5. Modern app architecture View ViewModel UseCase Repository Remote Data Source Local Data Source
6. Sample chat application
7. class ChatViewModel( private val getChatMessagesUseCase: GetChatMessagesUseCase, private val sendTextMessageUseCase: SendTextMessageUseCase, private val user: User, private val coroutineDispatcherProvider: CoroutineDispatcherProvider ) : ViewModel() { data class State( val messages: List<MessageUiModel> ) private val _state = MutableStateFlow(State(emptyList())) val state: StateFlow<State> = _state.asStateFlow() init { viewModelScope.launch(coroutineDispatcherProvider.main) { getChatMessagesUseCase(user).map { State(MessageUiModelMapper.fromMessagesList(it)) }.collect { newState -> _state.value = newState } } } fun onTextMessageSent(text: String) = viewModelScope.launch( coroutineDispatcherProvider.main ) { sendTextMessageUseCase(text, user) } } View ViewModel UseCase Repository DataSource
8. class ChatViewModel( private val getChatMessagesUseCase: GetChatMessagesUseCase, private val sendTextMessageUseCase: SendTextMessageUseCase, private val user: User, private val coroutineDispatcherProvider: CoroutineDispatcherProvider ) : ViewModel() { data class State( val messages: List<MessageUiModel> ) private val _state = MutableStateFlow(State(emptyList())) val state: StateFlow<State> = _state.asStateFlow() init { viewModelScope.launch(coroutineDispatcherProvider.main) { getChatMessagesUseCase(user).map { State(MessageUiModelMapper.fromMessagesList(it)) }.collect { newState -> _state.value = newState } } } fun onTextMessageSent(text: String) = viewModelScope.launch( coroutineDispatcherProvider.main ) { sendTextMessageUseCase(text, user) } } View ViewModel UseCase Repository DataSource
9. class ChatViewModel( private val getChatMessagesUseCase: GetChatMessagesUseCase, private val sendTextMessageUseCase: SendTextMessageUseCase, private val user: User, private val coroutineDispatcherProvider: CoroutineDispatcherProvider ) : ViewModel() { data class State( val messages: List<MessageUiModel> ) private val _state = MutableStateFlow(State(emptyList())) val state: StateFlow<State> = _state.asStateFlow() init { viewModelScope.launch(coroutineDispatcherProvider.main) { getChatMessagesUseCase(user).map { State(MessageUiModelMapper.fromMessagesList(it)) }.collect { newState -> _state.value = newState } } } fun onTextMessageSent(text: String) = viewModelScope.launch( coroutineDispatcherProvider.main ) { sendTextMessageUseCase(text, user) } } View ViewModel UseCase Repository DataSource
10. View class GetChatMessagesUseCase( private val chatMessagesRepository: ChatMessagesRepository ) { operator fun invoke(user: User): Flow<List<Message>> { return chatMessagesRepository.getChatMessages(user).map { messagesList -> messagesList.sortedBy { it.date.time } } } } class SendTextMessageUseCase( private val chatMessagesRepository: ChatMessagesRepository ) { suspend operator fun invoke(text: String, user: User) { if (isValid(text)) { val message = MessageFactory.fromText(text) chatMessagesRepository.sendChatMessage(message, user) } else { throw InvalidMessageException("Message is too long") } } private fun isValid(text: String): Boolean { return text.length < 180 } } ViewModel UseCase Repository DataSource
11. class GetChatMessagesUseCase( private val chatMessagesRepository: ChatMessagesRepository ) { operator fun invoke(user: User): Flow<List<Message>> { return chatMessagesRepository.getChatMessages(user).map { messagesList -> messagesList.sortedBy { it.date.time } } } } class SendTextMessageUseCase( private val chatMessagesRepository: ChatMessagesRepository ) { suspend operator fun invoke(text: String, user: User) { if (isValid(text)) { val message = MessageFactory.fromText(text) chatMessagesRepository.sendChatMessage(message, user) } else { throw InvalidMessageException("Message is too long") } } private fun isValid(text: String): Boolean { return text.length < 180 } } View ViewModel UseCase Repository DataSource
12. class ChatMessagesRepository( private val chatDataSource: ChatDataSource ) { private val _cachedChatMessages = MutableStateFlow<List<Message>>(emptyList()) fun getChatMessages(user: User): Flow<List<Message>> = flow { if (_cachedChatMessages.value.isEmpty()) { _cachedChatMessages.emit(chatDataSource.getChatMessages(user)) } emitAll(_cachedChatMessages) } suspend fun sendChatMessage(message: Message, user: User) { _cachedChatMessages.apply { value = value + message } chatDataSource.sendChatMessage(message, user) } } View ViewModel UseCase Repository DataSource
13. class ChatMessagesRepository( private val chatDataSource: ChatDataSource ) { private val _cachedChatMessages = MutableStateFlow<List<Message>>(emptyList()) fun getChatMessages(user: User): Flow<List<Message>> = flow { if (_cachedChatMessages.value.isEmpty()) { _cachedChatMessages.emit(chatDataSource.getChatMessages(user)) } emitAll(_cachedChatMessages) } suspend fun sendChatMessage(message: Message, user: User) { _cachedChatMessages.apply { value = value + message } chatDataSource.sendChatMessage(message, user) } } View ViewModel UseCase Repository DataSource
14. interface ChatDataSource { suspend fun getChatMessages(user: User): List<Message> suspend fun sendChatMessage(message: Message, user: User) } View ViewModel UseCase Repository DataSource
15. How do you write unit tests for this architecture?
16. Just mock every layer
17. class ChatViewModelTest { private val getChatMessagesUseCase: GetChatMessagesUseCase = mockk() private val sendTextMessageUseCase: SendTextMessageUseCase = mockk() private val user = User(id = "test_id", name = "test name") private val firstMessage = Message("id1", "1st message", Date().minusDays(1), Message.Type.Received) private val secondMessage = Message("id2", "2nd message", Date(), Message.Type.Received) private val messagesList = listOf(firstMessage, secondMessage) private val viewModel by lazy { ChatViewModel( getChatMessagesUseCase, sendTextMessageUseCase, user, TestCoroutineDispatcherProvider() ) } // .. }
18. class ChatViewModelTest { // ... @Test fun `it emits the state`() { every { getChatMessagesUseCase(user) } returns flowOf(messagesList) viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( text = "1st message", type = Message.Type.Received ), MessageUiModel( text = "2nd message", type = Message.Type.Received ) ) ) } @Test fun `it sends a text message`() { viewModel.onTextMessageSent("text message") coVerify(exactly = 1) { sendTextMessageUseCase(text = "text message", user = user) } } }
19. class ChatViewModelTest { // ... @Test fun `it emits the state`() { every { getChatMessagesUseCase(user) } returns flowOf(messagesList) viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( text = "1st message", type = Message.Type.Received ), MessageUiModel( text = "2nd message", type = Message.Type.Received ) ) ) } @Test fun `it sends a text message`() { viewModel.onTextMessageSent("text message") coVerify(exactly = 1) { sendTextMessageUseCase(text = "text message", user = user) } } }
20. class GetChatMessagesUseCaseTest { private val chatMessagesRepository: ChatMessagesRepository = mockk() private val user = User(id = "test_id", name = "test name") private val firstMessage = Message("id1", "1st message", Date().minusDays(1), Message.Type.Received) private val secondMessage = Message("id2", "2nd message", Date(), Message.Type.Received) private val messagesList = listOf(secondMessage, firstMessage) private val getChatMessagesUseCase = GetChatMessagesUseCase(chatMessagesRepository) @Test fun `it emits the list of chat messages`() { every { chatMessagesRepository.getChatMessages(user) } returns flowOf(messagesList) runTest { getChatMessagesUseCase(user).first() shouldBeEqual listOf( firstMessage, secondMessage ) } } }
21. class GetChatMessagesUseCaseTest { private val chatMessagesRepository: ChatMessagesRepository = mockk() private val user = User(id = "test_id", name = "test name") private val firstMessage = Message("id1", "1st message", Date().minusDays(1), Message.Type.Received) private val secondMessage = Message("id2", "2nd message", Date(), Message.Type.Received) private val messagesList = listOf(secondMessage, firstMessage) private val getChatMessagesUseCase = GetChatMessagesUseCase(chatMessagesRepository) @Test fun `it emits the list of chat messages`() { every { chatMessagesRepository.getChatMessages(user) } returns flowOf(messagesList) runTest { getChatMessagesUseCase(user).first() shouldBeEqual listOf( firstMessage, secondMessage ) } } }
22. class SendTextMessageUseCaseTest { private val chatMessagesRepository: ChatMessagesRepository = mockk() private val user = User(id = "test_id", name = "test name") private val sendTextMessageUseCase = SendTextMessageUseCase(chatMessagesRepository) @Test fun `it sends a valid text message`() { val validTextMessage = "text message" val message = MessageFactory.fromText(validTextMessage) coEvery { chatMessagesRepository.sendChatMessage(message, user) } just Runs runTest { sendTextMessageUseCase(validTextMessage, user) coVerify(exactly = 1) { chatMessagesRepository.sendChatMessage(message, user) } } } @Test fun `it does not send an invalid text message`() { val invalidTextMessage = (1..180).joinToString("") { "a" } coEvery { chatMessagesRepository.sendChatMessage(any(), user) } just Runs runTest { sendTextMessageUseCase(invalidTextMessage, user) coVerify(exactly = 0) { chatMessagesRepository.sendChatMessage(any(), user) } } } }
23. class SendTextMessageUseCaseTest { private val chatMessagesRepository: ChatMessagesRepository = mockk() private val user = User(id = "test_id", name = "test name") private val sendTextMessageUseCase = SendTextMessageUseCase(chatMessagesRepository) @Test fun `it sends a valid text message`() { val validTextMessage = "text message" val message = MessageFactory.fromText(validTextMessage) coEvery { chatMessagesRepository.sendChatMessage(message, user) } just Runs runTest { sendTextMessageUseCase(validTextMessage, user) coVerify(exactly = 1) { chatMessagesRepository.sendChatMessage(message, user) } } } @Test fun `it does not send an invalid text message`() { val invalidTextMessage = (1..180).joinToString("") { "a" } coEvery { chatMessagesRepository.sendChatMessage(any(), user) } just Runs runTest { sendTextMessageUseCase(invalidTextMessage, user) coVerify(exactly = 0) { chatMessagesRepository.sendChatMessage(any(), user) } } } }
24. class SendTextMessageUseCaseTest { private val chatMessagesRepository: ChatMessagesRepository = mockk() private val user = User(id = "test_id", name = "test name") private val sendTextMessageUseCase = SendTextMessageUseCase(chatMessagesRepository) @Test fun `it sends a valid text message`() { val validTextMessage = "text message" val message = MessageFactory.fromText(validTextMessage) coEvery { chatMessagesRepository.sendChatMessage(message, user) } just Runs runTest { sendTextMessageUseCase(validTextMessage, user) coVerify(exactly = 1) { chatMessagesRepository.sendChatMessage(message, user) } } } @Test fun `it does not send an invalid text message`() { val invalidTextMessage = (1..180).joinToString("") { "a" } coEvery { chatMessagesRepository.sendChatMessage(any(), user) } just Runs runTest { sendTextMessageUseCase(invalidTextMessage, user) coVerify(exactly = 0) { chatMessagesRepository.sendChatMessage(any(), user) } } } } object MessageFactory { fun fromText(text: String): Message { return Message( id = UUID.randomUUID().toString(), text = text, date = Date(), type = Message.Type.Pending ) } }
25. class SendTextMessageUseCaseTest { private val chatMessagesRepository: ChatMessagesRepository = mockk() private val user = User(id = "test_id", name = "test name") private val sendTextMessageUseCase = SendTextMessageUseCase(chatMessagesRepository) @Test fun `it sends a valid text message`() { val validTextMessage = "text message" // val message = MessageFactory.fromText(validTextMessage) coEvery { chatMessagesRepository.sendChatMessage(any(), user) } just Runs runTest { sendTextMessageUseCase(validTextMessage, user) coVerify(exactly = 1) { chatMessagesRepository.sendChatMessage(any(), user) } } } @Test fun `it does not send an invalid text message`() { val invalidTextMessage = (1..180).joinToString("") { "a" } coEvery { chatMessagesRepository.sendChatMessage(any(), user) } just Runs runTest { sendTextMessageUseCase(invalidTextMessage, user) coVerify(exactly = 0) { chatMessagesRepository.sendChatMessage(any(), user) } } } }
26. class SendTextMessageUseCaseTest { private val chatMessagesRepository: ChatMessagesRepository = mockk() private val user = User(id = "test_id", name = "test name") private val sendTextMessageUseCase = SendTextMessageUseCase(chatMessagesRepository) @Test fun `it sends a valid text message`() { val validTextMessage = "text message" // val message = MessageFactory.fromText(validTextMessage) coEvery { chatMessagesRepository.sendChatMessage(any(), user) } just Runs runTest { sendTextMessageUseCase(validTextMessage, user) coVerify(exactly = 1) { chatMessagesRepository.sendChatMessage(any(), user) } } } @Test fun `it does not send an invalid text message`() { val invalidTextMessage = (1..180).joinToString("") { "a" } coEvery { chatMessagesRepository.sendChatMessage(any(), user) } just Runs runTest { sendTextMessageUseCase(invalidTextMessage, user) coVerify(exactly = 0) { chatMessagesRepository.sendChatMessage(any(), user) } } } }
27. class ChatMessagesRepositoryTest { private val chatDataSource: ChatDataSource = mockk() private val user = User(id = "test_id", name = "test name") private val firstMessage = Message("id1", "1st message", Date().minusDays(1), Message.Type.Received) private val secondMessage = Message("id2", "2nd message", Date(), Message.Type.Received) private val messagesList = listOf(firstMessage, secondMessage) private val chatMessagesRepository = ChatMessagesRepository(chatDataSource) // ... }
28. class ChatMessagesRepositoryTest { // ... @Test fun `it emits the list of chat messages`() { coEvery { chatDataSource.getChatMessages(user) } returns messagesList runTest { chatMessagesRepository.getChatMessages(user).first() shouldBeEqual messagesList } } @Test fun `it sends a chat message`() { val message = MessageFactory.fromText("text message") coEvery { chatDataSource.sendChatMessage(message, user) } just Runs runTest { chatMessagesRepository.sendChatMessage(message, user) coVerify(exactly = 1) { chatDataSource.sendChatMessage(message, user) } } } // ... }
29. class ChatMessagesRepositoryTest { // ... @Test fun `it emits the list of chat messages`() { coEvery { chatDataSource.getChatMessages(user) } returns messagesList runTest { chatMessagesRepository.getChatMessages(user).first() shouldBeEqual messagesList } } @Test fun `it sends a chat message`() { val message = MessageFactory.fromText("text message") coEvery { chatDataSource.sendChatMessage(message, user) } just Runs runTest { chatMessagesRepository.sendChatMessage(message, user) coVerify(exactly = 1) { chatDataSource.sendChatMessage(message, user) } } } // ... }
30. class ChatMessagesRepositoryTest { // ... @Test fun `it updates the list of chat messages after sending a message`() { val message = MessageFactory.fromText("text message") coEvery { chatDataSource.getChatMessages(user) } returns messagesList coEvery { chatDataSource.sendChatMessage(message, user) } just Runs runTest { chatMessagesRepository.getChatMessages(user).first() shouldBeEqual listOf( firstMessage, secondMessage ) chatMessagesRepository.sendChatMessage(message, user) chatMessagesRepository.getChatMessages(user).first() shouldBeEqual listOf( firstMessage, secondMessage, message ) } } }
31. Problems of mocking every layer
32. Tests are tied to the implementation
33. Tests become overspecified
34. Mocks bind you to a specific dependency graph
35. Tests become the obstacle to refactoring
36. What do the experts say about mocks?
37. “Mockist tests are coupled to the implementation of a method. Coupling to the implementation interferes with refactoring, since implementation changes are much more likely to break tests.” “I don’t see any compelling bene fi ts for mockist TDD, and am concerned about the consequences of coupling tests to implementation.I really like the fact that while writing the test you focus on the result of the behavior, not how it’s done.” Martin Fowler
38. Robert C. Martin (Uncle Bob) “You have to mock out all the data pathways in the interaction; and that can be a complex task. This creates two problems. 1. The setup code can get extremely complicated. 2. The mocking structure become tightly coupled to implementation details causing many tests to break when those details are modi fi ed.” “The need to mock every class interaction forces an explosion of polymorphic interfaces whose sole purpose is to allow mocking.” “I recommend that you mock sparingly. Find a way to test — design a way to test — your code so that it doesn’t require a mock.”
39. “I mock almost nothing. If I can’t fi gure out how to test e ffi ciently with the real stuff, I fi nd another way of creating a feedback loop for myself.” “If you have mocks returning mocks, returning mocks, your test is completely coupled to the implementation, not the interface, but the exact implementation. Of course you can’t change anything without breaking the test.” Kent Beck
40. The BDD approach
41. Given - When - Then
42. Given("I have exchanged messages with a user") { fakeDataSourceResponse() Then("They are displayed") { viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( text = "1st message", type = Message.Type.Received ) ) ) } // ... }
43. Given("I have exchanged messages with a user") { fakeDataSourceResponse() Then("They are displayed") { viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( text = "1st message", type = Message.Type.Received ) ) ) } // ... }
44. Given("I have exchanged messages with a user") { // ... When("I send a text message") { var textMessage = "" beforeEach { viewModel.onTextMessageSent(textMessage) } And("It is valid") { textMessage = "2nd message" Then("It is displayed") { viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( text = "1st message", type = Message.Type.Received ), MessageUiModel( text = "2nd message", type = Message.Type.Pending ) ) ) } } // ... } }
45. Given("I have exchanged messages with a user") { // ... When("I send a text message") { var textMessage = "" beforeEach { viewModel.onTextMessageSent(textMessage) } And("It is valid") { textMessage = "2nd message" Then("It is displayed") { viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( text = "1st message", type = Message.Type.Received ), MessageUiModel( text = "2nd message", type = Message.Type.Pending ) ) ) } } // ... } }
46. Given("I have exchanged messages with a user") { // ... When("I send a text message") { var textMessage = "" beforeEach { viewModel.onTextMessageSent(textMessage) } And("It is valid") { textMessage = "2nd message" Then("It is displayed") { viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( text = "1st message", type = Message.Type.Received ), MessageUiModel( text = "2nd message", type = Message.Type.Pending ) ) ) } } // ... } }
47. Given("I have exchanged messages with a user") { // ... When("I send a text message") { var textMessage = "" beforeEach { viewModel.onTextMessageSent(textMessage) } // ... And("It is not valid") { textMessage = (1..180).joinToString("") { "a" } Then("It is not displayed") { viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( "1st message", Message.Type.Received ) ) ) } } } }
48. Given("I have exchanged messages with a user") { // ... When("I send a text message") { var textMessage = "" beforeEach { viewModel.onTextMessageSent(textMessage) } // ... And("It is not valid") { textMessage = (1..180).joinToString("") { "a" } Then("It is not displayed") { viewModel.state.value shouldBeEqual ChatViewModel.State( messages = listOf( MessageUiModel( "1st message", Message.Type.Received ) ) ) } } } }
49. Which layers did we test?
50. Is this a unit test or an integration test?
51. A unit test is a test that runs in isolation from other tests Kent Beck
52. In sociable unit tests, the tested unit can rely on other classes Martin Fowler
53. The unit is an isolated module. It’s a black box where you talk to what is exposed. Ian Cooper
54. Unit tests are not about testing a single class View ViewModel UseCase Repository Remote Data Source Local Data Source Feature / Module in BDD
55. When to mock?
56. Benefits of BDD
57. Refactoring won’t break the tests Martin Fowler “Refactoring is the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure.”
58. Tests are the documentation
59. TDD: Red - Green - Refactor
60. Less code to write and maintain
61. Additional resources Classical and Mockist Testing - Martin Fowler https://martinfowler.com/articles/mocksArentStubs.html#ClassicalAndMockistTesting On the Diverse And Fantastical Shapes of Testing - Martin Fowler https://martinfowler.com/articles/2021-test-shapes.html When to Mock - Robert C. Martin (Uncle Bob) https://blog.cleancoder.com/uncle-bob/2014/05/10/WhenToMock.html
62. Additional resources TDD, Where Did It All Go Wrong - Ian Cooper https://youtu.be/EZ05e7EMOLM Is TDD dead? - Martin Fowler, Kent Beck, DHH https://youtu.be/z9quxZsLcfo unit-testing-diet GitHub repository https://github.com/steliosfran/unit-testing-diet
63. Do not mock every layer
64. Test the behavior, not the implementation
65. “Eat food. Not too much. Mostly plants.” “Write tests. Not too much. Mostly on the ViewModel.”