Fantastic tests and where to find them

Presented on DroidCon SF 23

The more code we write the more we raise the risk of having issues on our apps. Our job as engineers is to mitigate those risks and how can we do that? With tests.

We all agree that tests are vital to build a successful app but how can we know that we are testing the right things? How do we measure the quality of our tests?

In this session we will go through these questions in a pragmatic approach, diving into every type of test in every aspect of the development phase. From communicating with backend to UI components, we will go through strategies and approaches on how to properly test each piece of code.

Transcript

1. FANTASTIC TESTS AND WHERE TO FIND THEM
2. WHO AM I? 🧔 DANIEL HOROWITZ 🧑💻 WORKING ON SPOTIFY 🇪🇸 BARCELONA / FROM 🇧🇷
3. 🗓 AGENDA 📖 FUNDAMENTALS 🎻 THE CLASSICS ♟ TESTING STRATEGY
4. TEST FUNDAMENTALS
5. UNIT TESTS ARE F.I.R.S.T - TIM OTTINGER AND JEFF LANGR https://medium.com/pragmatic-programmers/unit-tests-are- f irst-fast-isolated-repeatable-self-verifying-and-timely-a83e8070698e
6. TEST DESIDERATA - KENT BECK https://medium.com/@kentbeck_7670/test-desiderata-94150638a4b3 https://www.youtube.com/watch?v=5LOdKDqdWYU
7. TIM OTTINGER AND JEFF LANGR UNIT TESTS ARE F.I.R.S.T
8. FAST TEST SPEED WILL DETERMINE HOW OFTEN THEY RUN Photo by Tim Rüßmann on Unsplash
9. ISOLATED EACH TEST SHOULD FAIL FOR A SINGLE REASON DAVE ASTEL’S RECOMMENDATION: “ONE ASSERTION PER TEST” Photo by Jordan Steranka on Unsplash
10. REPEATABLE IF NOTHING CHANGES, SHOULD ALWAYS RETURN THE SAME RESULT Photo by Crawford Jolly on Unsplash
11. SELF-VERIFYING IT SHOULD BE CLEAR WHETHER A TEST PASSED OR FAILED Photo by Glen Carrie on Unsplash
12. TIMELY YOU SHOULD ALWAYS KNOW WHAT YOU’RE TRYING TO BUILD BEFORE YOU BUILD IT Photo by Jon Tyson on Unsplash
13. THOROUGH CODE SHOULD BE TESTED EXTENSIVELY FOR POSITIVE AND NEGATIVE CASES Photo by Ryota Nagasaka on Unsplash
14. - KENT BECK TEST DESIDERATA
15. TEST DESIDERATA - BY KENT BECK 📖 TEST FUNDAMENTALS TEST DESIDERATA 🗺 12 TEST PROPERTIES 🏷 NOT ALL PROPERTIES MUST BE PRESENT IN EVERY TEST ⚖ PROPERTIES ARE TRADEOFFS
16. WRITABLE TESTS SHOULD BE CHEAP TO WRITE RELATIVE TO THE COST OF THE CODE BEING TESTED Photo by Umberto on Unsplash
17. READABLE TESTS SHOULD BE COMPREHENSIBLE FOR READER, INVOKING THE MOTIVATION FOR WRITING THIS PARTICULAR TEST Photo by Syd Wachs on Unsplash
18. INSPIRING PASSING THE TESTS SHOULD INSPIRE CONFIDENCE Photo by orbtal media on Unsplash
19. TEST DESIDERATA - KENT BECK “TESTS SHOULD BE COUPLED TO THE BEHAVIOUR OF CODE AND DECOUPLED FROM THE STRUCTURE OF CODE”
20. DECOUPLING FROM CODE STRUCTURE EXAMPLE: CONVERT VOTE AVERAGE TO ⭐ { "id": 464052, "popularity": 4749.437, "poster_path": "/8UlWHLMpgZm9bx6QYh0NFoq67TZ.jpg", "title": "John Wick: Chapter 4", "vote_average": 7.2 }
21. DECOUPLING FROM CODE STRUCTURE EXAMPLE: CONVERT VOTE AVERAGE TO ⭐ internal fun Double.toStars(): String { val starsCount = Math.round(this / 2.0).toInt() return "⭐".repeat(starsCount) }
22. DECOUPLING FROM CODE STRUCTURE EXAMPLE: CONVERT VOTE AVERAGE TO ⭐ ✅ internal fun Double.toStars(): String { val starsCount = Math.round(this / 2.0).toInt() return "⭐".repeat(starsCount) } @Test fun `given voteAverage 6 should return 3 stars`() { val actual = 6.0.toStars() val expected = "⭐⭐⭐" assertEquals(expected, actual) }
23. DECOUPLING FROM CODE STRUCTURE EXAMPLE: CONVERT VOTE AVERAGE TO ⭐ internal fun Double.toStars(): String { val starsCount = Math.round(this / 2.0).toInt() return "⭐".repeat(starsCount) } @Test fun `given voteAverage 6 should return 3 stars`() { val actual = 6.0.toStars() val expected = "⭐⭐⭐" assertEquals(expected, actual) }
24. DECOUPLING FROM CODE STRUCTURE EXAMPLE: CONVERT VOTE AVERAGE TO ⭐ ✅ internal fun Double.toStars(): String { val starsCount = (this / 2.0).roundToInt() return "⭐".repeat(starsCount) } @Test fun `given voteAverage 6 should return 3 stars`() { val actual = 6.0.toStars() val expected = "⭐⭐⭐" assertEquals(expected, actual) }
26. THE CLASSICS
27. E2E 🎻 THE CLASSICS TEST SCOPES UNIT TESTS INTEGRATION
28. REAL DEVICES 🎻 THE CLASSICS TEST EXECUTION SCOPES LOCAL ENV EMULATORS
29. MARTIN FOWLER - MOCKS AREN’T STUBS FAKES VS MOCKS MOCKS Objects pre-programmed with expectations which form a speci f ication of the calls they are expected to receive. FAKES Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).
30. MOCKS VS FAKES - ANDROID COMMUNITY https://dev.to/jameson/fakes-mocks-on-android-well-partner-that-depends-h6n
31. 🎻 THE CLASSICS FAKES VS MOCKS Mocks ❌ Tightly coupled to code structure ❌ Could be costly to initialise ❌ Tend to be overused Fakes ✅ Focus on behaviour instead of implementation ✅ Lightweight ✅ Reusable
32. 🎻 THE CLASSICS FAKES VS MOCKS interface MoviesDataSource { suspend fun fetchNextPage() : List<Movie> }
33. 🎻 THE CLASSICS FAKES VS MOCKS interface MoviesDataSource { suspend fun fetchNextPage() : List<Movie> } class GetPopularMovies @Inject constructor( private val dataSource: MoviesDataSource ) { suspend operator fun invoke(): List<Movie> = dataSource.fetchNextPage() }
34. 🎻 THE CLASSICS FAKES VS MOCKS interface MoviesDataSource { suspend fun fetchNextPage() : List<Movie> } class GetPopularMovies @Inject constructor( private val dataSource: MoviesDataSource ) { suspend operator fun invoke(): List<Movie> = dataSource.fetchNextPage() } @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) }
35. 🎻 THE CLASSICS FAKES VS MOCKS @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) } @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) } @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) } @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) } @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) } @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) }
36. 🎻 THE CLASSICS FAKES VS MOCKS :movie
37. 🎻 THE CLASSICS FAKES VS MOCKS :movie MovieDataSource Movie Model …
38. 🎻 THE CLASSICS FAKES VS MOCKS :movie :featureA :featureB :featureC :featureD UseCase UseCase UseCase UseCase UseCase UseCase UseCase UseCase
39. 🎻 THE CLASSICS FAKES VS MOCKS :movie :featureA :featureB :featureC :featureD UseCase UseCase UseCase UseCase UseCase UseCase UseCase UseCase
40. 🎻 THE CLASSICS FAKES VS MOCKS :movie :featureA :featureB :featureC :featureD UseCase UseCase UseCase UseCase UseCase UseCase UseCase UseCase :movie_test
41. 🎻 THE CLASSICS FAKES VS MOCKS :movie_test FakeMovieDataSource … …
42. 🎻 THE CLASSICS FAKES VS MOCKS @Test fun `given successful network response should get popular movies correctly`() = runTest { val mockDataSource: MoviesDataSource = mock() whenever(mockDataSource.fetchNextPage()).thenReturn(listOf(movie())) val getPopularMovies = GetPopularMovies(mockDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) } @Test fun `given successful network response should get popular movies correctly`() = runTest { val fakeDataSource = FakeDataSource(listOf(movie())) val getPopularMovies = GetPopularMovies(fakeDataSource) val actual = getPopularMovies() val expected = listOf(movie()) assertEquals(expected, actual) }
43. 🎻 THE CLASSICS TEST CONVENTIONS ✍ MEANINGFUL NAMING ⚛ ONE ASSERTION PER TEST 🏗 GIVEN, WHEN, THEN STRUCTURE
44. ✍ MEANINGFUL NAMING TEST CONVENTIONS @Test fun `given 200 response with correct field types should return domain model correctly`() @Test fun `should fetch data`() ✅ ❌
45. 🏗 GIVEN, WHEN, THEN STRUCTURE TEST CONVENTIONS @Test fun `given a list of movies when vm starts should load content`() = runTest { sut.state.test(this).use { observer -> givenMovies(listOf(movie())) sut.handle(Load) val expected = listOf(Loading, Content(listOf(discoverViewEntity()))) assertEquals(observer.values, expected) } }
46. TEST CONVENTIONS GIVEN WHEN THEN @Test fun `given a list of movies when vm starts should load content`() = runTest { sut.state.test(this).use { observer -> givenMovies(listOf(movie())) sut.handle(Load) val expected = listOf(Loading, Content(listOf(discoverViewEntity()))) assertEquals(observer.values, expected) } } 🏗 STRUCTURE
47. 🎻 THE CLASSICS TEST COVERAGE TEST COVERAGE BY MARTIN FOWLER https://martinfowler.com/bliki/TestCoverage.html
48. TESTING STRATEGY
49. DATA PRESENTATION DOMAIN
50. PRESENTATION DOMAIN Business Logic M a p p e r Use case Use case Use case View Model Composable Composable Composable UI Elements Screen Controller DATA Data Source Server M a p p e r Repository Network data source Retro f it
51. ♟TESTING STRATEGY - FULLY TEST NETWORK INTEGRATION - FOCUS ON ISOLATION - PROPAGATE ERRORS CORRECTLY DATA LAYER DATA Data Source Server M a p p e r Repository Network data source Retro f it
52. ♟TESTING STRATEGY DATA LAYER class MovieDetailsNetworkDataSource @Inject constructor(private val api: MovieApi) : MovieDetailsDataSource { override suspend fun fetchMovieDetails(id: String) = api.fetchMovieDetails(id).toMovieDetails() } @GET("movie/{movieId}") suspend fun fetchMovieDetails(@Path("movieId") movieId: String): MovieDetailsDTO
53. ♟TESTING STRATEGY DATA LAYER @Test fun `given 200 response with correct field types should return domain model correctly`() = runTest { val api: MovieApi = mock() whenever(api.fetchMovieDetails("id")).thenReturn(movieDetailsDto()) val sut = MovieDetailsNetworkDataSource(api) val actual = sut.fetchMovieDetails("id") val expected = movieDetails( id = "436969", homepage = "https://www.thesuicidesquad.net", title = "The Suicide Squad", voteAverage = 8.1, runtime = 132, imdbUrl = "https://www.imdb.com/title/tt6334354" ) assertEquals(expected, actual) } class MovieDetailsNetworkDataSource @Inject constructor(private val api: MovieApi) : MovieDetailsDataSource { override suspend fun fetchMovieDetails(id: String) = api.fetchMovieDetails(id).toMovieDetails() } @GET("movie/{movieId}") suspend fun fetchMovieDetails(@Path("movieId") movieId: String): MovieDetailsDTO
54. ♟TESTING STRATEGY DATA LAYER @Test fun `given 200 response with correct field types should return domain model correctly`() = runTest { val api: MovieApi = mock() whenever(api.fetchMovieDetails("id")).thenReturn(movieDetailsDto()) val sut = MovieDetailsNetworkDataSource(api) val actual = sut.fetchMovieDetails("id") val expected = movieDetails( id = "436969", homepage = "https://www.thesuicidesquad.net", title = "The Suicide Squad", voteAverage = 8.1, runtime = 132, imdbUrl = "https://www.imdb.com/title/tt6334354" ) assertEquals(expected, actual) } class MovieDetailsNetworkDataSource @Inject constructor(private val api: MovieApi) : MovieDetailsDataSource { override suspend fun fetchMovieDetails(id: String) = api.fetchMovieDetails(id).toMovieDetails() } @GET("movie/{movieId}") suspend fun fetchMovieDetails(@Path("movieId") movieId: String): MovieDetailsDTO
55. TESTING STRATEGY DATA LAYER
56. ♟TESTING STRATEGY DATA LAYER @Serializable data class MovieDetailsDTO( val id: String, val title: String, @SerialName("poster_path") val posterPath: String, @SerialName("backdrop_path") val backdropPath: String, val tagline: String, val overview: String, val homepage: String, @SerialName("vote_average") val voteAverage: Double, val runtime: Int, val imdbId: String, ) { “homepage": "https://www.thesuicidesquad.net", "id": "436969", "imdb_id": "tt6334354", "original_language": "en", "original_title": "The Suicide Squad", "overview": "Supervillains Harley Quinn, Bloodsport, Peacemaker an Reve prison join the super-secret, super-shady Task Force X as the infused island of Corto Maltese.", "popularity": 7243.123, }
57. ♟TESTING STRATEGY DATA LAYER @Serializable data class MovieDetailsDTO( val id: String, val title: String, @SerialName("poster_path") val posterPath: String, @SerialName("backdrop_path") val backdropPath: String, val tagline: String, val overview: String, val homepage: String, @SerialName("vote_average") val voteAverage: Double, val runtime: Int, @SerialName("imdb_id") val imdbId: String, ) { “homepage": "https://www.thesuicidesquad.net", "id": 436969, "imdb_id": "tt6334354", "original_language": "en", "original_title": "The Suicide Squad", "overview": "Supervillains Harley Quinn, Bloodsport, Peacemaker an Reve prison join the super-secret, super-shady Task Force X as the infused island of Corto Maltese.", "popularity": 7243.123, }
58. FULLY TESTING NETWORK
59. DATA LAYER FULLY TESTING NETWORK @Test fun `given 200 response with correct field types should return domain model correctly`() = runTest { val api: MovieApi = mock() val sut = MovieDetailsNetworkDataSource(api) val actual = sut.fetchMovieDetails("id") val expected = movieDetails( id = "436969", homepage = "https://www.thesuicidesquad.net", title = "The Suicide Squad", voteAverage = 8.1, runtime = 132, imdbUrl = "https://www.imdb.com/title/tt6334354" ) assertEquals(expected, actual) }
60. DATA LAYER FULLY TESTING NETWORK @Test fun `given 200 response with correct field types should return domain model correctly`() = runTest { mockWebServer.enqueueResponse("movie-details-200.json", 200) val actual = sut.fetchMovieDetails("id") val expected = movieDetails( id = "436969", homepage = "https://www.thesuicidesquad.net", title = "The Suicide Squad", voteAverage = 8.1, runtime = 132, imdbUrl = "https://www.imdb.com/title/tt6334354" ) assertEquals(expected, actual) }
61. DATA LAYER FULLY TESTING NETWORK @Test fun `given 200 response with correct field types should return domain model correctly`() = runTest { mockWebServer.enqueueResponse("movie-details-200.json", 200) val actual = sut.fetchMovieDetails("id") val expected = movieDetails( id = "436969", homepage = "https://www.thesuicidesquad.net", title = "The Suicide Squad", voteAverage = 8.1, runtime = 132, imdbUrl = "https://www.imdb.com/title/tt6334354" ) assertEquals(expected, actual) } - TEST INTEGRATIONS WITH RESPONSE PARSERS - USE REAL RESPONSES - GREAT FOR POLYMORPHIC RESPONSES
62. DATA PRESENTATION DOMAIN Business Logic Use case Use case Use case View Model Composable Composable Composable UI Elements Screen Controller Data Source Server M a p p e r Repository Network data source Retro f it M a p p e r
63. ♟TESTING STRATEGY - FOCUS ON READABLE AND ISOLATED TESTS - MORE FAKES, LESS MOCKS - FREE OF DEPENDENCIES DOMAIN DOMAIN Business Logic Use case Use case Use case
64. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS class GetHighRatedMovies @Inject constructor( private val dataSource: MoviesDataSource ) { suspend operator fun invoke(): List<Movie> = dataSource.fetchNextPage().filter { it.voteAverage > 8 } }
65. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS class GetHighRatedMovies @Inject constructor( private val dataSource: MoviesDataSource ) { suspend operator fun invoke(): List<Movie> = dataSource.fetchNextPage().filter { it.voteAverage > 8 } } data class Movie( val id: String, val title: String, val overview: String, val image: String, val voteCount: Int, val voteAverage: Double )
66. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS @Test fun `given successful network response should filter out low rated movies`() = runTest { val fakeDataSource = FakeDataSource(listOf(?)) val getHighRatedMovies = GetHighRatedMovies(fakeDataSource) val actual = getHighRatedMovies() val expected = listOf(?) assertEquals(expected, actual) }
67. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS @Test fun `given successful network response should filter out low rated movies`() = runTest { val fakeDataSource = FakeDataSource(listOf( Movie( id = "1", title = "foo", overview = "bar", image = "", voteCount = 0, voteAverage = 9.0 ), Movie( id = "2", title = "foo", overview = "bar", image = "",
68. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS @Test fun `given successful network response should filter out low rated movies`() = runTest { val fakeDataSource = FakeDataSource(listOf( Movie( id = "1", title = "foo", overview = "bar", image = "", voteCount = 0, voteAverage = 9.0 ), Movie( id = "2", title = "foo", overview = "bar", image = "", voteCount = 0, voteAverage = 8.0 ), ) val getHighRatedMovies = GetHighRatedMovies(fakeDataSource) val actual = getHighRatedMovies() val expected = listOf(?) assertEquals(expected, actual) }
69. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS @Test fun `given successful network response should filter out low rated movies`() = runTest { val fakeDataSource = FakeDataSource( listOf( movie(id = "1", voteAverage = 9.0), movie(id = "2", voteAverage = 7.0) ) ) val getHighRatedMovies = GetHighRatedMovies(fakeDataSource) val actual = getHighRatedMovies() val expected = listOf(movie(id = "1", voteAverage = 9.0)) assertEquals(expected, actual) }
70. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS @Test fun `given successful network response should filter out low rated movies`() = runTest { val fakeDataSource = FakeDataSource( listOf( movie(id = "1", voteAverage = 9.0), movie(id = "2", voteAverage = 7.0) ) ) val getHighRatedMovies = GetHighRatedMovies(fakeDataSource) val actual = getHighRatedMovies() val expected = listOf(movie(id = "1", voteAverage = 9.0)) assertEquals(expected, actual) } internal fun movie( id: String = "id", overview: String = "id", title: String = "id", voteAverage: Double = 0.0, voteCount: Int = 0, image: String = "https://image.tmdb.org/t/p/w500posterPath" ) = Movie(id, title, overview, image, voteCount, voteAverage)
71. ♟TESTING STRATEGY READABLE AND ISOLATED TESTS @Test fun `given successful network response should filter out low rated movies`() = runTest { val fakeDataSource = FakeDataSource( listOf( movie(id = "1", voteAverage = 9.0), movie(id = "2", voteAverage = 7.0) ) ) val getHighRatedMovies = GetHighRatedMovies(fakeDataSource) val actual = getHighRatedMovies() val expected = listOf(movie(id = "1", voteAverage = 9.0)) assertEquals(expected, actual) } internal fun movie( id: String = "id", overview: String = "id", title: String = "id", voteAverage: Double = 0.0, voteCount: Int = 0, image: String = "https://image.tmdb.org/t/p/w500posterPath" ) = Movie(id, title, overview, image, voteCount, voteAverage)
72. DATA DOMAIN Business Logic Use case Use case Use case PRESENTATION M a p p e r View Model Composable Composable Composable UI Elements Screen Controller Data Source Server M a p p e r Repository Network data source Retro f it
73. ♟TESTING STRATEGY - TESTABLE UI LAYER (E.G. UDF) - AVOID ASYNCHRONICITY ON TESTS - “PURE” UI TESTS VS INTEGRATED TESTS PRESENTATION PRESENTATION M a p p e r View Model Composable Composable Composable UI Elements Screen Controller
74. PURE UI - Tests view in isolation - Ideally map each state to one assertion type - Simple view interactions INTEGRATED - Tests integration between all layers - mock server data PRESENTATION ♟TESTING STRATEGY
75. @Test fun givenErrorShouldRecoverFromItCorrectly() { val movieId = "123" composeTestRule.movieDetailsRobot { isErrorDisplayed() mockWebServer.registerApiRequest( HttpRequest("/${MovieApi.PATH}/$movieId", HttpMethod.GET), getJsonContent("movie-details-200.json") ) clickOnRetry() areItemsDisplayedCorrectly( "THE SUICIDE SQUAD", "Supervillains Harley Quinn, Bloodsport…”, "⭐ 8.1", "🕒 132 min" ) } } @Test fun givenContentStateShouldDisplayItemsCorrectly() { val state = Content( movieDetailsViewEntity( title = "THE SUICIDE SQUAD", overview = "Supervillains Harley Quinn, Bloodsport…”, voteAverage = "⭐ 8.1", runtime = "🕒 132 min" ) ) initScreenWithState(state) composeTestRule.movieDetailsRobot { with(state.viewEntity) { areItemsDisplayedCorrectly(title, overview, voteAverage, runtime) } } } “PURE” UI INTEGRATED PRESENTATION ♟TESTING STRATEGY
76. THANK YOU @HOROWITZD