Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

It's time to up your test game

It's time to up your test game

On paper, everyone agrees that tests are important. And yet, more often than not, tests are neglected. In the rare case where we take the time to write them, more often than not it’s considered a cumbersome task, and given less attention than the "real code".

Tests are often as important, if not more so, as the code that will be published on the Play Store. This talk will start with describing a productive mindset when writing tests, followed by a few examples of common pitfalls and how to avoid them, and will conclude with a couple of advanced testing paradigms that can help make your tests cleaner and better.

Xavier Gouchet

September 10, 2020
Tweet

More Decks by Xavier Gouchet

Other Decks in Programming

Transcript

  1. It’s time to UP your test game Xavier F. Gouchet,

    Mobile RUM Team Lead @xgouchet / @datadoghq
  2. Monitor your Web and Mobile applications from your users’ perspective

    • Performance Improvement • Error Management and Alerting • Customer Support • Application Analytics About… Datadog Real User Monitoring 2
  3. Causes for poor tests Lazy code It’s “just test”, why

    should we put too much effort in it? Not thought through Let’s just test the nominal case, what could go wrong? Tests without intent We need to increase our coverage, but what are we testing exactly ? 6
  4. Keep your best practices • Code style and naming conventions

    • KISS • SOLID • Use Patterns • Avoid Hardcoded Values✘ 8
  5. Keep in mind the goals of testing • Verify that

    the code works • Make the contract explicit • Guide the development (TDD) Watch your tests fail✘ • Prevent regressions • Ease maintenance 9
  6. What are you actually testing • Each test has a

    single purpose • Test name reflects the intent ◦ `given X when Y then Z` ◦ `A performs B on C event` 11
  7. Think of any way the code might break • Rule

    of thumb ◦ 1+ test for nominal case ◦ 1+ test for each input (invalid, null) ◦ 1+ test for error state(s) • Add a test for every new bug 12
  8. What could go wrong here? @Test fun `test get host`()

    { // Given val email = "[email protected]" // When val host = extractHost(email) // Then assertThat(host).isEqualTo("example.com") } 15
  9. Constants are especially bad in Tests • You’re only testing

    one value … Every … single … time! • Humans are bad at generating “randomness” 16
  10. The main concept ARRANGE Generate “random” data ACT Perform some

    test ASSERT Verify the output / callbacks matches the test data 18
  11. Using random test data… • Overall a wider range of

    data is tested • Understand boundaries of valid/invalid inputs • Write generic tests without assumptions • Find edge cases 19
  12. Basic tests class FooTest { @get:Rule val forge = ForgeRule()

    @IntForgery var i: Int = 0 // … } 21
  13. Basic tests class FooTest { @get:Rule val forge = ForgeRule()

    @IntForgery var i: Int = 0 // … } 22
  14. Basic tests class FooTest { @get:Rule val forge = ForgeRule()

    @IntForgery var i: Int = 0 // … } 23
  15. Generate any kind of data • Boolean • Numeric •

    String • Collections (List, Set, Map) • Enums 24
  16. Adapting to your use cases • Keep the control over

    the generation of primitives • Generate custom class instances 25
  17. Controlling your random data class FooTest { @get:Rule val forge

    = ForgeRule() @IntForgery(min = 13, max = 42) var i: Int = 0 // … } 26
  18. Controlling your random data class FooTest { @get:Rule val forge

    = ForgeRule() @IntForgery(min = 13, max = 42) var i: Int = 0 // … } 27
  19. Randomize your own classes class BookForgeryFactory : ForgeryFactory<Book> { override

    fun getForgery(forge: Forge): Book { return Book( bookId = forge.getForgery<UUID>(), title = forge.anAlphabeticalString(), author = forge.anAlphabeticalString(), email = forge.aStringMatching("\w+@\w+.com") ) } } 28
  20. Randomize your own classes class BookForgeryFactory : ForgeryFactory<Book> { override

    fun getForgery(forge: Forge): Book { return Book( bookId = forge.getForgery<UUID>(), title = forge.anAlphabeticalString(), author = forge.anAlphabeticalString(), email = forge.aStringMatching("\w+@\w+.com") ) } } 29
  21. Randomize your own classes class BookForgeryFactory : ForgeryFactory<Book> { override

    fun getForgery(forge: Forge): Book { return Book( bookId = forge.getForgery<UUID>(), title = forge.anAlphabeticalString(), author = forge.anAlphabeticalString(), email = forge.aStringMatching("\w+@\w+.com") ) } } 30
  22. Randomize your own classes class BookForgeryFactory : ForgeryFactory<Book> { override

    fun getForgery(forge: Forge): Book { return Book( bookId = forge.getForgery<UUID>(), title = forge.anAlphabeticalString(), author = forge.anAlphabeticalString(), email = forge.aStringMatching("\w+@\w+.com") ) } } 31
  23. Randomize your own classes class BookForgeryFactory : ForgeryFactory<Book> { override

    fun getForgery(forge: Forge): Book { return Book( bookId = forge.getForgery<UUID>(), title = forge.anAlphabeticalString(), author = forge.anAlphabeticalString(), email = forge.aStringMatching("\w+@\w+.com") ) } } 32
  24. Keep the random within a controlled range class FooTest {

    @get:Rule val forge = ForgeRule() .withFactory(BookFactory()) @Forgery lateinit var book: Book // … } 33
  25. Keep the random within a controlled range class FooTest {

    @get:Rule val forge = ForgeRule() .withFactory(BookFactory()) @Forgery lateinit var book: Book // … } 34
  26. Keep the random within a controlled range class FooTest {

    @get:Rule val forge = ForgeRule() .withFactory(BookFactory()) @Forgery lateinit var book: Book // … } 35
  27. Making tests reproducible class FooTest { @get:Rule val forge =

    ForgeRule(0xdeadfa11) @IntForgery var i: Int = 0 // … } 37
  28. Making tests reproducible class FooTest { @get:Rule val forge =

    ForgeRule(0xdeadfa11) @IntForgery var i: Int = 0 // … } 38
  29. An 80% code coverage doesn’t mean “80% of your code

    is tested”; it means “at least 20% of your code is not tested” 42
  30. The Mutation Testing Algorithm • Have some (valid) tests •

    Mutate the production code i.e. introduce some bugs • Run the tests again • … • Profit 45
  31. Add the plugin to your build.gradle apply plugin: 'pl.droidsonroids.pitest' pitest

    { targetClasses = ['com.example.*'] outputFormats = ['XML', 'HTML'] } 47
  32. if (x > y) … if (a && b) …

    return t i++ Before / After Mutation if (x ≤ y) … if (a || b) … return null i-- 49
  33. Think about your tests as an incident response team; Mutation

    testing is similar to running a drill emergency. 50
  34. It won’t find bugs in the code, just reveal test

    issues Useful but not critical Only simulates atomic faults 51
  35. “You need to be as confident in the tests you

    code as you are in the code you test.” 53
  36. • Test is code • Vary your test inputs •

    Make sure your tests are actually testing something Being confident in your tests 54
  37. Thanks! Any questions? @xgouchet / @datadoghq 57 Presentation template by

    SlidesCarnival · Icons by Aleksandra Wolska @tutsii159