Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Testing Best Practices [ko]

Sa-ryong Kang
September 25, 2021

Android Testing Best Practices [ko]

Sa-ryong Kang

September 25, 2021
Tweet

More Decks by Sa-ryong Kang

Other Decks in Technology

Transcript

  1. Android Testing Best Practices - 실제 프로젝트에 도움이 되는 테스트

    코드 구현 강사룡, Partner Developer Advocate @Google
  2. Why should I write test code? • 코딩 생산성 증대:

    확신을 갖고 시스템을 수정할 수 있음 • 효율적으로 버그를 잡을 수 있음 • 코드 변경(특히 refactoring)을 쉽게 할 수 있음 • modular한 설계에 도움: single responsibility
  3. Why should we write test code? • QA doesn’t scale

    • 협업을 위해 반드시 필요 ◦ 여름휴가 중에 내 코드에서 장애가 난다면? ▪ or 몇 개월 뒤의 내가 갑자기 내 코드를 다시 보게 된다면? ◦ 협업을 촉진: code owner가 아니더라도 코드를 수정할 수 있음 ▪ 문서로서의 테스트 코드. 테스트 케이스만 보면 특정 시스템의 기능과 의도, 올바른 사용법을 단번에 파악 가능 ▪ 원래 의도와 어긋나게 수정했다고 해도 테스트가 실패하므로 금방 알아챌 수
  4. Google의 테스트 정책 • 높은 Test Coverage*: 여기서 테스트 커버리지는

    오직 small-sized test로만 측정함 • Beyoncé Rule: “If you liked it, then you shoulda put a test on it.” * 프로덕트 별로 차이가 있음
  5. When to use real device for testing • DO -

    안정성 / 호환성 테스트, 성능 테스트 • DON'T - Unit tests: 할 수 있는 한 JVM에서 실행해야 함. 그리고 되지 않는 것은 simulator에서. 실 기기 테스트를 하기 전에 먼저 이런 것들을 고려해볼 것 ◦ 격리된 테스트 환경 구축 (Hermetic testing) ◦ Protocol testing for client-server compatibility
  6. 테스트 코드를 처음/다시 해본다면? - 1단계는.. • 작은, 독립적인 부분부터

    시작 ◦ 예: 계산/변환 로직을 가진 클래스 ◦ 참고: 테스트 주도 개발(TDD by Example) [link]
  7. 2단계: 실제로 (더) 도움이 되는 테스트 코드 작성 • 실제/의사

    디버깅 과정에서 테스트 코드를 적용 ◦ 구글의 경우, 장애가 발생하면.. i. 문제가 된 PR을 찾아서 롤백한다 ii. 문제 PR에 장애를 재현하기 위한 테스트 코드를 작성 → 물론 그 테스트는 fail iii. 테스트가 성공하도록 코드를 수정
  8. 실제에 가까운 단위 테스트 만들기 • 의존성은 어떻게 해결하나? ◦

    예: SQLite, REST/gRPC call ◦ Real code > Fake >> Mock/Spy/Stub ◦ 1순위: 의존성 관계에 있는 진짜 코드를 사용 - 예: in-memory DB ▪ prefer realism over isolation ◦ 2순위: 라이브러리에 의해 제공되는 표준 fake를 사용 ◦ 3순위: 위의 방법이 불가할 때, mock 사용 ◦ Hilt! - d.android.com/training/dependency-injection/hilt-testing
  9. Mocking Best Practice • type-safe한 matcher를 활용할 것 (hamcrest, truth

    등 + built-in) • interaction보다 state를 체크할 것 (appendix 참조) • 필요시 shared code를 적절히 사용할 것 (appendix 참조) • Android API를 mocking하지 말 것 ◦ Robolectric! via androidx.test ◦ Fragment 독립 생성, Life Cycle 제어 등 많은 개선이 있었음
  10. 3단계: 패자 부활전! • 테스트 코드의 유지보수가 너무 어렵다 •

    이전에 잘 돌아갔던 테스트가 어느 순간 fail 된다 • 이유는? Brittle test
  11. 대원칙: Unchanging Test • Test should not be changed by

    following reasons: ◦ Pure refactorings ◦ New features ◦ Bug fixes • Exception: behavior changes
  12. Best Practice #1 Test via Public APIs fun processTransaction(transaction: Transaction)

    { if (isValid(transaction)) { saveToDatabase(transaction) } } private fun isValid(t: Transaction): Boolean = return t.amount < t.sender.balance private fun saveToDatabase(t: Transaction) { val s = "${t.sender}, ${t.recipient()}, ${t.amount()}" database.put(t.getId(), s) }
  13. fun processTransaction(transaction: Transaction) { if (isValid(transaction)) { saveToDatabase(transaction) } }

    private fun isValid(t: Transaction): Boolean = return t.amount < t.sender.balance private fun saveToDatabase(t: Transaction) { val s = "${t.sender}, ${t.recipient()}, ${t.amount()}" database.put(t.getId(), s) }
  14. Bad @Test fun emptyAccountShouldNotBeValid() { assertThat(processor.isValid(newTransaction().setSender(EMPTY_ACCOUNT))) .isFalse() } @Test fun

    shouldSaveSerializedData() { processor.saveToDatabase(newTransaction() .setId(123) .setSender("me") .setRecipient("you") .setAmount(100)) assertThat(database.get(123)).isEqualTo("me,you,100") }
  15. Good @Test fun shouldTransferFunds() { processor.setAccountBalance("me", 150) processor.setAccountBalance("you", 20) processor.processTransaction(newTransaction()

    .setSender("me") .setRecipient("you") .setAmount(100)) assertThat(processor.getAccountBalance("me")).isEqualTo(50) assertThat(processor.getAccountBalance("you")).isEqualTo(120) }
  16. Good @Test fun shouldNotPerformInvalidTransactions() { processor.setAccountBalance("me", 50) processor.setAccountBalance("you", 20) processor.processTransaction(newTransaction()

    .setSender("me") .setRecipient("you") .setAmount(100)) assertThat(processor.getAccountBalance("me")).isEqualTo(50) assertThat(processor.getAccountBalance("you")).isEqualTo(20) }
  17. Best Practice #3 Make Your Tests Complete and Concise //

    Bad @Test fun shouldPerformAddition() { val calculator = Calculator(RoundingStrategy(), "unused", ENABLE_COSINE_FEATURE, 0.01, calculusEngine, false) val result = calculator.calculate(newTestCalculation()) assertThat(result).isEqualTo(5) // Where did this number come from? }
  18. • “테스트 본문은, 테스트하고자 하는 것을 정확히 알 수 있는

    정보를 모두 갖고 있어야 하고, 반대로 불필요한 내용은 감춰야 한다.” ◦ 더 자세한 설명: https://testing.googleblog.com/2014/03/testing-on-toilet-what-makes-good-test.html @Test fun shouldPerformAddition() { val calculator = newCalculator() val result = calculator.calculate(newCalculation(2, Operation.PLUS, 3)) assertThat(result).isEqualTo(5) } Best Practice #3 Make Your Tests Complete and Concise
  19. Best Practice #4 Test Behaviors, Not Methods // Code to

    be tested fun displayTransactionResults(user: User, transaction: Transaction) { ui.showMessage("You bought a " + transaction.itemName) if (user.balance < LOW_BALANCE_THRESHOLD) { ui.showMessage("Warning: your balance is low!") } }
  20. Bad: test methods @Test fun testDisplayTransactionResults() { transactionProcessor.displayTransactionResults( newUserWithBalance( LOW_BALANCE_THRESHOLD.plus(dollars(2))),

    Transaction("Some Item", dollars(3))) assertThat(ui.getText()).contains("You bought a Some Item") assertThat(ui.getText()).contains("your balance is low") }
  21. Good: test behaviors @Test fun displayTransactionResults_showsItemName() { transactionProcessor.displayTransactionResults( User(), Transaction("Some

    Item")) assertThat(ui.getText()).contains("You bought a Some Item") } @Test fun displayTransactionResults_showsLowBalanceWarning() { transactionProcessor.displayTransactionResults( newUserWithBalance( LOW_BALANCE_THRESHOLD.plus(dollars(2))), Transaction("Some Item", dollars(3))) assertThat(ui.getText()).contains("your balance is low") }
  22. Best Practice #6 Don’t Put Logic in Tests // Bad

    @Test fun shouldNavigateToAlbumsPage() { val baseUrl = "http://photos.google.com/" val nav = Navigator(baseUrl) nav.goToAlbumPage() assertThat(nav.getCurrentUrl()).isEqualTo(baseUrl + "/albums") }
  23. Good @Test fun shouldNavigateToPhotosPage() { val nav = Navigator("http://photos.google.com/"); nav.goToPhotosPage()

    assertThat(nav.getCurrentUrl())) .isEqualTo("http://photos.google.com//albums") // Oops! }
  24. Best Practice #7 DAMP, Not DRY • DAMP: Descriptive And

    Meaningful Phrases • DRY: Don't Repeat Yourself
  25. Bad @Test fun shouldAllowMultipleUsers() { val users = createUsers(false, false)

    val forum = createForumAndRegisterUsers(users) validateForumAndUsers(forum, users) } @Test public void shouldNotAllowBannedUsers() { val users = createUsers(true) val forum = createForumAndRegisterUsers(users) validateForumAndUsers(forum, users) } // Lots more tests... private fun createUsers(boolean... banned): List<User> { // ... } // ...
  26. Good @Test fun shouldAllowMultipleUsers() { val user1 = newUser().setState(State.NORMAL).build() val

    user2 = newUser().setState(State.NORMAL).build() val forum = Forum() forum.register(user1) forum.register(user2) assertThat(forum.hasRegisteredUser(user1)).isTrue() assertThat(forum.hasRegisteredUser(user2)).isTrue() }
  27. Good @Test fun shouldNotRegisterBannedUsers() { val user = newUser().setState(State.BANNED).build() val

    forum = Forum() try { forum.register(user) } catch(ignored: BannedUserException) {} assertThat(forum.hasRegisteredUser(user)).isFalse() }
  28. References • 발표 슬라이드: https://speakerdeck.com/saryong • Software Engineering At Google

    ◦ https://abseil.io/resources/swe_at_google.2.pdf • Test apps on Android ◦ d.android.com/training/testing • Google Codelab - Android Testing Basics ◦ d.android.com/codelabs/advanced-android-kotlin-training-testing-basics • Google Testing Blog: https://testing.googleblog.com/
  29. Best Practice #2 Test State, Not Interactions // Bad @Test

    fun shouldWriteToDatabase() { accounts.createUser("foobar") verify(database).put("foobar") }
  30. Best Practice #5 Structure tests to emphasize behaviors (Good) @Test

    fun transferFundsShouldMoveMoneyBetweenAccounts() { // Given two accounts with initial balances of $150 and $20 val account1 = newAccountWithBalance(usd(150)) val account2 = newAccountWithBalance(usd(20)) // When transferring $100 from the first to the second account bank.transferFunds(account1, account2, usd(100)) // Then the new account balances should reflect the transfer assertThat(account1.getBalance()).isEqualTo(usd(50)) assertThat(account2.getBalance()).isEqualTo(usd(120)) }
  31. Another good example @Test fun shouldTimeOutConnections() { // Given two

    users val user1 = newUser() val user2 = newUser() // And an empty connection pool with a 10-minute timeout val pool = newPool(Duration.minutes(10)) // When connecting both users to the pool pool.connect(user1) pool.connect(user2) // Then the pool should have two connections assertThat(pool.getConnections()).hasSize(2) // When waiting for 20 minutes clock.advance(Duration.minutes(20)) // Then the pool should have no connections assertThat(pool.getConnections()).isEmpty() // And each user should be disconnected assertThat(user1.isConnected()).isFalse() assertThat(user2.isConnected()).isFalse()
  32. Best Practice #8 No Shared Value // Bad private val

    ACCOUNT_1 = Account.newBuilder() .setState(AccountState.OPEN).setBalance(50).build() private val ACCOUNT_2 = Account.newBuilder() .setState(AccountState.CLOSED).setBalance(0).build() private val ITEM = Item.newBuilder() .setName("Cheeseburger").setPrice(100).build() // Hundreds of lines of other tests... @Test fun canBuyItem_returnsFalseForClosedAccounts() { assertThat(store.canBuyItem(ITEM, ACCOUNT_1)).isFalse() } // ...
  33. Good private fun newContact(): Contact.Builder = Contact.newBuilder() .setFirstName("Grace") .setLastName("Hopper") .setPhoneNumber("555-123-4567")

    @Test fun fullNameShouldCombineFirstAndLastNames() { val contact = newContact() .setFirstName("Ada") .setLastName("Lovelace") .build() assertThat(contact.getFullName()).isEqualTo("Ada Lovelace") }
  34. Best Practice #9 Shared Setup // Bad private lateinit var

    nameService: NameService private lateinit var userStore: UserStore @Before fun setUp() { nameService = NameService() nameService.set("user1", "Donald Knuth") userStore = UserStore(nameService) } // [... hundreds of lines of tests ...] @Test fun shouldReturnNameFromService() { val user = userStore.get("user1") assertThat(user.getName()).isEqualTo("Donald Knuth") }
  35. Good private lateinit nameService: NameService private lateinit userStore: UserStore @Before

    fun setUp() { nameService = NameService() nameService.set("user1", "Donald Knuth") userStore = UserStore(nameService) } @Test fun shouldReturnNameFromService() { nameService.set("user1", "Margaret Hamilton") val user = userStore.get("user1") assertThat(user.getName()).isEqualTo("Margaret Hamilton") }
  36. Good example of helper method fun assertUserHasAccessToAccount(user: User, account: Account)

    { for (long userId : account.getUsersWithAccess()) { if (user.id == userId) { return } } fail("${user.name} cannot access ${account.name}") }