Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing for Success in the Real World

Testing for Success in the Real World

Does that code you just wrote actually work? How do you know? How do your teammates know? You did write tests for that, right? What kind did you write though? Unit? Integration? System? End-to-End? What about mocking and stubbing? I know, you only changed two lines of a legacy piece of the app… but still, how do you know this didn’t break anything?

Ugh. Let’s face it testing your application is difficult and tedious. Where can you get the most bang for your buck? What’s the 20% of work that gets you 80% of the return? In this session you’ll learn where you can focus your attention to gain the most traction in your testing endeavors. From mocking api calls, to juxtaposing the benefits of unit testing vs end-to-end testing, we’ll cover it all.

Donn Felker

August 15, 2019
Tweet

More Decks by Donn Felker

Other Decks in Technology

Transcript

  1. 4 1999-2003: Wild West Cowboy Coder ▪ What’s a test?

    ▪ “It works on my box.” ▪ Yeah, I tested it. Just hit “refresh” and you’ll see that it still works. ▪ Didn’t really know what testing was/how to do it.
  2. Debugging and Manual Verification Testing ▪ alert(“in here”) ▪ console.log(“in

    here”) ▪ Response.Write(“in here”) ▪ echo “in here” ▪ Log.d(“MainActivity”, “in here”) 5
  3. 12 “YO, MY WEBSITE IS COVERED IN SHIT! IT’S NOT

    A GOOD LOOK, SON! WASSUP!?” - A HIP HOP PRODUCER
  4. 13

  5. 14

  6. 16 try { mysql_connect("localhost", "someuser", "somepass"); $selected = mysql_select_db("mydb"); //

    ... } catch (Exception $e) { echo "<h1>Oh shit.</h1>"; } I FORGOT TO REMOVE THIS. THAT IS A BUG. A BAD ONE.
  7. 17

  8. 18

  9. 20

  10. 2003-Current: Test. Test. Test. If it doesn’t have a test,

    it’s legacy code. Write tests for existing features, new features, etc. Testing creates confidence. 21
  11. 2003-2009: Testing with .NET and Rails ▪ .NET - nUnit

    ▫ RhinoMocks for mocks ▫ Selenium for UI Automation ▪ Ruby on Rails- Test::Unit ▫ Dynamic Language Tricks for Mocks ▫ Selenium / Capybara for UI Automation 22
  12. 2009-2011: Built over 30 Android Apps ▪ Over 10MM+ Installs

    ▪ All a Hodge Podge Spaghetti Mess ▪ ALL HAD ZERO TESTS 23
  13. 25

  14. AUTOMATED TESTING Proper test coverage increases your confidence when you

    make a change to your code base and it is verified via a continuous integration server. 27
  15. 32 Unit Testing ▪ Usually best for lower level components

    that don’t require other dependencies to do their job. ▪ Example: ▫ A Calculator ▫ Simple Objects (POJO/POKO) ▫ Mapper/Adapter Pattern Implementations
  16. 33 @Test fun `calculator can add`() { val calc =

    Calculator() assertThat(calc.add(2, 2)).isEqualTo(4) }
  17. 34 class Calculator { fun add(first: Int, second: Int) =

    first + second // other functions } No outside integrations or systems to rely on. It’s a unit of work that can be tested.
  18. 35 data class Customer( val firstName: String, val lastName: String,

    val age: Int, val level: String, val favoriteColor: String ) data class CustomerViewModel( val fullName: String, val age: Int )
  19. interface Mapper<InputType, ReturnType> { fun map(input: InputType): ReturnType } class

    CustomerViewModelMapper : Mapper<Customer, CustomerViewModel> { override fun map(c: Customer): CustomerViewModel { return CustomerViewModel("${c.firstName} ${c.lastName}", c.age) } }
  20. 37 @Test fun `should be able to map a customer

    view model from a customer object`() { val customer = Customer("Donn", "Felker", 66, "platinum", "Blue") val mapper: Mapper<Customer, CustomerViewModel> = CustomerViewModelMapper() val vm: CustomerViewModel = mapper.map(customer) assertThat(vm.fullName).isEqualTo("${customer.firstName} ${customer.lastName}") assertThat(vm.age).isEqualTo(customer.age) }
  21. 38 ARRANGE.ACT.ASSERT. @Test fun `should be able to map a

    customer view model from a customer object`() { // Arrange val customer = Customer("Donn", "Felker", 66, "platinum", "Blue") val mapper: Mapper<Customer, CustomerViewModel> = CustomerViewModelMapper() // Act val vm: CustomerViewModel = mapper.map(customer) // Assert assertThat(vm.fullName).isEqualTo("${customer.firstName} ${customer.lastName}") assertThat(vm.age).isEqualTo(customer.age) }
  22. 41 class OrderAggregator(private val legacyApi: LegacyApi, private val api: Api)

    { fun ordersFor(id: Int): List<Order> { val legacyOrders = legacyApi.getOrders(id) val newOrders = api.getOrders(id) return newOrders.union(legacyOrders).toList() } }
  23. 42 Integration Test: Verifying Values @Test fun `when both apis

    are called both the orders are combined and returned`() { val legacyApi: LegacyApi = mock() val api: Api = mock() val legacyOrders = listOf(Order(1), Order(2)) whenever(legacyApi.getOrders(anyInt())).thenReturn(legacyOrders) val newOrders = listOf(Order(8), Order(9)) whenever(api.getOrders(anyInt())).thenReturn(newOrders) val aggregator = OrderAggregator(legacyApi, api) val aggregatedOrders = aggregator.ordersFor(42) val expected = newOrders.union(legacyOrders).toList() assertEquals(expected, aggregatedOrders) }
  24. 43 class OrderAggregator(private val legacyApi: LegacyApi, private val api: Api)

    { fun ordersFor(id: Int): List<Order> { val legacyOrders: List<Order> = listOf() //legacyApi.getOrders(id) val newOrders = api.getOrders(id) return newOrders.union(legacyOrders).toList() } }
  25. 44 Integration Test: Verifying Behavior @Test fun `both apis should

    be called when retrieving orders`() { val legacyApi: LegacyApi = mock() val api: Api = mock() val aggregator = OrderAggregator(legacyApi, api) aggregator.ordersFor(42) verify(legacyApi).getOrders(42) verify(api).getOrders(42) }
  26. 45 class OrderAggregator(private val legacyApi: LegacyApi, private val api: Api)

    { fun ordersFor(id: Int): List<Order> { val legacyOrders = legacyApi.getOrders(id) val newOrders = api.getOrders(id) return newOrders.union(legacyOrders).toList() } }
  27. 46

  28. 48

  29. 54

  30. 55 Declaring Package Bankruptcy (When your app has so many

    1-star reviews that you cannot recover from it so you have to abandon the package name and upload a new one.)
  31. Business Logic/etc TEST THE WHOLE STACK (END TO END) Remote

    APIs/Services UI 60 Bluetooth Camera APIs Remote APIs Networking etc
  32. 62

  33. HOW TO MAKE A TEST HERMETIC ▪ Remove the variance

    & external influence ▫ Remove the network ▫ Remove shared files ▪ Sanitize the Testing Environment ▪ Create Repeatable Environments 63
  34. Bluetooth APIS → BlueTooth Delegate Camera APIS → Camera Delegate

    Contact Picker → Contact Picker Delegate Content Resolver → … yup ... a delegate DELEGATE PATTERN 67
  35. 68 public void onActivityResult(int reqCode, int resultCode, Intent data) {

    super.onActivityResult(reqCode, resultCode, data); switch (reqCode) { case (PICK_CONTACT): if (resultCode == Activity.RESULT_OK) { Uri contactData = data.getData(); Cursor c = managedQuery(contactData, null, null, null, null); if (c.moveToFirst()) { String id = c.getString(c.getColumnIndexOrThrow(ContactsContract.Contacts._ID)); String hasPhone = c.getString(c.getColumnIndex(ContactsContract.Contacts.HAS_PHONE_NUMBER)); try { if (hasPhone.equalsIgnoreCase("1")) { Cursor phones = getContentResolver().query( ContactsContract.CommonDataKinds.Phone.CONTENT_URI, null, ContactsContract.CommonDataKinds.Phone.CONTACT_ID + " = " + id, null, null); phones.moveToFirst(); String cNumber = phones.getString(phones.getColumnIndex("data1")); System.out.println("number is:" + cNumber); txtphno.setText("Phone Number is: "+cNumber); } String name = c.getString(c.getColumnIndex(ContactsContract.Contacts.DISPLAY_NAME)); txtname.setText("Name is: "+name); } catch (Exception ex) { st.getMessage(); } } } break; } }
  36. 69 data class Contact(val name: String, val number: String) interface

    ContactRepository { fun getContactFor(uri: Uri): Contact } class ContentResolverDelegate(context: Context) : ContactRepository { override fun getContactFor(uri: Uri): Contact { val contactData = uri.data val c = context.contentResolver... if (c.moveToFirst()) { // blah blah blah } } }
  37. 70 class FakeContactRepository : ContactRepository { lateinit var contact: Contact

    fun loadContact(contact: Contact) { this.contact = contact } override fun getContactFor(uri: Uri): Contact { return contact } }
  38. 71 override protected fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent)

    { // inject the contactRepository via DI val contact = contactRepository.getContactFor(data.data) // do stuff with the contact }
  39. 72

  40. 73 class ContactTest { @Inject var repository: FakeContactRepository @get:Rule var

    activityRule: ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java) @Before fun setup() { // do injection with your DI of choice so you can get a handle on the fake repository } @Test fun verifyThatUserCanSendMessageToFriendThatWasPickedFromContactPicker() { val stubResult = Intent() stubResult.data = Uri.EMPTY val activityResult = Instrumentation.ActivityResult(RESULT_OK, stubResult) Intents.Intending(IntentMatchers.hasAction(Intent.ACTION_PICK)).respondWith(activityResult) val expectedContact = Contact("Donn Felker", "888-867-5309") repository.loadContact(expectedContact) activityRule.launchActivity(intent) // Starts contact picker, the intents api kicks in and takes over // returning our expected intent onView(withId(R.id.choose_friend)).perform(click()) onView(withId(R.id.friend_header)).check(matches(withText("Send to: Donn Felker"))) onView(withId(R.id.friend_number)).check(matches(withText("Number: 888-867-5309"))) } }
  41. 74 Other Option - More work - More E2E ▪

    Write some test code to manually insert (and remove on tear down) contacts into the android content resolver. ▪ Very difficult, if not impossible, for things like bluetooth, wireless, etc ▪ Removing more of the “system” does make it hermetic, but at a cost.
  42. In E2E Tests Remove/Replace only what you absolutely have to.

    If it a component makes a test(s) flakey it might be a good candidate. 75
  43. Things you can do with these tools ▪ Return known

    responses ▪ Return HTTP errors ▪ Slow down the network ▪ Simulate very difficult HTTP Scenarios ▪ Get a reliable/measurable speed in your E2E tests that is not relate to network ▪ Make your API calls hermetic in test 80
  44. E2E HEURISTICS 82 Get the most out of Espresso by

    using these tools, tips and frameworks to speed up and stabilize your tests.
  45. USE ANDROID TEST ORCHESTRATOR 85 Minimal shared state. Each test

    runs in its own Instrumentation instance. Therefore, if your tests share app state, most of that shared state is removed from your device's CPU or memory after each test. Crashes are isolated. Even if one test crashes, it takes down only its own instance of Instrumentation, so the other tests in your suite still run. https://developer.android.com/training/testing/junit-runner#using-android-test-orchestrator
  46. SHARD YOUR TESTS Parallelize your tests. Specify how many shards

    and how many tests to run. Run on various devices 86 https://source.android.com/devices/tech/test_infra/tradefed/architecture/advanced/sharding
  47. 87 ANDROID_SERIAL=emulator-5554 ./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.numShards=3 -Pandroid.testInstrumentationRunnerArguments.shardIndex=0 // Runs first half

    of the tests adb -s DEVICE_1_SERIAL shell am instrument -w -e numShards 2 -e shardIndex 0 > device_1_results // Runs second half of the tests adb -s DEVICE_2_SERIAL shell am instrument -w -e numShards 2 -e shardIndex 1 > device_2_results VIA ADB VIA GRADLE
  48. USE FLANK (ONLY IF YOU USE Firebase Test Lab) 88

    Massively parallel Android and iOS test runner for Firebase Test Lab Built in Sharding Built in retry for flakey tests Set max run time per shard FTL Does cost $$$ https://developer.android.com/training/testing/junit-runner#using-android-test-orchestrator
  49. 96 SOMETIMES YOU DON’T KNOW WHAT YOU DON’T KNOW. Do

    it when you can, if you can’t … write tests afterwards that cover your changes.
  50. 100 REMEMBER USERS DON’T CARE THAT YOUR UNIT TESTS PASS.

    THEY CARE THAT YOUR APP WORKS. SO ... WRITE SOME E2E TESTS.
  51. 101