Test-Driven Android Development at GDG Devfest Istanbul 2014

Everybody knows testing is important, so let’s focus on test-driven development, testing best practices and the most useful Android testing libraries in our quest to improve the user experience and developer happiness. In this talk you'll get an overview of how several types of testing (unit, integration, UI testing) fit into an Android project. The Android team has improved the testing story for Android during the years, but the Android testing framework can be complemented with several other tools and libraries to help you build better quality applications. And let's not forget that the ease and thoroughness of your testing is also determined by the architecture of your application. We'll show you how to get the most out of Android's testing infrastructure, in order to enable you to build great quality apps.

Transcript

1. Test-Driven Android Development
2. Testing
3. Android & Testing 01 02 03 Why? What? How?
7. Your host Filip Maelbrancke Consultant @ AppFoundry filip.maelbrancke@appfoundry.be @fmaelbrancke
9. Testing Avoid problems
10. TESTING Manual testing is tedious Humans are not very good at testing But computers are...
11. “Computers are designed to do simple repetitive tasks. The second you have humans doing repetitive tasks, all the computers get together late at night and laugh at you...” Neil Ford
12. Manual vs Automated
13. Manual vs Automated
14. Manual vs Automated
15. Mitigate issues? Be pro-active… and test.
16. EXCUSES We never make mistakes! The functionality is trivial Tests slow us down Management won’t let us WE = HERO TRIVIAL SLOW MANAGEMENT ǩ
17. TEST TRAITS (UNIT) TESTS ISOLATED REPEATABLE FAST SELF-DOCUMENTING ž
18. Repeatability (no matter what development stack he uses) in which they are being run Tests need to be run by every developer ♂ Tests must not rely on the environment
19. Speed The shorter, the less distraction less of an interruption to our workflow
20. Speed The shorter, the less distraction less of an interruption to our workflow
21. Self-documenting Testable code = clear Testable code = easy to follow we can just look at the test No need to explain how a component works No need to write documentation tests = usage examples
22. Tests should also be … Easy to execute Easy to debug Realistic Fluent API
23. Unit tests are real measure of project health and code quality
24. Android dev
25. Test types Unit tests Ŷ Functional tests Ŷ Acceptance tests Ŷ Integration tests Ŷ Regression tests Ŷ
26. d.android.com
27. Instrumentation Process Test package InstrumentationTestRunner Application package Test case classes JUnit
28. Instrumentation Test Activity state saving and restoration // Start the main activity of the application under test
 mActivity = getActivity();
 // Get a handle to the Activity object's main UI widget, a Spinner
 mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner);
 // Set the Spinner to a known position
 mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION);
 // Stop the activity - The onDestroy() method should save the state of the Spinner
 mActivity.finish();
 // Re-start the Activity - the onResume() method should restore the state of the Spinner
 mActivity = getActivity();
 // Get the Spinner's current position
 int currentPosition = mActivity.getSpinnerPosition();
 // Assert that the current position is the same as the starting position
 assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition);
29. Testing demo Simple master-detail app ☼
30. Assert! Don’t assume Ŷ Assert Ŷ
31. Assert assertEquals assertTrue assertFalse fail assertNull assertNotSame assertSame assertNotNull Assert
32. ViewAssertions
33. Expressiveness 
34. Hamcrest assertThat(audience, is(payingAttention())) Verification Mocking Assertions
35. Hamcrest Library of matchers Ŷ Syntactic sugar Ŷ Assert assertThat(someString, is(equalTo(“Expected”))); actual value expectation on the value (Matcher)
36. Hamcrest Assert Hamcrest assertEquals(“Conversion failed”, “Expected”, someString); assertThat(someString, is(equalTo(“Expected”)));
37. Hamcrest: collections Assert assertEquals(person, bigbangCharacters.getPersons().iterator().next()); Hamcrest assertThat(bigbangCharacters.getPersons(), hasItem(person));
38. Hamcrest matchers Core anything
 is
 … Logical allOf
 anyOf
 not Object equalTo
 instanceOf
 (not)nullValue Custom Beans hasProperty Text equalToIgnoringCase
 startsWith
 … Number greaterThan(OrEqualTo)
 closeTo Collections hasItem, hasItems
 hasItemInArray
39. View Matchers Hamcrest Ŷ allOf() / anyOf() ➪ give more power Ŷ TestRunner // assert that textview does not start with with "XYZ" and contains "ABC" anywhere
 onView(withId(R.id.celsius_textview))
 .check(matches(allOf(withText(not(startsWith("XYZ"))), withText(containsString("ABC")))));
 // assert that textview ends with "XYZ" or contains "ABC" anywhere
 onView(withId(R.id.celsius_textview))
 .check(matches(anyOf(withText(endsWith("XYZ")), withText(containsString("ABC")))));
40. Hamcrest Lots of useful matchers Ŷ Custom matchers Ŷ not a number public void testSquareRootOfMinusOneIsNotANumber() {
 assertThat(Math.sqrt(-1), is(notANumber()));
 }
41. Hamcrest custom matcher Hamcrest matcher /**
 * Hamcrest matcher: test if a double has the value NaN (not a number)
 */
 public class IsNotANumber extends TypeSafeMatcher<Double> {
 
 @Override
 public boolean matchesSafely(Double number) {
 return number.isNaN();
 }
 
 public void describeTo(Description description) {
 description.appendText("not a number");
 }
 
 @Factory
 public static <T> Matcher<Double> notANumber() {
 return new IsNotANumber();
 }
 }
43. Espresso
44. Espresso Easy API for test authors Find view Do stuff with view Check some state Without • waitUntil(timeout) • boilerplate
45. Espresso API onView(Matcher<View>) onData(Matcher<Object>) onView(Matcher<View>) onView(Matcher<View>) withId() / withText() is() / instanceOf() click() / enterText() / scrollTo() matches(Matcher<View>) isDisplayed / isEnabled / hasFocus withText / … Find stuff Do stuff Check
46. Espresso Espresso onView(withId(R.id.some_id)).perform(click()); 
 
 onView(withText("Hi there”)).check(matches(isDisplayed())); 
 
 onView(withId(R.id.some_id)).check(matches(withText(containsString(“someString”)))); 
 
 onData(is(someObject)).perform(click());
47. Espresso API onView(Matcher<View>) onData(Matcher<Object>) Espresso withId() withText() ViewMatchers perform() check() ViewInteraction / DataInteraction click() enterText() scrollTo() ViewActions matches(Matcher<View>) doesNotExist() ViewAssertions find view to operate on composed of © Stephan Linzner (Google)
48. View Matchers Espresso uses Hamcrest Matchers Espresso Matchers = Hamcrest Ŷ Make assertions on views Ŷ Filter views Ŷ Reuse existing Matchers Ŷ Create your own Matchers Ŷ from hamcrest-core and hamcrest-library
49. View Matchers provides Android specific View Matchers ViewMatchers Ŷ • click() • scrollTo() • writeText() Basic view actions Ŷ
50. Custom ViewAction Espresso ViewAction /**
 * Clears view text by setting {@link EditText}s text property to "".
 */
 public class ClearTextAction implements ViewAction {
 
 @Override
 public Matcher<View> getConstraints() {
 return allOf(isDisplayed(), isAssignableFrom(EditText.class));
 }
 
 @Override
 public void perform(UiController uiController, View view) {
 ((EditText) view).setText("");
 }
 
 @Override
 public String getDescription() {
 return "Clear the EditText component";
 }
 }
51. Custom ViewAction Espresso ViewAction /**
 * Your own custom View Actions.
 */
 public class CustomViewActions {
 
 private CustomViewActions() {
 }
 
 /**
 * Returns an action that clears text on the view. Extra thoroughly!
 */
 public static ViewAction clearTextExtraThoroughly() {
 return new ClearTextAction();
 }
 }
52. Testing demo Testing a temperature converter ☼
53. Use Custom ViewAction Espresso ViewAction public void testClearCelsiusTextField() {
 onView(withText("Celsius")).perform(clearTextExtraThoroughly());
 }
54. Reliability through synchronization between Instrumentation thread and UI thread for background resources Espresso synchronizes AsyncTasks default thread pool • wait until app is idle • perform operation on the UI thread • wait until completion • check result for custom implementations
 IntentService
 HandlerThread
 ThreadPool Synchronization Ŷ Synchronization Ŷ Synchronization Ŷ AsyncTask Ŷ IdlingResource interface Ŷ
55. Espresso synchronization Espresso IdlingResource public class IdleMonitor implements IdlingResource {
 
 @Override
 public String getName() {
 return IdleMonitor.class.getSimpleName();
 }
 
 @Override
 public boolean isIdleNow() {
 // return true if resource is idle
 return false;
 }
 
 @Override
 public void registerIdleTransitionCallback(ResourceCallback resourceCallback) {
 // store a reference to the resourceCallback
 // notify resourceCallback when idle
 }
 } Espresso.registerIdlingResources(idlingResource);
56. Robolectric
57. Android testing AndroidTestCase needs an instance of the emulator to run • spin up emulator • deploy the APK • run the actual tests Big time sink: Ŷ
58. Android testing Android core libraries depend upon the actual Android operating system
59. Robolectric that would otherwise need an emulator / device (mock implementations of Android core libraries) too slow / hard for unit testing without the baggage of a device Replaces the behavior of code Ŷ Write JUnit tests Ŷ When? Ŷ Does this by using Shadow classes Ŷ InstrumentationTestCase Ŷ vs real device / speedy emulator Ŷ http://robolectric.org/
60. Robolectric Robolectric 
 public class MyActivityTest {
 
 @Test
 public void clickingButton_shouldChangeResultsViewText() throws Exception {
 MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();
 
 Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);
 TextView results = (TextView) activity.findViewById(R.id.results_text_view);
 
 pressMeButton.performClick();
 String resultsText = results.getText().toString();
 assertThat(resultsText, equalTo("Testing Android Rocks!"));
 }
61. Robolectric setup • Robolectric • Novoda gradle-android-test-plugin see Paul Blundell Tests run inside the JVM Ŷ Gradle - Maven Ŷ Configure it yourself Ŷ Gradle plugin Ŷ http://blog.blundell-apps.com/android-gradle-app-with-robolectric-junit-tests/
62. Robolectric setup
63. Robolectric setup Robolectric setup apply plugin: 'java'
 
 dependencies {
 def androidModule = project(':appfoundry-app')
 compile androidModule
 
 testCompile rootProject.ext.libraries.junit
 testCompile rootProject.ext.libraries.robolectric
 
 testCompile androidModule.android.applicationVariants.toList().first().javaCompile.classpath
 testCompile androidModule.android.applicationVariants.toList().first().javaCompile.outputs.files
 testCompile files(androidModule.plugins.findPlugin("com.android.application").getBootClasspath())
 
 
 }
64. Robolectric Annotate your class @RunWith(RobolectricTestRunner.class)
 public class MyActivityTest {
 
 @Test
 public void clickingButton_shouldChangeResultsViewText() throws Exception {
 MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();
 
 Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);
 TextView results = (TextView) activity.findViewById(R.id.results_text_view);
 
 pressMeButton.performClick();
 String resultsText = results.getText().toString();
 assertThat(resultsText, equalTo("Testing Android Rocks!"));
 }
65. Robolectric
66. Android testing JUnit 4 Support JUnit 4 test @RunWith(JUnit4.class)
 public class AndroidJUnit4Test {
 
 Conference devoxx;
 
 @Before
 public void setUp() {
 devoxx = Conferences.newInstance(Conference.DEVOXX);
 devoxx.start();
 }
 
 @After
 public void tearDown() {
 devoxx.stop();
 }
 
 @Test
 public void testPreconditions() {
 assertThat(devoxx, is(notNullValue()));
 }
 
 @Test
 public void devoxxShouldBeGreat() {
 assertThat(devoxx, is(awesome()));
 }
67. Mocking & Stubbing
68. Mockito in isolation Change runtime implementation Ŷ That can be predictably tested Ŷ Verify behaviour Ŷ Mockito Ŷ
69. Mockito the behaviour of your components during a test Java mocking framework Ŷ Mock dependencies Ŷ Inject Mocks to validate Ŷ Mock function return values Ŷ https://code.google.com/p/mockito/
70. Mockito Mockito @Test
 public void testMockedConverter() throws Exception {
 // expectations
 final float expected_result = 32.0f;
 
 // given
 ConverterActivity activity = Robolectric.buildActivity(ConverterActivity.class).create().get();
 activity.onCreate(new Bundle());
 EditText celsiusField = (EditText) activity.findViewById(R.id.celsius_textview);
 celsiusField.setText("0");
 
 CelsiusFahrenheitConverter mockConverter = Mockito.mock(CelsiusFahrenheitConverter.class);
 Mockito.when(mockConverter.convertCelsiusToFahrenheit(Mockito.anyFloat())).thenReturn(expected_result);
 activity.setConverter(mockConverter);
 
 // when
 Button convertButton = (Button) activity.findViewById(R.id.conversion_button);
 convertButton.performClick();
 
 // then
 Mockito.verify(mockConverter, Mockito.times(1)).convertCelsiusToFahrenheit(Mockito.anyFloat());
 TextView fahrenheitField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 String fahrenheitString = fahrenheitField.getText().toString();
 assertThat(fahrenheitString, equalTo(String.valueOf(expected_result)));
 }
71. The Monkey
72. evil monkey…
73. Monkey pseudo-random stream of user events Generates events Ŷ clicks Ŷ touches Ŷ gestures Ŷ system-level events Ŷ Stress test your app Ŷ
74. Monkey number of events to attempt operational constraints, such as restricting the test to a single package Event types and frequencies command-line tool Ŷ basic configuration Ŷ $ adb shell monkey -p your.package.name -v 500 Ŷ
75. AssertJ Android for checking assertions to read / write http://square.github.io/assertj-android/ Syntactic sugar Ŷ Extension of the AssertJ library Ŷ Makes tests easier Ŷ Fluent syntax Ŷ
76. AssertJ Android Assertions for Android objects Ŷ + support libraries Ŷ
77. AssertJ Android Regular JUnit AssertJ Android assertEquals(View.GONE, view.getVisibility()); assertThat(view).isGone(); expected: <8> but was: <4> Expected visibility <gone> but was <invisible>.
78. AssertJ Android Regular JUnit assertEquals(View.VISIBLE, layout.getVisibility()); assertEquals(VERTICAL, layout.getOrientation()); assertEquals(4, layout.getChildCount()); assertEquals(SHOW_DIVIDERS_MIDDLE, layout.getShowDividers()); AssertJ Android assertThat(layout).isVisible() .isVertical() .hasChildCount(4) .hasShowDividers(SHOW_DIVIDERS_MIDDLE);
79. Testing network resources
80. Testing network resources Unit tests Ŷ Integration tests Ŷ Retrofit Ŷ
81. Testing network resources how to trigger edge cases like API rate limit exceeded? HTTP calls should be asynchronous … Hit a real server? Ŷ Brittle tests! Ŷ Slow tests Ŷ Network or server is down Ŷ Incomplete tests Ŷ Further complicated in Android Ŷ
82. Testing network resources Use a test server Ŷ Mock the Retrofit interface Ŷ
83. Using a test server • virtually no changes or code required in your app codebase • can be shared across multiple platforms (Android, iOS, web, …) • another dependency (that can fail) • something new to figure out • not always easy to trigger errors or edge cases • what if ‘state’ is needed? • ‘slow’ tests execution, still HTTP calls needed PROs Ŷ CONs Ŷ
84. Mocking the Retrofit interface • robust: no network errors, timeouts, … • easy to generate errors / edge cases • easy to control server-side state • write your dummy data in Java • extra test code needed • you don’t test the JSON deserialization (but you can add unit tests) • mock a part of your application ➪ not a true integration test PROs Ŷ CONs Ŷ You trust the test suites of Retrofit, Gson, … and mock out the data
85. Network testing Mock network interface ‘Fake’-‘real’ server
86. Mocking the Retrofit interface MockRestAdapter sample
87. Mocking the Retrofit interface Retrofit mock public interface GitHub {
 @GET("/repos/{owner}/{repo}/contributors")
 List<Contributor> contributors(@Path("owner") String owner, @Path("repo") String repo);
 }
88. Mocking the Retrofit interface Canned data /** A mock implementation of the {@link GitHub} API interface. */
 static class MockGitHub implements GitHub {
 private final Map<String, Map<String, List<Contributor>>> ownerRepoContributors;
 
 public MockGitHub() {
 ownerRepoContributors = new LinkedHashMap<String, Map<String, List<Contributor>>>();
 
 // Seed some mock data.
 addContributor("square", "retrofit", "John Doe", 12);
 addContributor("square", "retrofit", "Bob Smith", 2);
 addContributor("square", "retrofit", "Big Bird", 40);
 addContributor("square", "picasso", "Proposition Joe", 39);
 addContributor("square", "picasso", "Keiser Soze", 152);
 }
89. Mocking the Retrofit interface Mock @Override public List<Contributor> contributors(String owner, String repo) {
 Map<String, List<Contributor>> repoContributors = ownerRepoContributors.get(owner);
 if (repoContributors == null) {
 return Collections.emptyList();
 }
 List<Contributor> contributors = repoContributors.get(repo);
 if (contributors == null) {
 return Collections.emptyList();
 }
 return contributors;
 }
90. Mocking the Retrofit interface Mock // Create a very simple REST adapter which points the GitHub API endpoint.
 RestAdapter restAdapter = new RestAdapter.Builder()
 .setEndpoint(API_URL)
 .build();
 
 // Wrap our REST adapter to allow mock implementations and fake network delay.
 MockRestAdapter mockRestAdapter = MockRestAdapter.from(restAdapter);
 
 // Instantiate a mock object so we can interact with it later.
 MockGitHub mockGitHub = new MockGitHub();
 // Use the mock REST adapter and our mock object to create the API interface.
 GitHub gitHub = mockRestAdapter.create(GitHub.class, mockGitHub);
 
 // Query for some contributors for a few repositories.
 printContributors(gitHub, "square", "retrofit");
 printContributors(gitHub, "square", "picasso");
91. Network testing Mock network interface ‘Fake’-‘real’ server
92. MockWebServer predictable requests-responses Scriptable web server Ŷ okhttp (square) Ŷ Specify responses Ŷ Test HTTP clients Ŷ Verify requests Ŷ Like Mockito (for HTTP stack) Ŷ • script your mocks • run app code • verify that expected requests were made
93. MockWebServer MockWebServer // Create a MockWebServer. These are lean enough that you can create // a new instance for every unit test. MockWebServer mockWebServer = new MockWebServer();
 mockWebServer.play();
 
 // Setup the MockWebServer & schedule a response
 MockResponse mockResponse = new MockResponse();
 mockResponse.setResponseCode(500);
 mockWebServer().enqueue(mockResponse);
 
 // Setup your networking lib. Ask the server for its url.
 … mockWebServer.getUrl(“/“);
 
 DO YOUR TEST
94. Wiremock predictable requests-responses wiremock.org Ŷ Tom Akehurst Ŷ Reproduce real-world scenarios Ŷ Increase reliability Ŷ Stateful scenarios Ŷ Non-Android specific Ŷ
95. Wiremock via HTTP/JSON API • _admin/mappings/new • mappings folder HTTP server Ŷ configure responses Ŷ use with Java API Ŷ
96. Wiremock Wiremock JSON API { "request": { "method": "GET", "url": "/api/mytest" }, "response": { "status": 200, "body": "More content\n" } } wiremock.org documentation
97. Wiremock Wiremock Java API stubFor(get(urlEqualTo("/some/thing")) .willReturn(aResponse() .withStatus(404) .withBody(“No!!”))); wiremock.org documentation
98. Wiremock Wiremock Java test @Test public void exampleTest() { // make your app do requests to http://localhost:8080 stubFor(get(urlEqualTo("/my/resource")) .willReturn(aResponse() .withStatus(200) .withBody("<response>Some content</response>"))); onView(withId(R.id.submit_button)).perform(click()); onView(withText(“error message”)).check(matches(isDisplayed())); verify(postRequestedFor(urlMatching(“/my/resource/[a-z0-9]+”))); } wiremock.org documentation
99. Wiremock … out-of-the-box Android support is coming… Stateful Ŷ Errors / delays Ŷ Wiremock in the test APK Ŷ Wiremock as server on network Ŷ
100. Application ARCHITECTURE
102. Software design patterns Good code = testable code Ŷ Dependency injection Ŷ
103. Software design patterns Architecting Android Applications with Dagger Dependency injection Ŷ Dagger Ŷ Avoid reflection at runtime Ŷ Define dependencies Ŷ Jake Wharton Ŷ https://parleys.com/play/529bde2ce4b0e619540cc3ae
104. Dependency injection Decoupling Ŷ Swap mocks in for tests Ŷ
105. AUTOMATION &
 REPORTING
106. Automated distributed testing Automated distributed instrumentation testing Screenshots share with the designers…? in parallel Run tests on multiple devices Reports / logs captured for each test
107. Spoon https://square.github.io/spoon/ Automate test execution Ŷ across multiple devices Ŷ Aggregation of screenshots Ŷ Aggregate the results Ŷ
108. Spoon High level tests overview Ŷ Test failure specific Ŷ to a single device or all devices
109. Spoon: device view Results of each test on one device Ŷ Useful for tracking down Ŷ device-specific failures of individual tests
110. Spoon: test view Results of single test on all devices Ŷ
111. Spoon: screenshots Visual inspection of test execution Ŷ screenshots Spoon.screenshot(activity, "initial_state"); /* Normal test code... */ Spoon.screenshot(activity, "after_login"); http://square.github.io/spoon/sample/
112. Build pipeline Checkout / compile Unit tests Test coverage Code analysis Create deployable artifact Deploy for automatic QA test Trigger automated QA stage
113. Build pipeline tools Build (maven - gradle) Dependency repo (nexus - artifactory) Testing framework (JUnit - ...) Test coverage (Cobertura - Emma - Jacoco) Code analysis (Checkstyle, findbugs, pmd, Android Lint) Creation of deployable artifact (buildtool, artifact repo) Trigger next stage
114. Continuous integration benefits without additional effort Fast feedback - fewer errors Ŷ Test everything on every (nightly) build Ŷ Regression tests Ŷ Less manual testing Ŷ
115. Jenkins
116. Jenkins
117. Continuous integration • Jenkins • Bamboo Travis … Server Ŷ Hosted Ŷ https://travis-ci.org
118. Test coverage open-source toolkit for measuring and reporting Java code coverage Metrics Ŷ Code safely Ŷ JaCoCo Ŷ Find risky code Ŷ
119. Android test coverage
120. Coverage 88%
121. Demo https://github.com/filipmaelbrancke/devfest-istanbul-2014
122. References
123. Suggested reading Android Application Testing Guide Diego Torres Milano (9781849513500) Robotium Automated Testing for Android Hrushikesh Zadgaonkar (9781782168010) The Busy Coder’s Guide to Android Development Mark Murphy - http://commonsware.com/Android/
124. Suggested reading Test Driven Development: By Example Beck, Kent (978-0321146533) Continuous Integration: Improving Software Quality and Reducing Risk Duvall, Paul M. et al. (978-0321336385) Working Effectively with Legacy Code Feathers, Michael (978-0131177055)
125. Questions? Filip Maelbrancke Consultant @ AppFoundry filip.maelbrancke@appfoundry.be @fmaelbrancke