Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DevFestFL 2017 - Espresso: A Screenshot is Wor...

Sam Edwards
November 11, 2017

DevFestFL 2017 - Espresso: A Screenshot is Worth 1,000 Words

Sam Edwards

November 11, 2017
Tweet

More Decks by Sam Edwards

Other Decks in Technology

Transcript

  1. Sam Edwards @HandstandSam • Android GDE • Architected and leading

    Espresso efforts for Capital One Wallet • Opinions in this talk are my own. http://handstandsam.com
  2. Outline • What is Espresso? • Screenshots - Why? How?

    When? The Cost? • How to get “Screenshots for Free” • Tips and Tricks
  3. What is Espresso? • A library provided by Google for

    Android UI testing • Built on Android Test Instrumentation • Aware of the idle state and therefore FAST • Easy to read and write
  4. NEVER Thread.sleep() • AsyncTask Threadpool ◦ Leverage for Networking and

    RxJava • Idling Resources - Use CountingIdlingResource ◦ increment(); and decrement(); https://developer.android.com/reference/android/support/test/espresso/contrib/CountingIdlingResource.html
  5. Oops, Your Test Failed android.support.test.espresso.NoMatchingViewException: No views in hierarchy found

    matching: with text: “DONE!” View Hierarchy: +>DecorView{id=-1, visibility=VISIBLE, width=720, height=1280, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=3} | +->ActionBarOverlayLayout{id=16909230, res-name=decor_content_parent, visibility=VISIBLE, width=720, height=1184, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2} | +-->FrameLayout{id=16908290, res-name=content, visibility=VISIBLE, width=720, height=1024, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=160.0, child-count=1} | +--->TextView{id=2131230720, res-name=greeting, visibility=VISIBLE, width=656, height=960, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=32.0, y=32.0, text=Hello world!, input-type=0, ime-target=false, has-links=false} | +-->ActionBarContainer{id=16909231, res-name=action_bar_container, visibility=VISIBLE, width=720, height=112, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=48.0, child-count=2}
  6. Ahh, I See Why android.support.test.espresso.NoMatchingViewException: No views in hierarchy found

    matching: with text: “DONE!” View Hierarchy: +>DecorView{id=-1, visibility=VISIBLE, width=720, height=1280, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=3} | +->ActionBarOverlayLayout{id=16909230, res-name=decor_content_parent, visibility=VISIBLE, width=720, height=1184, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2} | +-->FrameLayout{id=16908290, res-name=content, visibility=VISIBLE, width=720, height=1024, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false,
  7. Screenshots on FAILURES Espresso.setFailureHandler(new FailureHandler() { @Override public void handle(Throwable

    throwable, Matcher<View> matcher) { ScreenshotHelper.takeScreenshotForcefully("test_failed"); try { new DefaultFailureHandler(applicationContext).handle(throwable, matcher); } catch (Exception e) { logger.error(e.getMessage(), e); throw new RuntimeException(e); } } });
  8. Screenshots on FAILURES Espresso.setFailureHandler(new FailureHandler() { @Override public void handle(Throwable

    throwable, Matcher<View> matcher) { ScreenshotHelper.takeScreenshotForcefully("test_failed"); try { new DefaultFailureHandler(applicationContext).handle(throwable, matcher); } catch (Exception e) { logger.error(e.getMessage(), e); throw new RuntimeException(e); } } });
  9. Screenshots on FAILURES Espresso.setFailureHandler(new FailureHandler() { @Override public void handle(Throwable

    throwable, Matcher<View> matcher) { ScreenshotHelper.takeScreenshotForcefully("test_failed"); try { new DefaultFailureHandler(applicationContext).handle(throwable, matcher); } catch (Exception e) { logger.error(e.getMessage(), e); throw new RuntimeException(e); } } });
  10. NO ONE Sees The Value of Your Tests… YET •

    Your Boss • Product Owners • Designers • Manual Testers
  11. That’s because they can’t tell what your tests are doing

    and don’t find any use for them. To them it’s just console output.
  12. User Interfaces are VISUAL • A User Interface is something

    very hard to represent in a non-visual way. • Exercise: ◦ Imagine you need to tell someone who is blind what the Mona Lisa painting looks like. What would you say? https://en.wikipedia.org/wiki/Mona_Lisa
  13. Test Runners and HTML Reports • Run Instrumentation Commands •

    Collect Logs • Collect Screenshots • Generate HTML Reports
  14. Spoon Test Runner java -jar spoon-runner-1.7.0-jar-with-dependencies.jar \ --apk example-app.apk \

    --test-apk example-tests.apk • Run instrumentation tests in parallel • Shard tests across multiple devices • Beautiful HTML reports
  15. UiAutomator File outputFile = Spoon.screenshot(activity, tag, testClass, testMethod); uiDevice.takeScreenshot(outputFile); •

    Works inside and outside of your app • API 18+ (Jelly Bean MR2) https://developer.android.com/topic/libraries/testing-support-library/in dex.html#UIAutomator
  16. Other Alternatives • Firebase Test Lab’s “Screenshotter” ◦ https://firebase.google.com/docs/test-lab/test-screenshots •

    Fastlane’s “Screengrab” ◦ https://github.com/fastlane/fastlane/tree/master/screengrab
  17. Typical Espresso Test @RunWith(AndroidJUnit4.class) public class LoginTest { @Rule public

    ActivityTestRule<LoginActivity> activityRule = new ActivityTestRule(LoginActivity.class); @Test public void testLogin(){ onView(withId(R.id.username)).perform(typeText("sam")); onView(withText("LOGIN")).perform(click()); onView(withId(R.id.home_view_pager)).check(matches(isDisplayed())); } }
  18. Typical Espresso Test With Screenshots @RunWith(AndroidJUnit4.class) public class LoginTest {

    @Rule public ActivityTestRule<LoginActivity> activityRule = new ActivityTestRule(LoginActivity.class); @Test public void testLogin(){ onView(withId(R.id.username)).perform(typeText("sam")); takeScreenshot("entered_username"); takeScreenshot("clicking_login"); onView(withText("LOGIN")).perform(click()); takeScreenshot("homescreen_shown"); onView(withId(R.id.home_view_pager)).check(matches(isDisplayed())); } }
  19. Robot Testing Pattern View Presenter Model Test What Test Test

    Test Test Test Test X X X X X X X View Changed
  20. Robot Testing Pattern View Presenter Model Test What Test Test

    Test Test Test Test Robot X View Changed
  21. Espresso Test With a Robot @RunWith(AndroidJUnit4.class) public class LoginTest {

    @Rule public ActivityTestRule<LoginActivity> activityRule = new ActivityTestRule(LoginActivity.class); @Test public void testLogin(){ new LoginRobot().username("sam").login().assertHomeScreenShown(); } }
  22. Example Robot static final Matcher<View> VIEW_MATCHER_USERNAME_EDIT_TEXT = withId(R.id.username); static final

    Matcher<View> VIEW_MATCHER_LOGIN_BUTTON = withText("LOGIN"); static final Matcher<View> VIEW_MATCHER_HOME_SCREEN = withId(R.id.home_view_pager); public LoginRobot username(String username) { onView(VIEW_MATCHER_USERNAME_EDIT_TEXT).perform(clearText(), typeText(username), closeSoftKeyboard()); takeScreenshot("username_entered"); return this; } public LoginRobot login() { onView(VIEW_MATCHER_LOGIN_BUTTON).check(matches(isDisplayed())); takeScreenshot("logging_in"); onView(VIEW_MATCHER_LOGIN_BUTTON).perform(click()); return this; } public LoginRobot assertHomeScreenShown() { onView(VIEW_MATCHER_HOME_SCREEN).check(matches(isDisplayed())); takeScreenshot("home_screen_shown"); return this; }
  23. Example Robot static final Matcher<View> VIEW_MATCHER_USERNAME_EDIT_TEXT = withId(R.id.username); static final

    Matcher<View> VIEW_MATCHER_LOGIN_BUTTON = withText("LOGIN"); static final Matcher<View> VIEW_MATCHER_HOME_SCREEN = withId(R.id.home_view_pager); public LoginRobot username(String username) { onView(VIEW_MATCHER_USERNAME_EDIT_TEXT).perform(clearText(), typeText(username), closeSoftKeyboard()); takeScreenshot("username_entered"); return this; } public LoginRobot login() { onView(VIEW_MATCHER_LOGIN_BUTTON).check(matches(isDisplayed())); takeScreenshot("logging_in"); onView(VIEW_MATCHER_LOGIN_BUTTON).perform(click()); return this; } public LoginRobot assertHomeScreenShown() { onView(VIEW_MATCHER_HOME_SCREEN).check(matches(isDisplayed())); takeScreenshot("home_screen_shown"); return this; }
  24. Example Robot static final Matcher<View> VIEW_MATCHER_USERNAME_EDIT_TEXT = withId(R.id.username); static final

    Matcher<View> VIEW_MATCHER_LOGIN_BUTTON = withText("LOGIN"); static final Matcher<View> VIEW_MATCHER_HOME_SCREEN = withId(R.id.home_view_pager); public LoginRobot username(String username) { onView(VIEW_MATCHER_USERNAME_EDIT_TEXT).perform(clearText(), typeText(username), closeSoftKeyboard()); takeScreenshot("username_entered"); return this; } public LoginRobot login() { onView(VIEW_MATCHER_LOGIN_BUTTON).check(matches(isDisplayed())); takeScreenshot("logging_in"); onView(VIEW_MATCHER_LOGIN_BUTTON).perform(click()); return this; } public LoginRobot assertHomeScreenShown() { onView(VIEW_MATCHER_HOME_SCREEN).check(matches(isDisplayed())); takeScreenshot("home_screen_shown"); return this; }
  25. Example Robot static final Matcher<View> VIEW_MATCHER_USERNAME_EDIT_TEXT = withId(R.id.username); static final

    Matcher<View> VIEW_MATCHER_LOGIN_BUTTON = withText("LOGIN"); static final Matcher<View> VIEW_MATCHER_HOME_SCREEN = withId(R.id.home_view_pager); public LoginRobot username(String username) { onView(VIEW_MATCHER_USERNAME_EDIT_TEXT).perform(clearText(), typeText(username), closeSoftKeyboard()); takeScreenshot("username_entered"); return this; } public LoginRobot login() { onView(VIEW_MATCHER_LOGIN_BUTTON).check(matches(isDisplayed())); takeScreenshot("logging_in"); onView(VIEW_MATCHER_LOGIN_BUTTON).perform(click()); return this; } public LoginRobot assertHomeScreenShown() { onView(VIEW_MATCHER_HOME_SCREEN).check(matches(isDisplayed())); takeScreenshot("home_screen_shown"); return this; }
  26. When to Take Screenshots • Assertions ◦ Assertion(s) ◦ SCREENSHOT

    • Actions (Alters screen state) ◦ Displayed Assertion ◦ SCREENSHOT ◦ Action public LoginRobot login() { onView(VIEW_MATCHER_LOGIN_BUTTON).check(matches(isDisplayed())); takeScreenshot("logging_in"); onView(VIEW_MATCHER_LOGIN_BUTTON).perform(click()); return this; } public LoginRobot assertHomeScreenShown() { onView(VIEW_MATCHER_HOME_SCREEN).check(matches(isDisplayed())); takeScreenshot("home_screen_shown"); return this; }
  27. When to Take Screenshots • Assertions ◦ Assertion(s) ◦ SCREENSHOT

    • Actions (Alters screen state) ◦ Displayed Assertion ◦ SCREENSHOT ◦ Action public LoginRobot login() { onView(VIEW_MATCHER_LOGIN_BUTTON).check(matches(isDisplayed())); takeScreenshot("logging_in"); onView(VIEW_MATCHER_LOGIN_BUTTON).perform(click()); return this; } public LoginRobot assertHomeScreenShown() { onView(VIEW_MATCHER_HOME_SCREEN).check(matches(isDisplayed())); takeScreenshot("home_screen_shown"); return this; }
  28. When Should I Take Screenshots? • Well, it depends… but

    ALWAYS on FAILURE • To generate reports for product and design teams • On small/low-res emulators with mdpi screen density for smaller screenshots.
  29. Use Screenshots Wisely • Programmatically configurable • Change config without

    recompiling ◦ Leverage Instrumentation Args ▪ --e screenshots=true ▪ --e screenshots=false
  30. Easy Screenshots • Screenshots on Failure for FREE • Settings:

    ALL, FAILURE_ONLY or NONE ◦ Can be changed: ▪ Programmatically ▪ Via instrumentation args • Configurable ◦ Reporting Framework ◦ Screenshot Implementation
  31. “Can’t write to sdcard” error • Use @GrantPermissionRule from latest

    Android Test Support Library • Declare in app’s Manifest: ◦ READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE
  32. Target Audience for Espresso “Espresso is targeted at developers, who

    believe that automated testing is an integral part of the development lifecycle. While it can be used for black-box testing, Espresso’s full power is unlocked by those who are familiar with the codebase under test.”
  33. Execution Time Overhead in Minutes 384x640 mdpi Nexus 5 Nexus

    6P Falcon +16.53% +62.31% +114.39% UiAutomator +28.93% +116.15% +168.87%
  34. Report Size Overhead in MB 384x640 mdpi Nexus 5 Nexus

    6P Falcon +17.35% +66.41% +90.13% UiAutomator +18.41% +64.77% +89.19%
  35. Get Currently RESUMED Activity private static Activity getCurrentActivityInstance() { final

    Activity[] activity = new Activity[1]; getInstrumentation().runOnMainSync(new Runnable() { public void run() { Collection resumedActivities = ActivityLifecycleMonitorRegistry.getInstance() .getActivitiesInStage(RESUMED); if (resumedActivities.iterator().hasNext()) { Activity currentActivity = (Activity) resumedActivities.iterator().next(); activity[0] = currentActivity; logger.trace("Activity obtained of type: " + currentActivity.getClass().getName()); } } }); return activity[0]; }
  36. Get Test Method & Class from Current Stack Trace public

    static StackTraceElement getTestMethodStackTraceElement() { StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace(); for (int i = 4; i < stackTrace.length; i++) { StackTraceElement stackTraceElement = stackTrace[i]; try { Method method = Class.forName(stackTraceElement.getClassName()).getMethod( stackTraceElement.getMethodName()); Annotation annotation = method.getAnnotation(Test.class); if (annotation != null) { return stackTraceElement; } } catch (Exception e) { // That's fine, will look at the next stack trace element } } return stackTrace[4]; }
  37. Get Test Method & Class from Current Stack Trace public

    static StackTraceElement getTestMethodStackTraceElement() { StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace(); for (int i = 4; i < stackTrace.length; i++) { StackTraceElement stackTraceElement = stackTrace[i]; try { Method method = Class.forName(stackTraceElement.getClassName()).getMethod( stackTraceElement.getMethodName()); Annotation annotation = method.getAnnotation(Test.class); if (annotation != null) { return stackTraceElement; } } catch (Exception e) { // That's fine, will look at the next stack trace element } } return stackTrace[4]; }
  38. Screenshot-tests-for-android by Facebook public class MyTests { @Test public void

    doScreenshot() { View view = mLayoutInflater.inflate(R.layout.my_layout, null, false); ViewHelpers.setupView(view) .setExactWidthDp(300) .layout(); Screenshot.snap(view) .record(); } }
  39. Robot Testing Pattern • A Test Architecture Pattern for Maintainability

    • Learned about it from Jake Wharton’s talk: ◦ VIDEO: https://realm.io/news/kau-jake-wharton-testing-robots/ ◦ SLIDES: https://speakerdeck.com/jakewharton/testing-robots-kotlin-night-may-2016
  40. Create a Custom Test Runner • Control test runner behavior

    without recompiling • Turn on/off all screenshots with a command-line parameter ◦ --e screenshots=true ◦ --e screenshots=false
  41. Define Your “LOW BAR” Device • Android Emulator Virtual Device

    • 384x640 • mdpi - 160 dots per inch • Able to run in continuous integration
  42. Test Butler by LinkedIn “Reliable Android testing, at your service.”

    • Stabilizes the Android emulator ◦ Disables animations ◦ Disables crash & ANR dialogs ◦ Locks the keyguard, WiFi radio, and CPU to ensure they don't go to sleep unexpectedly while tests are running. • Handles changing global emulator settings and holds relevant permissions so your app doesn't have to. ◦ Enable/disable WiFi ◦ Change device orientation ◦ Set location services mode ◦ Set application locale https://github.com/linkedin/test-butler
  43. Create a Custom Test Runner • Control test runner behavior

    without recompiling • Turn on/off all screenshots with a command-line parameter ◦ --e screenshots=true ◦ --e screenshots=false
  44. Create a Custom Test Runner public class MyAndroidJUnitRunner extends AndroidJUnitRunner

    { @Override public void onCreate(Bundle arguments) { // process you parameters here. super.onCreate(arguments); } } defaultConfig { testInstrumentationRunner "com.handstandsam.MyAndroidJUnitRunner" }
  45. Create a Custom Test Runner public class MyAndroidJUnitRunner extends AndroidJUnitRunner

    { @Override public void onCreate(Bundle arguments) { // process you parameters here. super.onCreate(arguments); } } defaultConfig { testInstrumentationRunner "com.handstandsam.MyAndroidJUnitRunner" }
  46. GIFs • Generated by default • Processing overhead ◦ 11

    Seconds for 21 Screenshots -> • Disable using “--no-animations”