Automated Mobile Acceptance Testing Workshop - mdevcon 2013

Mobile Acceptance Testing Workshop mdevcon 2013 did you do the
setup steps? github.com/moredip/mdevcon-workshop Thursday, March 14, 13

Pete Hodgson @ph1 thepete.net [email protected] Thursday, March 14, 13

Thursday, March 14, 13

enough about me, what about you? Thursday, March 14, 13

The Plan Thursday, March 14, 13

The Plan some theory, but centered on practical know-how lots
of opportunities for hands on Thursday, March 14, 13

The Plan focus will be iOS and Frank but almost
all concepts are generally applicable Thursday, March 14, 13

Agenda the value of testing acceptance testing fundamentals selecting bits
of UI interacting with bits of UI inspecting bits of UI “advanced stuff” Thursday, March 14, 13

The Goal leave here feeling empowered to write your first
automated acceptance tests Thursday, March 14, 13

Bonus Goal learn some acceptance testing ProTipstm Thursday, March 14,
13

interrupt, raise your hand, ask questions Please Thursday, March 14,
13

Manual Testing Thursday, March 14, 13

launch the app > cd your_app_directory > frank launch LAUNCHING
IN THE SIMULATOR... Thursday, March 14, 13

manual test script Scenario: adding a new counter - launch
app - tap + to add a new counter - choose a CountUp counter - enter “My Group” as a group name - enter “My Counter” as a counter name - tap Save - check you are back on the main screen - check you see your group and counter name - check that tapping the counter makes it count Thursday, March 14, 13

manual test script Scenario: adding a new counter - launch
app - tap + to add a new counter - choose a CountUp counter - enter “My Group” as a group name - enter “My Counter” as a counter name - tap Save - check you are back on the main screen - check you see your group and counter name - check that tapping the counter makes it count again Thursday, March 14, 13

what have we discovered? - testing is a way to
explore the app - testing can be tedious - humans tend to optimize - humans tend to make mistakes Thursday, March 14, 13

what else? Thursday, March 14, 13

did we find any bugs? Thursday, March 14, 13

what else would you test? Thursday, March 14, 13

running our first automated test Thursday, March 14, 13

running our first automated test > cd your_app_directory > frank
build > cucumber Frank/features Thursday, March 14, 13

what just happened? Thursday, March 14, 13

iOS app acceptance tests Thursday, March 14, 13

iOS app acceptance tests Frank server Frank driver HTTP test
scripts Thursday, March 14, 13

why do we test? Thursday, March 14, 13

why do we test? interacting with the app to see
what it does Thursday, March 14, 13

“does it work?” Thursday, March 14, 13

“does it still work?” Thursday, March 14, 13

“what’s it supposed to do anyway?” Thursday, March 14, 13

manual testing is tedious ... Thursday, March 14, 13

and we’re not very good at it ... manual testing
is tedious ... Thursday, March 14, 13

but computers are ... and we’re not very good at
it ... manual testing is tedious ... Thursday, March 14, 13

“Computers are designed to do simple repetitive tasks. The second
you have humans doing repetitive tasks, all the computers get together late at night and laugh at you…” - Neal Ford Thursday, March 14, 13

why automate your tests Thursday, March 14, 13

why automate your tests - more time for exploratory QA

- consistency in our testing Thursday, March 14, 13

- consistency in our testing - drives focus on requirements Thursday, March 14, 13

- consistency in our testing - drives focus on requirements - protect against regressions Thursday, March 14, 13

- consistency in our testing - drives focus on requirements - protect against regressions - enable courageous change Thursday, March 14, 13

- consistency in our testing - drives focus on requirements - protect against regressions - enable courageous change - what else? Thursday, March 14, 13

types of automated testing Thursday, March 14, 13

Mobile App Thursday, March 14, 13

Mobile App End User Thursday, March 14, 13

Mobile App End User Backend Services Thursday, March 14, 13

Unit Test Mobile App End User Backend Services Thursday, March
14, 13

Integration Test Mobile App End User Backend Services Thursday, March
14, 13

Acceptance Test Mobile App End User Backend Services Thursday, March
14, 13

Acceptance Test Mobile App End User Backend Services The App

Acceptance Unit Integration Thursday, March 14, 13

a few reliable tests are better than lots of flakey
tests ProTiptm Thursday, March 14, 13

run your tests as frequently as you can ProTiptm Thursday,
March 14, 13

the UI testing triad Thursday, March 14, 13

select a bit o UI Thursday, March 14, 13

select a bit o UI simulate interaction with it inspect
state o it Thursday, March 14, 13

UI select interact inspect Thursday, March 14, 13

symbiote > cd your_app_directory > frank launch > frank inspect

symbiote Thursday, March 14, 13

demo! Thursday, March 14, 13

view selectors Thursday, March 14, 13

view selectors - a little language to specify which views
to inspect/ interact with - like CSS or XPath Thursday, March 14, 13

view marked:‘A Group’ by accessibility label Thursday, March 14, 13

accessibility labels are your friend ProTiptm Thursday, March 14, 13

view marked:‘A Group’ by accessibility label Thursday, March 14, 13

tableViewCell by view class Thursday, March 14, 13

tableViewCell label drill down Thursday, March 14, 13

putting it together Thursday, March 14, 13

tableViewCell label marked:’A Counter’ putting it together Thursday, March 14,
13

avoid brittle view selection ProTiptm Thursday, March 14, 13

record & playback: not a sustainable approach ProTiptm Thursday, March
14, 13

user simulation touch(...) type_into_keyboard(...) set_orientation(...) double_tap(...) tap_and_hold(...) drag_with_initial_delay(...) Thursday, March
14, 13

touch(...) Thursday, March 14, 13

touch(...) view selector Thursday, March 14, 13

frank console > cd your_app_directory > frank launch > frank
console connecting to app... connected [1] pry(#<Frank::Console>)> Thursday, March 14, 13

frank console > touch “button marked:‘Add’” Thursday, March 14, 13

view introspection frankly_map(view_selector, method) runs an ObjectiveC method on all
matching UIView instances Thursday, March 14, 13

back in the console Thursday, March 14, 13

back in the console > selector = “button marked:‘Edit’” >
element_exists( selector ) => true > frankly_map( selector, ‘isEnabled’ ) => [true] > frankly_map( selector, ‘isHidden’ ) => [false] > frankly_map( selector, ‘accessibilityFrame’ ) => [{"size"=>{"width"=>50, "height"=>30}, "origin"=>{"x"=>5, "y"=>27}}] Thursday, March 14, 13

interact inspect select UI Thursday, March 14, 13

putting it all together Thursday, March 14, 13

back in the console > touch “view marked:‘Add’” > touch
“label marked:'CountUp'” > touch “view:'UITextFieldLabel' marked:'Group Name'” > type_into_keyboard “My Group” > touch “view:'UITextFieldLabel' marked:'Counter Name'” > type_into_keyboard “My Counter” > touch “button marked:'Save'” > element_exists “view marked:’My Group’” > element_exists “view marked:’My Counter’” > touch “view marked:’My Counter’” Thursday, March 14, 13

other platforms Thursday, March 14, 13

Android Robotium Calabash Thursday, March 14, 13

Mobile Web WebDriver (aka Selenium 2) with Capybara (if you’re
using ruby) Thursday, March 14, 13

which WebDriver? Thursday, March 14, 13

which WebDriver? iPhoneDriver ~ or ~ a desktop WebDriver (e.g.
chromedriver) Thursday, March 14, 13

Headless capybara-webkit poltergeist ghostdriver htmlunitdriver Thursday, March 14, 13

cucumber Thursday, March 14, 13

cucumber write tests which describe behavior in terms of user
value Thursday, March 14, 13

cucumber Given... When... Then... Thursday, March 14, 13

cucumber Given I have a simple counter And I have
counted to 10 When I reset the counter Then the counter should be at 0 Thursday, March 14, 13

How does that work? Thursday, March 14, 13

How does that work? Regular Expressions (!!!) Thursday, March 14,
13

Cucumber steps Given I have a simple counter And I
have counted to 10 When I reset the counter Then the counter should be at 0 Thursday, March 14, 13

Cucumber step definitions Thursday, March 14, 13

Cucumber step definitions When /^I reset the counter$/ do touch(
“view marked:‘Zero Counts’” ) end Thursday, March 14, 13

so, what does Cucumber do for us? Thursday, March 14,
13

... ... touch “button marked:'CountUp'” touch “view marked:'Group Name'” type_into_keyboard
“my group” touch “view marked:'Counter Name'” type_into_keyboard “my counter” touch “button marked:‘Save’” ... ... Thursday, March 14, 13

When I create a “my counter” counter Thursday, March 14,
13

When /^I create a "(.*?)" counter$/ do |counter| touch “button
marked:'CountUp'” touch “view marked:'Group Name'” type_into_keyboard DEFAULT_GROUP_NAME touch “view marked:'Counter Name'” type_into_keyboard counter touch “button marked:‘Save’” end Thursday, March 14, 13

“foo” When I create a counter Thursday, March 14, 13

“bar” When I create a counter Thursday, March 14, 13

“baz” When I create a counter Thursday, March 14, 13

Don’t write test scripts in cucumber ProTiptm Thursday, March 14,
13

Cucumber should be high-level and user-focussed ProTiptm Thursday, March 14,
13

embrace patterns like Page Object ProTiptm Thursday, March 14, 13

race conditions Thursday, March 14, 13

did we log in? touch “button marked:‘Login’” check_element_exists “view marked:‘Welcome
back!’” Thursday, March 14, 13

what about the network? touch
“button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” Thursday, March 14, 13

what about the network? touch “button marked:‘Login’”
check_element_exists “view marked:‘Welcome back!’” Thursday, March 14, 13

what about the network? touch “button marked:‘Login’”
check_element_exists “view marked:‘Welcome back!’” fail Thursday, March 14, 13

what about the network? touch
“button marked:‘Login’” check_element_exists “view marked:‘Welcome back!’” fail Thursday, March 14, 13

a tempting solution touch “button marked:‘Login’” sleep 2 check_element_exists “view
marked:‘Welcome back!’” Thursday, March 14, 13

marked:‘Welcome back!’” PROBLEM slow tests Thursday, March 14, 13

marked:‘Welcome back!’” PROBLEM slow tests PROBLEM fragile tests Thursday, March 14, 13

wait_until SOLUTION: Thursday, March 14, 13

wait_until touch “button marked:‘Login’” wait_until do element_exists “view marked:‘Welcome back!’”
end Thursday, March 14, 13

wait_until touch “button marked:‘Login’” wait_for_element_to_exist do “view marked:‘Welcome back!’” end

Spin Assert ProTiptm Thursday, March 14, 13

wrapping up the value of testing (feedback) definition of an
acceptance test selecting/interacting/inspecting frank console and symbiote “advanced stuff” Thursday, March 14, 13

this slide deck will be at: http://slides.thepete.net Thursday, March 14,
13

further research - ‘Page Object’ pattern - Spin Asserts -
Quarantine broken tests - acceptance testing Journeys - Continuous Integration - faking out external services Thursday, March 14, 13

helpful resources testingwithfrank.com the cucumber book the rspec book scripting
w. Ruby book specification by example book rubymine (good IDE for ruby/cuke dev.) eradicating non- determinism in tests (good blog post) creating maintainable acceptance tests (presentation slides and video) the example tests in the Frank repo Thursday, March 14, 13

more questions? @ph1 http://blog.thepete.net [email protected] me, at the bar feedback
plz! http://rate.thepete.net Thursday, March 14, 13

Automated Mobile Acceptance Testing Workshop - ...

Automated Mobile Acceptance Testing Workshop - mdevcon 2013

More Decks by Pete Hodgson

Featured

Transcript