UI Regression Testing for Fun and Profit

Presented at Ruby Melbourne.

Transcript

1. UI REGRESSION TESTING FOR FUN AND PROFIT
2. Hi there I’m @plasticine Lead Frontend dev at Ferocia • pixelbloom.com • ferocia.com.au
3. ! TOOT TOOT We built and shipped a responsive, mobile-first Rails banking app, with native wrappers for iOS and Android. (it’s pretty cool, you should check it out) To do this we partnered up with Bendigo Bank.
5. ! THE FEAR IS REAL • Dedicated Test Team for our App(s) • Code lockdowns • Lonnnnnnggggg test cycle • Manual (functional & visual) Regression testing • Multi platform / version / browser Fabulously slow, often subjective, huge communication overhead
6. BETTER LIVING THROUGH TESTS Suddenly I’m not (as) scared anymore! Stop trying to hit me and hit me!
7. FEAR REDUCTION Internally we test our code a lot. • Ruby unit tests • Javascript unit tests across multiple devices/platforms • Exhaustive test coverage JS / Ruby • Feature testing via Selenium • Blackbox (no knowledge) • Remote against prod-like envs
8. But... THINGS THAT CAN’T BE AUTOMATED ARE PRETTY MUCH THE WORST THING EVER
11. WHERE IS YOUR TESTING GOD NOW?
13. “Is…is there a pony, dancing on the screen on that page?”
14. “Ahh, yeah — didn’t test that. Didn’t think of that one”
16. PROBLEM Frontend dev is hard, and CSS is a firehose of danger and failure • Manually testing the visual appearance of your UI is incredibly slow (sorry test team) • Automated browser testing won’t tell you if something doesn’t look right • A CSS diff is pretty much magic once it exceeds about 20 LOC
17. HELPFUL THINGS • CSS architecture (BEM, OOCSS, etc.) • Selenium testing (half the picture but it helps!) • Browserstack / Saucelabs (really level up you slowness)
19. WHAT I WANT describe 'My UI Component' do it { is_expected.to look_the_same } end
20. AUTOMATED UI TESTING Turns out there are tools out there that can help: • Huxley: (Python, Selenium/Webdriver) • Wraith: (Node.js, Wizards?) • Green Onion: (Ruby, Selenium/Webdriver)
21. SIMULACRUM github.com/plasticine/simulacrum • it to feel “unit-testy”, no new language to learn • it to have a simple workflow which fits in with existing tools • it to build on common, accepted tools (rspec, selenium, capybara) • it to be able to run tests on mobile handsets via remote systems like Browserstack and Saucelabs
22. SIMULACRUM github.com/plasticine/simulacrum # spec/ui/button_spec.rb describe 'My Component' do component :my_component do |options| options.url = '/my_component.html' options.delta_threshold = 1 # 1% pixel change allowed end it { is_expected.to look_the_same } end
23. SIMULACRUM github.com/plasticine/simulacrum Specs are just reference images that represent known good state.
24. SIMULACRUM github.com/plasticine/simulacrum It’s just selenium so you can run remote browser or device tests by using a supporting driver.
25. Questions
26. SNEAKY BONUS PLUG! WE’RE HIRING! Frontend, Backend, ObjC! Whatever! Come and talk to me if you want to work on cool, big, challenging stuff with an awesome team. Check out our work on: demo.bendigobank.com.au !