Front-end Testing for Skeptics

Paul Graham once quipped that "Web 2.0" really meant "JavaScript now works". Nearly ten years later, more and more functionality of our web applications is written in JavaScript. But for those of us who came of age when JavaScript was unreliable, it's been preferable to test the server-side, while leaving the UI a thin-as-possible shell. Testing the front-end was error prone and brittle.

However, when you're delivering a JavaScript widget hundreds of thousands of times a day on diverse third party websites, it needs to work. So: we need to test it, and those tests need to be as painless as possible to write and maintain.

This is a session for front-end testing skeptics (like me): It is possible to create tests that drive your web UI (JavaScript and all) that are automated, fast, reliable, headless -- no browser required -- and written in pure Ruby instead of some obtuse syntax. We'll explore the challenges and gotchas of testing the front-end and walk through an example that meets the above goals.

Transcript

1. Front-end Testing Luke Francl for Skeptics
3. View Controller Model Model Model Unit tests System tests
4. HTML/CSS/JavaScript Controller Model Model Model HTTP
5. “JavaScript now works.” – Paul Graham, defining Ajax
6. ”Search that works.”
9. M A N N I N G Ben Vinegar Anton Kovalyov FOREWORD BY Paul Irish
10. 8/12 9/12 10/12 11/12 12/12 1/13 2/13 3/13 4/13 Search Volume Conveniently missing Y axis
11. 8/12 9/12 10/12 11/12 12/12 1/13 2/13 3/13 4/13 Total Search Engines Conveniently missing Y axis
12. Improve Mobile
13. Improve Mobile ...and DON‘T BREAK ANYTHING
15. Scenario: Testing front-end code Given that I am a programmer And that I can code Ruby Then writing tests in English makes me sad
18. brew install phantomjs
19. # Gemfile group :test, :development do gem 'rspec-rails', '~> 2.13' gem 'capybara', '~> 2.1' gem 'poltergeist', '~> 1.2' end github.com/look/frontend-testing-for-skeptics
20. # spec_helper.rb require 'capybara/rspec' require 'capybara/poltergeist' Capybara.register_driver :poltergeist do |app| Capybara::Poltergeist::Driver.new(app, :js_errors => true, :inspector => true) end Capybara.javascript_driver = :poltergeist github.com/look/frontend-testing-for-skeptics
21. describe 'JavaScript errors', :js => true do context 'home page' do it 'does not raise a JavaScript error' do visit root_path fill_in "country", :with => 'Uni' end end end Minimum Viable Test
22. describe 'JavaScript errors', :js => true do context 'home page' do it 'does not raise a JavaScript error' do visit root_path fill_in "country", :with => 'Uni' end end end Minimum Viable Test
23. it 'shows off the Capybara DSL' do visit '/path' # use text or DOM ID click_on('Login') fill_in('Username', :with => 'RailsConf') page.should have_css('button.red') # Finders wait and retry # if the element isn't found find('button.red').click # Run JavaScript in the page context page.evaluate_script("1 == '1'").should be_true end
24. span.twitter-typeahead input#country input.tt-hint span.tt-dropdown-menu span.tt-dataset-countries span.tt-suggestions span.tt-suggestion span.tt-suggestion
25. describe 'country autocomplete', :js => true do it 'shows autocompletions and redirects' do visit root_path fill_in "country", :with => 'norw' find("body").click # HACK find('.tt-suggestion').click current_path.should == country_path('NO') end end
26. 1) country autocomplete with match shows autocompletions and redirects to the selected country Failure/Error: find('.tt-suggestion').click Capybara::ElementNotFound: Unable to find css ".tt-suggestion"
27. page.driver.debug
30. describe 'country autocomplete', :js => true do it 'shows no results' do visit root_path fill_in "country", :with => 'zzz' # Sometimes it may be easier to # use JavaScript directly! js = '$(".tt-suggestions").children().size() == 0' page.evaluate_script(js).should be_true end end
31. expect do click_on '#some-ajax-thing' page.should have_css(".some-result-class") end.to have_clean_global_scope
32. RSpec::Matchers.define :have_clean_global_scope do |*allowed| match do |event_proc| # globals_js is a string containing a JavaScript function # that returns an array of global variable names before_globals = page.evaluate_script(globals_js) event_proc.call after_globals = page.evaluate_script(globals_js) diff = after_globals - before_globals # ignore objects in allowed diff = diff - allowed diff.empty? end end
33. describe 'xss', :js => true do let(:title) do "Title <script>window.xss = true;</script>" end let(:document) { Document.create(:title => title) } it "escapes JavaScript" do visit document_path(document) find("h1").should have_content(title) page.evaluate_script('window.xss').should_not be_true end end
34. Gotchas
35. it "should show autocomplete without leaking globals" do # force capybara to wait until the script gets added to the DOM page.find("script[src*='/assets/swiftype']") page.find("link[rel='stylesheet'][href*='/assets/swiftype']") wait_until { page.evaluate_script('typeof $stjq === "undefined"') == false } expect do click_on '#st-search-input' # wait for click before filling in the field sleep(1) fill_in 'st-search-input', :with => 'content' wait_for_ajax!('$stjq') wait_until { page.has_css?('.swiftype-widget div.autocomplete ul li') } end.to have_clean_global_scope end Capybara 1
36. Capybara 2 it "should show autocomplete without leaking globals" do page.find("script[src*='/embed.js']") page.find("script[src*='/assets/swiftype']") page.find("link[rel='stylesheet'][href*='/assets/swiftype']") page.current_scope.synchronize do if page.evaluate_script("typeof $stjq === 'undefined'") raise Capybara::ExpectationNotMet.new("Script wasn't true") end end expect do click_on '#st-search-input' fill_in 'st-search-input', :with => 'content' page.has_css?('.swiftype-widget div.autocomplete ul li') end.to have_clean_global_scope end
37. # Use Capybara's waiting mechanism until a # JavaScript expression evaluates to true. def synchronize_javascript(script) current_node.synchronize do unless evaluate_script(script) raise ExpectationNotMet.new("Script wasn't true") end end end Kyle VanderBeek http://bit.ly/YrBG7t
39. Transactions and database setup Some Capybara drivers need to run against an actual HTTP server. Capybara takes care of this and starts one for you in the same process as your test, but on another thread. Selenium is one of those drivers, whereas RackTest is not. – Capybara README
40. Next steps: JavaScript unit testing describe("recursive factorial", function() { it("is correct", function() { expect(factorial(10)).toBe(3628800); }); });
42. Full stack integration testing with Rails 3, Cucumber, RSpec, QUnit and Capybara Matthew O'Riordan, April 17, 2011 blog.mattheworiordan.com/post/4701529828/
44. Photo: Noah Scalin modeset/teabag
45. $ teabag Starting the Teabag server... Teabag running default suite at http://127.0.0.1:52681/(...) FFF Failures: 1) Factorial without memoization base case is correct Failure/Error: Expected 0 to be 1. 2) Factorial without memoization recursion case is correct Failure/Error: Expected 0 to be 3628800. 3) Factorial with memoization has the same result as the (...) Failure/Error: Expected 3628800 to be 0. Finished in 0.00500 seconds 3 examples, 3 failures
46. Next steps: Perceptual Diffs
47. Next steps: Perceptual Diffs
48. Next steps: Perceptual Diffs
49. Next steps: Perceptual Diffs
50. Brett Slatkin
51. Brett Slatkin Expected Actual Diff
52. Skeptic no more?