Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gherkin* and Cucumber*

Gherkin* and Cucumber*

Gherkin* and Cucumber*: a new test case path composition approach to testing Ruby on Rails web applications.

Roberto Zen

July 15, 2014
Tweet

More Decks by Roberto Zen

Other Decks in Technology

Transcript

  1. Gherkin* and Cucumber* A new test case path composition approach


    to testing Ruby on Rails web applications Trento, 15 July 2014 Graduant: Roberto Zen Supervisor: Prof. Maurizio Marchese University of Trento Department of Information Engineering and Computer Science Research carried on at ICT4G
 Bruno Kessler Foundation Co-Supervisor: Prof. Adolfo Villafiorita
  2. Feature: Example ! Scenario: Login Given I am on the

    login page When I sign in as “[email protected]” Then I should see “You are now logged in” Given /^I am on the login page$/ do
 visit login_page end ! When /^I sign in as ([\w]*@example.com)$/ do |email|
 fill_in “e-mail” with: email fill_in “password” with: “password” click_on “Sign in” end Step definitions Test cases in Gherkin Feature file 1
 2 3 4 5 6 1
 2 3 4 5 6 7 8 9 step
  3. Test cases execution flow Token ::= A | B A

    ::= aA | B B ::= nil Ragel lexer.rl lexer.rb while true do ! when 0 then … ! if … then goto 9; ! end transition_table.txt + feature file Given /^I am on .. $/ do end ! When /^ I sign in/$ do end The Gherkin gem Output The Cucumber gem Feature step definitions Feature: Login Scenario: Login failed Given I am on .. When I sign in .. Then I should see … parser.rb lexer.scan( transition_table.txt)
  4. Limits of tests in BDD Feature: Example ! Scenario: Login

    Given I am on the login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “[email protected]” And I sign out Then I should be redirected to the login page ! Scenario: A user publishes a new donation Given I am logged in as “[email protected]” When I publish a new donation And I sign out Then I should be redirected to the login page 1
 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
  5. Feature: Example ! Scenario: Login Given I am on the

    login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “[email protected]” And I sign out Then I should be redirected to the login page ! Scenario: A user publishes a new donation Given I am logged in as “[email protected]” When I publish a new donation And I sign out Then I should be redirected to the login page 1
 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Limits of tests in BDD
  6. Aim of the thesis Limit the number of duplicate steps

    following the DRY principle. Generate automatically new test cases. Improve the validation of web applications.
  7. Feature: Example ! Scenario: Login Given I am on the

    login page When I sign in as “[email protected]” Then I should see “Available Offers” Next_Scenario: Logout ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page Test cases in Gherkin* 1
 2 3 4 5 6 7 8 9 10 11 12
  8. Graph generation Feature: Example ! Scenario: Login Given I am

    on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1
 2 3 4 5 6 7 8 9 10 11 12 Scenario: Login
  9. Graph generation Scenario: Login Scenario: New Donation Feature: Example !

    Scenario: Login Given I am on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1
 2 3 4 5 6 7 8 9 10 11 12
  10. Graph generation Scenario: Login Scenario: Logout Feature: Example ! Scenario:

    Login Given I am on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1
 2 3 4 5 6 7 8 9 10 11 12 Scenario: New Donation
  11. Scenario: Login Scenario: Logout Total number of scenarios: |scenarios| +

    |P| Test case path detections P = {! Login→s1→Logout! Login→s2→Logout Login→s3→Logout Login→s2→s1→Logout Login→s2→s3→Logout
 ! } Scenario: s1 Scenario: s2 Scenario: s3
  12. The AST built by Cucumber* Scenario: Login Scenario: Logout Scenario:

    Login Logout Feature: Example Stepi … … Stepn Stepk … … Stepm {Stepi … … Stepn} ∪ {Stepk … … Stepm}
  13. Results and Remarks Feature: Example ! Scenario: Login Given I

    am on the login page When I sign in as “[email protected]” Then I should see “Available Offers” Next_Scenario: New donation Next_Scenario: Logout ! Scenario: New donation Given I am logged in as “[email protected]” When I publish a new donation Then I should see it in the list of donations Next_Scenario: Logout ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: New donation Given I am logged in as “[email protected]” When I publish a new donation And I sign out Then I should be redirected to the login page ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “[email protected]” And I sign out Then I should be redirected to the login page Scenarios:
 |scenarios| + |P| = 3 + 2 = 5 Scenarios: 4 1
 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
  14. Feature: Example ! Scenario: Login Given I am on the

    login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: New donation Given I am logged in as “[email protected]” When I publish a new donation Then I should see it in the list of donations ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login→New Donation→Logout! ! ! Given I am on the login page! ! ! When I sign in as “[email protected]”! ! ! Then I should see “Available Offers”! ! ! Given I am on the login page! ! ! When I publish a new donation! ! ! Then I should see it in the list of donations! ! ! Given I am on the login page! ! ! When I sign out! ! ! Then I should be redirected to the login page! ! ! Scenario: Login→Logout! ! ! Given I am on the login page! ! ! When I sign in as “[email protected]”! ! ! Then I should see “Available Offers”! ! ! Given I am on the login page! ! ! When I sign out! ! ! Then I should be redirected to the login page Results and Remarks
  15. • A new keyword in the Gherkin’s language. • Three

    new phases in Cucumber:
 • Graph generation • Test case paths detection • Scenario generation Conclusion
  16. • Analyze the performance of Cucumber*. • Find a way

    to limit multiple Given-When-Then sentences in output. • Overcome the limit of infeasible paths in the generated graph. Future work