Share
Embed
Gherkin* and Cucumber*
Gherkin* and Cucumber*: a new test case path composition approach to testing Ruby on Rails web applications.
Roberto Zen
July 15, 2014
Transcript
19 slides
-
Gherkin* and Cucumber* A new test case path composition approach
to testing Ruby on Rails web applications Trento, 15 July 2014 Graduant: Roberto Zen Supervisor: Prof. Maurizio Marchese University of Trento Department of Information Engineering and Computer Science Research carried on at ICT4G Bruno Kessler Foundation Co-Supervisor: Prof. Adolfo Villafiorita -
Feature: Example ! Scenario: Login Given I am on the
login page When I sign in as “[email protected]” Then I should see “You are now logged in” Given /^I am on the login page$/ do visit login_page end ! When /^I sign in as ([\w]*@example.com)$/ do |email| fill_in “e-mail” with: email fill_in “password” with: “password” click_on “Sign in” end Step definitions Test cases in Gherkin Feature file 1 2 3 4 5 6 1 2 3 4 5 6 7 8 9 step -
Test cases execution flow Token ::= A | B A
::= aA | B B ::= nil Ragel lexer.rl lexer.rb while true do ! when 0 then … ! if … then goto 9; ! end transition_table.txt + feature file Given /^I am on .. $/ do end ! When /^ I sign in/$ do end The Gherkin gem Output The Cucumber gem Feature step definitions Feature: Login Scenario: Login failed Given I am on .. When I sign in .. Then I should see … parser.rb lexer.scan( transition_table.txt) -
Limits of tests in BDD Feature: Example ! Scenario: Login
Given I am on the login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “[email protected]” And I sign out Then I should be redirected to the login page ! Scenario: A user publishes a new donation Given I am logged in as “[email protected]” When I publish a new donation And I sign out Then I should be redirected to the login page 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 -
Feature: Example ! Scenario: Login Given I am on the
login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “[email protected]” And I sign out Then I should be redirected to the login page ! Scenario: A user publishes a new donation Given I am logged in as “[email protected]” When I publish a new donation And I sign out Then I should be redirected to the login page 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Limits of tests in BDD -
Aim of the thesis Limit the number of duplicate steps
following the DRY principle. Generate automatically new test cases. Improve the validation of web applications. -
Feature: Example ! Scenario: Login Given I am on the
login page When I sign in as “[email protected]” Then I should see “Available Offers” Next_Scenario: Logout ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page Test cases in Gherkin* 1 2 3 4 5 6 7 8 9 10 11 12 -
Graph generation Feature: Example ! Scenario: Login Given I am
on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1 2 3 4 5 6 7 8 9 10 11 12 Scenario: Login -
Graph generation Scenario: Login Scenario: New Donation Feature: Example !
Scenario: Login Given I am on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1 2 3 4 5 6 7 8 9 10 11 12 -
Graph generation Scenario: Login Scenario: Logout Feature: Example ! Scenario:
Login Given I am on the … When I sign in as … Then I should see … Next_Scenario: New Donation Next_Scenario: Logout ! Scenario: Logout Given I am logged in as… When I sign out Then I should be redirected … 1 2 3 4 5 6 7 8 9 10 11 12 Scenario: New Donation -
Test case path detections Scenario: Login Scenario: Logout Scenario: s1
Scenario: s2 Scenario: s3 -
Scenario: Login Scenario: Logout Total number of scenarios: |scenarios| +
|P| Test case path detections P = {! Login→s1→Logout! Login→s2→Logout Login→s3→Logout Login→s2→s1→Logout Login→s2→s3→Logout ! } Scenario: s1 Scenario: s2 Scenario: s3 -
The AST built by Cucumber* Scenario: Login Scenario: Logout Scenario:
Login Logout Feature: Example Stepi … … Stepn Stepk … … Stepm {Stepi … … Stepn} ∪ {Stepk … … Stepm} -
Results and Remarks Feature: Example ! Scenario: Login Given I
am on the login page When I sign in as “[email protected]” Then I should see “Available Offers” Next_Scenario: New donation Next_Scenario: Logout ! Scenario: New donation Given I am logged in as “[email protected]” When I publish a new donation Then I should see it in the list of donations Next_Scenario: Logout ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page Feature: Example ! Scenario: Login Given I am on the login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: New donation Given I am logged in as “[email protected]” When I publish a new donation And I sign out Then I should be redirected to the login page ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login and logout Given I am on the login page When I sign in as “[email protected]” And I sign out Then I should be redirected to the login page Scenarios: |scenarios| + |P| = 3 + 2 = 5 Scenarios: 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 -
Feature: Example ! Scenario: Login Given I am on the
login page When I sign in as “[email protected]” Then I should see “Available Offers” ! Scenario: New donation Given I am logged in as “[email protected]” When I publish a new donation Then I should see it in the list of donations ! Scenario: Logout Given I am logged in as “[email protected]” When I sign out Then I should be redirected to the login page ! Scenario: Login→New Donation→Logout! ! ! Given I am on the login page! ! ! When I sign in as “[email protected]”! ! ! Then I should see “Available Offers”! ! ! Given I am on the login page! ! ! When I publish a new donation! ! ! Then I should see it in the list of donations! ! ! Given I am on the login page! ! ! When I sign out! ! ! Then I should be redirected to the login page! ! ! Scenario: Login→Logout! ! ! Given I am on the login page! ! ! When I sign in as “[email protected]”! ! ! Then I should see “Available Offers”! ! ! Given I am on the login page! ! ! When I sign out! ! ! Then I should be redirected to the login page Results and Remarks -
• A new keyword in the Gherkin’s language. • Three
new phases in Cucumber: • Graph generation • Test case paths detection • Scenario generation Conclusion -
• Analyze the performance of Cucumber*. • Find a way
to limit multiple Given-When-Then sentences in output. • Overcome the limit of infeasible paths in the generated graph. Future work