Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Behat for Characterization Tests

Michael Heap
November 15, 2017

Behat for Characterization Tests

Once an API ships it doesn't matter how it should behave—how it actually behaves is the important part. Users depend on the existing behavior, and we need a way to ensure it doesn't change. Behat is a tool that was built to help design software, but it’s actually a great tool for capturing existing behavior too. We’ve used these tools to gain confidence to refactor 5+-year-old apps by capturing the existing behavior before making changes. I want to share the secrets we learned with you.

Michael Heap

November 15, 2017
Tweet

More Decks by Michael Heap

Other Decks in Programming

Transcript

  1. @mheap #phpworld $$$ $$$ $$$ Billing Rating Auth Report Masking

    Encoding public function provides() { return [Client ::class]; }
  2. @mheap https://twitter.com/brianwisti/status/503987766032494592 YOU ARE IN A LEGACY CODEBASE > RUN

    TESTS YOU HAVE NO TESTS > READ SPEC YOU HAVE NO SPEC > WRITE FIX YOU ARE EATEN BY AN ELDER CODE HACK.
  3. @mheap Legacy Code is “valuable code that we feel afraid

    to change” JB Rainsberger - Surviving Legacy Code with Golden Master and Sampling
  4. @mheap #phpworld class SalesUtil { const BQ = 1000.0; const

    BCR = 0.20; const OQM1 = 1.5; const OQM2 = self ::OQM1 * 2; static public function calculate($tSales) { if ($tSales <= static ::BQ) { return $tSales * static ::BCR; } else if ($tSales <= static ::BQ*2) { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1; } else { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1 + ($tSales - static ::BQ * 2) * static ::BCR * static ::OQM2; } } }
  5. @mheap #phpworld class SalesUtil { const BQ = 1000.0; const

    BCR = 0.20; const OQM1 = 1.5; const OQM2 = self ::OQM1 * 2; static public function calculate($tSales) { if ($tSales <= static ::BQ) { return $tSales * static ::BCR; } else if ($tSales <= static ::BQ*2) { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1; } else { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1 + ($tSales - static ::BQ * 2) * static ::BCR * static ::OQM2; } } }
  6. @mheap #phpworld class SalesUtil { const BQ = 1000.0; const

    BCR = 0.20; const OQM1 = 1.5; const OQM2 = self ::OQM1 * 2; static public function calculate($tSales) { if ($tSales <= static ::BQ) { return $tSales * static ::BCR; } else if ($tSales <= static ::BQ*2) { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1; } else { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1 + ($tSales - static ::BQ * 2) * static ::BCR * static ::OQM2; } } }
  7. @mheap #phpworld class SalesUtil { const BQ = 1000.0; const

    BCR = 0.20; const OQM1 = 1.5; const OQM2 = self ::OQM1 * 2; static public function calculate($tSales) { if ($tSales <= static ::BQ) { return $tSales * static ::BCR; } else if ($tSales <= static ::BQ*2) { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1; } else { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1 + ($tSales - static ::BQ * 2) * static ::BCR * static ::OQM2; } } }
  8. @mheap #phpworld use PHPUnit\Framework\TestCase; class SalesUtilTest extends TestCase { public

    function testX() { $this ->assertEquals(null, SalesUtil ::calculate(1000)); } }
  9. @mheap #phpworld use PHPUnit\Framework\TestCase; class SalesUtilTest extends TestCase { public

    function testX() { $this ->assertEquals(null, SalesUtil ::calculate(999)); } }
  10. @mheap #phpworld use PHPUnit\Framework\TestCase; class SalesUtilTest extends TestCase { public

    function testX() { $this ->assertEquals(null, SalesUtil ::calculate(1000)); } }
  11. @mheap #phpworld There was 1 failure: 1) SalesUtilTest ::testX Failed

    asserting that 200 matches expected null. /Users/michael/development/oss/characterisation-tests-examples/sales-util/test/SalesUtilTest.php:8 FAILURES! Tests: 1, Assertions: 1, Failures: 1.
  12. @mheap #phpworld use PHPUnit\Framework\TestCase; class SalesUtilTest extends TestCase { public

    function testSalesLTE1000Pay20PercentCommission() { $this ->assertEquals(200, SalesUtil ::calculate(1000)); } }
  13. @mheap #phpworld class SalesUtil { const BQ = 1000.0; const

    BCR = 0.20; const OQM1 = 1.5; const OQM2 = self ::OQM1 * 2; static public function calculate($tSales) { if ($tSales <= static ::BQ) { return $tSales * static ::BCR; } else if ($tSales <= static ::BQ*2) { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1; } else { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1 + ($tSales - static ::BQ * 2) * static ::BCR * static ::OQM2; } } }
  14. @mheap #phpworld There was 1 failure: 1) SalesUtilTest ::testX Failed

    asserting that 500.0 matches expected null. /Users/michael/development/oss/characterisation-tests-examples/sales-util/test/SalesUtilTest.php:13 FAILURES! Tests: 2, Assertions: 2, Failures: 1.
  15. @mheap #phpworld public function test2000Gets500Commission() { $this ->assertEquals(500, SalesUtil ::calculate(2000));

    } public function testX() { $this ->assertEquals(null, SalesUtil ::calculate(1600)); }
  16. @mheap #phpworld There was 1 failure: 1) SalesUtilTest ::testX Failed

    asserting that 380.0 matches expected null. /Users/michael/development/oss/characterisation-tests-examples/sales-util/test/SalesUtilTest.php:19 FAILURES! Tests: 3, Assertions: 3, Failures: 1.
  17. @mheap #phpworld class SalesUtil { const BQ = 1000.0; const

    BCR = 0.20; const OQM1 = 1.5; const OQM2 = self ::OQM1 * 2; static public function calculate($tSales) { if ($tSales <= static ::BQ) { return $tSales * static ::BCR; } else if ($tSales <= static ::BQ*2) { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1; } else { return static ::BQ * static ::BCR + ($tSales - static ::BQ) * static ::BCR * static ::OQM1 + ($tSales - static ::BQ * 2) * static ::BCR * static ::OQM2; } } }
  18. @mheap #phpworld Input Expected 0 0 10 2 500 100

    1000 200 1001 200.3 1500 350 1982 494.6 1999.9 499.97 2000 500 2001 500.9 5000 3200 7890 5801 10000 7700
  19. @mheap #phpworld Given I have 1 "Red Bucket" in my

    basket And I have 2 "Large Spades" in my basket When I add 1 "Red Bucket" in my basket Then I should see 2 "Red Buckets" in my basket And I should see 2 "Large Spades" in my basket
  20. @mheap #phpworld Given I have 1 "Red Bucket" in my

    basket And I have 2 "Large Spades" in my basket When I add 1 "Red Bucket" in my basket Then I should see 2 "Red Buckets" in my basket And I should see 2 "Large Spades" in my basket
  21. @mheap #phpworld Given I have 1 "Red Bucket" in my

    basket And I have 2 "Large Spades" in my basket When I add 1 "Red Bucket" in my basket Then I should see 2 "Red Buckets" in my basket And I should see 2 "Large Spades" in my basket
  22. @mheap #phpworld Given I have 1 "Red Bucket" in my

    basket And I have 2 "Large Spades" in my basket When I add 1 "Red Bucket" in my basket Then I should see 2 "Red Buckets" in my basket And I should see 2 "Large Spades" in my basket
  23. @mheap #phpworld Given I have 1 "Red Bucket" in my

    basket And I have 2 "Large Spades" in my basket When I add 1 "Red Bucket" in my basket Then I should see 2 "Red Buckets" in my basket And I should see 2 "Large Spades" in my basket
  24. @mheap #phpworld $username = $_GET['username'] ?? error('username is required'); $key

    = $_GET['key'] ?? error('key is required'); $validUsers = [ "michael" => "kangar00s", "oscar" => "phparch17" ]; $validKey = $validUsers[strtolower($username)] ?? error('could not find user', 404); if ($key !== $validKey){ error('invalid apikey'); } output([ 'success' => 'true', 'user' => ['name' => $username] ]);
  25. @mheap #phpworld $username = $_GET['username'] ?? error('username is required'); $key

    = $_GET['key'] ?? error('key is required'); $validUsers = [ "michael" => "kangar00s", "oscar" => "phparch17" ]; $validKey = $validUsers[strtolower($username)] ?? error('could not find user', 404); if ($key !== $validKey){ error('invalid apikey'); } output([ 'success' => 'true', 'user' => ['name' => $username] ]);
  26. @mheap #phpworld $username = $_GET['username'] ?? error('username is required'); $key

    = $_GET['key'] ?? error('key is required'); $validUsers = [ "michael" => "kangar00s", "oscar" => "phparch17" ]; $validKey = $validUsers[strtolower($username)] ?? error('could not find user', 404); if ($key !== $validKey){ error('invalid apikey'); } output([ 'success' => 'true', 'user' => ['name' => $username] ]);
  27. @mheap #phpworld error('username is required'); error('key is required'); error('could not

    find user', 404); error('invalid apikey'); output([ 'success' => 'true', 'user' => ['name' => $username] ]); 1. Missing username 2. Missing key 3. Invalid username 4. Invalid key 5. Successful authentication
  28. @mheap #phpworld Scenario: No Username When I make a "GET"

    request to "/" Then the response status code should be "400" And the "error" property equals "username is required"
  29. @mheap #phpworld Scenario: No API Key When I make a

    "GET" request to "/?username=foo" Then the response status code should be "400" And the "error" property equals "key is required”
  30. @mheap #phpworld Scenario: Invalid Username When I make a "GET"

    request to "/?username=bananas&key=foo" Then the response status code should be "404" And the "error" property equals "could not find user” Scenario: Invalid API Key When I make a "GET" request to "/?username=michael&key=foo" Then the response status code should be "400" And the "error" property equals "invalid apikey"
  31. @mheap #phpworld Scenario: Successful login When I make a "GET"

    request to "/?username=michael&key=kangar00s" Then the response status code should be "200" And the "success" property equals "true" And the "user.name" property equals "michael”
  32. @mheap #phpworld Scenario Outline: Successful login When I make a

    "GET" request to "/?username=<name>&key=<key>" Then the response status code should be "200" And the "success" property equals "true" And the "user.name" property equals "<name>" Examples: | name | key | | michael | kangar00s | | oscar | phparch16 |
  33. @mheap #phpworld Given that header property "X-CustomAuthKey" is "Secret123" And

    that header property "Accept" is "application/json+vnd.v2" And that the request body is valid JSON ''' { "alpha":"beta", "count":3, "collection":["a","b","c"] } ''' When I make a "POST" request to "/account/balance" Then the response status code should be "200" And the "X-Remaining" header property equals "18" And the "balance.usd" property equals"1832.54" And the "balance.gbp" property equals "13.99" And the "balance.eur" property equals "19422.18"
  34. @mheap #phpworld When I make a "POST" request to "/account/balance"

    Then the response status code should be “200" And the "X-Remaining" header property equals "18" And the response body contains the JSON data ''' { "in_credit": true, "balance": { "usd": 1832.54, "gbp": 13.99, "eur": 19422.18 } } '''
  35. @mheap #phpworld When I make a "POST" request to "/account/balance"

    Then the response status code should be "200" And the "X-Remaining" header property equals "18" And the value of the "balance.usd" property matches the pattern “/^\d{1,3}\. \d{2}$/" And the value of the “last_updated” property matches the pattern "/^[0-9]{4} [\-][0-9]{2}[\-][0-9]{2} [0-9]{2}[:][0-9]{2}[:][0-9]{2}$/"
  36. @mheap #phpworld # behat.yml default: extensions: DataSift\BehatExtension: base_url: "http: //localhost:8080/"

    suites: default: contexts: - 'DataSift\BehatExtension\Context\RestContext'
  37. @mheap #phpworld function get_balance($username) { return $client ->get( “http: //localhost:88/billing/check_balance/".$username

    ) ->getBody(); } $balance = get_balance($username); if ($username != 'michael') { error('wrong username'); } if (!$balance['has_credit']) { error('no credit remaining'); } success(['data' => 'valid account'])
  38. @mheap #phpworld Given Mountebank is running And a mock exists

    at "/billing/check_balance" it should return "200" with the body: ''' { "has_credit": false, "credit_limit": 0 } ''' And the mocks are created When I make a "GET" request to "/auth?username=michael" Then the response status code should be "403" And the response is JSON And the response body JSON equals ''' { "error": "no credit remaining" } '''
  39. @mheap #phpworld Given Mountebank is running And a mock exists

    at "/billing/check_balance" it should return "200" with the body: ''' { "has_credit": true, "credit_limit": 10000 } ''' And the mocks are created When I make a "GET" request to "/auth?username=michael" Then the response status code should be "200" And the response is JSON And the response body JSON equals ''' { "data": "valid account" } '''
  40. @mheap #phpworld $ composer require --dev behat/mink-extension $ composer require

    --dev behat/mink-selenium2-driver $ java -Dwebdriver.chrome.driver=chromedriver -jar selenium- server-standalone-3.7.1.jar
  41. @mheap #phpworld # behat.yml default: extensions: Behat\MinkExtension: base_url: 'https: //wikipedia.org'

    sessions: default: selenium2: browser: "chrome" suites: my_suite: contexts: - \Behat\MinkExtension\Context\MinkContext
  42. @mheap #phpworld Scenario: Searching for a page that does exist

    Given I am on "/wiki/Main_Page" When I fill in "search" with "Behavior Driven Development" And I press "searchButton" Then I should see "agile software development"
  43. @mheap #phpworld Scenario: Searching for a page that does exist

    Given I am on "/wiki/Main_Page" When I fill in "search" with "Behavior Driven Development" And I press "searchButton" Then I should see "agile software development"
  44. @mheap #phpworld Scenario: Searching for a page that does exist

    Given I am on "/wiki/Main_Page" When I search for "Behavior Driven Development" Then I should see "agile software development"
  45. @mheap #phpworld Scenario: Searching for a page that does exist

    Given I am on "/wiki/Main_Page" When I search for "Behavior Driven Development" Then I should see "agile software development" 1 scenario (1 undefined) 3 steps (1 passed, 1 undefined, 1 skipped) 0m3.63s (10.62Mb) >> my_suite suite has undefined steps. Please choose the context to generate snippets: [0] None [1] FeatureContext [2] Behat\MinkExtension\Context\MinkContext
  46. @mheap #phpworld [0] None [1] FeatureContext [2] Behat\MinkExtension\Context\MinkContext > 1

    --- FeatureContext has missing steps. Define them with these snippets: /** * @When I search for :arg1 */ public function iSearchFor($arg1) { throw new PendingException(); }
  47. @mheap #phpworld [0] None [1] FeatureContext [2] Behat\MinkExtension\Context\MinkContext > 1

    u features/bootstrap/FeatureContext.php - `I search for "Behavior Driven Development"` definition added
  48. @mheap #phpworld Feature: Test Scenario: Searching for a page that

    does exist Given I am on "/wiki/Main_Page" When I search for "Behavior Driven Development" TODO: write pending definition Then I should see "agile software development" 1 scenario (1 pending) 3 steps (1 passed, 1 pending, 1 skipped) 0m2.23s (10.61Mb)
  49. @mheap #phpworld /** @BeforeScenario */ public function gatherContexts( Behat\Behat\Hook\Scope\BeforeScenarioScope $scope

    ) { $environment = $scope ->getEnvironment(); $this ->minkContext = $environment ->getContext( 'Behat\MinkExtension\Context\MinkContext' ); }
  50. @mheap #phpworld /** * Fills in form field with specified

    id|name|label|value * Example: When I fill in "username" with: "bwayne" * Example: And I fill in "bwayne" for "username" * * @When /^( ?:|I )fill in "(?P<field>( ?:[^"]| \\")*)" with "(?P<value>( ?:[^"]| \\")*)"$/ * @When /^( ?:|I )fill in "(?P<field>( ?:[^"]| \\")*)" with:$/ * @When /^( ?:|I )fill in "(?P<value>( ?:[^"]| \\")*)" for "(?P<field>( ?:[^"]| \\")*)"$/ */ public function fillField($field, $value) { $field = $this ->fixStepArgument($field); $value = $this ->fixStepArgument($value); $this ->getSession() ->getPage() ->fillField($field, $value); }
  51. @mheap #phpworld /** * Presses button with specified id|name|title|alt|value *

    Example: When I press "Log In" * Example: And I press "Log In" * @When /^( ?:|I )press "(?P<button>( ?:[^"]| \\")*)"$/ */ public function pressButton($button) { $button = $this ->fixStepArgument($button); $this ->getSession() ->getPage() ->pressButton($button); }
  52. @mheap #phpworld /** * @When I search for :term */

    public function iSearchFor($term) { $this ->minkContext ->fillField('search', $term); $this ->minkContext ->pressButton('searchButton'); }
  53. @mheap #phpworld $ ./vendor/bin/behat Feature: Test Scenario: Searching for a

    page that does exist Given I am on "/wiki/Main_Page" When I search for "Behavior Driven Development" Then I should see "agile software development" 1 scenario (1 passed) 3 steps (3 passed)
  54. @mheap #phpworld Scenario: Successful login When I make a "GET"

    request to "/?username=michael&key=kangar00s" Then the response status code should be "200" And the "success" property equals "true" And the "user.name" property equals "michael”
  55. @mheap #phpworld Scenario: Successful login When I log in as

    "Michael" with the password "kangar00s" Then the response status code should be "200" And the "success" property equals "true" And the "user.name" property equals "michael”
  56. @mheap #phpworld public function toJson(){ return json_encode(["id" => $this ->id]);

    } —— public function test_json_formatting_is_correct() { $order = new Order(1); $this ->assertMatchesSnapshot($order ->toJson()); }
  57. @mheap #phpworld $ ./vendor/bin/phpunit There was 1 incomplete test: 1)

    ExampleTest ::test_json_formatting_is_correct Snapshot created for ExampleTest __test_json_formatting_is_correct
  58. @mheap #phpworld $ ./vendor/bin/phpunit 1) ExampleTest ::test_json_formatting_is_correct Failed asserting that

    two strings are equal. --- Expected +++ Actual @@ @@ -'{"id": 1}' +'{"id": "Q1"}' FAILURES! Tests: 1, Assertions: 1, Failures: 1.