Share
Embed
Dealing with Testing Fatigue #mm18pl
Slides from Meet Magento Poland 2018
30 Minutes Presentation. Video will be available later, follow https://twitter.com/meetmagentopl
Fabian Schmengler
September 10, 2018
Transcript
61 slides
-
Fabian Schmengler Developer and partner at integer_net, Germany Testing Magento
since 2011 Magento Master 2017-2018 @fschmengler -
Did you know? 3 / 60 #MM18PL Fabian Schmengler />
Dealing With Testing Fatigue @fschmengler -
What I'm hearing a lot: “ I'd really like to
write more tests but I don't get any practical value from them. It takes too much time, or it is too hard. Therefore I don't enjoy it or stopped doing it. (I totally understand that!) 4 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Another popular opinion: ‘ Integration tests are superior to unit
tests. Do not waste your time with unit tests. (at least they are writing tests!) 5 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
6 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing
Fatigue @fschmengler -
7 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing
Fatigue @fschmengler -
Why all the frustration? 8 / 60 #MM18PL Fabian Schmengler
/> Dealing With Testing Fatigue @fschmengler -
http://xunitpatterns.com/ "Test smells" Project smells Behavior smells Code smells 9
/ 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Why all the frustration? The project smells Buggy tests High
test maintenance cost Production bugs Developers not writing tests 10 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Why all the frustration? The behavior smells Fragile Test Erratic
Test Slow Test Frequent Debugging Manual Intervention 11 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
I've been through it, too... 12 / 60 #MM18PL Fabian
Schmengler /> Dealing With Testing Fatigue @fschmengler -
...but did I give up? 13 / 60 #MM18PL Fabian
Schmengler /> Dealing With Testing Fatigue @fschmengler -
...but did I give up? NOPE There's a way out!
13 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
(It's not easy!) 14 / 60 #MM18PL Fabian Schmengler />
Dealing With Testing Fatigue @fschmengler -
First and foremost: Learn! Testing is a different skill than
coding 15 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Test the right things! No, "everything" is not the answer
16 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Write testable code! If code is hard to test, make
it easy to test 17 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Write testable code Dependency Injection 18 / 60 #MM18PL Fabian
Schmengler /> Dealing With Testing Fatigue @fschmengler -
Write testable code Dependency Inversion 19 / 60 #MM18PL Fabian
Schmengler /> Dealing With Testing Fatigue @fschmengler -
Write testable code Ports & Adapters 20 / 60 #MM18PL
Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Improve test architecture Know your (anti-)patterns 21 / 60 #MM18PL
Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Test phases 1. Setup - set up xture (instantiate objects)
2. Exercise - interact with subject under test (SUT) 3. Verify - assertions 4. Teardown - clean up xture 22 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Fixture setup strategy Prebuilt xture Shared xture Fresh xture 23
/ 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Fixture setup strategy Prebuilt xture e.g. Magento integration test database
Shared xture Fresh xture 24 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Fixture setup strategy Prebuilt xture Shared xture shared by whole
test suite OR shared by test case Fresh xture 25 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Fixture setup strategy Prebuilt xture Shared xture Fresh xture Inline
in test method Implicitly in setUp() 26 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing smell Copy and paste Duplicated logic in test code
dev/tests/integration/testsuite/Magento/Customer/_files/customer.php dev/tests/integration/testsuite/Magento/Customer/_files/two_customers.php dev/tests/integration/testsuite/Magento/Customer/_files/three_customers.php dev/tests/integration/testsuite/Magento/Customer/_files/twenty_one_customers.php 27 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem increasing cost of changes Possible solutions Extract method refactoring
In setup: Creation method for fresh xture Finder method for shared xture In veri cation: Custom assertion method 28 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem increasing cost of changes Possible solutions Extract class refactoring
In setup: Test data builder In veri cation: Custom constraint 29 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing pattern Constraint class IsMeaningOfLife extends \PHPUnit\Framework\Constraint { protected function
matches($other) { return $other === 42; } protected function failureDescription($other) { return sprintf( "%s is not the meaning of life, the universe and everything", $other ); } } 30 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing pattern Constraint Usage $this->assertThat(42, new IsMeaningOfLife()); 31 / 60
#MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing pattern Test data builder Creates xture, reusable across tests
32 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Test data builder Example CustomerBuilder::aCustomer() ->withEmail('[email protected]') ->withCustomAttributes( [ 'my_custom_attribute' =>
42 ] ) ->build() https://github.com/tddwizard/magento2- xtures 33 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing smell Irrelevant information 34 / 60 #MM18PL Fabian Schmengler
/> Dealing With Testing Fatigue @fschmengler -
$customer->setWebsiteId( 1 )->setId( 1 )->setEntityTypeId( 1 )->setAttributeSetId( 1 )->setEmail( '[email protected]'
)->setPassword( 'password' )->setGroupId( 1 )->setStoreId( 1 )->setIsActive( 1 )->setFirstname( 'Firstname' )->setLastname( 'Lastname' )->setDefaultBilling( 1 )->setDefaultShipping( 1 )->setRpToken( '8ed8677e6c79e68b94e61658bd756ea5' )->setRpTokenCreatedAt( date('Y-m-d H:i:s') ); 35 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Hard to read test as documentation Possible solutions Extract
creation methods or assertions with relevant parameters For xtures: Test data builder 36 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing Smell Hard-coded test data public function testGetTranslationFileTimestamp() { $this->fileManagerMock->expects($this->once())
->method('getTranslationFileTimestamp') ->willReturn(1445736974); $this->assertEquals(1445736974, $this->model->getTranslationFileTimestamp()); } 37 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Hard to understand cause and effect Purpose of typical
test data (foo, bar, 1, 42) is unclear Possible solutions Introduce variables or constants 38 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing smell Mystery guest Set up or veri cation relies
on information that's not visible in the test Filename Database record General xture 39 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Mystery Guest Filename public function testBundleImport() { // import data
from CSV file $pathToFile = __DIR__ . '/../../_files/import_bundle.csv'; // ... } 40 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Mystery Guest Database record / General Fixture /** * @magentoAppArea
adminhtml * @magentoDataFixture Magento/Reports/_files/viewed_products.php */ public function testExecute() { $this->dispatch('backend/admin/dashboard/productsViewed/'); $this->assertEquals(200, $this->getResponse()->getHttpResponseCode()); $actual = $this->getResponse()->getBody(); $this->assertContains('Simple Product', $actual); } 41 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Hard to understand cause and effect Risk of somebody
else editing external source without knowing how it's used Possible solutions Fresh xture for each test Test data builders instead of xture scripts Create les in test For shared xture: Accurately named nder methods 42 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Shared Fixture Using attribute $this->assertTrue($this->product->isNew(), 'Product should be new'); Using
nder method $this->assertTrue( $this->getRecentlyAddedProduct()->isNew(), 'Product should be new' ); private function getRecentlyAddedProduct() { return $this->product; } 43 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing smell Eager test $savedStockItem->setQty(null); $savedStockItem->save(); $savedStockItem->setQty(2); $savedStockItem->save(); $this->assertEquals('2.0000', $savedStockItem->load($savedStockItemId)->getQty());
$savedStockItem->setQty(0); $savedStockItem->save(); $this->assertEquals('0.0000', $savedStockItem->load($savedStockItemId)->getQty()); $savedStockItem->setQty(null); $savedStockItem->save(); $this->assertEquals(null, $savedStockItem->load($savedStockItemId)->getQty()); 44 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Hard to tell apart setup, exercise and veri cation
Harder to localize defects Possible solutions Single condition tests 45 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing pattern Given...When...Then... $this->given_stock_item_with_qty(2); $this->when_stock_item_is_changed_to(0); $this->then_loaded_stock_item_qty_should_be('0.0000'); Readable high level test
methods Clear phases: setup (given), exercise (when) and veri cation (then) Convert variables ($savedStockItem) to properties Additional methods can also start with and_ 46 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing smell Excessive Mocking $baseCurrency->expects()->method('getCode')->willReturn('USD'); $baseCurrency->expects()->method('getCurrencySymbol')->willReturn('$'); $baseCurrency->expects()->method('getRate')->with('AUD')->willReturn('0.80'); $store->expects()->method('getBaseCurrency')->willReturn($baseCurrency); $store->expects()->method('getDefaultCurrency')->willReturn($baseCurrency); $store->expects()->method('getAvailableCurrencyCodes')->willReturn(['AUD']);
$this->storeManager->expects()->method('getStore')->willReturn($store); 47 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Coupling test to implementation Test is hard to read
Possible solutions Use real domain objects where possible Refactor production code Use custom stub or fake objects with simpler setup 48 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing pattern Fake object Test speci c implementation without external
dependencies 49 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Fake object interface CustomerSession { public function isLoggedIn() : bool;
} class FakeSession implements CustomerSession { private $loggedIn = false; public function isLoggedIn() : bool { return $this->loggedIn; } // test methods: public function login() { $this->loggedIn = true; } public function logout() { $this->loggedIn = false; } } 50 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Fake object PHPUnit stub: $session = $this->createMock(CustomerSession::class); $session->method('isLoggedIn')->willReturn(true); Custom fake:
$session = new FakeSession(); $session->login(); more succinct reusable accross tests 51 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing Smell Meaningless test names public function testProcessValidStoreCodeCase1() { //
... } public function testProcessValidStoreCodeCase2() { // ... } public function testProcessValidStoreCodeCase3() { // ... } 52 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Failures in test results are unclear Possible solutions Write
meaningful test names Use named keys in data provider arrays return [ 'meaningful description of case 1' => [ ... ], 'meaningful description of case 2' => [ ... ], ] 53 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing Smell Meaningless assertion messages Failed asserting that 0 is
1. Failed asserting that false is true. 54 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Failures in test results are unclear Possible solutions use
the $message parameter in assert methods to add custom message 55 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Testing smell Conditional test logic public function testSend($configValue, $forceSyncMode, $isComment,
$sendingResult) { // ... if (!$isComment) { // ... } // ... if (!$configValue || $forceSyncMode) { // ... if ($sendingResult) { // ... } else { // ... } } else { // ... } } 56 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Problem Harder to understand the test Harder to write the
test correctly Possible solutions Write separate test cases, extract duplicated logic Make sure, tests are deterministic 57 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
More smells Test logic in production General xture Using resources
(DB, HTTP) in unit tests Indirect testing "Inappropiate intimacy" 58 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Conclusion Maintainability and stability of tests can be greatly improved
by taking care of the test architecture. Watch out for these typical problems. Get rid of them. 59 / 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler -
Image sources Head in sand: tropical.pete @Flickr CC-BY-SA 2.0 60
/ 60 #MM18PL Fabian Schmengler /> Dealing With Testing Fatigue @fschmengler