Lean Test Framework for Web Testing

Automating tests on a web front end is not the most important level in the test automation pyramid, but it is critical from a user perspective. If it is not done with a lean architecture and good use of software engineering practices, we will have a lot of maintenance and abandonment of tests in this layer. A test framework using Selenium WebDriver with the Page Object model; some design patterns; parallel execution for multiple browsers, using containers that can auto-scale and support a large test suite; and putting all this together in a test pipeline may be a good strategy for smashing the bugs and maintaining the high quality of your application without add complexity on the test framework.

The project that contains this architecture tips can be found at https://github.com/eliasnogueira/selenium-java-lean-test-achitecture

Transcript

1. ELIAS NOGUEIRA @eliasnogueira
2. THE STACK LANGUAGE TESTING INFRA DATA GENERATION javafaker LOG & REPORTS Allure
3. BASIC ITEMS FOR A TEST ARCHITECTURE with focus on web automation BASE ARCHITECTURE • Clean Architecture • Design Pattern • Testing Patterns PAGE OBJECT MODEL • Page Objects • Page Factory • Abstraction • Waiting Strategy LOGS AND REPORTS • Exception logs • General reports • Evidence DATA GENERATION • Fake • Static creation • Dynamic creation PARALLEL EXECUTION • Infrastructure • Containers PIPELINE • Execution strategy
4. an abstract class that will take care of commons actions in your automated tests FACTORY the Design Pattern to create, in our case, browser instances LISTENER a non-intrusive way to know what is happening during the test execution BASE TEST BASE ARCHITECTURE to apply DRY and KISS CONFIG a way to manage the configuration files
5. Smart use of inheritance • test inherits common test actions One test case per class • provide an easy way to add more tests • ease division of tests in suites BASE TEST CLASS TEST 1 TEST 2 TEST N • browser initialization/close • open/close database, logs … • connect/disconnect servers • login/logout app BASE TEST CLASS extends
6. Apply Factory Design Pattern will help us to create a browser instance and make easy the parallel execution in many environments. BROWSER FACTORY CHROME FIREFOX EDGE FACTORY CLASS SAFARI
7. Using TestNG we can use some listeners that allow modifying (or just watch) the test behaviors. Helpful o watch the test lifecycle and do something. LISTENERS MY LISTENER • test start • test finish • on test fail • on test skipped • on start • on finish • on success TEST 1 @MyTestListener TEST 2 @MyTestListener TEST N @MyTestListener
8. But the best approach is to add the listener to the Base Test class, so all the tests that extend this class will have the listener. LISTENERS MY LISTENER • test start • test finish • on test fail • on test skipped • on start • on finish • on success TEST 1 TEST 2 TEST N BASE TEST CLASS @MyTestListener
9. There are many ways to create a configuration management with different configuration file types. We must try to avoid: • Call multiple times the file (should be a singleton) • Provide a way to load multiples files based on your requirements CONFIGURATION MANAGEMENT
10. CONFIGURATION MANAGEMENT CONFIG MGMT MY CONFIG 1 MY CONFIG 2 MY CONFIG 3 Should be singleton There are many ways to create a configuration management with different configuration file types. We must try to avoid: • Call multiple times the file (should be a singleton) • Provide a way to load multiples files based on your requirements
11. way to add more readability service class LOAD STRATEGY make the code wait for async executions FLUENT INTERFACE enable you to create tests in a fluent way PAGE FACTORY PAGE OBJECTS MODEL more maintainability and readability
12. PAGE OBJECTS Page Object is a class that serves as an interface to a page of your web page. The class provides methods to do page actions. Tests will use these methods. Image from https://martinfowler.com/bliki/PageObject.html
13. PAGE OBJECTS SEARCH PAGE FLIGHT SELECTION PAGE PAYMENT PAGE Let’s say we have to test an application that sells flight tickets. We would have those 3 pages.
14. PAGE OBJECTS PAGE OBJECT FLIGHT SELECTION PAGE OBJECT SEARCH PAGE OBJECT PAYMENT SEARCH PAGE FLIGHT SELECTION PAGE PAYMENT PAGE We will create a class for each page and add all the interactions we want to do with it. It’s called a Page Object.
15. PAGE OBJECTS PAGE OBJECT FLIGHT SELECTION PAGE OBJECT SEARCH PAGE OBJECT PAYMENT SEARCH PAGE FLIGHT SELECTION PAGE PAYMENT PAGE SUCCESSFUL BOOK TEST For a successful test, we would consume those 3 Page Objects.
16. PAGE OBJECTS PAGE OBJECT FLIGHT SELECTION PAGE OBJECT SEARCH PAGE OBJECT PAYMENT INVALID DATE TEST SEARCH PAGE FLIGHT SELECTION PAGE PAYMENT PAGE For an invalid date test, we would consume only one Page Object.
17. PAGE OBJECTS PAGE OBJECT FLIGHT SELECTION PAGE OBJECT SEARCH PAGE OBJECT PAYMENT SEARCH PAGE FLIGHT SELECTION PAGE PAYMENT PAGE PAYMENT PROBLEM TEST For a payment problem simulation test, we would use those 3 Page Objects.
18. PAGE OBJECTS PAGE OBJECT FLIGHT SELECTION PAGE OBJECT SEARCH PAGE OBJECT PAYMENT SEARCH PAGE FLIGHT SELECTION PAGE PAYMENT PAGE PAYMENT PROBLEM TEST INVALID DATE TEST SUCCESSFUL BOOK TEST CHANGE If the Frontend developer do some changes in the Search Page that affects the tests, we just need to change the Page Object for that page, not the tests.
19. LOAD STRATEGY A Load Strategy is responsible for wait for a certain time by any event on the web page, most of the time related to async requests (Ajax). PAUSE IMPLICITLY any type of sleep that will pause the execution you won’t know, in your code witch action will wait EXPLICITLY AJAX LOCATOR the best choice to use with Page Factory strategy with this strategy, you can see in the code which element will take time
20. FLUENT INTERFACE Creates a method chaining to perform a series of actions to make the code more readable and easier to use. @Test public void testWithoutFluentInterface() { GeneralMenuPage menu = new GeneralMenuPage(); menu.clickinExperience(); menu.clickInOurFleet(); menu.clickInSeatingCharts(); } @Test public void testWithFluentInterface() { GeneralMenuPage menu = new GeneralMenuPage(); menu.clickinExperience().clickInOurFleet().clickInSeatingCharts(); }
21. know all the exceptions to solve the problems root-cause GENERAL REPORTS evidence and executive reports EXCEPTION LOGS LOGS AND REPORTS because we need to know about any error
22. By using any log strategy, saving a log file, we can understand the common errors that occurred during the test execution. These errors can be of: • assertion errors • timeout exceptions • locator exception • an exception on your architecture EXCEPTION LOGS If you want to analyze test errors across teams a good way is using Elasticsearch with Grafana/Kibana or using Report Portal.
23. GENERAL REPORTS Generate xUnit reports to attach on your CI/CD and, rapidly, see the test status.
24. GENERAL REPORTS Create an executive report to provide information and evidence about the test execution. This report may contain screenshots when an error occurs to help to analyze the root cause of a problem.
25. pass the responsibility of non- sensitive data generation to a framework STATIC/DYNAMIC GENERATION create sensitive data and put under your control FAKES DATA GENERATION solve one of the biggest problems
26. Ability to create an approach to generate non-sensitive data for your test without the necessity to manually change the test data in each execution. There’re a lot of tools to create this type of data. FAKE GENERATION Example with javafaker Faker faker = new Faker(new Locale("pt-BR")); faker.name().fullName(); faker.address().fullAddress(); faker.internet().emailAddress(); faker.business().creditCardNumber(); faker.date().birthday();
27. When the data cause different behaviors in your application. STATIC / DYNAMIC GENERATION A Static approach can be implemented with any kind of solution, like: • Files • CSV | JSON | TXT | YML • Database • Mock A Dynamic approach can be created according to your context. Used to remove the maintenance of test data • Queries in a database • Consume data from a static poll
28. run many tests at the same time in a chosen target GRID AND AUTO-SCALE using the proper containers, we can speed up the test execution PARALLELISM PARALLEL EXECUTION to speed up your test execution
29. maven-surefire- plugin Have an ability to control how many threads we need inside the pom.xml Junit Experimental classes in JUnit 4 and 5 ParallelComputer and ParallelExecution ConfigurationStrategy TestNG Control the parallelism thought the suites in any level of tests (class, methods, etc..) Parallelism, under test, is the ability to perform the same test in different conditions (browser, devices, etc...) or different tests at the same time. PARALLELISM
30. GRID SCHEMA Node Windows Node MacOSX Node Linux Test Script Hub send capabilities understands the capabilities and send to the proper node
31. WAYS TO CREATE A GRID LOCAL Uses machines inside an infrastructure. Can be a bare-metal desktop or a virtual machine CLOUD Uses a cloud infrastructure platform to create virtual machines CONTAINERS Uses containers (locally or cloud-based) to create the infrastructure and support orchestration
32. CONTAINERS TO AUTO-SCALE • Uses a custom container elgalu/selenium that provides: • live Preview with VNC • video recording • dashboard • automatic auto-scale containers based on the number of tests CURRENTLY NOT BEING DEVELOPED ANYMORE IN ORDER TO SELENIUM GRID 4 ADOPTION
33. create a pipeline for any type of test execution DIVIDE ALL TYPES OF EXECUTION PIPELINE make the execution process clear
34. FUNCTIONAL TEST ACCEPTANCE TEST SMOKE TEST DIVIDE ALL TYPES OF EXECUTION WEB PART IN THE PIPELINE Most important tests from a business perspective Most used user scenarios Assure that critical functionalities works
35. THANK YOU! https://github.com/eliasnogueira/public-speaking