Apidays Helsinki 2024 - Bridging the Gap Between Backend and Frontend API Testing with K6 by Ayush Goyal, Grafana Labs

Bridging the Gap Between Backend and Frontend API Testing with K6
Ayush Goyal, Senior Software Engineer - Grafana Labs

Apidays Helsinki & North 2024 - Connecting Physical and Digital: Sustainable APIs for the Era of AI, Super and Quantum Computing (May 28 and 29, 2024)

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

Transcript

1. Supercharging Performance Testing: Bridging the Gap Between Backend and Frontend with k6 Ayush Goyal Senior Software Engineer @ Grafana Labs
3. Agenda How we’ll structure our time • What is Performance Testing? • Introduction to Protocol-Based Load Testing • Frontend and browser testing • Bridging Backend and Frontend testing - Hybrid testing • Exploring k6: Features and Capabilities • Benefits of running hybrid tests in Grafana K6 cloud
4. What is Performance Testing? It measures qualitative aspects of a user's experience of a system, such as its responsiveness and reliability.
5. Why should we do performance testing? • Improve user experience. • Prepare for unexpected demand. • Increase confidence in the application. • Assess and optimize infrastructure.
6. What is Load testing?
7. Load testing is the process of putting demand on a system and measuring its response
8. 1. Smoke Testing 2. “Average” load test 3. Stress Testing 4. Soak Testing 5. Spike Testing 6. Breakpoint test Different types of Load testing
9. 1. Smoke Testing • Minimal Load (around 5 VUs or less) • Short duration (few seconds to couple of minutes) • Verify test script does not have any errors • Verify system-under-test is properly operational • Run it first after any change to test script or application
11. 2. “Average” load test • Assess how the system behaves under expected normal conditions. • Typically increases the throughput or VUs gradually and keeps that average load for some time. • Also called day-in-life test or volume test
13. 3. Stress testing • Assesses how the system performs when loads are heavier than usual. • Main difference from the Average Load Test is the higher load. • Load should be higher than what the system experiences on average. • Ideal to run only after successful average load test using the same script
15. 4. Soak Testing • It is another variation of the Average-Load test which focuses on extended periods. • The peak load duration extends several hours or even days. • Should be executed after successfully running smoke and average load test
17. 4. Spike Testing • It verifies system can survive sudden and massive traffic. • Extremely high loads in a shot interval of time. • Generally recommended to execute when the system expects to receive a sudden rush of activity.
19. 5. Breakpoint testing • The aim of this test is to find the system limits • Gradually increase load to identify the capacity limits of the system. • Recommended to run only after system is known to be functioning under all other load types. • Avoid breakpoint tests in elastic cloud environments.
21. Frontend Testing
22. What is Frontend testing? • Verifies application performance on the interface level. • Concerned with the end-user experience of an application, usually involving a browser. • Primarily measures a single user's experience of the system • It has metrics that are distinct from backend performance testing like core web vitals - LCP (Largest Contentful Paint), CLS (Cumulative Layout Shift)
23. Browser testing with K6
24. Browser testing with K6 • Allows automating browser actions for end-to-end web testing • Collects frontend performance metrics as part of your existing k6 tests • Browser level API has rough compatibility with Playwright for easier migration. • Easy to mix browser-level scripts with existing protocol-level scripts to implement a hybrid approach to performance testing.
25. Hybrid Testing
26. Issues with only doing: • Focuses solely on backend without any regard to user experience. • Difficult to add and test complex user flows • Difficult to maintain as the usage grows Backend Testing VS • Expensive and resource intensive to create high amount of load. • Tests only for handful of users, hence does not tell how UI will behave under extreme stress. Frontend Testing
27. Benefits of Hybrid testing Enhance user experience by monitoring browser performance metrics alongside existing protocol metrics. Identify blind spots and errors with browser-based performance testing that can uncover browser-specific issues missed by protocol-level testing. Facilitate cross-team collaboration by enabling developers, test automation engineers, and SDETs to utilize a shared tool for performance testing.
28. What is K6?
29. Tests as Code - Programmable
30. Portable open source load testing tool Same k6 test script for multiple execution modes
31. No NodeJS JAVASCRIPT RUNTIME GOJA GOLANG Script.JS Performant
32. Other testing protocols Other testing cases Extensible k6.io/docs/extensions/getting-started/explore/ Other storage options JAVASCRIPT RUNTIME GOJA GOLANG Script.JS xk6 EXTENSIONS
33. k6 Concepts
34. Configurable options → https://k6.io/docs/using-k6/k6-options/reference/ Options CLI flags ENV variables Script options Default Config file Order of preference
35. Test life cycle / Test data -> https://k6.io/docs/using-k6/test-lifecycle/
36. Built-in Metrics Custom Metrics new Trend(‘metric_name’); new Rate(‘metric_name’); new Counter(‘metric_name’); new Gauge(‘metric_name’);
37. Running a hybrid test with K6 on local and cloud
39. Benefits of running hybrid tests in cloud
40. 1. Test for users from different geographic locations • Easily distribute your virtual users across the world with a simple configuration. • Simulate real world traffic generated from different parts of the world.
41. 2. Easily compare results with previous test runs • Compare test run results to find regressions and fix performance bottlenecks faster • Learn More on test comparison.
42. 3. Custom dashboard from test run results • Using Grafana cloud k6 adds a “K6” datasource to your stack which can be queried to get specific test run metrics to create custom dashboards for better tracking and visualizations on your test runs.
43. 4. No infrastructure maintenance for browsers • Running browser tests requires you to have, either ◦ Beefy machines to support multiple browsers in the same instance, or ◦ Use costly cloud browsers and maintain custom connection logic • With Grafana K6 browser in Cloud, run browser tests in cloud without worrying about the infrastructure changes.
44. Thank you