How To Manage 5000+ Tests Efficiently

Facing the challenges of testing large-scale applications isn't an easy task. In this talk, we will explore the distinct differences between testing smaller and larger software, and why standard approaches often fall short when confronted with 4000+ tests in an application.

We will delve into a real-life case study, illuminating the transformation journey of a heavy, slow-moving test suite into a high-speed, efficient system. This discussion will highlight the strategic shifts, practical techniques, and insightful methodologies employed and the hurdles overcome along the way.

Ultimately, participants will leave with an enhanced understanding of managing and scaling test suites in large application environments. Together, we will reveal how, with the proper perspective and tools, we can make our test suites not just manageable but highly efficient, driving the success of our large-scale software projects.

Transcript

1. TESTING IS
2. A JOURNEY TESTING IS
3. I started writing tests in 2014.
4. It was hard!
5. It was hard! What? Where? When? Why?
6. It takes time. A lot of time. Years.
7. Today I enjoy writing tests.
8. TEACHING TESTING HAS BECOME BETTER
9. TEACHING TESTING HAS BECOME BETTER
10. TEACHING TESTING HAS BECOME BETTER
15. Couple hundred tests are always fast.
16. Managing 5000+ Tests Efficiently
17. Christoph Rumpel @christophrumpel
19. 5370 Tests
20. 27869 Assertions
21. 50 Mins
22. Tests Lots of tests We can grow fast POSITIVE
23. Couldn't run whole test suite locally Relied on GH actions Performance was already bad NEGATIVE
24. Crappy developers? Bad setup? Project grew too fast? Part of the journey WHY ❌ ❌ ✅ ✅
25. PERFORMANCE IS
26. A JOURNEY PERFORMANCE IS
27. 50 Mins
28. SETUP TESTS
29. Database Migrations Sqlite MySQL Other Refresh Database DB Transactions SETUP Seeders Yes No
30. OUR SETUP Sqlite MySQL Other Database Migrations Refresh Database DB Transactions Seeders Yes No
31. PREPARE COMMAND
32. Set DB table names in config Run Landlord migrations Run Tenant migrations Run Test Playbooks PREPARE COMMAND
33. THE BIGGEST BOTTLENECK WAS ALRADY DEALT WITH
34. THE BIGGEST BOTTLENECK WAS ALRADY DEALT WITH #
35. SINGLE TESTS
38. Before
39. After
44. Before After
45. Before After
46. Before After
47. Before After sec 2
48. After
49. After
50. 45 Mins
51. THE HOLY GRAIL OF PERFORMANCE
52. Pretty easy these days. PARALLEL TESTING
53. BUT IT DEPENDS
54. SOMETIMES PARALLEL TESTING
55. IS A JOURNEY SOMETIMES PARALLEL TESTING
57. Failing Tests 122
58. PARALLEL ISSUES
59. PARALLEL ISSUES
60. PARALLEL ISSUES AppServiceProvider
61. PARALLEL ISSUES
63. 5 Mins
64. WHATS NEXT? GitHub Actions with Parallel Include Prepare Command Dump Schema
65. Managing 5000+ Tests Efficiently
66. Managing 5000+ Tests Write tests Improve the setup Parallel Testing Don't forget single tests Re-Check Performance
67. THANK YOU @christophrumpel