Test Smarter Not Harder: New Testing Strategies

Automated testing helps you improve app quality in multiple ways. For example, it helps you perform validation, catching regressions, and verifying compatibility. A good testing strategy lets you take advantage of automated testing to focus on an important benefit: developer productivity.

Teams achieve higher levels of productivity when they use a systematic approach to testing paired with infrastructure enhancements. Doing so provides timely feedback on how the code behaves. A good testing strategy does the following:

Catches issues as early as possible.
Executes quickly.
Provides clear indications when something needs to be fixed.
This slide will help you decide what types of tests to implement, where to run them and how often.

Transcript

1. Test Smarter Not Harder : New Testing Strategies Yangon Thaw Zin Toe
2. Why testing strategies matter?
3. Nightmare releasing to Production without testing
4. helps you perform validation, catching regressions, and verifying compatibility Teams achieve higher levels of productivity when they use a systematic approach to testing paired with infrastructure enhancements. App Quality Developer Productivity When the bug is caught by a unit test, it's typically fixed in minutes, so the cost is low. Catch issues Early.
5. Agenda Understanding, the old 3-layer model and its behavior. Unit, Component, Feature, Application and Release Candidate Tests. Testing Pyramid Scalable Testing Strategies Challenge of Top-Heavy Strategy. Decide Test Category How to scale testing for better feedback loops. Test Infrastructure
6. Unit Testing Integration Testing E2E Testing 10 % 20 % 70 % Test Ratios can be difference based on the App Testing Pyramid
7. Scalable Testing Strategies
8. Unit Tests Component Tests Feature Tests Application Tests Release Candidate Tests New Categories of Test
9. Unit Tests Component Tests Feature Tests Application Tests Release candidate Tests Isolation Integration
10. What is Unit Tests? A unit test is • Run on the Host machine • Verifies a single functional unit of logic with no dependencies on the Android framework. #1 Categories of Test Unit Unit Test
11. What is Component Test? A component test is • verifies the functionality or appearance of a module or component independently from other components. • Unlike unit tests, covers more than just single methods or classes. #2 Categories of Test Unit Component Test Unit Unit
12. What is Feature Test? • test verifies the interaction of two or more independent components or modules. • Feature tests are larger and more complex, and usually operate at the feature level. #2 Categories of Test
14. ( 1 ) Feature Tests Start Feature Test: Identify multiple components (e.g., UI, API, navigation). ( 2 ) Mock Dependencies: Mock API responses for Login Scenarios ( 3 ) UI Behavior Tests: Test state changes for invalid and valid inputs.
15. ( 4 ) Feature Tests Verify State Management: Make sure proper error messages and navigation flows. ( 5 ) Debug and Fix issues: Analyze test failure and resolve issues before release.
16. What is Application Test? • Check the app’s overall functionality using a debug build with testing hooks for integration. • Example: UI behavior test to verify configuration changes in a foldable, localization and accessibility tests #2 Categories of Test
18. What is Release Candidate Test? • Release build tests validate the app’s functionality using an optimized, production-like version. • These end-to-end tests ensure everything works as expected in a near-production environment, without exposing real user accounts or live backends. • Example: Critical User Journey and Performance testing #2 Categories of Test
20. Scope Network Access Execution Build type Lifecycle Unit Single method or class with minimal dependencies. No Local Debuggable Pre-merge Component Module or component Level Multiple classes together No Local Robolectric Emulator Debuggable Pre-merge Feature Feature Level Integration with components Mocked Local Robolectric Emulator Devices Debuggable Pre-merge Application Application Level Integration with features and/or services owned by other teams Mocked Staging server Prod server Emulator Devices Debuggable Pre-merge Post-merge Release candidate Application level Integration with features and/or services owned by other teams Prod server Emulator Devices Minified release build Post-merge Pre-release
21. Decide the test category
22. Case Study: Testing a Sign-In Flow • Unit Test: Validate form input. • Component Test: Enable button based on input validity. • Feature Test: Handle login responses. • Application Test: Verify navigation post-login. • Release Candidate Test: Test on real devices.
23. Unit Tests - Sign-In Flow • Email Validation • Password Validation • Error Handling
24. System Design Consideration for Unit Test Functional Requirements • Email Validation: Ensures the email matches the expected regex pattern. • Password Validation: Checks if the password meets complexity requirements. • Error Handling: Ensures validators return meaningful errors. Non-Functional Requirements (Optional) • Security: Ensure no sensitive data leaks during validation. • Performance: Validators must run efficiently for frequent use in UI state updates.
25. Component Tests - Sign-In Flow • the “Login” button is enabled only when both the email and password fields are valid. • the form updates the UI state correctly based on user input and validation logic. • A form following a UX specification
26. System Design Consideration for Component Test Functional Requirements • The “Login” button is disabled when the email or password is invalid. • Show a “Loading…” indicator during authentication. Non-Functional Requirements • Error catch when API was getting wrong. • The UI responds within 100ms to user input changes. • System supports up to 10,000 concurrent logins without degradation.
27. Feature Tests - Sign-In Flow • Integration with the auth manager - The UI that sends credentials to an auth manager and receives responses that can contain different errors.
28. System Design Consideration for Feature Test Functional Requirements • Authenticate user credentials (email and password) using the AuthManager and return appropriate responses (success or error). • Validate email format (e.g., user@example.com) and password rules (e.g., at least 8 characters, not empty) before sending credentials to the AuthManager. Non-Functional Requirements • Load Testing - Use FakeAuthManager in JVM tests to simulate multiple concurrent login requests
29. Application Tests - Sign-In Flow • Sign-in dialog: A screen showing the sign-in form when the login button is pressed. • Profile Update Screen: A screen that allows users to update their profile information, such as name, email, and password, and save changes.
30. System Design Consideration for Application Test Functional Requirements • The sign-in dialog should open when “Login” is pressed and close on success or dismissal for user navigation. • Email and password fields must be validated, enabling the “Login” button only for valid inputs to avoid invalid requests. Non-Functional Requirements • Test Accessibility for error Message when trying to login. • Test Localization when trying to login.
31. Release Candidate Tests • Critical user journey: Signing in A complete sign-in flow using a test account against a staging server and navigate to Home Screen. • Run nightly tests using a minified release build.
32. System Design Consideration for Release Candidate Test Functional Requirements • Validate correct username/password allows login. • Validate UI adjusts for different languages (e.g., Thai, Myanmar, English). • Verify screen reader announces correct labels for elements. Non-Functional Requirements • Measure login response time under different network conditions. • Ensure large translations don’t break layouts. • Test usability with high-contrast and large text modes.
33. Test Infrastructure
34. Unit Tests Environment • Runs on the developer’s local machine. • Use a JVM without any Android framework dependencies. • Eg - Logic validation, off-by-one errors in functions. Trigger • Every commit (Git hooks pre-commit) • Part of the initial CI pipeline to ensure that changes don’t break basic functionality. Key Point: These are fast, isolated tests designed to catch bugs in individual methods or classes.
35. Component Tests Environment • Runs locally on the developer’s machine or in a Robolectric environment. • May use a lightweight emulator to Android components. Trigger • Every commit (Git hooks pre-commit) • Scoped to only the affected module or feature for efficiency. Key Point: Tests at the module or component level to make sure independent components that want as expected.
36. Feature Tests Environment • Executed in local environments, emulators, or mocked environments. • May require access to specific hardware or integrations with other components. Trigger • Pre-merge, before merging or submitting a change. • feature-level changes interact properly with dependent components. Key Point: Validates end-to-end interactions for a feature to catch integration issues early.
37. Feature Tests Environment • Executed in local environments, emulators, or mocked environments. • May require access to specific hardware or integrations with other components. Trigger • Pre-merge, before merging or submitting a change. • feature-level changes interact properly with dependent components. Key Point: Validates end-to-end interactions for a feature to catch integration issues early.
38. Application Tests Environment • Tests run on local machines, emulators, one phone, and one foldable device. • Staging servers may be used for mocking backends or services. Trigger • Post-merge, after changes are integrated into the main branch. • Ensures the application behaves correctly as a whole with the merged changes. Key Point: Full-scale integration tests verifying app-wide behavior, configuration changes, and screen compatibility.
39. Release Candidate Tests Environment • Extensive testing on real devices including: 8 different phones, 1 foldable, and 1 tablet. • Real Production-like environments with release builds. Trigger • Pre-release, during nightly runs or before final production deployment. • Comprehensive testing to ensure stability and compatibility across a wide range of devices. Key Point: Ensures the app meets performance, usability, and compatibility standards in production-ready builds.
40. Thank you for your Concerns. Yangon
41. Thaw Zin Toe (P Tut) Software Engineer at Nimble