Moving Forward with JUnit 5

DroidKaigi 2018, Tokyo, Japan

Transcript

1. Moving Forward with JUnit 5 Marcel Schnelle @marcelschnelle TenTen Ltd.
2. Everybody knows JUnit.
3. Everybody knows JUnit. class FooTest { @Test fun someTest() { assertEquals(4, 2 + 2) } }
5. dependencies { testImplementation "junit:junit:4.12" } Released in December 2014! (JDK 8: March 2014)
6. July 2015
7. July 2015
8. Rewrite: Why?
9. Two types of limitation inside the current JUnit 4 codebase Use-Site Coupling Ambiguous Evolution
10. Use-Site Coupling Today's tooling depends on JUnit Internals
11. Use-Site Coupling Today's tooling depends on JUnit Internals class FooTest { @Test fun someTest() { assertEquals(5, 2 + 2) } }
12. class FooTest { @Test fun someTest() { assertEquals(5, 2 + 2) } } java.lang.AssertionError: expected:<5> but was:<4> Expected :5 Actual :4 <Click to see difference> Use-Site Coupling Today's tooling depends on JUnit Internals
13. // junit.framework.ComparisonFailure.java public class ComparisonFailure extends AssertionFailedError { private String fExpected; private String fActual; public ComparisonFailure (String message, String expected, String actual) { super (message); fExpected = expected; fActual = actual; } } Use-Site Coupling Today's tooling depends on JUnit Internals
14. // junit.framework.ComparisonFailure.java public class ComparisonFailure extends AssertionFailedError { private String fExpected; private String fActual; public ComparisonFailure (String message, String expected, String actual) { super (message); fExpected = expected; fActual = actual; } } private String fExpected; private String fActual; fExpected = expected; fActual = actual; Use-Site Coupling Today's tooling depends on JUnit Internals
15. // junit.framework.ComparisonFailure.java public class ComparisonFailure extends AssertionFailedError { private String expected; private String actual; public ComparisonFailure (String message, String expected, String actual) { super (message); this.expected = expected; this.actual = actual; } } private String expected; private String actual; this.expected = expected; this.actual = actual; Use-Site Coupling Today's tooling depends on JUnit Internals
16. // junit.framework.ComparisonFailure.java public class ComparisonFailure extends AssertionFailedError { private String expected; private String actual; public ComparisonFailure (String message, String expected, String actual) { super (message); this.expected = expected; this.actual = actual; } } Use-Site Coupling Today's tooling depends on JUnit Internals
17. // junit.framework.ComparisonFailure.java public class ComparisonFailure extends AssertionFailedError { private String expected; private String actual; public ComparisonFailure (String message, String expected, String actual) { super (message); this.expected = expected; this.actual = actual; } } // Compile local JUnit $ mvn install ... Use-Site Coupling Today's tooling depends on JUnit Internals
18. class FooTest { @Test fun someTest() { assertEquals(5, 2 + 2) } } Use-Site Coupling Today's tooling depends on JUnit Internals
19. class FooTest { @Test fun someTest() { assertEquals(5, 2 + 2) } } java.lang.AssertionError: expected:<null> but was:<null> Expected :null Actual :null <Click to see difference> Use-Site Coupling Today's tooling depends on JUnit Internals
20. Two types of limitation inside the current JUnit 4 codebase Use-Site Coupling Ambiguous Evolution
21. Ambiguous Evolution Conflicting requirements reduced clarity
22. @RunWith(Parameterized::class) class BarTest { // ... } Ambiguous Evolution Conflicting requirements reduced clarity
23. @RunWith(Parameterized::class) class BarTest { @Mock lateinit var user: User // ... } @RunWith(Parameterized::class) class BarTest { @Mock lateinit var user: User // ... } Ambiguous Evolution Conflicting requirements reduced clarity
24. @RunWith(MockitoJUnitRunner::class) @RunWith(Parameterized::class) class BarTest { @Mock lateinit var user: User // ... } @RunWith(MockitoJUnitRunner::class) @RunWith(Parameterized::class) class BarTest { } Ambiguous Evolution Conflicting requirements reduced clarity
25. // @RunWith(MockitoJUnitRunner::class) @RunWith(Parameterized::class) class BarTest { @Mock lateinit var user: User @Rule val mockitoRule = MockitoJUnitRule.rule() // ... } // @RunWith(MockitoJUnitRunner::class) @RunWith(Parameterized::class) class BarTest { @Rule val mockitoRule = MockitoJUnitRule.rule() } Ambiguous Evolution Conflicting requirements reduced clarity
26. // @RunWith(MockitoJUnitRunner::class) @RunWith(Parameterized::class) class BarTest { @Mock lateinit var user: User @Rule val mockitoRule = MockitoJUnitRule.rule() // ... } Runner vs Rule: Two concepts for test extension Ambiguous Evolution Conflicting requirements reduced clarity
28. JUnit 5 = Platform + Jupiter + Vintage
29. JUnit 5 = Platform + Jupiter + Vintage junit-platform-engine junit-platform-launcher
30. Platform • Dedicated access point for plugin & build tools developers • TestEngine interface for implementors
 (e.g. Spock, Spek, Cucumber) • Launcher API to kick off test runs
31. JUnit 5 = Platform + Jupiter + Vintage junit-jupiter-api junit-jupiter-engine junit-platform-engine junit-platform-launcher
32. Jupiter • New Programming Model for JUnit tests • Annotations & Extension Point API • JupiterTestEngine
33. JUnit 5 = Platform + Jupiter + Vintage junit-jupiter-api junit-jupiter-engine junit-vintage-engine junit-platform-engine junit-platform-launcher
34. Vintage • Backwards Compatibility layer • Execution of JUnit 3 & 4 @Test classes
35. Your tests junit-jupiter-api junit-4.12 <other framework> junit-vintage-engine junit-jupiter-engine <other engine> junit-platform-engine junit-platform-launcher Android Studio, Gradle, etc. written with uses finds & executes implements discovers & runs
36. API Overview
37. JUnit 4 JUnit Jupiter @org.junit.Test → @org.junit.jupiter.api.Test @Ignore → @Disabled @Category(Class) → @Tag(String) Comparisons with JUnit 4: Basic Annotations
38. Comparisons with JUnit 4: Lifecycle Annotations JUnit 4 JUnit Jupiter @BeforeClass → @BeforeAll @Before → @BeforeEach @After → @AfterEach @AfterClass → @AfterAll
39. JUnit Jupiter @Nested @TestFactory @ParameterizedTest & <Source> @ExtendWith(Extension) Comparisons with JUnit 4: New Annotations @DisplayName(String)
40. Comparisons with JUnit 4: New Annotations @DisplayName(String) class FooTest { @Test fun checkIfSomethingWorksEvenIfItShouldnt() { // ... } }
41. Comparisons with JUnit 4: New Annotations @DisplayName(String) class FooTest { @Test fun checkIfSomethingWorksEvenIfItShouldnt() { // ... } }
42. Comparisons with JUnit 4: New Annotations @DisplayName(String) class FooTest { @DisplayName("Something works, even if it shouldn't") @Test fun checkIfSomethingWorksEvenIfItShouldnt() { // ... } }
43. Comparisons with JUnit 4: New Annotations @DisplayName(String) class FooTest { @DisplayName("Something works, even if it shouldn't") @Test fun checkIfSomethingWorksEvenIfItShouldnt() { // ... } }
44. Comparisons with JUnit 4: New Annotations @DisplayName(String) @DisplayName("Unexpected Non-Issue Tests") class FooTest { @DisplayName("Something works, even if it shouldn't") @Test fun checkIfSomethingWorksEvenIfItShouldnt() { // ... } }
45. Comparisons with JUnit 4: New Annotations @DisplayName(String) @DisplayName("Unexpected Non-Issue Tests") class FooTest { @DisplayName("Something works, even if it shouldn't") @Test fun checkIfSomethingWorksEvenIfItShouldnt() { // ... } }
46. JUnit Jupiter @DisplayName(String) @TestFactory @ParameterizedTest & <Source> @ExtendWith(Extension) Comparisons with JUnit 4: New Annotations @Nested
47. @Nested Comparisons with JUnit 4: New Annotations @DisplayName("Given a Calculator") class CalculatorTests { }
48. @Nested Comparisons with JUnit 4: New Annotations @DisplayName("Given a Calculator") class CalculatorTests { lateinit var calculator: Calculator @BeforeEach fun beforeEach() { this.calculator = Calculator() } }
49. @Nested Comparisons with JUnit 4: New Annotations @DisplayName("Given a Calculator") class CalculatorTests { lateinit var calculator: Calculator @BeforeEach fun beforeEach() { this.calculator = Calculator() } @Nested @DisplayName("When using 'Plus' Operator") inner class PlusOperator { } }
50. @Nested Comparisons with JUnit 4: New Annotations @DisplayName("Given a Calculator") class CalculatorTests { lateinit var calculator: Calculator @BeforeEach fun beforeEach() { this.calculator = Calculator() } @Nested @DisplayName("When using 'Plus' Operator") inner class PlusOperator { @Test @DisplayName("Then Computing 1 and 2 equals 3") fun oneAndTwoIsThree() { val actual = calculator.compute(1, 2, PLUS) Assertions.assertEquals(3, actual) } } }
51. @Nested Comparisons with JUnit 4: New Annotations @DisplayName("Given a Calculator") class CalculatorTests { lateinit var calculator: Calculator @BeforeEach fun beforeEach() { this.calculator = Calculator() } @Nested @DisplayName("When using 'Plus' Operator") inner class PlusOperator { @Test @DisplayName("Then Computing 1 and 2 equals 3") fun oneAndTwoIsThree() { val actual = calculator.compute(1, 2, PLUS) Assertions.assertEquals(3, actual) } } }
52. JUnit Jupiter @DisplayName(String) @Nested @ParameterizedTest & <Source> @ExtendWith(Extension) Comparisons with JUnit 4: New Annotations @TestFactory
53. @TestFactory Comparisons with JUnit 4: New Annotations class FooTest { @TestFactory fun createDynamicTests(): List<DynamicTest> = listOf( dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }, dynamicTest("Another One") { assertEquals(0, 2 - 2) } ) }
54. @TestFactory Comparisons with JUnit 4: New Annotations class FooTest { @TestFactory fun createDynamicTests(): List<DynamicTest> = listOf( dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }, dynamicTest("Another One") { assertEquals(0, 2 - 2) } ) } @TestFactory fun createDynamicTests(): List<DynamicTest>
55. @TestFactory Comparisons with JUnit 4: New Annotations class FooTest { @TestFactory fun createDynamicTests(): List<DynamicTest> = listOf( dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }, dynamicTest("Another One") { assertEquals(0, 2 - 2) } ) } dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }
56. @TestFactory Comparisons with JUnit 4: New Annotations class FooTest { @TestFactory fun createDynamicTests(): List<DynamicTest> = listOf( dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }, dynamicTest("Another One") { assertEquals(0, 2 - 2) } ) } dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) } Factory method for dynamic tests: fun dynamicTest(String, () -> Unit)
57. @TestFactory Comparisons with JUnit 4: New Annotations class FooTest { @TestFactory fun createDynamicTests(): List<DynamicTest> = listOf( dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }, dynamicTest("Another One") { assertEquals(0, 2 - 2) } ) }
58. @TestFactory Comparisons with JUnit 4: New Annotations class FooTest { @TestFactory fun createDynamicTests(): List<DynamicTest> = listOf( dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }, dynamicTest("Another One") { assertEquals(0, 2 - 2) } ) }
59. @TestFactory Comparisons with JUnit 4: New Annotations class FooTest { @TestFactory fun createDynamicTests(): List<DynamicTest> = listOf( dynamicTest("A Dynamic Test") { assertEquals(4, 2 + 2) }, dynamicTest("Another One") { assertEquals(0, 2 - 2) } ) } • Tests generated at runtime • Individual DynamicTest objects do not partake in the lifecycle of a test case (@BeforeEach etc.) • Generally, prefer @ParameterizedTest
60. JUnit Jupiter @DisplayName(String) @Nested @TestFactory @ExtendWith(Extension) Comparisons with JUnit 4: New Annotations @ParameterizedTest & <Source>
61. @ParameterizedTest & <Source> Comparisons with JUnit 4: New Annotations class FooTest { @ParameterizedTest @CsvSource("Alice, 5", "Bob, 3", "Charles, 7") fun expectedNameLengths(name: String, expectedLen: Int) { Assertions.assertEquals(expectedLen, name.length) } }
62. @ParameterizedTest & <Source> Comparisons with JUnit 4: New Annotations class FooTest { @ParameterizedTest @CsvSource("Alice, 5", "Bob, 3", "Charles, 7") fun expectedNameLengths(name: String, expectedLen: Int) { Assertions.assertEquals(expectedLen, name.length) } } @ParameterizedTest
63. @ParameterizedTest & <Source> Comparisons with JUnit 4: New Annotations class FooTest { @ParameterizedTest @CsvSource("Alice, 5", "Bob, 3", "Charles, 7") fun expectedNameLengths(name: String, expectedLen: Int) { Assertions.assertEquals(expectedLen, name.length) } } @CsvSource("Alice, 5", "Bob, 3", "Charles, 7") fun expectedNameLengths(name: String, expectedLen: Int)
64. @ParameterizedTest & <Source> Comparisons with JUnit 4: New Annotations class FooTest { @ParameterizedTest @CsvSource("Alice, 5", "Bob, 3", "Charles, 7") fun expectedNameLengths(name: String, expectedLen: Int) { Assertions.assertEquals(expectedLen, name.length) } }
65. @ParameterizedTest & <Source> Comparisons with JUnit 4: New Annotations class FooTest { @ParameterizedTest @CsvSource("Alice, 5", "Bob, 3", "Charles, 7") fun expectedNameLengths(name: String, expectedLen: Int) { Assertions.assertEquals(expectedLen, name.length) } }
66. Comparisons with JUnit 4: New Annotations class FooTest { @ParameterizedTest @CsvSource("Alice, 5", "Bob, 3", "Charles, 7") fun expectedNameLengths(name: String, expectedLen: Int) { Assertions.assertEquals(expectedLen, name.length) } } • Different possibilities for parameter providers: •@CsvSource •@CsvFileSource •@EnumSource •@MethodSource •@ValueSource • …or implement your own @ParameterizedTest & <Source>
67. JUnit Jupiter @DisplayName(String) @Nested @TestFactory @ParameterizedTest & <Source> Comparisons with JUnit 4: New Annotations @ExtendWith(Extension)
68. Comparisons with JUnit 4: New Annotations @ExtendWith(Extension) • Unified API for Test Extension • Replacement for @RunWith & @Rule • Repeatable annotation → allows composition • Different interfaces provide hooks into the system • Usage Example: Integration with existing libraries • Complex topic deserving of its own talk
70. • JUnit 5 Gradle Plugin only works in pure-Java projects • mannodermaus/android-junit5 • Unit Test Integration for Android projects
71. buildscript { dependencies { classpath "de.mannodermaus.gradle.plugins:android-junit5:1.0.30" } } apply plugin: "de.mannodermaus.android-junit5" dependencies { testImplementation junit5.unitTests() }
72. buildscript { dependencies { classpath "de.mannodermaus.gradle.plugins:android-junit5:1.0.30" } } apply plugin: "de.mannodermaus.android-junit5" dependencies { testImplementation junit5.unitTests() } dependencies { testImplementation junit5.unitTests() } Bundled dependencies for easy setup
73. • New: Experimental Instrumentation Test Support • Successor to JUnit 4 ActivityTestRule • API 26+ • Full Android migration is going to take a while longer…
74. android.testOptions { junitPlatform { instrumentationTests.enabled true } } dependencies { androidTestImplementation junit5.instrumentationTests() }
75. @RunWith(AndroidJUnit4::class) class MyActivityTest { @Rule val rule = ActivityTestRule(MyActivity::class.java) @Test fun testSomething() { rule.launchActivity(null) onView(withId(R.id.textView)).check(...) rule.finishActivity() } } Espresso Test with JUnit 4
76. @ActivityTest(MyActivity::class) class MyActivityTest { @Test fun testSomething(tested: Tested<MyActivity>) { tested.launchActivity() onView(withId(R.id.textView)).check(...) tested.finishActivity() } } Espresso Test with JUnit 5
77. @ActivityTest(MyActivity::class) class MyActivityTest { @Test fun testSomething(tested: Tested<MyActivity>) { tested.launchActivity() onView(withId(R.id.textView)).check(...) tested.finishActivity() } } Espresso Test with JUnit 5 @ActivityTest(MyActivity::class)
78. @ActivityTest(MyActivity::class) class MyActivityTest { @Test fun testSomething(tested: Tested<MyActivity>) { tested.launchActivity() onView(withId(R.id.textView)).check(...) tested.finishActivity() } } Espresso Test with JUnit 5 @ActivityTest(MyActivity::class) Test Extension with custom Configuration Parameters: •targetPackage •launchFlags •launchActivity •…
79. @ActivityTest(MyActivity::class) class MyActivityTest { @Test fun testSomething(tested: Tested<MyActivity>) { tested.launchActivity() onView(withId(R.id.textView)).check(...) tested.finishActivity() } } Espresso Test with JUnit 5 fun testSomething(tested: Tested<MyActivity>)
80. @ActivityTest(MyActivity::class) class MyActivityTest { @Test fun testSomething(tested: Tested<MyActivity>) { tested.launchActivity() onView(withId(R.id.textView)).check(...) tested.finishActivity() } } Espresso Test with JUnit 5 fun testSomething(tested: Tested<MyActivity>) Access to Activity under test, successor to old ActivityTestRule: •Tested#launchActivity •Tested#finishActivity •Tested#getActivityResult
81. @ActivityTest(MyActivity::class) class MyActivityTest { @Test fun testSomething(tested: Tested<MyActivity>) { tested.launchActivity() onView(withId(R.id.textView)).check(...) tested.finishActivity() } } Espresso Test with JUnit 5 @ActivityTest(MyActivity::class) class MyActivityTest { @Test fun testSomething(tested: Tested<MyActivity>) { tested.launchActivity() onView(withId(R.id.textView)).check(...) tested.finishActivity() } }
82. The future is bright for Testing.
83. It'll take some more time before JUnit 5 becomes common practice in Android.
84. Tools today already make it possible to become an early adopter.
85. https://github.com/mannodermaus/android-junit5
86. https://www.mytenten.com/jobs
87. Moving Forward with JUnit 5