Surviving Dependency Hell with Maven

As a developer advocate working with customers, Ray has seen all sorts of issues due to dependency conflicts. Dependency conflicts come in many different forms and have different impacts on your applications. This presentation examines common causes of a dependency conflict, how you can mitigate it as a library developer, and how end users can resolve it. It also covers what Google has been documenting in terms of best practices and what tools it has created to help, based on its learnings.

Transcript

1. Surviving Dependency Hell With Maven
2. 2 @aalmiray @saturnism Ray Tsang Developer Advocate @ Google Java Champion Spring Cloud GCP JDeferred Contributors to JLBP.dev @saturnism | saturnism.me
3. 3 @aalmiray @saturnism Robert Scholte Java Champion Founder, Source Ground Apache Maven PMC Chair Member Expert Group JSR 376 - Jigsaw @rfscholte
4. 4 @aalmiray @saturnism Andres Almiray Seasoned Sourceror @ Oracle Java Champion Hackergarten JDeferred & many others! @aalmiray | andresalmiray.com
5. 5 @aalmiray @saturnism Let's see the code
6. 6 @aalmiray @saturnism NoSuchMethodError NoSuchFieldError NoClassDefFoundError
7. 7 @aalmiray @saturnism mvn dependency:tree mvn dependency:tree -Dverbose=true mvn dependency:tree -Dverbose=true -Dincludes=...
8. 8 @aalmiray @saturnism Example 1 Translate Truth Guava 19 Guava 21 Which version to use?
9. 9 @aalmiray @saturnism Any dependency there can be only ONE version Classpath - First class wins Maven - Nearest wins
10. 10 @aalmiray @saturnism Maven does not understand Semver Nor compatibility
11. 11 @aalmiray @saturnism Use Maven Enforcer Convergence vs Upper bound http://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html (See Code!)
12. 12 @aalmiray @saturnism Ensuring only ONE version of the dependency in tree Exclusions, or Dependency Management (See Code)
13. 13 @aalmiray @saturnism Upper bound if higher version is backwards compatible [JLBP-7] Make breaking transitions easy [JLBP-10] Maintain API stability as long as needed for consumers (Guava 21 and up are backwards compatible)
14. 14 @aalmiray @saturnism If upper version is breaking lower version… Or, system classpath has an incompatible version (Hadoooooop)
15. 15 @aalmiray @saturnism Shade + Relocation :( Classloaders OSGi
16. 16 @aalmiray @saturnism GA libraries don't depend on non-GA APIs alpha, beta, RC, 0.xx, @UnstableApi, @Beta, @Internal [JLBP-4] Avoid dependencies on unstable libraries and features
17. 17 @aalmiray @saturnism Major release, breaking changes Use new Group ID or Artifact ID - different coordinate! AND Use a new package name [JLBP-6] Rename artifacts and packages together
18. 18 @aalmiray @saturnism How do we change versions in transitive dependencies?
19. 19 @aalmiray @saturnism Maven Dependency Management
20. 20 @aalmiray @saturnism Let's see Example 3e
21. 21 @aalmiray @saturnism Example 1 PubSub Firebase guava-jdk5 17.0 guava 28.0
22. 22 @aalmiray @saturnism Ban Duplicate Classes https://www.mojohaus.org/extra-enforcer-rules/banDuplicateClasses.html
23. 23 @aalmiray @saturnism 2 artifacts should not have overlapping classes [JLBP-5] Avoid dependencies that overlap classes with other dependencies [JLBP-19] Place each package in only one module
24. 24 @aalmiray @saturnism [JLBP-11] Keep dependencies up to date!
25. 25 @aalmiray @saturnism Let's fix all of these And see the next example 3c
26. 26 @aalmiray @saturnism Example Pubsub Trace grpc-stub 1.10.1 grpc-all 1.0.1 grpc-core 1.0.1 grpc-... 1.0.1
27. 27 @aalmiray @saturnism Version Misalignment
28. 28 @aalmiray @saturnism How do we change versions in transitive dependencies?
29. 29 @aalmiray @saturnism Example Pubsub Trace grpc-stub 1.10.1 grpc-all 1.10.1 grpc-core 1.10.1 grpc-... 1.10.1 grpc-bom 1.10.1
30. 30 @aalmiray @saturnism Multi-module projects Produce a BOM! [JLBP-15] Publish a BOM for multi-module projects
31. 31 @aalmiray @saturnism See an example!
32. 32 @aalmiray @saturnism Power → Responsibility BOM should not overreach! ONLY modules within the multi-project project
33. 33 @aalmiray @saturnism gRPC also uses another method Strict Version Ranges for inter-module dependencies, e.g. [1.10.1]
34. 34 @aalmiray @saturnism Linkage Checker Static Analysis to identify missing links https://github.com/GoogleCloudPlatform/cloud-opensource-java/tree/master/enforcer-rules
35. 35 @aalmiray @saturnism Bigger issues with Dependency Management...
36. 36 @aalmiray @saturnism What you build & test with Is not what consumers get!
37. 37 @aalmiray @saturnism Flatten the POM https://github.com/mojohaus/flatten-maven-plugin/pull/93
38. 38 @aalmiray @saturnism Bonus mvn dependency:analyze - find dependencies you don't need Maven 3.6.3 - fixes our exclusion fix in Maven 3.6.2
39. 39 @aalmiray @saturnism Enforcer Rules Enforcer Version Convergence or Upperbound Ban Duplicate Classes Linkage Checker Manage Transitive Versions <dependencyManagement/> BOM for Multi-Module Import BOM Maven Flatten Plugin
40. 40 @aalmiray @saturnism Visit the best practices at JLBP.dev Thanks! Andres Almiray @aalmiray Ray Tsang @saturnism Maven used by 60% to 80% of Java projects 92 projects, 50+ plugins, lots of libraries - Contribution Appreciated!