Jigsaw our savior

Learn how jigsaw will save us from our sins

Transcript

1. #DevoxxF R 1 Rémi Forax Université Paris Est Marne la vallée Jigsaw will save us Jigsaw will save us
2. #DevoxxFR Me, Myself and I Maitre de conférence University Paris Est Marne la vallée Developer: OpenJDK, ASM, Tatoo, Dragon, … Java Community Process Expert – Invokedynamic (JSR 292) – Lambda (JSR 335 + java.util/java.util.stream) – Jigsaw (JSR 376)
3. #DevoxxFR Dramatic Introduction
4. #DevoxxFR No reliable configuration ClassPath ? – Linear scan :( – If two versions of the same package exist ? => Jar Hell Example: CLASSPATH = …:asm-2.3.jar:…:asm-3.1.jar A newly introduced class of ASM 3.1 will use an old class of ASM 2.3 => compile time information is different from runtime info
5. #DevoxxFR No way to scale Down the platform Some libraries tend to be too big – OpenJDK 8 rt.jar => 66 Mb – Guava 1.9 => 2.3 Mb – Spring Context => 1.1 Mb on Maven Central IoT anyone ? and JDK classes are privileged :(
6. #DevoxxFR No Strong Encapsulation 2013 is a very bad year for Java security The JDK has only one line of defense :(
7. #DevoxxFR
8. #DevoxxFR The Jigsaw Sandwich Compile time Runtime
9. #DevoxxFR Meet the Dragon An interpreter of a toy language def(fib: \ n "returns the fibonacci number of n" if(n < 2 1 fib(n - 1) + fib(n – 2))) A small application delivered as a 'fat' jar (dragon.jar) – 244 classes (dependencies included) – 20 packages • 3 auto-generated from the grammar • 14 from tatoo-runtime.jar – 1 external dependency (tatoo-runtime.jar)
10. #DevoxxFR Jigsawification !
11. #DevoxxFR jdeps – find package dependencies $ /usr/jdk/jdk-9/bin/jdeps dragon.jar dragon.jar -> java.base fr.umlv.dragon.ast (dragon.jar) -> fr.umlv.dragon.grammar.tools -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.grammar.lexer (dragon.jar) -> fr.umlv.tatoo.runtime. ... (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.parser (dragon.jar) -> fr.umlv.tatoo.runtime.parser (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.tools (dragon.jar) -> fr.umlv.dragon.grammar.lexer -> fr.umlv.dragon.grammar.parser -> fr.umlv.dragon.rt -> fr.umlv.tatoo.runtime. … (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.main (dragon.jar) -> fr.umlv.dragon.ast -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.rt (dragon.jar) -> java ...
12. #DevoxxFR Packages dependencies dragon.rt dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser fr.umlv.tatoo.runtime.* dragon.ast dragon.main java.*
13. #DevoxxFR Modules dependencies dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
14. #DevoxxFR dragon.rt module-info.java module fr.umlv.dragon.rt { requires java.base; exports fr.umlv.dragon.rt; } dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
15. #DevoxxFR Create module fr.umlv.dragon.rt javac -d build/modules \ -modulesourcepath src \ $(find src/fr.umlv.dragon.rt -name "*.java") jar --create \ --file mlib/fr.umlv.dragon.rt-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.rt .
16. #DevoxxFR dragon.rt module-info.java module fr.umlv.dragon.rt { // requires java.base; exports fr.umlv.dragon.rt; } dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
17. #DevoxxFR dragon.grammar module-info.java module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
18. #DevoxxFR Create module dragon.grammar javac -d build/modules \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ -classpath lib/tatoo-runtime.jar \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") This doesn't compile !!
19. #DevoxxFR Mixing modules and jar Modules in modulepath do not read (see) classes in classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo-runtime.jar OK ! NO !
20. #DevoxxFR Automatic modules (migration mode) Plain old jar in modulepath – can be used in requires – can read modules – can read any jars from the classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo.runtime-1.0.jar another-jar.jar modulepath the jar also needs to be renamed ! module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; }
21. #DevoxxFR Create module dragon.grammar mv lib/tatoo-runtime.jar \ mlib/fr.umlv.tatoo.runtime-1.0.jar javac -d build/modules \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") It works !
22. #DevoxxFR dragon.ast module-info.java dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; }
23. #DevoxxFR dragon.main module-info.java dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
24. #DevoxxFR Create dragon.main module The main class is specified when creating the jar as usual $ javac -d build/modules \ -modulesourcepath src \ -modulepath mlib \ $(find src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.main-1.0.jar \ --module-version=1.0 \ --main-class fr.umlv.dragon.main.Main \ -C build/modules/fr.umlv.dragon.main
25. #DevoxxFR requires is not transitive by default ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
26. #DevoxxFR requires public to the rescue ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
27. #DevoxxFR require public The public API of the module fr.umlv.dragon.ast refers to classes from a required module package fr.umlv.dragon.ast; import ... import fr.umlv.dragon.rt.Block; public class ASTBuilder { • public static Block buildAST(Reader reader) { TerminalEvaluator<CharSequence> terminalEvaluator = new TokenEvaluator(); ASTEvaluator grammarEvaluator = new ASTEvaluator(); Analyzers.run(reader, terminalEvaluator, grammarEvaluator, null, null); return grammarEvaluator.script; } ... } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }
28. #DevoxxFR require public only works with one hop dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
29. #DevoxxFR
30. #DevoxxFR Restricted export module fr.umlv.dragon.grammar exports methods that are specific to the Tatoo parser compiler what if I want to change the parser in the future ? => restrict the visibility of a module !
31. #DevoxxFR Only dragon.ast can 'requires' dragon.grammar ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools to fr.umlv.dragon.ast; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
32. #DevoxxFR sun.misc.Unsafe in 9 ? $ javap -m java.base module-info module java.base { …, exports sun.misc; exports jdk.internal.misc to java.rmi, java.sql, jdk.charsets, … } $ javap -m jdk.unsupported module-info module jdk.unsupported { exports sun.misc; } Keep sun.misc in 9, remove features when a public replacement APIs exist New internal unsafe API (hidden)
33. #DevoxxFR Discovering usage of internal APIs $ jdeps -jdkinternals jruby-9.0.5.0/lib/jruby.jar jruby.jar -> java.base jruby.jar -> java.management org.jruby.management.BeanManagerImpl (jruby.jar) -> sun.management.Agent JDK internal API (java.management) org.jruby.util.encoding.ISO_8859_16 (jruby.jar) -> sun.nio.cs.US_ASCII JDK internal API (java.base) $ jdeps -jdkinternals gradle/lib/*.jar gradle/lib/plugins/*.jar gradle-core-2.12.jar -> java.base org.gradle.internal.filewatch.jdk7.WatchServiceRegistrar (gradle-core-2.12.jar) -> com.sun.nio.file.ExtendedWatchEventModifier JDK internal API (java.base) -> com.sun.nio.file.SensitivityWatchEventModifier JDK internal API (java.base) bsh-2.0b4.jar -> java.desktop bsh.util.AWTConsole (bsh-2.0b4.jar) -> java.awt.peer.ComponentPeer JDK internal API (java.desktop) -> java.awt.peer.TextComponentPeer JDK internal API (java.desktop) ...
34. #DevoxxFR Module Service I want to be able to plug different kinds of interpreter for dragon – AST interpreter, – Bytecode interpreter, etc. Need to decouple the interpreter interface from its implementation At least one interpreter should be present at runtime
35. #DevoxxFR Module Service / Code package fr.umlv.dragon.ast; public interface Interpreter { Object interpret(Array<Parameter> parameters, Block body, ...); } package fr.umlv.dragon.main; … public static void main(String[] args) { Interpreter interpreter = ServiceLoader.load(Interpreter.class).iterator().next(); } package fr.umlv.dragon.eval; public class InterpreterImpl implements Interpreter { … } interface an implementation usage
36. #DevoxxFR dragon.rt java.base dragon.ast dragon.main module fr.umlv.dragon.eval { requires fr.umlv.dragon.ast // export fr.umlv.tatoo.eval; provides fr.umlv.dragon.ast.Interpreter to fr.umlv.dragon.eval.InterpreterImpl; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast uses fr.umlv.dragon.ast.Interpreter;; } Uses / Provides … with dragon.eval
37. #DevoxxFR Run dragon As jars (Java 9 is compatible with Java <9) $ /usr/jdk/jdk-9/bin/java -classpath \ mlib/fr.umlv.tatoo.runtime-1.0.jar: \ mlib/fr.umlv.dragon.grammar-1.0.jar: \ mlib/fr.umlv.dragon.ast-1.0.jar: \ mlib/fr.umlv.dragon.rt-1.0.jar: \ mlib/fr.umlv.dragon.eval-1.0.jar \ mlib/fr.umlv.dragon.main-1.0.jar \ fr.umlv.dragon.main.Main As modular jars $ java --modulepath mlib \ --m fr.umlv.dragon.main
38. #DevoxxFR
39. #DevoxxFR jlink Create a custom runtime image from a set of root modules Image is specific for a platform – Intel 64bits linux, ARM 32bits Windows, etc For small devices or in the cloud (jlink is not mandatory !)
40. #DevoxxFR jlink and automatic module ! $ jlink --modulepath /usr/jdk/jdk-9-jigsaw/jmods:mlib --addmods "fr.umlv.dragon.main,fr.umlv.dragon.eval" \ --output image Error: jdk.tools.jlink.plugin.PluginException : module-info.class not found for fr.umlv.tatoo.runtime module Automatic modules can see classes from the classpath so the world in not closed :(
41. #DevoxxFR Jigsawification by hand $ jdeps -genmoduleinfo tmp lib/tatoo-runtime.jar writing to tmp/tatoo.runtime/module-info.java $ cat tmp/tatoo.runtime/module-info.java module tatoo.runtime { exports fr.umlv.tatoo.runtime.ast; exports fr.umlv.tatoo.runtime.buffer; exports fr.umlv.tatoo.runtime.lexer; exports fr.umlv.tatoo.runtime.lexer.rules; exports fr.umlv.tatoo.runtime.log; exports fr.umlv.tatoo.runtime.parser; exports fr.umlv.tatoo.runtime.regex; exports fr.umlv.tatoo.runtime.tools; ... }
42. #DevoxxFR Jigsawification by hand (2) $ javac -d build/modules \ -modulesourcepath tmp \ fr.umlv.tatoo.runtime/module-info.java $ jar --update \ --file mlib/fr.umlv.tatoo.runtime-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.tatoo.runtime \ module-info.class
43. #DevoxxFR jlink my dragon Generating a custom image for dragon $ rm -fr image $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --strip-debug \ --output image
44. #DevoxxFR jlink image - Run dragon $ ls -R image image: bin conf lib release image/bin: fr.umlv.dragon.main java keytool image/lib: amd64 classlist jexec modules security tzdb.dat … image/lib/amd64/server: classes.jsa libjsig.so libjvm.so Xusage.txt Run the executable $ ./image/bin/fr.umlv.dragon.main Run with the 'java' wrapper $ ./image/bin/java -m fr.umlv.dragon.main
45. #DevoxxFR Compact VM / AOT On small device, use the compact VM, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --strip-debug \ --vm=compact \ --output image and Oracle is working on a way to pre-JIT the code https://www.youtube.com/watch?v=Xybzyv8qbOc
46. #DevoxxFR
47. #DevoxxFR Startup time :( (as today)
48. #DevoxxFR Size on disc (as today)
49. #DevoxxFR Jigsaw Goals Reliable configuration • No classpath anymore • compile time / runtime fidelity Scale down the platform • downsized unit of re-use Strong encapsulation • Separate public types from implementation • Enhanced security Closed World • Ahead of time optimization
50. #DevoxxFR Questions ? Code is available at https://github.com/forax/dragon/tree/jigsaw
51. #DevoxxFR Supplementary slides
52. #DevoxxFR • jmod ? • Module Layer ?
53. #DevoxxFR Command line java/javac Override root modules configuration -addmods module to add root modules -upgrademodulepath to replace some root modules Patch a module -Xpatch:module=files patch existing classes of a module Add a dependency (can be done by reflection too) -XaddReads:module=anotherModule Break encapsulation -XaddExports:module/package=anotherModule
54. #DevoxxFR jlink plugin Jlink has an experimental plugin API By example, to generate a CDS file, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --plugin-module-path mlib \ --genclasslist=image/lib/classlist2 \ --strip-debug \ --output image
55. #DevoxxFR jlink plugin - code import jdk.tools.jlink.plugin.TransformerPlugin; public class ClassListGeneratorPlugin implements TransformerPlugin { public String getName() { return "genclasslist"; } public boolean hasArguments() { return true; } ... public void configure(Map<String, String> config) { … } public void visit(Pool in, Pool out) { ... } }
56. #DevoxxFR jlink plugin – Service Use the mechanism of Service integrated with jigsaw modules module fr.umlv.dragon.jlinkplugin { requires jdk.jlink; provides jdk.tools.jlink.plugin.TransformerPlugin with fr.umlv.dragon.jlinkplugin.ClassListGeneratorPlugin; } $ javap -m jdk.jlink module-info module jdk.jlink { … exports jdk.tools.jlink.builder; exports jdk.tools.jlink.plugin; exports jdk.tools.jlink; uses jdk.tools.jlink.plugin.TransformerPlugin; }
57. #DevoxxFR jlink plugin – create module A module that defines a service is compiled like any other modules $ javac -d build/modules \ -modulesourcepath jlink-plugin/src \ -modulepath mlib \ $(find jlink-plugin/src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.jlinkplugin-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.jlinkplugin .