Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kogito + Quarkus: A Marriage made on a Cloud

Kogito + Quarkus: A Marriage made on a Cloud

Full recording https://youtu.be/hY_eERFX_DQ

A hands-on walkthrough of the Kogito codebase with a focus on the interaction with Quarkus.

About this Event

KieLive#23: Kogito + Quarkus: the Marriage Made on a Cloud
Quarkus is the secret sauce that makes Kogito cloud-native. But like every special ingredient, you need to learn how to use it! In this hands-on walkthrough we will get an overview of the design principles of our framework, and we will learn some of the nuances of writing a native application or even a Quarkus extension. This presentation is intended for a technical audience, but it is open to everyone: maybe you are the next Kogito contributor?

Link to the live streaming: http://red.ht/KieLive23

About the invited speaker:

During my PhD, I researched language design and implementation at University of Milan. After three years at UniCredit Bank's R&D department, I have joined Red Hat: I work on the Drools rule engine, the jBPM platform and the Kogito project.

Twitter: https://twitter.com/evacchi

About the KIE Lives:
The KIE Live Series is composed of live streamings that bring technical information and updates about business automation delivered by the projects under the KIE umbrella: Drools, jBPM, OptaPlanner, and Kogito.

Problems like process automation, decision automation, resource planning solution are the main topics, and of course, we always have in mind recent technology concepts like cloud-native application target for any type of cloud (private/public/hybrid/edge). You can expect to hear from business automation experts who code or/and deliver business automation within big enterprises across the world.

Join us to share, learn, and grow together.

Knowledge is everything

Edoardo Vacchi

February 02, 2021
Tweet

More Decks by Edoardo Vacchi

Other Decks in Programming

Transcript

  1. • Edoardo Vacchi @evacchi • Research @ UniMi / MaTe

    • Research @ UniCredit R&D • Kogito / Drools / jBPM @ Red Hat
  2. • You are a team mate ( hey! ) •

    You are a wannabe contributor ( wow! ) • You are just curious ( welcome! )
  3. • Kogito • Compile-Time vs Run-Time • Quarkus Extensions •

    Bytecode vs Source-Code Generation • GraalVM Native image • Scaffolding and Troubleshooting
  4. • There is always a pre-processing phase where you prepare

    your program for execution • Then, there's actual process execution phase
  5. • Will that configuration change across runs? • Do you

    have to repackage the application to bundle the new configuration?
  6. • You are building an Immutable Dockerized Microservice in the

    Cloud • You are not going to load new classes at run-time • You don’t need Runtime Dependency Injection
  7. Build-time • Pre-process configuration • Generate code with configuration options

    pre-loaded • Resolve dependencies • Generate “wiring” code -- e.g. factories, startup methods etc.
  8. Assets Wiring Pre-Built Assets REST/Event Handlers ∙ “Drools Canonical Model”

    ∙ BPMN Fluent API ∙ ... ∙ DRL ∙ BPMN ∙ DMN ∙ ... ∙ JAX-RS Endpoints ∙ Cloud Event Reactive Consumers ∙ ... ∙ CDI Producers ∙ Bean Factories ∙ ...
  9. REST Resources class MyModelProcessResource { @Inject Process<MyModel> p; } class

    MyRuleUnitResource { @Inject RuleUnit<MyUnitData> r; } Wiring Code (CDI etc.) Application.java ApplicationConfig.java Processes.java Decisions.java ... (Inferred) Data Model (process variables, DMN ItemDefinitions, DMN context, Rule Unit descriptors...) e.g. MyModel, MyUnitData Typed Knowledge Asset API (and related wiring) exposes the engine e.g. Process<MyModel>, RuleUnit<MyUnitData>, DMNDecision
  10. REST Resources class MyModelProcessResource { @Inject Process<MyModel> p; } class

    MyRuleUnitResource { @Inject RuleUnit<MyUnitData> r; } Wiring Code (CDI etc.) Application.java ApplicationConfig.java Processes.java Decisions.java ... (Inferred) Data Model (process variables, DMN ItemDefinitions, DMN context, Rule Unit descriptors...) e.g. MyModel, MyUnitData Typed Knowledge Asset API (and related wiring) exposes the engine e.g. Process<MyModel>, RuleUnit<MyUnitData>, DMNDecision Implementation Detail ! Heavy Rework in Progress ! Do Not Rely on The Details !
  11. • @BuildStep • annotates a method • method parameters: required

    items • return type: provided result • You can think of it as of • a CDI @Produces method or • a Spring @Factory • With a twist: • You can return more than one element at once (with a “trick”) @BuildStep public SomeBuildItem myBuildStep( SomeOtherBuildItem items ) { ... }
  12. all: foo.o bar.o cc -o out foo.o bar.o foo.o: foo.c

    $(CC) -c foo.c -o foo.o bar.o: bar.c $(CC) -c bar.c -o bar.o
  13. • Think of them as CDI/Spring beans • A special

    interface that represents things that can be • produced (provided) • consumed (required) @BuildStep public SomeBuildItem generate( SomeOtherBuildItem items) { ... }
  14. • Think of them as CDI/Spring beans • A special

    interface that represents things that can be • produced (provided) • consumed (required) @BuildStep public SomeBuildItem generate( SomeOtherBuildItem items) { ... } @BuildStep public SomeBuildItem generate( Collection<SomeOtherBuildItem> items) { ... } You can depend on a collection !
  15. @BuildStep public void generate( SomeBuildItem items, BuildProducer<SomeOtherBuildItem> bp) { ...

    bp.produce(t); } BuildProducer<GeneratedBeanBuildItem> generatedBeans, BuildProducer<NativeImageResourceBuildItem> resource, BuildProducer<ReflectiveClassBuildItem> reflectiveClass, BuildProducer<GeneratedResourceBuildItem> genResBI
  16. @BuildStep public void generate( SomeBuildItem items, BuildProducer<SomeOtherBuildItem> bp) { ...

    bp.produce(t); } BuildProducer<GeneratedBeanBuildItem> generatedBeans, BuildProducer<NativeImageResourceBuildItem> resource, BuildProducer<ReflectiveClassBuildItem> reflectiveClass, BuildProducer<GeneratedResourceBuildItem> genResBI You can have as many as you want ! Simulates multiple return values !
  17. In the body of the methods // real work occurs

    here: invoke the code-generation procedure Collection<GeneratedFile> generatedFiles = appGen.generate(); // Write files to disk dumpFilesToDisk(appPaths, generatedFiles);
  18. private void registerResources(Collection<GeneratedFile> generatedFiles, BuildProducer<NativeImageResourceBuildItem> resource, BuildProducer<GeneratedResourceBuildItem> genResBI) { for

    (GeneratedFile f : generatedFiles) { if (f.getType() == GeneratedFile.Type.GENERATED_CP_RESOURCE) { genResBI.produce(new GeneratedResourceBuildItem(f.relativePath(), f.contents())); resource.produce(new NativeImageResourceBuildItem(f.relativePath())); } } }
  19. • Avoid scanning class-path • Instead use a serialized index

    • Faster lookup: no scanning! IndexView index = combinedIndexBuildItem.getIndex(); DotName classDotName = DotName.createSimple(className); return !index.getAnnotations(classDotName).isEmpty() || index.getClassByName(classDotName) != null;
  20. • Quarkus general approach is to generate byte-code • In

    Kogito we generate Java source code • In some cases we compile it on-the-fly
  21. • Native binary compilation • Restriction: “closed-world assumption” • Limitations

    on reflection • No dynamic code loading: forbidden ClassLoader#defineClass(...byte[]...) • Allows more aggressive optimization (pruning) • Static initializers are not lazy* ! • Evaluated at build time ! * in Quarkus
  22. • Native does not prevent dynamic code loading! • e.g.

    dlopen() for dynamic .so/.dll/.dylib loading • JVM mode does not prevent pruning! • e.g. ProGuard • These are just choices of the native image compiler
  23. META-INF/ └── native-image └── groupID └── artifactID └── native-image.properties └──

    reflection-config.json └── resource-config.json └── jni-config.json └── proxy-config.json ReflectiveClassBuildItem
  24. @TargetClass(ASMAccessorOptimizer.class) final class ASMAccessorOptimizer_Target { @Substitute public Accessor optimizeAccessor( ParserContext

    pCtx, char[] property, int start, int offset, Object ctx, Object thisRef, VariableResolverFactory factory, boolean rootThisRef, Class ingressType) { throw new UnsupportedOperationException(); } ... }
  25. @Recorder class HelloRecorder { public void sayHello(String name) { System.out.println("Hello"

    + name); } } @Record(RUNTIME_INIT) @BuildStep public void helloBuildStep(HelloRecorder recorder) { recorder.sayHello("World"); }
  26. public class Example { static { System.out.println("hello"); } public static

    void main(String... args) { System.out.println("world"); } }
  27. $ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB

    [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB
  28. $ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB

    [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB
  29. the check on a static method may not be equivalent

    from the perspective of the native image compiler, which does aggressive code pruning at build time. If the field is static, then the entire non-true branch is basically treated as unreachable; otherwise the native image compiler will try to visit the other branch too, which in some cases may mark as reachable code things that would not really be reached at runtime. this may not be an issue here, but it may be if that code eventually leads to some "banned" method call (e.g. ClassLoader#defineClass()), causing the native image build to fail.
  30. private static final Collection<InputStreamReader> readers = List.of( readResource("some/path..."), ... );

    private static InputStreamReader readResource(InputStream stream) { return new java.io.InputStreamReader(stream); }
  31. private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private

    static final Collection<InputStreamReader> readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }
  32. private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private

    static final Collection<InputStreamReader> readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }
  33. private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private

    static final Collection<InputStreamReader> readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }
  34. private static final Collection<InputStreamReader> readers = List.of(...); private static InputStreamReader

    readResource(java.io.InputStream stream) { try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }
  35. private boolean isNativeImage() { return System.getProperty("org.graalvm.nativeimage.imagecode") != null; } private

    static final Collection<InputStreamReader> readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!isNativeImage()) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } } Not Equivalent!
  36. I don’t want to contribute to the extension. What else

    can I do? - contribute documentation - contribute examples - hang out on the Zulip channel / mailing list and help other users out!