Share
Embed
Eclipse Vert.x for Dj fun and for profit!
Talk on Vert.x given at the JUG Summer Camp 2017 / La Rochelle, France.
Julien Ponge
September 15, 2017
Transcript
78 slides
-
Eclipse Vert.x for Dj fun and for profit! Speaker :
Julien Ponge - @jponge -
Julien Ponge Maitre de Conférences “Delegated consultant to Red Hat”
on Vert.x Eclipse Golo + extensive F/OSS background ! https://julien.ponge.org/ " @jponge # @jponge https://www.mixcloud.com/hclcast/ -
Outline ✓ Reactive? Vert.x? ✓ Vert.x core ✓ Reactive programming
with Vert.x / RxJava ✓ BoilerVroom -
Reactive systems Reactive streams Reactive programming Reactive “Responding to stimuli”
Manifesto, Actor, Messages Resilience, Elasticity, Scalability, Asynchronous, non-blocking Data flow Back-pressure Non-blocking Data flow Events, Observable Spreadsheets Akka, Vert.x Akka Streams, Rx v2, Reactor, Vert.x Reactor, Reactive Spring, RxJava, Vert.x -
Eclipse Vert.x Open source project started in 2012 Created by
Tim Fox Eclipse / Apache licensing A toolkit for building reactive applications for the JVM Built on top of Netty ! https://vertx.io " @vertx_project -
Outline ✓ Vert.x concurrency model ✓ Message passing on the
event bus ✓ Dealing with asynchronous events ✓ Failover demo -
while (isRunning) { String line = bufferedReader.readLine(); switch (line.substring(0, 4))
{ case "ECHO": bufferedWriter.write(line); break // ... // other cases ( ...) // ... default: bufferedWriter.write("UNKW Unknown command"); } } -
C1 “When you have a line of text, call C2”
Something else with no blocking call either C2 -
Verticles ' ' ' public class SomeVerticle extends AbstractVerticle
{ @Override public void start() throws Exception { } @Override public void stop() throws Exception { } } class SomeVerticle : AbstractVerticle() { override fun start() { } override fun stop() { } } exports.vertxStart = function() { } exports.vertxStop = function() { } -
' ( Configuration ' ' Network events come from acceptor
threads Verticle Deploy Deploy -
' “Regular verticle” (on event-loop) ) ' Worker verticle (1
thread) Multi-thread worker verticle ' Worker pool -
' , ' Http server verticle Database client verticle
Event Bus - “Details for user 1234?” Send to “user.db” -
' , ' Http server verticle Database client verticle
Event Bus - “Details for user 1234?” Send to “user.db” Consume from “user.db” -
' , ' Http server verticle Database client verticle
Event Bus - , “Details for user 1234?” “{data}” -
- Asynchronous messaging “foo.bar”, “foo-bar”, “foo/bar”, … Point to point
(with possible response back) Publish / subscribe -
- Distributed across Vert.x nodes Hazelcast, Ignite, Infinispan, … TCP
bridge interface Go, Python, C, JavaScript, Swift, C#, … SockJS bridge Seamless frontend / backend messaging -
, “Primitive” types String, int, double, … JSON Object/Array Polyglot
applications, clean boundaries Custom codecs For advanced usages -
EventBus eb = vertx.eventBus(); eb.consumer("ping-address", message -> { System.out.println("Received message:
" + message.body()); message.reply("pong!"); }); EventBus eb = vertx.eventBus(); vertx.setPeriodic(1000, v -> { eb.send("ping-address", "ping!", reply -> { if (reply.succeeded()) { System.out.println("Received reply " + reply.result().body()); } else { System.out.println("No reply"); } }); }); -
switch (message.headers().get("action")) { case "all-pages": fetchAllPages(message); break; case "get-page": fetchPage(message);
break; case "create-page": createPage(message); break; case "save-page": savePage(message); break; case "delete-page": deletePage(message); break; default: message.fail(ErrorCodes.BAD_ACTION.ordinal(), "Bad action: " + action); } -
private void deletePage(Message<JsonObject> message) { dbClient.getConnection(car -> { if (car.succeeded())
{ SQLConnection connection = car.result(); JsonArray data = new JsonArray().add(message.body().getString("id")); connection.updateWithParams(sqlQueries.get(SqlQuery.DELETE_PAGE), data, res -> { connection.close(); if (res.succeeded()) { message.reply(new JsonObject().put("result", "ok")); } else { reportQueryError(message, res.cause()); } }); } else { reportQueryError(message, car.cause()); } }); } -
switch (message.headers().get("action")) { case "all-pages": fetchAllPages(message); break; case "get-page": fetchPage(message);
break; case "create-page": createPage(message); break; case "save-page": savePage(message); break; case "delete-page": deletePage(message); break; default: message.fail(ErrorCodes.BAD_ACTION.ordinal(), "Bad action: " + action); } . If lots of actions… -
@ProxyGen public interface WikiDatabaseService { // ( ...) @Fluent WikiDatabaseService
savePage(int id, String markdown, Handler<AsyncResult<Void >> resultHandler); @Fluent WikiDatabaseService deletePage(int id, Handler<AsyncResult<Void >> resultHandler); static WikiDatabaseService createProxy(Vertx vertx, String address) { return new WikiDatabaseServiceVertxEBProxy(vertx, address); } // ( ...) } Proxy + handler source code will be generated Parameters from a JSON document Handlers for replies Generated proxy -
dbService = WikiDatabaseService.createProxy(vertx, "wikidb.queue"); private void pageDeletionHandler(RoutingContext context) { dbService.deletePage(Integer.valueOf(context.request().getParam("id")),
reply -> { if (reply.succeeded()) { context.response().setStatusCode(303); context.response().putHeader("Location", "/"); context.response().end(); } else { context.fail(reply.cause()); } }); } -
Callbacks RxJava 1 + 2 Quasar (vertx-sync) Kotlin coroutines (core)
(codegen) (codegen) (codegen) (codegen) -
foo.a(1, res1 -> { if (res1.succeeded()) { bar.b("abc", 1, res2
-> { if (res.succeeded()) { baz.c(res3 -> { dosomething(res1, res2, res3, res4 -> { // (...) }); }); } }); } }); “Callback hell” -
jdbc.rxGetConnection().flatMap(conn -> { // Now chain some statements using flatmap
composition Single<ResultSet> resa = conn.rxUpdate("CREATE TABLE test(col VARCHAR(20))") .flatMap(result -> conn.rxUpdate("INSERT INTO test (col) VALUES ('val1')")) .flatMap(result -> conn.rxUpdate("INSERT INTO test (col) VALUES ('val2')")) .flatMap(result -> conn.rxQuery("SELECT * FROM test")); return resa.doAfterTerminate(conn ::close); }).subscribe(resultSet -> { // Subscribe to the final result System.out.println("Results : " + resultSet.getRows()); }, err -> { System.out.println("Database problem"); err.printStackTrace(); }); -
try (SQLConnection conn = awaitResult(jdbc ::getConnection)) { // Create a
table Void v = awaitResult(h -> conn.execute("CREATE TABLE test(col VARCHAR(20))", h)); // Insert some stuff for (int i = 0; i < 10; i ++) { int ii = i; UpdateResult res = awaitResult(h -> conn.update("INSERT INTO test (col) VALUES ('val" + ii + "')", h)); System.out.println("Rows updated: " + res.getUpdated()); } // Select the results ResultSet res = awaitResult(h -> conn.query("SELECT * FROM test", h)); System.out.println("Selected " + res.getNumRows() + " results"); res.getResults().forEach(System.out ::println); } -
val consumer = vertx.eventBus().localConsumer<String>("a.b.c") consumer.handler { message -> println("Consumer received:
${message.body()}") message.reply("pong") } // Wait until the consumer has properly registered asyncResult<Void> { h -> consumer.completionHandler(h) } // Send a message and wait for a reply val reply = asyncResult<Message<String >> { h -> vertx.eventBus().send("a.b.c", "ping", h) } println("Reply received: ${reply.body()}") -
RxJava Data and events flows Organising transformation of data and
coordination of events Makes most sense with many sources of events -
Motivation Future<List<T >> is not appropriate Dealing with latencies Functional
programming influence -
Iterable / Observable try { for (String item : it)
{ ➊ } ➌ } catch (Throwable e) { ➋ } observable.subscribe(item -> { ➊ // onNext }, error -> { ➋ // onError }, () -> { ➌ // onCompleted }); -
Rxified APIs Each API type (annotated with @VertxGen) has its
prefix io.vertx replaced by io.vertx.rxjava io.vertx.core.Vertx / io.vertx.rxjava.Vertx etc… -
RxJava 2 Better performances Based on reactive-streams Null values forbidden
More reactive types Observable / Flowable Single / Maybe / Completable Upcoming in 3.5.0 -
MIDI Signals Channel control Note on / off Start /
Stop Pitchbend Clock (…) https://github.com/cotejp/webmidi https://webaudio.github.io/web-midi-api/ https://github.com/jponge/webmidi-sequencer # https://github.com/jponge/boiler-vroom 24 clock messages per bar -
3 4 5 6 IceCast VLC 2 Chrome / DJ
Booth app Clients 7 NuProcess MIDI Event Bus Event Bus -
3 4 5 6 IceCast VLC Clients 7 OGG/Vorbis OGG/Vorbis
MP3 MP3 HTTP 1.0 HTTP 1.1 HTTP Client Event bus → Chunked HTTP -
Future<Void> icecastFuture = Future.future(); icecastProcess = Process.create(vertx, "icecast", asList("-c", “etc/icecast.xml"));
icecastProcess.start(process -> icecastFuture.complete()); icecastProcess.exitHandler(exitCode -> { String message = "icecast exited with status code " + exitCode; if (!icecastFuture.isComplete()) { icecastFuture.fail(message); } logger.warn(message); }); icecastProcess.stdout().handler(logger ::info); icecastProcess.stderr().handler(logger ::warn); Reactive processes -
HttpClientOptions options = new HttpClientOptions() .setDefaultHost("localhost") .setDefaultPort(8001); vertx.createHttpClient(options) .getNow("/stream.mp3", response
-> { eventBus.publish(ANNOUNCE_DESTINATION, CONNECTED_MESSAGE); logger.info("Connected to the VLC transcoder"); response.handler(buffer -> eventBus.publish("boilervroom.audiostream", buffer)); response.exceptionHandler(t -> { logger.error(t); eventBus.publish(ANNOUNCE_DESTINATION, DISCONNECTED_MESSAGE); }); response.endHandler(v -> { logger.warn("Connection closed to the VLC transcoder"); eventBus.publish(ANNOUNCE_DESTINATION, DISCONNECTED_MESSAGE); }); }); Icecast to subscribers -
router.get("/audiostream").handler(context -> { logger.info("New streaming client: " + context.request().remoteAddress()); HttpServerResponse
response = context.response(); response.setStatusCode(200); response.setChunked(true); response.putHeader("Content-Type", "audio/mpeg"); MessageConsumer<Buffer> consumer = eventBus.consumer("boilervroom.audiostream"); consumer.bodyStream().handler(buffer -> { if (!response.writeQueueFull()) { response.write(buffer); } }); ( ...) Back-pressure -
SockJSHandler sockJSHandler = SockJSHandler.create(vertx); PermittedOptions permittedOptions = new PermittedOptions() .setAddressRegex(“boilervroom
\\ ..+"); BridgeOptions bridgeOptions = new BridgeOptions() .addInboundPermitted(permittedOptions) .addOutboundPermitted(permittedOptions); sockJSHandler.bridge(bridgeOptions); router.route("/eventbus /*").handler(sockJSHandler); SockJS event bus bridge: “killer feature” SockJS Event Bus -
const eventBus = new EventBus("/eventbus") traktorIn.addListener("controlchange", 5, (event) => {
eventBus.publish("boilervroom.vu-meter", { type: "volume-level", value: event.value }) }) eventBus.publish("boilervroom.fromtraktor", { type: "filter", number: 1, value: (event.value !== 0) }) -
ITopic<Object> topic = Hazelcast.newHazelcastInstance().getTopic("vu-meter"); vertx.eventBus().consumer("boilervroom.vu-meter", event -> { JsonObject body
= (JsonObject) event.body(); topic.publish(body.getInteger("value")); }); Publish audio levels using Hazelcast -
Unified end-to-end reactive model + ecosystem (not just APIs…) For
all kinds of distributed applications (even the small-scale ones) Flexible toolkit, not a framework (your needs, your call) -
https: //youtu.be/ZkWsilpiSqw 8 Applications réactives avec Eclipse Vert.x 9 Building
Reactive Microservices in Java https: //goo.gl/ep6yB9 https: //youtu.be/ApGNp4uHKdY 8 Microservices réactifs avec Eclipse Vert.x et Kubernetes 9 Guide to async programming with Vert.x for Java developers https: //goo.gl/AcWW3A