Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ITT 2018 - Dejan Bosanac - Scalable Cloud Messa...

ITT 2018 - Dejan Bosanac - Scalable Cloud Messaging

For successful implementation of distributed systems, flexible and scalable messaging layer is one of the most important components. Setting a static messaging infrastructure and provisioning it manually doesn’t fit well with the cloud-native development model most organisations are adopting lately.

enmasse.io provides an open source solution for deploying your own messaging infrastructure in the cloud. It’s based on proven standards and technologies, like AMQP 1.0 and Kubernetes. Additionally, the platform provides a wide range of features expected of the cloud ready messaging solution. Features range from scalability and elasticity to multi-tenancy, simple management and monitoring.

This session will cover the EnMasse project in details. It will start by covering messaging concepts and how they fit the cloud environment. Then we’ll continue with the platform architecture and see how various features are implemented. Finally, we’ll discuss deployment and monitoring options necessary to run a platform like this in the production.

Istanbul Tech Talks

April 17, 2018
Tweet

More Decks by Istanbul Tech Talks

Other Decks in Programming

Transcript

  1. Messaging for the cloud Building a scalable messaging service on

    Kubernetes and OpenShift Dejan Bosanac Senior Software Engineer, Red Hat @dejanb 1
  2. 2 Outline •  What is messaging? •  How to scale

    messaging in general •  EnMasse messaging cloud platform
  3. 3 •  Sending messages ◦  Internally in distributed systems ◦ 

    Externally between systems •  Communication at the application level •  Messages go from sender/producer to receiver/consumer ◦  Asynchronously ◦  Time decoupling What is messaging?
  4. 6 •  Multi Protocol Broker ◦  AMQP 1.0, MQTT, STOMP,

    OpenWire, Artemis Core ◦  JMS 2.0 (API) •  https://activemq.apache.org/artemis/ •  Started as HornetQ JBoss project in 2009 •  In 2014 donated to Apache ActiveMQ (Sub project ActiveMQ Artemis) ActiveMQ Artemis
  5. 8 •  Great performance due to ◦  Reactive Architecture ◦ 

    Efficient message journal •  High-availability: ◦  Shared storage ◦  Replicated journal ActiveMQ Artemis
  6. 9 •  International Standard (ISO/IEC ISO 19464) •  Binary Protocol

    •  Rich feature set: ◦  conversation multiplexing ◦  advanced flow control ◦  Type system ◦  QoS Guarantees •  Symmetrical message exchange ◦  No Broker required AMQP 1.0
  7. 11 •  Lightweight AMQP 1.0 message router written in C

    •  http://qpid.apache.org/components/dispatch-router/ •  Provides flexible and scalable interconnect between AMQP endpoints •  Redundant paths •  Supports waypoints Qpid Dispatch Router
  8. 12 •  Store and Forward ◦  Queue ◦  Topic • 

    Direct ◦  Anycast ◦  Multicast (Broadcast) Addressing semantics
  9. 16 •  Open source cloud messaging running on Kubernetes and

    OpenShift •  enmasse.io •  github.com/enmasseproject/enmasse EnMasse Messaging-as-a-Service
  10. 17 •  Support for store-and-forward and direct messaging mechanisms • 

    Multiple communication patterns: request-response, publish-subscribe and competing consumers •  Scale and elasticity of message brokers •  AMQP 1.0 and MQTT support •  Simple setup, management and monitoring •  Multitenancy: manage multiple independent instances •  Deploy on-premise or in the cloud EnMasse Features
  11. 20 Address Space { "apiVersion": "v1/enmasse", "kind": "AddressSpace", "metadata": {

    "name": "myspace" }, "spec": { "type": "standard", "plan": "unlimited-standard" } }
  12. 21 Address Space Plan "addressPlans": [ "small-queue", "small-anycast" ],"resources": [

    { "name": "router", "min": "0.0", "max": "2.0" }, { "name": "broker", "min": "0.0", "max": "3.0" }, { "name": "aggregate", "min": "0.0", "max": "5.0" } ]
  13. 22 Address Plan "addressType": "queue", "metadata": { "name": "small-queue" },

    "requiredResources": [ { "name": "router", "credit": 0.2 }, { "name": "broker", "credit": 0.3 } ]
  14. 23 Address { "apiVersion": "enmasse.io/v1", "kind": "AddressList", "items": [ {

    "spec": { "address": "myqueue", "type": "queue", "plan": "small-queue" } } ] }
  15. MQTT support 26 •  MQTT gateway ◦  Handles connections with

    remote MQTT clients ◦  Bridges MQTT - AMQP protocols •  MQTT lwt ◦  Provides the “will testament” feature ◦  In charge to recover & send the “will” if client dies •  Doesn’t support ◦  QoS2 ◦  Retained messages
  16. Connecting 29 •  Use TLS with SNI on port 443

    Clients •  Qpid - http://qpid.apache.org/ ◦  JMS 2.0 client ◦  Python, C, C++, Ruby •  JavaScript RHEA client - https://github.com/amqp/rhea •  Vert.x Proton - https://github.com/vert-x3/vertx-proton
  17. Resources 30 •  EnMasse - http://enmasse.io •  ActiveMQ Artemis -

    https://activemq.apache.org/artemis/ •  Qpid Dispatch Router - http://qpid.apache.org/components/dispatch-router/