Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MQTT for system administrators (and for the IoT)

MQTT for system administrators (and for the IoT)

They say MQTT is a PUB/SUB protocol for the Internet of Things, which it was originally designed for, but it's also well suited for monitoring machines and services. Presentation given at BSDCan2019 in Ottawa

Avatar for Jan-Piet Mens

Jan-Piet Mens

May 18, 2019
Tweet

More Decks by Jan-Piet Mens

Other Decks in Technology

Transcript

  1. @jpmens: consultant, part-time admin, trainer, small-scale fiddler, loves plain text,

    and things which work. Contributes to Ansible, dreamed up OwnTracks, and chases bugs in open source DNS servers.
  2. MQTT MQTT is a standard, a TCP-based transport, for PUB/SUB

    messaging, designed for unreliable networks, binary payloads up to 256MB, (+2 bytes), fast, lightweight, ideal for low- bandwith, high-latency networks, TLS, authentication, ACLs, TLS- PSK, (payload encryption), keepalive, last will & testament, UTF-8 hierarchical topics, wildcards
  3. CLI utilities mosquitto_sub [-h localhost] [-p 1883] [--cafile file] [--cert

    file --key file] [-u username [-P password]]
 -v -t 'topic/#' subscribe publish mosquitto_pub ... [-r]
 -t topic -m message
  4. Language bindings C, C++, Clojure, Dart, Delphi, Erlang, Elixir, Go,

    Haskell, Java, JavaScript, LotusScript, Lua, .NET, Objective-C, OCaml, Perl, PHP, Python, REXX, Ruby, Smalltalk, Swift, Tcl, …
 
 COBOL
  5. Python API: PUB #!/usr/bin/env python import paho.mqtt.publish as mqtt mqtt.single('conf/hello',

    'Hello MQTT') $ mosquitto_sub -h localhost -v -t 'conf/#' conf/hello Hello MQTT payload topic
  6. Python API: SUB callbacks #!/usr/bin/env python import paho.mqtt.client as paho

    def on_connect(mosq, userdata, flags, rc): mqttc.subscribe("conf/+", 0) def on_message(mosq, userdata, msg): print "%s %s" % (msg.topic, str(msg.payload)) mqttc = paho.Client(userdata=None) mqttc.on_connect = on_connect mqttc.on_message = on_message mqttc.connect("localhost", 1883, 60) mqttc.loop_forever()
  7. Python API: SUB $ mosquitto_pub -t 'conf/thirsty' -m 'Beer time?'

    $ mosquitto_pub -t 'conf/catering' -m 'Coffee is ready' $ ./sub.py conf/thirsty Beer time? conf/catering Coffee is ready
  8. libmosquitto #include <stdio.h> #include <string.h> #include <mosquitto.h> #define MESSAGE "Goodbye,

    cruel world" int main(int argc, char *argv[]) { struct mosquitto *mosq; mosquitto_lib_init(); if ((mosq = mosquitto_new(NULL, true, NULL)) == NULL) { return fprintf(stderr, "Error: Out of memory.\n"); } if (mosquitto_connect(mosq, "192.168.1.130", 1883, 60) != 0) { return fprintf(stderr, "Unable to connect to MQTT broker\n"); } mosquitto_publish(mosq, NULL, /* mid */ "message/adieu", /* topic */ strlen(MESSAGE), /* payload length */ MESSAGE, /* payload */ 1, /* qos */ false); /* retain */ mosquitto_loop(mosq, -1, 1); mosquitto_disconnect(mosq); mosquitto_destroy(mosq); mosquitto_lib_cleanup(); return (0); }
  9. job monitor, reporting https://gist.github.com/jpmens/7101170 $ mosquitto_sub -v -t 'processes/#' processes/run.sh

    Starting processes/monitor/spec1 Starting processes/run.sh Still going strong at Tue Oct 22 15:49:07 CEST 2013 processes/run.sh That's it, folks! #!/bin/sh
 topic="processes/$(basename $0)" mqtt_opts="--quiet -h 192.168.1.130 -p 1883" mqtt() { mosquitto_pub ${mqtt_opts} -t "${topic}" -m "$*" || true } mqtt "Starting"
  10. “That is what I ask you to keep in mind

    as you read this. Think of the possibilities.”
 — Dan Langille
  11. tracking logins (2) #!/bin/sh export PAM_TYPE=open_session export PAM_USER=$LOGNAME export PAM_SERVICE=ssh

    export PAM_RHOST="$(echo $SSH_CLIENT | cut -d' ' -f1)" export PAM_TTY=$SSH_TTY /usr/local/bin/hare mqtt.ww.mens.de
  12. tracking logins (3) https://jpmens.net/2018/03/25/alerting-on-ssh-logins/ $ mosquitto_sub -v -t 'logging/#' -F

    '%I %J' 2019-03-14T10:19:54+0000 { "tst": 1552558794, "topic": "logging/hare", "qos": 0, "retain": 0, "payloadlen": 130, "payload": { "hostname": "canfb12", "remote": "192.168.33.123", "rhost": "192.168.33.1", "service": "sshd", "tst": 1552562392, "tty": null, "user": "jane" } }
  13. tracking logins (4) https://dan.langille.org/2018/04/15/using-mtqq-to-create-a-notification-network-mosquitto-mqttwarn-hare-and-hared/ Date: Thu, 14 Mar 2019 11:19:54

    +0100 From: MQTTwarn <jpm@localhost> Subject: SSH login on canfb12 X-Mailer: mqttwarn login via sshd by jane on canfb12 from 192.168.33.1 at 2019-03-14 12:19:52
  14. telegraf to mqtt [agent] interval = "10s" hostname = "bsdcan"

    [[outputs.mqtt]] servers = ["localhost:1883"] topic_prefix = "telegraf" batch = false data_format = "influx" [[inputs.dns_query]] servers = ["9.9.9.9"] domains = ["example.com"] record_type = "A" [[inputs.exec]] commands = ["./howmany.sh"] name_override = "users_on" data_format = "value" data_type = "integer"
  15. Wemos D1 mini EUR 4.00 Flash/RAM 4MB / 64 KB

    Voltage 3.3V Digital I/O 11 Analog 1
  16. Last Will & Testament #!/usr/bin/env python import paho.mqtt.subscribe as subscribe

    import os def on_message(client, userdata, m): print("%s %s" % (m.topic, m.payload)) lwt = { "topic" : "clients/{0}".format(os.path.basename(__file__)), "payload": "I am no longer" } subscribe.callback(on_message, "test/+", hostname="localhost", will=lwt)
  17. MQTT in the wild Graylog, beaver, Ansible, RabbitMQ, collectd, openHAB,

    Github, Wireshark, Flukso, RemakeElectric, Jenkins, Diamond, OwnTracks, Telegraf