This talk shows how to extract (structured) value from the huge amount of (unstructured) information that logs contain using InfluxData technologies.
Particularly the task is achieved using two pieces of code I wrote: the https://github.com/influxdata/go-syslog library and the Telegraf Syslog Input Plugin.
The slides demonstrate how to parse logs and to store consequent time-series data into InfluxDB.
At this point it is possible to visualize them via the new Chronograf's Log Viewer, eliciting new meaningful metrics to plot (eg., number of process OOM killed) processing them via a Kapacitor UDF.
The stack used to achieve this is:
- Telegraf with the syslog input plugin, which uses this blazing fast go-syslog parser
- Chronograf with its new Log Viewer
- InfluxDB
- Kapacitor
Companion source code and repository is at http://bit.ly/logs-2-metrics-influx-code