Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Time series & monitoring with InfluxDB and the ...

Paul Dix
October 14, 2017

Time series & monitoring with InfluxDB and the TICK stack

Intro talk to time series data, InfluxDB, and the TICK stack

Paul Dix

October 14, 2017
Tweet

More Decks by Paul Dix

Other Decks in Programming

Transcript

  1. Time series & monitoring with InfluxDB & the TICK stack

    Paul Dix paul@influxdb.com @pauldix
  2. Regular time series t0 t1 t2 t3 t4 t6 t7

    Samples at regular intervals
  3. Irregular time series t0 t1 t2 t3 t4 t6 t7

    Events whenever they come in
  4. Store: • OSS - MIT License • Written in Go

    • SQL-ish query language • Time Series Merge Tree storage engine & inverted index • Retention Policies • Continuous Queries
  5. Create a retention policy CREATE RETENTION POLICY <rp-name> ON <db-name>

    DURATION <duration> REPLICATION <n> [DEFAULT]
  6. Create a retention policy CREATE RETENTION POLICY <rp-name> ON <db-name>

    DURATION <duration> REPLICATION <n> [DEFAULT] CREATE RETENTION POLICY high_precision ON mydb DURATION 7d REPLICATION 3 DEFAULT
  7. Create a retention policy CREATE RETENTION POLICY <rp-name> ON <db-name>

    DURATION <duration> REPLICATION <n> [DEFAULT] CREATE RETENTION POLICY high_precision ON mydb DURATION 7d REPLICATION 3 DEFAULT Writes will go into this RP unless otherwise specified
  8. Discovery SHOW MEASUREMENTs SHOW MEASUREMENTS where host = 'serverA' SHOW

    TAG KEYS SHOW TAG KEYS from CPU SHOW TAG VALUES from CPU WITH KEY = 'region'
  9. Discovery SHOW MEASUREMENTs SHOW MEASUREMENTS where host = 'serverA' SHOW

    TAG KEYS SHOW TAG KEYS from CPU SHOW TAG VALUES from CPU WITH KEY = 'region' SHOW SERIES
  10. Discovery SHOW MEASUREMENTs SHOW MEASUREMENTS where host = 'serverA' SHOW

    TAG KEYS SHOW TAG KEYS from CPU SHOW TAG VALUES from CPU WITH KEY = 'region' SHOW SERIES SHOW SERIES where service = 'redis'
  11. Aggregates select percentile(90, value) from cpu where time > now()

    - 1d group by time(10m), region Group by a tag
  12. Where against Regex (field) select value from some_log_series where value

    =~ /.*ERROR.*/ and time > "2014-03-01" and time < "2014-03-03"
  13. Where against Regex (tag) select value from some_log_series where host

    =~ /.*asdf.*/ and time > "2014-03-01" and time < “2014-03-03" group by host
  14. Functions min max percentile first last stddev mean count sum

    median distinct count(distinct) difference moving_average
  15. Continuous queries CREATE CONTINUOUS QUERY "10m_event_count" ON mydb BEGIN SELECT

    count(value) INTO "6_months".events FROM events GROUP BY time(10m) END;
  16. Collect: • OSS - MIT License • Written in Go

    • Agent deployed across infrastructure • Input plugins - system, docker, postgres, mysql, cassandra, elastic, hadoop, redis, nginx, apache, etc. • Output plugins - InfluxDB, Graphite, Kafka, etc.
  17. Visualize: • OSS - AGPL License • Written in Go,

    React, dygraph • UI for administering TICK stack • Ad-hoc exploration and visualization • Create monitoring and alerting rules in Kapacitor • Query builder, TICK script editor, and more
  18. Process: • OSS - MIT License • Written in Go

    • Process, monitor, alert, act/execute • TICK script • Streaming & Batch • Store data back into InfluxDB • User Defined Functions • Service Discovery & Pull