Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Time Series (Cloud Native Krakó...

Introduction to Time Series (Cloud Native Kraków, April 2019)

Where you at this talk? Feedback can be left:

https://rawko.de/feedback

Time-Series has been the fastest growing database category, rated, by DBEngines, for over 2 years; yet, less than 15% store their time-series data in a time-series database. Do you?

One could, accurately, say that time-series data is as old as the universe; but it wasn't until the mid-19th century that the first article was published on the concept: A Comparison of the Fluctuations in the Price of Wheat and in the Cotton and Silk Imports into Great Britain by J. H. Poynting (March 1884).

Time-Series data is so natural and common that you actually consume, evaluate, and utilise it everyday; when you're:

- Paying for your morning coffee
- Sighing at the "Delayed" notice on your commute
- Hugging your coffee mug as you process your email inbox

In this talk we will look at the different types of time-series data and how to use that to drive observations, understanding, and automation.

"All data becomes an order of magnitude more interesting on the time dimension" - Lets see why.

David McKay

April 18, 2019
Tweet

More Decks by David McKay

Other Decks in Technology

Transcript

  1. Alcibiades suddenly raised the Athenian ensign in the admiral shop,

    and fell upon those galleys of the Peloponnesians …
  2. In the 14th century, things hadn’t actually advanced much more.

    The Black Book of Admiralty listed 2 signals: 1 flag or 2 flags Encoding
  3. Finally, in the late 17th century; a French system existed

    (Mahé de la Bourdonnais) with 10 coloured flags, representing 0-9 Encoding
  4. We take the alphabet and divide it into five parts,

    each consisting of five letters.
  5. The earliest form of a company which issued public shares

    was the case of the publicani during the Roman Republic. The Romans Did It
  6. Like modern joint-stock companies, the publicani were legal bodies independent

    of their members whose ownership was divided into shares, or partes. There is evidence that these shares were sold to public investors and traded in a type of over-the-counter market in the Forum, near the Temple of Castor and Pollux. The shares fluctuated in value, encouraging the activity of speculators, or quaestors.
  7. A Comparison of the Fluctuations in the Price of Wheat

    and in the Cotton and Silk Imports into Great Britain First Documented Time Series J. H. Poynting Journal of the Statistical Society of London Vol. 47, No. 1 (Mar., 1884), pp. 34-74
  8. What is all this? This is the first (or one

    of) paper that added the dimension of time to statistical mathematics
  9. What Will We Cover? ➔ Time Series Data ➔ Time

    Series Databases ➔ Getting to Know InfluxDB ➔ Value of Time Series Data ➔ Advancing Monitoring to Time Series
  10. Irregular (Events) ➔ Unpredictable ➔ Inconsistent Intervals What is Time

    Series Data? Regular (Metrics) ➔ Predictable ➔ Evenly Distributed
  11. Regular / Metrics ★ CPU Usage ★ Memory Usage ★

    Ping Time for Google.com ★ Number of Processes
  12. Irregular / Events ★ User Clicked Login ★ Authentication Failed

    ★ CI Published v1.3.1 ★ Network Cable Unplugged
  13. IoT / Sensor ➔ Thermostats ➔ Electric Engines ➔ Smart

    Things ➔ GPS ➔ Fitbits Real Time Analytics ➔ Website Tracking ➔ Stock Prices ➔ Currency Exchange Rates Use Cases for Time Series Monitoring ➔ Infrastructure ➔ Applications ➔ Third Party Services
  14. Time Series databases are optimized for collecting, storing, retrieving, and

    processing of Time Series data. Time Series Databases
  15. ➔ High Write Frequency ➔ Reads are range scans ➔

    TTL / Lifecycle Management ➔ Time Sensitive Time Series Databases
  16. Fields ➔ Not Indexed ➔ Multiple Data Types Tags &

    Fields Tags ➔ Indexed ➔ String Types
  17. The value of all time series data is directly correlated

    with the resolution that the data is available Value of Time Series Data
  18. Rollups with Continuous Queries CREATE CONTINUOUS QUERY "rollup_1h" ON "nasdaq"

    BEGIN SELECT mean(price) INTO yearly FROM weekly GROUP BY time(1h) END
  19. Service A Database A Service B Service B Service C

    Database B Database C Virtual Network Service Mesh Canary Ummm?
  20. Cloud Native Architectures Convenience Vs. Cost You can treat the

    symptoms for a while … Upgrade Your Monitoring
  21. ➔ Look at last weeks, months, and years of data

    ➔ Use tags to build correlation ➔ Get Statistical ◆ INTEGRAL() ◆ LINEAR_PREDICTION() ◆ DERIVATIVE() ◆ MOVING_AVERAGE() ◆ HOLT_WINTERS() Causality
  22. ➔ Use a TSDB ➔ Understand Cost / Select Tags

    Wisely ➔ Understand the resolution you need for 1m, 6m, > 12m Summary ➔ Rollup metrics ➔ Perform outlier detection on events ➔ Build automation, dashboarding, and reporting around your data (past, present, and future)