Upgrade to Pro — share decks privately, control downloads, hide ads and more …

An Applicative Application

Bucharest FP
October 01, 2018

An Applicative Application

Bucharest FP

October 01, 2018
Tweet

More Decks by Bucharest FP

Other Decks in Programming

Transcript

  1. An Applicative Application How to obtain OpenTSDB query batching by

    exploiting the applicative structure of a computation.
  2. Today's Roadmap • The Problem • OpenTSDB Overview — Time

    Series Data Storage • Business Use Case — Time Series Data Aggregation
  3. Today's Roadmap • The Problem • OpenTSDB Overview — Time

    Series Data Storage • Business Use Case — Time Series Data Aggregation • The Solution
  4. Today's Roadmap • The Problem • OpenTSDB Overview — Time

    Series Data Storage • Business Use Case — Time Series Data Aggregation • The Solution • Inspiration
  5. Today's Roadmap • The Problem • OpenTSDB Overview — Time

    Series Data Storage • Business Use Case — Time Series Data Aggregation • The Solution • Inspiration • Intuitions — Functions, Monads & Applicatives
  6. Today's Roadmap • The Problem • OpenTSDB Overview — Time

    Series Data Storage • Business Use Case — Time Series Data Aggregation • The Solution • Inspiration • Intuitions — Functions, Monads & Applicatives • Code
  7. Today's Roadmap • The Problem • OpenTSDB Overview — Time

    Series Data Storage • Business Use Case — Time Series Data Aggregation • The Solution • Inspiration • Intuitions — Functions, Monads & Applicatives • Code • Future Work
  8. http POST 'localhost:4242/api/put' << EOF [ { "metric": "metric-a", "timestamp":

    1538092800, "value": 10, "tags": { "t1": "v1", "t2": "v2" } }, { "metric": "metric-b", "timestamp": 1538092801, "value": 20, "tags": { "t1": "v1", "t2": "v2" } } ] EOF
  9. http POST 'localhost:4242/api/put' << EOF [ { "metric": "metric-a", "timestamp":

    1538092800, "value": 10, "tags": { "t1": "v1", "t2": "v2" } }, { "metric": "metric-b", "timestamp": 1538092801, "value": 20, "tags": { "t1": "v1", "t2": "v2" } } ] EOF
  10. http POST 'localhost:4242/api/put' << EOF [ { "metric": "metric-a", "timestamp":

    1538092800, "value": 10, "tags": { "t1": "v1", "t2": "v2" } }, { "metric": "metric-b", "timestamp": 1538092801, "value": 20, "tags": { "t1": "v1", "t2": "v2" } } ] EOF
  11. http POST 'localhost:4242/api/put' << EOF [ { "metric": "metric-a", "timestamp":

    1538092800, "value": 10, "tags": { "t1": "v1", "t2": "v2" } }, { "metric": "metric-b", "timestamp": 1538092801, "value": 20, "tags": { "t1": "v1", "t2": "v2" } } ] EOF
  12. http POST 'localhost:4242/api/put' << EOF [ { "metric": "metric-a", "timestamp":

    1538092800, "value": 10, "tags": { "t1": "v1", "t2": "v2" } }, { "metric": "metric-b", "timestamp": 1538092801, "value": 20, "tags": { "t1": "v1", "t2": "v2" } } ] EOF
  13. http POST 'localhost:4242/api/query' << EOF { "start": "2018/05/01 00:10:00", "end":

    "2018/05/01 00:10:05", "timezone": "UTC", "queries": [ { "metric": "metric-a", "aggregator": "none", "tags": {} }, { "metric": "metric-b", "aggregator": "none", "tags": {} } ] } EOF
  14. Business Use Case • We store historical data in OpenTSDB

    • Basis for end-of-month reports • Past: everything computed at end of month from raw OpenTSDB data
  15. Business Use Case • We store historical data in OpenTSDB

    • Basis for end-of-month reports • Past: everything computed at end of month from raw OpenTSDB data • Now: pre-compute aggregations continuously at various frequencies, e.g., 5 minutes, and use these for reporting
  16. Business Use Case • We store historical data in OpenTSDB

    • Basis for end-of-month reports • Past: everything computed at end of month from raw OpenTSDB data • Now: pre-compute aggregations continuously at various frequencies, e.g., 5 minutes, and use these for reporting • We call these aggregations roll-ups
  17. Business Use Case • We store historical data in OpenTSDB

    • Basis for end-of-month reports • Past: everything computed at end of month from raw OpenTSDB data • Now: pre-compute aggregations continuously at various frequencies, e.g., 5 minutes, and use these for reporting • We call these aggregations roll-ups • Decreases time to generate reports
  18. case class Datapoint( timestamp: Instant, value: Double, ) def rollup(

    metric0: List[Datapoint], metric1: List[Datapoint], ... metricN: List[Datapoint], ): Double
  19. def example1[A, B](f: A => B, a: A): B =

    f(a) def example2[F[_]: Applicative, A, B](f: A => B, a: A): F[B] = { val ff = Applicative[F].pure(f) val fa = Applicative[F].pure(a) ff.ap(fa) }
  20. def example1[A, B](f: A => B, a: A): B =

    f(a) def example2[F[_]: Applicative, A, B](f: A => B, a: A): F[B] = { val ff = Applicative[F].pure(f) val fa = Applicative[F].pure(a) ff.ap(fa) } def example3[F[_]: Applicative, A, B](f: F[A => B], a: F[A]): F[B] = f.ap(a)
  21. def example1[A, B](f: A => B, a: A): B =

    f.apply(a) def example2[F[_]: Applicative, A, B](f: A => B, a: A): F[B] = { val ff = Applicative[F].pure(f) val fa = Applicative[F].pure(a) ff.ap(fa) } def example3[F[_]: Applicative, A, B](f: F[A => B], a: F[A]): F[B] = f.ap(a)
  22. // Potential side-effects, but we don't know. def foo(): String

    = ??? def bar(): String = ??? // No data dependency between function calls. val a = foo() val b = bar()
  23. // Potential side-effects, but we don't know. def foo(): String

    = ??? def bar(a: String): String = ??? // Data-dependency between functions calls. val a = foo() val b = bar(a)
  24. // Effectful functions; encoded in signature. def foo(): Task[String] =

    ??? def bar(): Task[String] = ??? // No data dependency between function calls. val a = foo() val b = bar()
  25. // Effectful functions; encoded in signature. def foo(): Task[String] =

    ??? def bar(): Task[String] = ??? // No data dependency between function calls. val a = foo() val b = bar() val c = a.flatMap(_ => b) // sequential
  26. // Effectful functions; encoded in signature. def foo(): Task[String] =

    ??? def bar(): Task[String] = ??? // No data dependency between function calls. val a = foo() val b = bar() val c = a.flatMap(_ => b) // sequential val d = (a, b).mapN((a, b) => ...) c.unsafeRunSync()
  27. // Effectful functions; encoded in signature. def foo(): IO[String] =

    ??? def bar(a: String): IO[String] = ??? // Data-dependency between functions calls. val c = foo().flatMap(a => bar(a)) c.unsafeRunSync()
  28. Flashback Applicative lets us embed function-like DSLs into our programs.

    Applicative lets us perform global optimizations on the Abstract Syntax Tree of an EDSL.
  29. Roll-Ups are Functions Idea: come up with an EDSL to

    model time series data fetching and constrain its usage to an Applicative interface. Then, statically analyze the EDSL to batch and deduplicate issued queries before making the actual network calls.
  30. // Independent computations val mA = val resultAB = rollupAB(metricA(),

    metricB()) val resultAC = rollupAC(metricA(), metricC())
  31. // Optimize: common subexpression elimination val mA = metricA() val

    resultAB = rollupAB(mA, metricB()) val resultAC = rollupAC(mA, metricC())
  32. Observations Semantically-parallel Applicative instances don't bode well with Monad instances.

    If a type is a monadic, a lawful Applicative instance has to be sequential. This is why Cats exposes a Parallel type-class, inspired by PureScript, which allows client code to choose between semantically-sequential and semantically-parallel Applicative instances.