Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Data Science at Scale @ Barricade.io

Data Science at Scale @ Barricade.io

This talk describes the challenges with data science and how we run data analysis at scale at https://Barricade.io

David Coallier

November 04, 2015
Tweet

More Decks by David Coallier

Other Decks in Technology

Transcript

  1. And

  2. Speed Layer: U new behaviour from new data Batch Layer:

    All classified behaviour since T Serve Layer: Batch layer U Speed Layer
  3. Kafka Queue. Distributed messaging system Append-only log Consumers have offsets

    Partition for parallelism Replicate for redundancy Message order guaranteed, per-partition