Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Pat Helland and me: How to build stateful distr...

Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018

In 2007, Pat Helland published “Life Beyond Distributed Transactions: An Apostate’s Opinion,” in which he conducts a thought experiment on how to design a distributed database that can scale almost infinitely. While the paper explicitly addresses distributed database design, Sean Allen shows that the ideas are far more widely applicable, particularly in scaling stateful applications. Sean explores some of Helland’s ideas through practical examples from his experience building data processing systems using tools like Apache Storm and, more recently, developing a stateful distributed stream processor at Wallaroo Labs.

Sean T Allen

July 23, 2018
Tweet

More Decks by Sean T Allen

Other Decks in Technology

Transcript

  1. SEAN T. ALLEN VP OF ENGINEERING AT WALLAROO LABS MEMBER

    OF THE PONY CORE TEAM AUTHOR OF “STORM APPLIED” LOVER OF FRENCH STREET ART @SEANTALLEN @WALLAROOLABS @PONYLANG
  2. MARKET SPREAD REAL-TIME “SOMETHING AIN’T RIGHT” TRADE CHECKS Market Spread

    State Market Data Orders Update APPL Check MSFT Rejections
  3. MARKET SPREAD TWO SOURCES OF DATA Market Spread State Market

    Data Orders Update APPL Check MSFT Rejections
  4. STATE OBJECTS WITH A KEY FOR EACH OBJECT APPL IBM

    MSFT AMZN INTC NVDA Market Data Update
  5. STATE OBJECTS WHERE WE MAY FROM INCOMING DATA’S KEY APPL

    IBM MSFT AMZN INTC NVDA Market Data MSFT
  6. STATE OBJECTS TO THE STATE OBJECT FOR THAT KEY APPL

    IBM MSFT AMZN INTC NVDA Market Data MSFT