Scale Agnostic Python

Presented to the NYC Python meetup August 17, 2017

Transcript

1. SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT ANY CODE CHANGES*
2. SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF “STORM APPLIED” @SEANTALLEN
3. SCALE AGNOSTIC COMPUTING
4. THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
5. Scaling an application from “little data” to “a little more data” to “big data” is hard.
6. OPTIONS
7. Rewrite the application Rewrite to scale across multiple processes Example: “multiprocessing” Python library
8. Run more versions Smart loader to load data in. Perhaps via a job queue like Celery.
9. Go scale agnostic Write you application against a scale agnostic API that runs on a scale aware platform
10. SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
11. Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed Message Processing
12. “We handle the hard infrastructure plumbing so you don’t have to.” – The Wallaroo Team
13. LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s “Life Beyond Distributed Transactions”