Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale Agnostic Python
Search
Sean T Allen
August 17, 2017
Programming
0
240
Scale Agnostic Python
Presented to the NYC Python meetup August 17, 2017
Sean T Allen
August 17, 2017
Tweet
Share
More Decks by Sean T Allen
See All by Sean T Allen
On Deny Capabilities for Safe, Fast Actors
seantallen
1
120
Data corrupting architectures we know and love - VelocityConf San Jose 2019
seantallen
1
120
Data corrupting architectures we know and love
seantallen
0
120
Scale-Independent Python
seantallen
2
240
Pat Helland and Me: How to Build Stateful Distributed Applications That Can Scale Almost Infinitely - Velocity NY October 2018
seantallen
0
95
Adventures in Cgo Preformance
seantallen
0
670
Pat Helland and me: A talk about “Life Beyond Distributed Transactions: An Apostate’s Opinion”
seantallen
0
680
Pat Helland and me: How to build stateful distributed applications that can scale almost infinitely - Salesforce July 2018
seantallen
0
180
Pony: How I learned to stop worrying and embrace an unproven technology - OSCON 2018
seantallen
0
510
Other Decks in Programming
See All in Programming
AIのメモリー
watany
12
1.1k
型で語るカタ
irof
2
890
MCPで実現できる、Webサービス利用体験について
syumai
7
2.3k
GPUを計算資源として使おう!
primenumber
1
310
Streamlitで実現できるようになったこと、実現してくれたこと
ayumu_yamaguchi
2
250
kiroでゲームを作ってみた
iriikeita
0
110
11年かかって やっとVibe Codingに 時代が追いつきましたね
yimajo
0
230
Vibe Codingの幻想を超えて-生成AIを現場で使えるようにするまでの泥臭い話.ai
fumiyakume
21
9.8k
React は次の10年を生き残れるか:3つのトレンドから考える
oukayuka
41
16k
「次に何を学べばいいか分からない」あなたへ──若手エンジニアのための学習地図
panda_program
3
690
0から始めるモジュラーモノリス-クリーンなモノリスを目指して
sushi0120
0
230
Claude Code で Astro blog を Pages から Workers へ移行してみた
codehex
0
170
Featured
See All Featured
How to Think Like a Performance Engineer
csswizardry
25
1.8k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.5k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.9k
Become a Pro
speakerdeck
PRO
29
5.5k
We Have a Design System, Now What?
morganepeng
53
7.7k
RailsConf 2023
tenderlove
30
1.2k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
21
1.4k
Git: the NoSQL Database
bkeepers
PRO
431
65k
Raft: Consensus for Rubyists
vanstee
140
7k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
8
720
Facilitating Awesome Meetings
lara
54
6.5k
Transcript
SCALE AGNOSTIC PYTHON HOW TO SCALE YOUR PYTHON APPLICATION WITHOUT
ANY CODE CHANGES*
SEAN T. ALLEN VP OF ENGINEERING AT SENDENCE AUTHOR OF
“STORM APPLIED” @SEANTALLEN
SCALE AGNOSTIC COMPUTING
THE PROBLEM THE LITTLE APPLICATION THAT COULD… UNTIL IT COULDN’T.
Scaling an application from “little data” to “a little more
data” to “big data” is hard.
OPTIONS
Rewrite the application Rewrite to scale across multiple processes Example:
“multiprocessing” Python library
Run more versions Smart loader to load data in. Perhaps
via a job queue like Celery.
Go scale agnostic Write you application against a scale agnostic
API that runs on a scale aware platform
SENDENCE WALLAROO SCALE AGNOSTIC COMPUTING FOR PYTHON
Sendence Wallaroo Simple Scale Agnostic API Managed in-memory state Guaranteed
Message Processing
“We handle the hard infrastructure plumbing so you don’t have
to.” – The Wallaroo Team
LEARN MORE… sendence.com Sendence Engineering on Twitter: @SendenceEng Pat Helland’s
“Life Beyond Distributed Transactions”