Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale the Google Way
Search
Terrence Ryan
July 30, 2015
Technology
1
84
Scale the Google Way
A discussion of issues with Google scaling technologies
Terrence Ryan
July 30, 2015
Tweet
Share
More Decks by Terrence Ryan
See All by Terrence Ryan
Vms, Serverless, or Containers
tpryan
0
540
Go for PHP Developers
tpryan
2
960
Navigating Google Cloud Platform
tpryan
0
360
Which Engine?
tpryan
0
320
Introduction to Containers and Kubernetes
tpryan
2
340
Cloud Next 2017 Roundup
tpryan
2
120
LAMP in Containers
tpryan
1
230
GCP and IoT
tpryan
0
310
30,000,000 Requests in a Hour in the Cloud
tpryan
2
320
Other Decks in Technology
See All in Technology
watsonx.ai Dojo 環境準備について
oniak3ibm
PRO
0
340
スタッフエンジニアの道: The Staff Engineer’s Path
snoozer05
PRO
44
14k
開発生産性を始める前に開発チームができること / optim-improve-development-productivity.pdf
optim
0
110
不動産売買取引におけるAIの可能性とプロダクトでのAI活用
zabio3
0
270
忙しい人のためのLangGraph概要まとめ
__ymgc__
1
200
どこよりも遅めなWinActor Ver.7.5.0 新機能紹介
tamai_63
0
210
プロダクトエンジニアを支えるための開発生産性向上施策
tsukakei
0
150
GC24 Recap: Interface Internals
task4233
0
160
Discovering AI Models
picardparis
4
3.9k
PdMはどのように全てのスピードを上げられるか ~ 非連続進化のための具体的な取り組み ~
sansantech
PRO
4
1.3k
App Router を実プロダクトで採用して見えてきた勘所をちょっとだけ紹介
marokanatani
1
930
事前準備が肝!AI活用のための業務改革
layerx
PRO
1
390
Featured
See All Featured
Ruby is Unlike a Banana
tanoku
96
11k
Fashionably flexible responsive web design (full day workshop)
malarkey
401
65k
From Idea to $5000 a Month in 5 Months
shpigford
379
46k
VelocityConf: Rendering Performance Case Studies
addyosmani
322
23k
Visualization
eitanlees
142
15k
What’s in a name? Adding method to the madness
productmarketing
PRO
21
3k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
230
17k
Build your cross-platform service in a week with App Engine
jlugia
228
18k
It's Worth the Effort
3n
182
27k
Optimizing for Happiness
mojombo
375
69k
The Language of Interfaces
destraynor
153
23k
Thoughts on Productivity
jonyablonski
66
4.2k
Transcript
@tpryan Scale the Google Way Terry Ryan Developer Advocate
@tpryan Who are you?
None
None
None
None
None
None
None
@tpryan First Cluster History
@tpryan First Cluster •Every machine had a role History
@tpryan First Cluster •Every machine had a role •Every machine
had a name History
@tpryan First Rack History
@tpryan First Rack •Machines belong to groups History
@tpryan First Rack •Machines belong to groups •Slack capacity goes
to waste History
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000
@tpryan Today’s Numbers History
@tpryan Today’s Numbers •Over 3 Billion searches a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History
@tpryan Handling this growth took fundamental changes
Networking
70 POP’s 33 Countries B4 WAN
Jupiter Superblock
Jupiter Superblock • 1 Petabit/sec
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
• 10 Libraries of Congress a second
Storage
Colossus Datacenter wide storage
Processing
Productionized Containers
Productionized Containers • Developed as only practical way to manage
Google-scale compute
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.
@tpryan What does this mean for you?
@tpryan
@tpryan Big Table
@tpryan Big Table
@tpryan Big Table MapReduce
@tpryan Big Table MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan
@tpryan Google Cloud Platform
@tpryan Google Cloud Platform Compute
@tpryan Google Cloud Platform Compute Storage
@tpryan Google Cloud Platform Compute Connectivity Storage
@tpryan Google Cloud Platform Compute Connectivity Big Data Storage
@tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big
Data Storage Management
@tpryan Conclusions
None
None
Your App Here
@tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work
is licensed under a Creative Commons Attribution 2.0 Generic License.