Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale the Google Way
Search
Terrence Ryan
July 30, 2015
Technology
1
84
Scale the Google Way
A discussion of issues with Google scaling technologies
Terrence Ryan
July 30, 2015
Tweet
Share
More Decks by Terrence Ryan
See All by Terrence Ryan
Vms, Serverless, or Containers
tpryan
0
560
Go for PHP Developers
tpryan
2
980
Navigating Google Cloud Platform
tpryan
0
360
Which Engine?
tpryan
0
320
Introduction to Containers and Kubernetes
tpryan
2
340
Cloud Next 2017 Roundup
tpryan
2
120
LAMP in Containers
tpryan
1
230
GCP and IoT
tpryan
0
320
30,000,000 Requests in a Hour in the Cloud
tpryan
2
330
Other Decks in Technology
See All in Technology
30万人が利用するチャットをFirebase Realtime DatabaseからActionCableへ移行する方法
ryosk7
5
350
Figma Dev Modeで進化するデザインとエンジニアリングの協働 / figma-with-engineering
cyberagentdevelopers
PRO
1
430
Amazon FSx for NetApp ONTAPを利用するにあたっての要件整理と設計のポイント
non97
1
160
AWSコンテナ本出版から3年経った今、もし改めて執筆し直すなら / If I revise our container book
iselegant
15
4k
スプリントゴールにチームの状態も設定する背景とその効果 / Team state in sprint goals why and impact
kakehashi
2
100
分布で見る効果検証入門 / ai-distributional-effect
cyberagentdevelopers
PRO
4
700
Oracle Cloud Infrastructureデータベース・クラウド:各バージョンのサポート期間
oracle4engineer
PRO
27
12k
【技術書典17】OpenFOAM(自宅で極める流体解析)2次元円柱まわりの流れ
kamakiri1225
0
210
初心者に Vue.js を 教えるには
tsukuha
5
390
カメラを用いた店内計測におけるオプトインの仕組みの実現 / ai-optin-camera
cyberagentdevelopers
PRO
1
120
IaC運用を楽にするためにCDK Pipelinesを導入したけど、思い通りにいかなかった話
smt7174
1
110
新卒1年目が向き合う生成AI事業の開発を加速させる技術選定 / ai-web-launcher
cyberagentdevelopers
PRO
7
1.5k
Featured
See All Featured
GitHub's CSS Performance
jonrohan
1030
460k
Become a Pro
speakerdeck
PRO
24
5k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Imperfection Machines: The Place of Print at Facebook
scottboms
264
13k
How To Stay Up To Date on Web Technology
chriscoyier
788
250k
Keith and Marios Guide to Fast Websites
keithpitt
408
22k
RailsConf 2023
tenderlove
29
880
Fireside Chat
paigeccino
32
3k
BBQ
matthewcrist
85
9.3k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
167
49k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
231
17k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
37
1.8k
Transcript
@tpryan Scale the Google Way Terry Ryan Developer Advocate
@tpryan Who are you?
None
None
None
None
None
None
None
@tpryan First Cluster History
@tpryan First Cluster •Every machine had a role History
@tpryan First Cluster •Every machine had a role •Every machine
had a name History
@tpryan First Rack History
@tpryan First Rack •Machines belong to groups History
@tpryan First Rack •Machines belong to groups •Slack capacity goes
to waste History
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000
@tpryan Today’s Numbers History
@tpryan Today’s Numbers •Over 3 Billion searches a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History
@tpryan Handling this growth took fundamental changes
Networking
70 POP’s 33 Countries B4 WAN
Jupiter Superblock
Jupiter Superblock • 1 Petabit/sec
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
• 10 Libraries of Congress a second
Storage
Colossus Datacenter wide storage
Processing
Productionized Containers
Productionized Containers • Developed as only practical way to manage
Google-scale compute
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.
@tpryan What does this mean for you?
@tpryan
@tpryan Big Table
@tpryan Big Table
@tpryan Big Table MapReduce
@tpryan Big Table MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan
@tpryan Google Cloud Platform
@tpryan Google Cloud Platform Compute
@tpryan Google Cloud Platform Compute Storage
@tpryan Google Cloud Platform Compute Connectivity Storage
@tpryan Google Cloud Platform Compute Connectivity Big Data Storage
@tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big
Data Storage Management
@tpryan Conclusions
None
None
Your App Here
@tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work
is licensed under a Creative Commons Attribution 2.0 Generic License.