Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
From 1 to 20 million users the technical story ...
Search
Matthieu Moquet
June 06, 2015
Programming
1
1.7k
From 1 to 20 million users the technical story of BlaBlaCar
IPC Spring Berlin 2015
Matthieu Moquet
June 06, 2015
Tweet
Share
More Decks by Matthieu Moquet
See All by Matthieu Moquet
Migrating to containers
mattketmo
0
350
CQRS & Event Sourcing — LavaJUG
mattketmo
1
480
Running container at BlaBlaCar — While 42 (San-Francisco)
mattketmo
0
360
ForumPHP 2015
mattketmo
0
850
CQRS & Event Sourcing
mattketmo
7
1.3k
Take care of your logs with ELK
mattketmo
8
690
Event Sourcing
mattketmo
9
3.9k
Localization should not be a pain (anymore)
mattketmo
1
730
Understanding git
mattketmo
5
600
Other Decks in Programming
See All in Programming
2万ページのSSG運用における工夫と注意点 / Vue Fes Japan 2024
chinen
3
1.3k
アジャイルを支えるテストアーキテクチャ設計/Test Architecting for Agile
goyoki
7
2.8k
デプロイを任されたので、教わった通りにデプロイしたら障害になった件 ~俺のやらかしを越えてゆけ~
techouse
52
32k
とにかくAWS GameDay!AWSは世界の共通言語! / Anyway, AWS GameDay! AWS is the world's lingua franca!
seike460
PRO
1
540
gopls を改造したら開発生産性が高まった
satorunooshie
8
240
飲食業界向けマルチプロダクトを実現させる開発体制とリアルな現状
hiroya0601
1
390
Kotlin2でdataクラスの copyメソッドを禁止する/Data class copy function to have the same visibility as constructor
eichisanden
1
130
Vaporモードを大規模サービスに最速導入して学びを共有する
kazukishimamoto
4
4.3k
PLoP 2024: The evolution of the microservice architecture pattern language
cer
PRO
0
1.6k
のびしろを広げる巻き込まれ力:偶然を活かすキャリアの作り方/oso2024
takahashiikki
1
410
What’s New in Compose Multiplatform - A Live Tour (droidcon London 2024)
zsmb
1
330
Googleのテストサイズを活用したテスト環境の構築
toms74209200
0
270
Featured
See All Featured
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
41
2.1k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
664
120k
Making the Leap to Tech Lead
cromwellryan
132
8.9k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
107
49k
We Have a Design System, Now What?
morganepeng
50
7.2k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
Typedesign – Prime Four
hannesfritz
39
2.4k
RailsConf 2023
tenderlove
29
880
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
226
22k
Documentation Writing (for coders)
carmenintech
65
4.4k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
27
4.2k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
228
52k
Transcript
the technical story of From 1 to 20 million users
Matthieu Moquet @MattKetmo web engineer at
Leader ride-sharing service Our goal is to become the 1st
travel platform
Why this talk? ❖ History of the BlaBlaCar platform ❖
Overview our main technical choices ❖ Understand of our culture & methodologies
v1.0 2005–2006
The prototype
v2.0 (2008) – Plain PHP
2008 2010 2011 2012 v2.0
None
None
Homemade framework + No code convention + Messy = Not
maintainable in the long term
$ wc -‐l lib.trip.php Longest method: 1000+ lines 3678
None
No tests
Going further will be a pain Hiring will be hard
We need to change things
Refactor all the code
Symfony 2.0
Because community matters
Attract new talented & motivated people
Separation of Concerns Dependencies Injection Integration & Unit tests etc.
Embrace best practices
Update product design
None
Agility is our best strength
— Eberhardt Von Kuenheim « The big will not always
eat the small, but the fast ones will overtake the slow ones »
10+ deployments per day
DEVOPS… before it was cool « if you break it
you fix it »
Automation Automatize as much as possible to reduce the time
between dev and prod
Simple development workflow Master Branch
git push origin feat-awesome Deploy on staging environment is as
simple as
Staging env. for QA & product owners https://feat-awesome.staging.blablacar.com
Continuous Integration https://github.com/alexandresalome/behat-launcher
None
Localization Update translations on the fly
Frontend server Databases Backoffice Sync translations Update translations
Download last translations when app boot Update translations Mobile app
CDN Backoffice
openl10n.io
Progressive Rollout ❖ Open new countries with v3 one by
one ❖ It took about 2.5 years to run v3 everywhere ❖ Today we can deploy new features for a set of users (by attributes or random)
Monitoring
ELK
Monitoring
Departure to new horizons…
Opportunity to change backend But we still keep the same
primary database
Database Updates Sync replication & No Master SPOF
Better archiving strategy
Photo Storage Don’t store static BLOB into MySQL. Use an
elastic filesystem storage. MySQL AWS S3
elasticsearch ❖ Horizontally scalable ❖ Geo type index ❖ Aggregations
Varnish Fast reverse proxy cache 30% HIT/MISS Firemode to handle
high traffic (TTL) Be careful with authenticated user blocks (Javascript is your friend)
SPDY (HTTP 2.0) Fast API calls for (compliant) mobile clients
ngx_http_spdy_module
Asynchronous jobs Command Command Handlers Commands Queueing Ack
workers/ mail sms push
image-‐resize indexer cache-‐invalidation elasticsearch-‐indexation trip-‐publication ... github.com/swarrot
Horizontal scalability
Event dispatcher Every business events is dispatched into RabbitMQ Easy
to watch the events (in real-time or batchs)
Data Warehouse Log every business event into Hadoop user.register user.edit_bio
user.left_rating user.post_trip ...
user.register user.edit_bio user.left_rating user.post_trip ... user.register user.post_trip ... Real-Time dashboard
Log every business event into Elasticsearch
Such performances. Many users. Wow.
How to serve the product worldwide?
Datacenter centric One datastore to rule them all? It ain’t
gonna work. ?? ??
Monolithics Micro-Services
Today we are mainly Monolithics But we would love using
more Micro-Services Easier to deploy Development workflow Legacy database Smaller teams Faster deployments Easier to scale out
Gateway Principles (Clean Architecture) Decouple models (not db) Isolate business
& data accesses (in the Monolytics to better decouple in micro services) Restrictive rules Enter the
Gateway RegisterUser PublishTrip PostMessage UpdateBio PostRating Users Messages Trips Business
Data
But this is a long term project…
caching In the short term we invested on
Points of presence & Varnish
Mobile First
But how to cache the API? GET /api/trips?from=Paris&to=Berlin Authorization:
Bearer 7c82e855b0415f27bd92d HTTP/1.1 200 OK { "trips": [...] }
{ Reverse proxy is useless if only the app knows
the authorizations Reverse Proxy Client User Scopes Access Token
Let the reverse proxy check authorization directly Reverse Proxy But
here is a latency problem
Use of stateless token Reverse Proxy So we don’t need
a database anymore
Json Web Token
What mobile apps send GET /api/trips?from=Paris&to=Berlin Authorization: Bearer eyJhbGciOiJIUzI1NiI...
What backend servers receive GET /api/trips?from=Paris&to=Berlin X-‐Auth-‐User: 1337
X-‐Auth-‐Client: android X-‐Auth-‐Scope: user_info,messages
App gets an Access Token from the origin Reverse Proxy
App submit request with Access Token Reverse Proxy transform Access Token header into custom X-Auth headers
Cachable response HTTP/1.1 200 OK Content-‐Type application/json Vary:
X-‐Auth-‐Scope ! { "trips": [ ... ] }
Long term solution to deliver data close to the client
❖ Manage massive amounts of data ❖ High Availability ❖
Multi Datacenter replication
‣ Know the read requests before creating your data models
‣ Create as many tables (KeySpaces) than you have views ‣ Denormalize the data (no join allowed)
CQRS & Event Sourcing ❖ Separate Read & Write ❖
Eventual consistency ❖ But hard to do with legacy software / database See talk PHPTour 2015 at moquet.net
C* C* C* Primary Data C* Read Data User post
message Endpoint
Messaging (micro) service Expose an HTTP API
2008 2012 2013 2014 2015 2016 2 6 15 30+
… ?
blablatech.com We’re hiring Follow us
Thank You Slides available at moquet.net/talks/ipc-2015-blablacar Leave feedbacks at @MattKetmo