How Uber leverages JavaScript across its engineering orgs. From building frontend properties with a full JavaScript stack using express and react to the tools and infrastructure we use to automate best practices.
worldwide Jun ‘16 Launched self-driving car project Feb ‘15 May ‘12 1 Million rides worldwide 1 Billion rides worldwide Sep ‘16 First self-driving ride May ‘17 73 countries launched TODAY
• Simple interface with thorough documentation • Lends itself to fast prototyping and quick iteration • Asynchronous, nimble, flexible • Avoid concurrency challenges • Increasingly mature module ecosystem • Today we only use Node.js for web applications
and request for comments to entire eng org • Abstract, Architecture, UX, Ops, Security, Timeline • Start with scaffolding that is purpose built for our stack • Skip the boilerplate while following best practices • Deploy with standardized global infrastructure • Automate everything in a consistent way • Playbooks • Operationalize teams at scale
99.99% of all the mobile traffic with >1000 HTTP endpoints. • Acts as a router to 3400+ microservices • A single place for monitoring and documentation Marketplace Gateway Real-time API is the only interface
off 2nd Drop- off Logistics Challenges ... • How to optimize for overall marketplace efficiency? • How to dynamically price trips and balance efficiency? • How to maximize pick-up efficiency (pick up zones, pick up suggestions)? • How to minimize wait times for drivers and riders? • How to deal with different street configurations? • Combine different logistics products (Pool, Eats, Rush) UberPool is a traveling salesman problem on steroids
Jan 2011 - "Dispatch" - Node.JS/MongoDB Jan 2011 - "API" - Python/SQLAlchemy/MySQL Feb 2012 - Dispatch swaps MongoDB for Redis May 2012 - Dispatch adds on fallback Jan 2013 - First non-API Python services Feb 2013 - API switched to Postgres Mar 2014 - New Python services use MySQL Mar 2014 - Schemaless begins, must finish before Postgres collapse Sep 2014 - First Schemaless - trips out of Postgres Aug 2015 - Dispatch X.0 / Ringpop / Riak Jan 2016 - Go, Java, More Abstractions May 2017 - UberFx for Go, Cloud
can turn into a ball of mud quickly • Optimizes development velocity of early stage projects • Consolidates operational overhead in one place • Monolith isn’t as bad when there is a single team that owns it
Product-specific velocity • Move the business faster • Able to solve scaling issues • Systems as well as company • Training, documentation, tooling • Language/platform independence • Best tool for the job • Reliability of the system • Independent • Scalable • Testable Pros What’s good • Increased complexity • Naming + Discovery • Eventual consistency • Performance Impact • Distributed tracing • Testing • Operational overhead • Monitoring • Continuous Integration • Deployment • Documentation • Instrumentation • Security • Repeated code/effort Cons What’s not so good
to distributed applications. It maintains a consistent hash ring on top of a membership protocol and provides request forwarding as a routing convenience. TChannel TChannel is a networking framing protocol used for general RPC, supporting out- of-order responses at extremely high performance where intermediaries can make a forwarding decision quickly. Client libraries available in Go, Java, Node and Python. Jaeger Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It can be used for monitoring microservice-based architectures. Client libraries available in Go, Java, Node and Python using the OpenTracing standard.
ultra performant backend systems (p99 for max latency) • Early on it made it quick to iterate, but as the size of the team scaled the developer velocity started to slow down • Microservices enforce a tight interface so having static typing enables large teams to catches issues earlier. It really has an impact with 100+ devs. • Quick to learn, but easy to write poor quality code • Enables you to move fast, but allows for sloppy code • Great ecosystem of small libraries, but many were immature compared to Java/Go • Don't be afraid of writing c++ Growing pains learned the hard way
for browsers; using RPC is better for computer-to-computer requests • After a certain age, microservices should become immutable • Having multiple languages allows for team preferences, but segregates developers based on language and prevents easy re-use of code across services • Company > Team > Self • Monorepos allow for changes to be made across multiple services atomically, but prevent future open-sourcing and subset checkouts • Performance problems are difficult to debug cross-language without standardised service dashboards and observability tools • Logging should never slow production down; in a failure storm, the logging system should drop rather than delay • Everything is a tradeoff: Be intentional Growing pains learned the hard way
foundation, all of these applications could have been built off of entirely different tools and architectures, creating a higher likelihood of security vulnerabilities, duplicated effort across teams, technical debt, minimal UI consistency, etc.
with caching • Yeoman like scaffolding for bootstrap new apps with best practices • Customized middlewares for our Express based app framework • Authentication, Security (XSS, CSRF, CSP), Metrics, Logging, I18n, Errors, Instrumentation, Analytics • Bootstrap like UI component library build on top of React.js • RPC library for intelligently using http/tchannel • Shared Build + Test Gulp tasks with hot reloading, ES6/JSX, asset versioning and deployment pipelines, testing + linting • Universal rendering for server/client
that serve web pages) from our backend services (those that get data from some database) • Services can take advantage of universal rendering + tooling • You can utilize languages better suited for your backend services (Java, Go) • You can deploy and scale your services separately • You can reuse the backend service API for other clients
sites with flexible components you don’t have to maintain yourself • Standard UI components • Optimized for performance • Encapsulated style • Consistent look and feel throughout all components that fit brand • Analytics and instrumentation baked-in • Seamless upgrades to new versions
{ injectStyle } from 'styletron-utils'; // Create a Styletron instance const styletron = new Styletron(); const className = injectStyle(styletron, { color: 'red', display: 'inline-block', fontSize: '1.6em' }); // Css is injected into the page // and a class name is returned
React • react-map-gl: A React interface to MapboxGL-js • deck.gl: Layered WebGL approach system for visualization • luma.gl: A JavaScript WebGL Framework for Data Visualization
popular libraries like stack.gl • Shader library with a 64-bit floating point emulation package • Advanced debugging, tracing, error checking for WebGL luma.gl A JavaScript WebGL Framework for Data Visualization
other talks by engineers far smarter than myself • Lessons learned from scaling to 2000 engineers and 1000+ services - Matt Ranney • Uber Architecture: Moving Bits and Atoms at Scale - Andrii Iasynetskyi • Thanks to the many amazing people of the Marketplace Platform, Web Platform, Developer Platform, and Visualization Teams at Uber! • Come join us and work with these amazing people. Hiring globally (San Francisco, Amsterdam, Sofia, ...)
dustinwhittle
spatial_ai_network
minorun365
shotashiratori
twada
muziyoshiz
oomatomo
aiandrox
zigorou
qt_luigi
sat
rokuosan
lycorp_recruit_jp
bkeepers
soudai
trallard
phodgson
maggiecrowley
eileencodes
chriscoyier
jakevdp
bermonpainter
destraynor
keathley
notwaldorf
