How Uber leverages JavaScript across its engineering orgs. From building frontend properties with a full JavaScript stack using express and react to the tools and infrastructure we use to automate best practices.
worldwide Jun ‘16 Launched self-driving car project Feb ‘15 May ‘12 1 Million rides worldwide 1 Billion rides worldwide Sep ‘16 First self-driving ride May ‘17 73 countries launched TODAY
• Simple interface with thorough documentation • Lends itself to fast prototyping and quick iteration • Asynchronous, nimble, flexible • Avoid concurrency challenges • Increasingly mature module ecosystem • Today we only use Node.js for web applications
and request for comments to entire eng org • Abstract, Architecture, UX, Ops, Security, Timeline • Start with scaffolding that is purpose built for our stack • Skip the boilerplate while following best practices • Deploy with standardized global infrastructure • Automate everything in a consistent way • Playbooks • Operationalize teams at scale
99.99% of all the mobile traffic with >1000 HTTP endpoints. • Acts as a router to 3400+ microservices • A single place for monitoring and documentation Marketplace Gateway Real-time API is the only interface
off 2nd Drop- off Logistics Challenges ... • How to optimize for overall marketplace efficiency? • How to dynamically price trips and balance efficiency? • How to maximize pick-up efficiency (pick up zones, pick up suggestions)? • How to minimize wait times for drivers and riders? • How to deal with different street configurations? • Combine different logistics products (Pool, Eats, Rush) UberPool is a traveling salesman problem on steroids
Jan 2011 - "Dispatch" - Node.JS/MongoDB Jan 2011 - "API" - Python/SQLAlchemy/MySQL Feb 2012 - Dispatch swaps MongoDB for Redis May 2012 - Dispatch adds on fallback Jan 2013 - First non-API Python services Feb 2013 - API switched to Postgres Mar 2014 - New Python services use MySQL Mar 2014 - Schemaless begins, must finish before Postgres collapse Sep 2014 - First Schemaless - trips out of Postgres Aug 2015 - Dispatch X.0 / Ringpop / Riak Jan 2016 - Go, Java, More Abstractions May 2017 - UberFx for Go, Cloud
can turn into a ball of mud quickly • Optimizes development velocity of early stage projects • Consolidates operational overhead in one place • Monolith isn’t as bad when there is a single team that owns it
Product-specific velocity • Move the business faster • Able to solve scaling issues • Systems as well as company • Training, documentation, tooling • Language/platform independence • Best tool for the job • Reliability of the system • Independent • Scalable • Testable Pros What’s good • Increased complexity • Naming + Discovery • Eventual consistency • Performance Impact • Distributed tracing • Testing • Operational overhead • Monitoring • Continuous Integration • Deployment • Documentation • Instrumentation • Security • Repeated code/effort Cons What’s not so good
to distributed applications. It maintains a consistent hash ring on top of a membership protocol and provides request forwarding as a routing convenience. TChannel TChannel is a networking framing protocol used for general RPC, supporting out- of-order responses at extremely high performance where intermediaries can make a forwarding decision quickly. Client libraries available in Go, Java, Node and Python. Jaeger Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It can be used for monitoring microservice-based architectures. Client libraries available in Go, Java, Node and Python using the OpenTracing standard.
ultra performant backend systems (p99 for max latency) • Early on it made it quick to iterate, but as the size of the team scaled the developer velocity started to slow down • Microservices enforce a tight interface so having static typing enables large teams to catches issues earlier. It really has an impact with 100+ devs. • Quick to learn, but easy to write poor quality code • Enables you to move fast, but allows for sloppy code • Great ecosystem of small libraries, but many were immature compared to Java/Go • Don't be afraid of writing c++ Growing pains learned the hard way
for browsers; using RPC is better for computer-to-computer requests • After a certain age, microservices should become immutable • Having multiple languages allows for team preferences, but segregates developers based on language and prevents easy re-use of code across services • Company > Team > Self • Monorepos allow for changes to be made across multiple services atomically, but prevent future open-sourcing and subset checkouts • Performance problems are difficult to debug cross-language without standardised service dashboards and observability tools • Logging should never slow production down; in a failure storm, the logging system should drop rather than delay • Everything is a tradeoff: Be intentional Growing pains learned the hard way
foundation, all of these applications could have been built off of entirely different tools and architectures, creating a higher likelihood of security vulnerabilities, duplicated effort across teams, technical debt, minimal UI consistency, etc.
with caching • Yeoman like scaffolding for bootstrap new apps with best practices • Customized middlewares for our Express based app framework • Authentication, Security (XSS, CSRF, CSP), Metrics, Logging, I18n, Errors, Instrumentation, Analytics • Bootstrap like UI component library build on top of React.js • RPC library for intelligently using http/tchannel • Shared Build + Test Gulp tasks with hot reloading, ES6/JSX, asset versioning and deployment pipelines, testing + linting • Universal rendering for server/client
that serve web pages) from our backend services (those that get data from some database) • Services can take advantage of universal rendering + tooling • You can utilize languages better suited for your backend services (Java, Go) • You can deploy and scale your services separately • You can reuse the backend service API for other clients
sites with flexible components you don’t have to maintain yourself • Standard UI components • Optimized for performance • Encapsulated style • Consistent look and feel throughout all components that fit brand • Analytics and instrumentation baked-in • Seamless upgrades to new versions
{ injectStyle } from 'styletron-utils'; // Create a Styletron instance const styletron = new Styletron(); const className = injectStyle(styletron, { color: 'red', display: 'inline-block', fontSize: '1.6em' }); // Css is injected into the page // and a class name is returned
React • react-map-gl: A React interface to MapboxGL-js • deck.gl: Layered WebGL approach system for visualization • luma.gl: A JavaScript WebGL Framework for Data Visualization
popular libraries like stack.gl • Shader library with a 64-bit floating point emulation package • Advanced debugging, tracing, error checking for WebGL luma.gl A JavaScript WebGL Framework for Data Visualization
other talks by engineers far smarter than myself • Lessons learned from scaling to 2000 engineers and 1000+ services - Matt Ranney • Uber Architecture: Moving Bits and Atoms at Scale - Andrii Iasynetskyi • Thanks to the many amazing people of the Marketplace Platform, Web Platform, Developer Platform, and Visualization Teams at Uber! • Come join us and work with these amazing people. Hiring globally (San Francisco, Amsterdam, Sofia, ...)
dustinwhittle
blueb
kakehashi
chou500
ryosukeigarashi
eventhub
tohutohu
gumamon
isaoshimizu
oracle4engineer
sansantech
tanoku
irinanazarova
chrislema
mraible
jacobian
orderedlist
aleyda
quramy
samlambert
eileencodes
trishagee
kastner
