Upgrade to Pro — share decks privately, control downloads, hide ads and more …

"API First" Development

"API First" Development

API First Development at Etsy. Presented on August 15th at Converge.

Dan Miller

August 15, 2014
Tweet

More Decks by Dan Miller

Other Decks in Programming

Transcript

  1. 2005 2006 2007 2008 2009 2010 2011 2012 2013 GROSS

    MERCHANDISE SALES $0.17 M $26 M $87.5 M $180.6 M $314.3 M $525.6 M $895.1 M $1.35 B $3.8 M
  2. 2005 2014 ‘12 ‘11 ‘10 ‘09 ‘08 ‘07 ‘06 ‘13

    • LAMP in a Box • Explosive growth • Chiseling away at the monolith
  3. • Four Employees • One Web Server* • One Database

    Server
 * until getting Slashdotted in August, 2005 2005 2014 ‘12 ‘11 ‘10 ‘09 ‘08 ‘07 ‘06 ‘13
  4. • Etsy goes down all the time (often for a

    long time) • Code deployments are incredibly painful • Developer, DBA, Operations silos 2005 2014 ‘12 ‘11 ‘10 ‘09 ‘08 ‘07 ‘06 ‘13
  5. • New CTO • New CEO • New VP of

    Operations 2005 2014 ‘12 ‘11 ‘10 ‘09 ‘08 ‘07 ‘06 ‘13
  6. • Metrics Driven Development • DevOps • Continuous Deployment •

    Sharded Databases • Tons of Hiring 2005 2014 ‘12 ‘11 ‘10 ‘09 ‘08 ‘07 ‘06 ‘13
  7. • Deployed over 30 times per day (Deployinator) • Hundreds

    of thousands of graphs (StatsD) • 6x as many engineers • $895 million in Gross Merchandise Sales/year 2005 2014 ‘12 ‘11 ‘10 ‘09 ‘08 ‘07 ‘06 ‘13
  8. @jazzdan A REST API is a Slow API Uniform Interface

    ! “Individual resources are identified in requests” ! (Fielding 2000, §5.1.5)
  9. @jazzdan A REST API is a Slow API 1.GET /listing/12345

    2.GET /shop/98765 3.GET /user/cart 4.GET /shop/98765/reviews 5.GET /shop/98765/shipping_profiles 200 ms (best case) 200 ms (best case) 200 ms (best case) 200 ms (best case) 200 ms (best case)
  10. @jazzdan A REST API is a Slow API 1000 ms

    -900 ms (3G network + client rendering) —————————————————— 100ms
  11. @jazzdan A REST API is a Slow API 1.GET /listing/12345

    2.GET /shop/98765 3.GET /user/cart 4.GET /shop/98765/reviews 5.GET /shop/98765/shipping_profiles 200 ms (best case) 200 ms (best case) 200 ms (best case) 200 ms (best case) 200 ms (best case)
  12. @jazzdan Human Problems • Most Engineers speak PHP and JavaScript

    • iOS apps are in Objective-C, Android apps are in Java • 1/10 Engineers were native apps engineers
  13. @jazzdan Human Problems • Separate teams • iOS team for

    iPhone, iPad • Android team for Android phone and tablet
  14. @jazzdan Option 1: Learn on your Own • Curated guides

    and tutorials • Getting started with Objective-C at Etsy • Obj-C/Java style guides • Lunch and Learn videos • App architecture documentation
  15. @jazzdan Option 2: The Classroom • Week 1: Prerequisites •

    Week 2: Class • Week 3: Post-Class Hacking
  16. @jazzdan End Goal • Everyone will be a mobile engineer

    • All product teams can develop their products on all platforms: Desktop, iOS, Android • Infrastructure teams can triage issues on all platforms
  17. @jazzdan Move data closer to the user faster Origin <!DOCTYPE

    html> ! <html xmlns="http://www.w3.org/1999/xhtml" ! xml:lang="en" ! lang="en" ! xmlns:og="http://ogp.me/ns#" xmlns:fb="https:// www.facebook.com/2008/fbml" ! data-user-id="15515418" > ! <head> ! <title>Etsy :: Your place to buy and sell all things handmade</title>
  18. @jazzdan <!DOCTYPE html> ! <html xmlns="http://www.w3.org/1999/xhtml" ! xml:lang="en" ! lang="en"

    ! xmlns:og="http://ogp.me/ns#" xmlns:fb="https:// www.facebook.com/2008/fbml" ! data-user-id="15515418" > ! <head> ! <title>Etsy :: Your place to buy and sell all things handmade</title> Move data closer to the user faster Origin Edge {! "listing": {! "listing_id": 150002520,! "state": "active",! "user_id": 5029420,! "category_id": 69150467,! "title": "ETSY Buttons Variety Pack (Set of 6) copy 3",! ...! } 70% reduction in payload size to Edge
  19. @jazzdan v2 Row Oriented aka REST 12345 54321 active 5

    123478 Pug Hat 20.00 Cool Pug Hat USD shop_id currency_code price description title category_id user_id state listing_id { “listing_id”: 12345, “state”: “active”, “user_id”: 123478, “title: “Pug Hat”, “description”: “Cool Pug Hat” “price”: “20.00”, “currency_code”: “USD”, “shop_id”: 54321 }
  20. @jazzdan v3 Resource Oriented { listing_id: 12345, shop: { name:

    “Awesome shop”, url: “et.sy/abcdf” }, price: “20.00”, currency: “USD”, currency_symbol: “$”, … }
  21. @jazzdan A REST API is a Slow API 1000 ms

    -900 ms (network + client processing) —————————————————— 100ms
  22. @jazzdan A REST API is a Slow API 1000 ms

    -900 ms (network + client processing) —————————————————— 100ms = Bespoke
  23. @jazzdan Bespoke Endpoints etsy.com/listing/124740565 Bespoke Recent Shop Overview Related Seller

    Listing listing->id listing->id listing->id listing->user_id listing->shop_id listing->shop_id
  24. @jazzdan Technical Benefits • Same API endpoints power web and

    apps • Enables multi-colo • More easily understood codebase • Enable infrastructure experimentation