API Design: It's Not Rocket Surgery (PHPUK13)

The talk I gave at PHPUK13 on Friday 22nd February in (a very packed!) Track 3.

Transcript

1. API design It’s Not Rocket Surgery Dave Ingram @dmi PHP UK Conference 22nd February 2013
2. Who am I? • Senior operations engineer at DataSift • Was developer/RM at GroupSpaces for 41/2 years • Way too many projects • Involved in open source, including Phabricator • Occasionally found at London Hackspace • Twitter: @dmi • Github: dingram
3. Why am I giving this talk? • I’ve built a number of APIs • Both for GroupSpaces and my own projects • Sadly most are not public (yet!) • I’m also a consumer of many other APIs • Twitter • Foursquare • Tumblr • TfL • Spotify • . . . blah blah blah. . . • I’m opinionated
4. Feedback/thoughts via Twitter: #apirocketsurgery (or @dmi) http://joind.in/8 45
5. What this isn’t
6. T T REST REST REST REST REST REST REST EST EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
7. T T REST REST REST REST REST REST REST EST EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
8. T T REST REST REST REST REST REST REST EST EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
9. There’s so much more!
10. URLs
11. URLs verbs
12. URLs verbs headers
13. URLs verbs headers authentication
14. URLs verbs headers authentication formats
15. URLs verbs headers authentication formats integrity
16. URLs verbs headers authentication formats integrity documentation
17. DOCUMENTATION
18. DOCUMENTATION (later)
19. Part 1: URLs
20. Make them versioned
21. Make them versioned, hackable
22. Make them versioned, hackable & meaningful
23. http://api.com/v1/
24. http://api.com/v1/users/
25. http://api.com/v1/users/31337
26. http://api.com/v1/users/31337/posts/
27. Look at URLs from a consumer perspective
28. They don’t care about internal implementation details
29. Use common sense
30. Don’t be afraid of the query string
31. Use query arguments to filter output or for (some) alternate representations
32. Good: searching & verbosity Bad: output format control (XML vs JSON)
33. Good: http://api.com/v1/search?q=foobar Bad: http://api.com/v1/search/foobar
34. Good: http://api.com/v1/foo.json Bad: http://api.com/v1/foo?format=json
35. Part 2: Verbs
36. • GET = get() • PUT = setAll() / new Obj($id) • POST = new Obj() / doStuff() / set() • DELETE = delete() • HEAD ≈ getMetadata() • OPTIONS ≈ reflection/permissions
37. Must at least handle GET/POST
38. Can emulate PUT/DELETE
39. For example: POST http://api.com/v1/foo/42!PUT POST http://api.com/v1/foo/11!DELETE
40. Remember that POST can have side-effects and is never cached
41. PUT and DELETE are idempotent, which means they have the same effect even if they’re repeated
42. HEAD is rare and can probably be ignored
43. OPTIONS is used by CORS, but otherwise rare
44. CORS?
45. Cross-Origin Resource Sharing
46. A way to allow in-browser cross-origin XMLHTTPRequests Support: FF3.5+, Chrome 4+, Safari 4+, Opera 12+, IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+ http://www.w3.org/TR/cors/ http://caniuse.com/cors
47. Origin & Allow-Origin A way to allow in-browser cross-origin XMLHTTPRequests Support: FF3.5+, Chrome 4+, Safari 4+, Opera 12+, IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+ http://www.w3.org/TR/cors/ http://caniuse.com/cors
48. Part 3: Headers
49. Headers are important too! (and you can be clever)
50. Accept The MIME types the client will accept. No need to use file extensions to decide what content type to serve!* Accept-Language The languages the client will accept. No need to ask clients or (worse) just assume English responses.
51. Some headers reduce traffic (important for mobile)
52. • ETag – A unique tag for the content • If-(None-)Match – Check ETag • If-(Un)Modified-Since – Is it newer? • Cache-Control – Can it be cached? • Expires – How long is it valid? • Vary – Additional caching rules
53. Beware: some proxies may not pass custom headers
54. Beware: even some standard headers are not safe
55. Prefer headers, but accept other methods
56. Part 4: Authentication & Authorization
57. OAuth2 over HTTPS
58. Much simpler than OAuth1 (and similar to session cookies)
59. Many libraries for OAuth2 for many platforms as it’s a popular standard
60. For extra security use OAuth2-MAC
61. OAuth2-MAC uses signatures instead of bearer tokens, so secrets stay secret
62. Then again, the author of OAuth2 has now advised not upgrading from OAuth 1.0a and either using OAuth 1.0a for new sites or staying close to a large provider’s implementation
63. No need to worry about rate-limiting (to start with*) 3scale, Apiaxle, Mashery can help
64. Part 5: Formats
65. Sane default: JSON, plus envelope with metadata
66. { "meta": { "code": 2 , "dev_notes ": [ "This endpoint is deprecated" ] }, "response ": { ... } }
67. Metadata envelope helps with JSONP and limited clients that discard errors
68. Having metadata allows out-of-band information, like deprecation warnings
69. What about XML?
70. *Hypertext as the Engine of Application State HATEOAS* is a nice idea, although it’s very verbose (but useful for building an API explorer)
71. HATEOAS adds links to related endpoints in the response
72. Don’t forget: mobile bandwidth is still very limited
73. Timestamps • ISO-8601 2 12- 5- 3T19: : Z Human-readable, but needs parsing • UTC seconds since epoch: 1336 716 Easily machine-usable
74. Should your API really be human-readable? It’s better to help your consumers.
75. Part 6: Data integrity
76. Allow your consumers to cache data whenever possible
77. Encourage use of request headers: • GET: • If-Modified-Since • If-None-Match • POST/PUT/DELETE: • If-Unmodified-Since • If-Match
78. Return useful response headers: • Last-Modified • Expires • ETag • Cache-Control
79. Deal with preconditions and give correct response codes
80. Useful status codes: 4 5 Method Not Allowed 4 6 Not Acceptable 412 Precondition Failed 428 Precondition Required* 429 Too Many Requests* *New in RFC6585
81. For example: ETag and Last-Modified can help prevent race conditions
82. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type: text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
83. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type: text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
84. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type: text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
85. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type: text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
86. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
87. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
88. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
89. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
90. Part 7: Documentation
91. Documentation is important
92. If people can’t understand your API, they won’t use it
93. Use examples liberally. . . and make sure they’re both up-to-date and correct!
94. Consider linking your examples to your automated tests (You do have them, right?)
95. Assume nothing; explain everything (What is an ID? string[8]? int32? uint64?)
96. Be helpful: provide an interactive console e.g. Apiary, Apigee console, Mashery’s iodocs
97. And finally: Statistics
98. Record numbers for anything and everything you can imagine
99. Everybody loves graphs (and they’re important for understanding API usage and performance)
100. Things to measure Counters: HTTP status (200, 400, 500, . . . ) Tokens issued Token refreshes Clients created Client usage Auth successes Auth failures Auth grants Auth rejections DB queries API version/release Client type . . . Avg response time: Auth requests General requests Specific subsystems . . . Other Memory usage Client location . . .
101. Thanks! Any questions? Feedback via joind.in: http://joind.in/8 45 or #apirocketsurgery or @dmi Slides: http://www.dmi.me.uk/talks/ Built in L ATEX Inspired by: http://goo.gl/ mT55