Grokking HTTP (CoderFaire Nashville 2012)

Hypertext Transfer Protocol is the protocol of the Web. From static HTML pages to massive web services, everything we do as web developers has some relationship to this protocol. To effectively create services that use the Web, we need a deep understanding of HTTP. This talk goes beyond a surface understanding of GET and 200 OK to explore how an intimate knowledge of HTTP can lead to more efficient applications utilizing the Web in a RESTful way. We'll cover topics such as content negotiation, hypermedia, caching, and conditional requests, as well as recent developments in efforts to update HTTP.

Transcript

1. Ben Ramsey Grokking HTTP
2. Why HTTP?
3. Because you are a web developer.
4. HTTP is the Web.
5. • A client-server architecture • Atomic operations • Cacheable responses • A uniform and constrained interface • Layered system • Allows for code on demand
6. HTTP is RESTful.
7. • My favorite HTTP tools • Overview of the protocol • Content negotiation • Conditional requests • Caching • The future of HTTP Grokking HTTP
8. grok • /ˈɡrɒk/ To grok is to intimately and completely share the same reality or line of thinking with another physical or conceptual entity. Author Robert A. Heinlein coined the term in his best-selling 1961 book Stranger in a Strange Land. In Heinlein's view, grokking is the intermingling of intelligence that necessarily a!ects both the observer and the observed. —from Wikipedia, http://en.wikipedia.org/wiki/Grok
9. My Favorite Tools
10. HTTPie • Ditch cURL. Use HTTPie. • httpie.org • Perfect for testing and debugging APIs • Free; requires Python
11. Charles • I cannot recommend this enough! • charlesproxy.com • Perfect for debugging Ajax and Flash remoting (AMF) requests • Well worth the $50 license fee
12. Setting Headers <?php header('HTTP/1.1 201 Created'); header('Location: http://example.com/books/decd0562'); header('ETag: "decd0562-1"'); header('Last-Modified: Sun, 29 Jul 2012 23:26:49 GMT'); header('Content-Type: application/hal+json'); echo json_encode(array( '_links' => array( 'self' => array( 'href' => 'http://example.com/books/decd0562', ), ), 'author' => 'Stoyan Stefanov', 'id' => 'decd0562', 'isbn10' => '1449320198', 'isbn13' => '9781449320195', 'publisher' => 'O\'Reilly Media', 'title' => 'JavaScript for PHP Developers', 'year' => 2012, ));
13. Other Tools • PHP cURL functions • fopen wrappers for http/https • "le_get_contents() • "le_put_contents() • stream_context_create() • PHP pecl_http extension
14. Protocol Overview
15. RFC 2616
16. • GET • POST • PUT • DELETE • HEAD • OPTIONS • TRACE • CONNECT Methods
17. Safe Methods • GET & HEAD should not take action other than retrieval • These are considered safe • This allows user agents to represent POST, PUT, & DELETE in a special way
18. Idempotence • Side e!ects of N > 0 identical requests is the same as for a single request • GET, HEAD, PUT, and DELETE share this property • OPTIONS and TRACE are inherently idempotent
19. GET
20. GET /books/9790482c HTTP/1.1 Host: example.com Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
21. GET /books/9790482c HTTP/1.1 Host: example.com Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
22. GET /books/9790482c HTTP/1.1 Host: example.com Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
23. HTTP/1.1 200 OK Date: Sun, 29 Jul 2012 22:46:43 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 ETag: "9790482c-1" Last-Modified: Sun, 15 Jul 2012 16:34:23 GMT Content-Length: 254 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/9790482c" } }, "author": "Luke Welling, Laura Thomson", "id": "9790482c", "isbn10": "0672329166", "isbn13": "9780672329166", "publisher": "Pearson Education", "title": "PHP and MySQL Web Development", "year": 2008 }
24. HTTP/1.1 200 OK Date: Sun, 29 Jul 2012 22:46:43 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 ETag: "9790482c-1" Last-Modified: Sun, 15 Jul 2012 16:34:23 GMT Content-Length: 254 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/9790482c" } }, "author": "Luke Welling, Laura Thomson", "id": "9790482c", "isbn10": "0672329166", "isbn13": "9780672329166", "publisher": "Pearson Education", "title": "PHP and MySQL Web Development", "year": 2008 }
25. HTTP/1.1 200 OK Date: Sun, 29 Jul 2012 22:46:43 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 ETag: "9790482c-1" Last-Modified: Sun, 15 Jul 2012 16:34:23 GMT Content-Length: 254 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/9790482c" } }, "author": "Luke Welling, Laura Thomson", "id": "9790482c", "isbn10": "0672329166", "isbn13": "9780672329166", "publisher": "Pearson Education", "title": "PHP and MySQL Web Development", "year": 2008 }
26. HTTP/1.1 200 OK Date: Sun, 29 Jul 2012 22:46:43 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 ETag: "9790482c-1" Last-Modified: Sun, 15 Jul 2012 16:34:23 GMT Content-Length: 254 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/9790482c" } }, "author": "Luke Welling, Laura Thomson", "id": "9790482c", "isbn10": "0672329166", "isbn13": "9780672329166", "publisher": "Pearson Education", "title": "PHP and MySQL Web Development", "year": 2008 }
27. POST
28. POST /books HTTP/1.1 Host: example.com Content-Type: application/hal+json Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0 { "author": "Stoyan Stefanov", "isbn10": "1449320198", "isbn13": "9781449320195", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
29. HTTP/1.1 201 Created Date: Sun, 29 Jul 2012 23:26:49 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Location: http://example.com/books/decd0562 ETag: "decd0562-1" Last-Modified: Sun, 29 Jul 2012 23:26:49 GMT Content-Length: 239 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
30. HTTP/1.1 201 Created Date: Sun, 29 Jul 2012 23:26:49 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Location: http://example.com/books/decd0562 ETag: "decd0562-1" Last-Modified: Sun, 29 Jul 2012 23:26:49 GMT Content-Length: 239 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
31. HTTP/1.1 201 Created Date: Sun, 29 Jul 2012 23:26:49 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Location: http://example.com/books/decd0562 ETag: "decd0562-1" Last-Modified: Sun, 29 Jul 2012 23:26:49 GMT Content-Length: 239 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
32. HTTP/1.1 201 Created Date: Sun, 29 Jul 2012 23:26:49 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Location: http://example.com/books/decd0562 ETag: "decd0562-1" Last-Modified: Sun, 29 Jul 2012 23:26:49 GMT Content-Length: 239 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
33. PUT
34. PUT /books/decd0562 HTTP/1.1 Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0 Host: example.com If-Match: "decd0562-1" Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "pubDate": "September 22, 2012", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
35. PUT /books/decd0562 HTTP/1.1 Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0 Host: example.com If-Match: "decd0562-1" Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "pubDate": "September 22, 2012", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
36. PUT /books/decd0562 HTTP/1.1 Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0 Host: example.com If-Match: "decd0562-1" Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "pubDate": "September 22, 2012", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
37. HTTP/1.1 200 OK Date: Sun, 29 Jul 2012 23:47:59 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 ETag: "decd0562-2" Last-Modified: Sun, 29 Jul 2012 23:47:59 GMT Content-Length: 270 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "pubDate": "September 22, 2012", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
38. HTTP/1.1 200 OK Date: Sun, 29 Jul 2012 23:47:59 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 ETag: "decd0562-2" Last-Modified: Sun, 29 Jul 2012 23:47:59 GMT Content-Length: 270 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "pubDate": "September 22, 2012", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
39. HTTP/1.1 200 OK Date: Sun, 29 Jul 2012 23:47:59 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 ETag: "decd0562-2" Last-Modified: Sun, 29 Jul 2012 23:47:59 GMT Content-Length: 270 Content-Type: application/hal+json { "_links": { "self": { "href": "http://example.com/books/decd0562" } }, "author": "Stoyan Stefanov", "id": "decd0562", "isbn10": "1449320198", "isbn13": "9781449320195", "pubDate": "September 22, 2012", "publisher": "O'Reilly Media", "title": "JavaScript for PHP Developers", "year": 2012 }
40. DELETE
41. DELETE /books/decd0562 HTTP/1.1 Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0 Host: example.com If-Match: "decd0562-2"
42. HTTP/1.1 204 No Content Date: Mon, 30 Jul 2012 00:01:44 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Length: 0 Content-Type: application/hal+json
43. HTTP/1.1 204 No Content Date: Mon, 30 Jul 2012 00:01:44 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Length: 0 Content-Type: application/hal+json
44. Status Codes
45. • Informational (1xx) • Successful (2xx) • Redirection (3xx) • Client error (4xx) • Server error (5xx)
46. Content Negotiation
47. • Also called “conneg” • “the process of selecting the best representation for a given response when there are multiple representations available” • Two kinds: • server-driven • agent-driven
48. Server-driven Negotiation
49. GET /books/9790482c HTTP/1.1 Accept-Charset: utf-8 Host: example.com Accept-Language: en-us, en-gb;q=0.8, en;q=0.7 Accept-Encoding: gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
50. GET /books/9790482c HTTP/1.1 Accept-Charset: utf-8 Host: example.com Accept-Language: en-us, en-gb;q=0.8, en;q=0.7 Accept-Encoding: gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
51. GET /books/9790482c HTTP/1.1 Accept-Charset: utf-8 Host: example.com Accept-Language: en-us, en-gb;q=0.8, en;q=0.7 Accept-Encoding: gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
52. GET /books/9790482c HTTP/1.1 Accept-Charset: utf-8 Host: example.com Accept-Language: en-us, en-gb;q=0.8, en;q=0.7 Accept-Encoding: gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
53. GET /books/9790482c HTTP/1.1 Accept-Charset: utf-8 Host: example.com Accept-Language: en-us, en-gb;q=0.8, en;q=0.7 Accept-Encoding: gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0
54. HTTP/1.1 200 OK Date: Mon, 30 Jul 2012 02:42:26 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Language: en-us ETag: "9790482c-1" Vary: Accept,Accept-Charset,Accept-Language,Accept-Encoding Content-Encoding: gzip Content-Length: 213 Content-Type: application/hal+json; charset=utf-8 { ... }
55. HTTP/1.1 200 OK Date: Mon, 30 Jul 2012 02:42:26 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Language: en-us ETag: "9790482c-1" Vary: Accept,Accept-Charset,Accept-Language,Accept-Encoding Content-Encoding: gzip Content-Length: 213 Content-Type: application/hal+json; charset=utf-8 { ... }
56. HTTP/1.1 200 OK Date: Mon, 30 Jul 2012 02:42:26 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Language: en-us ETag: "9790482c-1" Vary: Accept,Accept-Charset,Accept-Language,Accept-Encoding Content-Encoding: gzip Content-Length: 213 Content-Type: application/hal+json; charset=utf-8 { ... }
57. HTTP/1.1 200 OK Date: Mon, 30 Jul 2012 02:42:26 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Language: en-us ETag: "9790482c-1" Vary: Accept,Accept-Charset,Accept-Language,Accept-Encoding Content-Encoding: gzip Content-Length: 213 Content-Type: application/hal+json; charset=utf-8 { ... }
58. HTTP/1.1 200 OK Date: Mon, 30 Jul 2012 02:42:26 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Language: en-us ETag: "9790482c-1" Vary: Accept,Accept-Charset,Accept-Language,Accept-Encoding Content-Encoding: gzip Content-Length: 213 Content-Type: application/hal+json; charset=utf-8 { ... }
59. Agent-driven Negotiation
60. GET /books/9790482c HTTP/1.1 Host: example.com User-Agent: HTTPie/0.2.0
61. HTTP/1.1 300 Multiple Choices Date: Mon, 30 Jul 2012 02:57:42 GMT Server: Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.3.10-1ubuntu3.2 Content-Length: 444 Content-Type: application/hal+json
62. { "_links": { "alternate": [ { "href": "http://example.com/books/9790482c.en-us.html", "hreflang": "en-us", "type": "text/html; charset=utf-8" }, { "href": "http://example.com/books/9790482c.en-us.json", "hreflang": "en-us", "type": "application/hal+json; charset=utf-8" }, { "href": "http://example.com/books/9790482c.en-us.xml", "hreflang": "en-us", "type": "application/hal+xml; charset=utf-8" } ], "self": { "href": "http://example.com/books/9790482c" } } }
63. Conditional Requests
64. Request Headers • If-Modi"ed-Since • If-Unmodi"ed-Since • If-Match • If-None-Match • If-Range
65. GET /books/9790482c HTTP/1.1 Host: example.com Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0 If-Modified-Since: Sun, 15 Jul 2012 16:34:23 GMT HTTP/1.1 304 Not Modified Date: Mon, 30 Jul 2012 03:39:51 GMT Server: Apache/2.2.22 (Ubuntu) Vary: Accept-Encoding
66. GET /books/9790482c HTTP/1.1 Host: example.com Accept-Encoding: identity, deflate, compress, gzip Accept: application/hal+json User-Agent: HTTPie/0.2.0 If-Modified-Since: Sun, 15 Jul 2012 16:34:23 GMT HTTP/1.1 304 Not Modified Date: Mon, 30 Jul 2012 03:39:51 GMT Server: Apache/2.2.22 (Ubuntu) Vary: Accept-Encoding
67. Caching
68. Response Headers • Expires • Cache-Control • max-age • s-maxage • public • private • no-cache • no-store • must-revalidate • proxy-revalidate
69. Cache-Control: max-age=3600, must-revalidate
70. The Future of HTTP
71. • HTTP Bis IETF Working Group http://tools.ietf.org/wg/httpbis/ • RFCs and speci"cations that extend HTTP: • WebDAV and related extensions • RFC 5789 (PATCH Method) • RFC 6266 (Use of Content-Disposition) • RFC 6585 (Additional Status Codes) • Method Registrations • Authentication Scheme Registrations • Permanent Message Header Fields
72. There’s too much in HTTP to fully grok it in one presentation.
73. • RFC 2616 (HTTP) • Mark Nottingham: mnot.net • Mark’s Caching Tutorial • RFC 2295 (Transparent Content Negotiation in HTTP) • RFC 2296 (HTTP Remote Variant Selection Algorithm) More Resources
74. Thank you. • Rate this talk: joind.in/6717 • Read my blog: benramsey.com • Follow me on Twitter: @ramsey • Slides: bram.se/cf12-http
75. Grokking HTTP Copyright © Ben Ramsey. Some rights reserved. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported. For uses not covered under this license, please contact the author.