Functional Web Apps with Webmachine

LambdaJam 2013: Webmachine Tutorial

1. Functional Web Apps with Sean Cribbs Chris Meiklejohn Thursday, July 11, 13
2. Download http://bit.ly/wmlj-pdf Clone github.com/cmeiklejohn/webmachine-tutorial http://bit.ly/wmlj-zip Thursday, July 11, 13
3. Introduction Thursday, July 11, 13
4. Have you ever... Thursday, July 11, 13
5. Have you ever... CGI Thursday, July 11, 13
6. Have you ever... CGI Servlet Thursday, July 11, 13
7. Have you ever... CGI Servlet Model-2 “MVC” Thursday, July 11, 13
8. request process response client server GET /something Thursday, July 11, 13
9. request process response client server GET /something Thursday, July 11, 13
10. Imperative : Actions :: Functional : Facts Thursday, July 11, 13
11. Imperative : Actions :: Functional : Facts what does it DO? Thursday, July 11, 13
12. Imperative : Actions :: Functional : Facts what does it DO? what IS it? Thursday, July 11, 13
13. HTTP Facts: Resources •Data or Service •Identified by URI •Decorated by representations and other properties/variances Thursday, July 11, 13
14. request state machine + functional predicates response client server GET /something the thing Thursday, July 11, 13
15. f(ReqData,State) -> {RetV,ReqData,State}. function behavior request/ response data resource state Resource functions are referentially transparent and have a uniform interface. Many resource functions are optional and use reasonable defaults. + + Thursday, July 11, 13
16. f(ReqData,State) -> {RetV,ReqData,State}. function behavior request/ response data resource state Resource functions are referentially transparent and have a uniform interface. Many resource functions are optional and use reasonable defaults. + + Thursday, July 11, 13
17. f(ReqData,State) -> {RetV,ReqData,State}. function behavior request/ response data resource state Resource functions are referentially transparent and have a uniform interface. Many resource functions are optional and use reasonable defaults. + + Thursday, July 11, 13
18. f(ReqData,State) -> {RetV,ReqData,State}. function behavior request/ response data resource state Resource functions are referentially transparent and have a uniform interface. Many resource functions are optional and use reasonable defaults. + + Thursday, July 11, 13
19. Thursday, July 11, 13
20. Start Thursday, July 11, 13
21. Start 200 OK Thursday, July 11, 13
22. Thursday, July 11, 13
23. validation & auth Thursday, July 11, 13
24. validation & auth content- negotiation Thursday, July 11, 13
25. validation & auth content- negotiation existence & redirection Thursday, July 11, 13
26. validation & auth content- negotiation existence & redirection conditional requests Thursday, July 11, 13
27. validation & auth content- negotiation existence & redirection conditional requests PUT/POST Thursday, July 11, 13
28. validation & auth content- negotiation existence & redirection conditional requests PUT/POST DELETE Thursday, July 11, 13
29. validation & auth content- negotiation existence & redirection conditional requests PUT/POST DELETE body Thursday, July 11, 13
30. Hello, World $ git checkout hello-world $ $EDITOR src/tweeter_resource.erl Thursday, July 11, 13
31. $ make $ ./start.sh $ $BROWSER http://localhost:8080 Build & Run Also Heroku compatible! (use foreman start) Thursday, July 11, 13
32. init([]) -> {ok, undefined}. to_html(ReqData, State) -> {"<html><body>Hello, new world</body></html>", ReqData, State}. Default resource Thursday, July 11, 13
33. init([]) -> {ok, undefined}. to_html(ReqData, State) -> {"<html><body>Hello, new world</body></html>", ReqData, State}. resource state Default resource Thursday, July 11, 13
34. init([]) -> {ok, undefined}. to_html(ReqData, State) -> {"<html><body>Hello, new world</body></html>", ReqData, State}. resource state iolist() Default resource Thursday, July 11, 13
35. •Use the resource state as the body, setting it in init/1 •Put the value of the Host request header inside the response body using an iolist() and: wrq:get_req_header(Key, ReqData) * Hint: header keys are lowercase strings Exercises Thursday, July 11, 13
36. UI Skeleton <Ctrl-C> a # if still running $ git checkout -f assets $ make; ./start.sh $ $BROWSER http://localhost:8080 Thursday, July 11, 13
37. We’ll come back to the resource at the end. Thursday, July 11, 13
38. Load Tweets in UI <Ctrl-C> a # if still running $ git checkout -f load-tweets $ make; ./start.sh <refresh browser> Thursday, July 11, 13
39. Media Types ...specify alternative or multiple formats (“representations”) for resources: text/html application/json image/jpeg Thursday, July 11, 13
40. Media Types ...specify alternative or multiple formats (“representations”) for resources: text/html application/json image/jpeg Thursday, July 11, 13
41. %% default implementation content_types_provided(ReqData, State) -> {[{"text/html", to_html}], ReqData, State}. %% As many types as you want! content_types_provided(ReqData, State) -> {[{"text/html", to_html}, {"application/json", to_json}, {"text/xml", to_xml}], ReqData, State}. media type body-producer function Media-types callback Thursday, July 11, 13
42. %% tweeter_wm_tweet_resource.erl routes() -> [{["tweets"], ?MODULE, []}]. %% tweeter_wm_asset_resource.erl routes() -> [{[""], ?MODULE, []}, {['*'], ?MODULE, []}]. Dispatching matches any number of segments Thursday, July 11, 13
43. %% tweeter_wm_tweet_resource.erl routes() -> [{["tweets"], ?MODULE, []}]. %% tweeter_wm_asset_resource.erl routes() -> [{[""], ?MODULE, []}, {['*'], ?MODULE, []}]. Dispatching path segments matches any number of segments Thursday, July 11, 13
44. %% tweeter_wm_tweet_resource.erl routes() -> [{["tweets"], ?MODULE, []}]. %% tweeter_wm_asset_resource.erl routes() -> [{[""], ?MODULE, []}, {['*'], ?MODULE, []}]. Dispatching path segments resource module matches any number of segments Thursday, July 11, 13
45. %% tweeter_wm_tweet_resource.erl routes() -> [{["tweets"], ?MODULE, []}]. %% tweeter_wm_asset_resource.erl routes() -> [{[""], ?MODULE, []}, {['*'], ?MODULE, []}]. Dispatching path segments resource module args to init/1 matches any number of segments Thursday, July 11, 13
46. %% tweeter_wm_tweet_resource.erl routes() -> [{["tweets"], ?MODULE, []}]. %% tweeter_wm_asset_resource.erl routes() -> [{[""], ?MODULE, []}, {['*'], ?MODULE, []}]. Dispatching path segments resource module args to init/1 matches any number of segments The dispatch list is set before starting up the server in tweeter_sup. Thursday, July 11, 13
47. %% definition -record(context, {tweet, tweets}). %% construction #context{}. %% {context, undefined, undefined} #context{tweets=[a,b,c]}. %% {context, undefined, [a,b,c]} %% pattern-matching and destructuring #context{tweets=Tweets} = Context. %% {context, _, Tweets} = Context. Tweets = Context#context.tweets. %% Tweets = element(3,Context). %% modification NewContext = Context#context{tweet="foo"}. %% NewContext = setelement(1, Context, "foo"). Records Thursday, July 11, 13
48. %% create a table {ok, TableID} = ets:new(table, [set]). %% make it public, multi-reader/writer {ok, TableID} = ets:new(table, [set, public, named_table, {read_concurrency, true}, {write_concurrency, true}]). %% read (lookup by key) ets:lookup(TableID, a). %% write ets:insert(TableID, {foo, bar}). %% query with abstract patterns ets:match(TableID, {'$1', bar}). ETS Thursday, July 11, 13
49. •Use curl to GET /tweets •Change the Accept header (-H option) to exclude application/json, compare the response. •Add the application/x-erlang-binary format to the resource. Use term_to_binary/1 to generate the body. Exercises Thursday, July 11, 13
50. Unique Tweet URLs <Ctrl-C> a # if still running $ git checkout -f tweet-urls $ make; ./start.sh Thursday, July 11, 13
51. Resource exists? 404 Not Found Redirection Creation 200 OK Condition validation Deletion Update / replace Usually used for fetching the internal representation of the resource. { { Thursday, July 11, 13
52. %% default implementation resource_exists(ReqData, State) -> {true, ReqData, State}. %% Dispatch rule that binds a path segment %% to the atom ‘key’ routes() -> [{["data", key], ?MODULE, []}]. %% Do a query to get the data using the %% bound dispatch path segment resource_exists(ReqData, State) -> Key = wrq:path_info(key, ReqData), case query_storage(Key) of undefined -> {false, ReqData, State}; Value -> {true, ReqData, State#state{data=Value}} end. resource_exists Thursday, July 11, 13
53. Exercises •Find the identifier of a tweet in the JSON, request the composite URL with curl. •Use curl to request a tweet that doesn’t exist. Thursday, July 11, 13
54. Create Tweets $ git checkout -f tweet-urls Thursday, July 11, 13
55. Creating Resources: PUT vs. POST Idempotent Client-speci!ed URI 204 No Content Non-Idempotent Server-generated URI 201 Created Thursday, July 11, 13
56. POSTing Resources 1. Allow the POST method 2. Specify that POST creates new resources 3. Generate a URL for the new resource 4. Accept the request body Thursday, July 11, 13
57. %% 1. Allow the POST method %% Default is ['HEAD', 'GET'] allowed_methods(ReqData, Context) -> {['HEAD', 'GET', 'POST'], ReqData, Context}. %% 2. Specify that POST creates new resources %% Default is false post_is_create(ReqData, Context) -> {true, ReqData, Context}. %% 3. Generate a URL for the new resource create_path(ReqData, Context) -> NewID = ets:update_counter(table, curr_id, {2, 1}), {"/steps/" ++ integer_to_list(NewID), ReqData, Context#context{id=NewID}}. Steps 1-3 Thursday, July 11, 13
58. %% 4. Accept the request body %% 4a. Specify the acceptable media types content_types_accepted(ReqData, Context) -> {[{"application/json", accept_json}], ReqData, Context}. %% 4b. Accept the negotiated type accept_json(ReqData, Context) -> Body = wrq:req_body(), {struct, Props} = mochijson2:decode(Body), ok = store(Context#context.id, Props), {true, ReqData, Context}. Step 4 Thursday, July 11, 13
59. •Use the browser UI to post tweets •Post a tweet using curl... •Sending a JSON body •Sending a non-JSON body *Hint: use -H and Content-Type Exercises Thursday, July 11, 13
60. Streaming Responses $ git checkout -f stream-pg2 Thursday, July 11, 13
61. Why Stream? •Less buffering, memory usage •Reduced latency, partial results •Long-lived connections Thursday, July 11, 13
62. %% Before to_text(ReqData, Context) -> {"Hello, world!", ReqData, Context}. %% After to_text(ReqData, Context) -> {{stream, {<<>>, fun stream/0}}, ReqData, Context}. %% Stream response as a "lazy sequence", with the %% Webmachine process waiting on messages. stream() -> receive {chat, Message} -> {[Message, "\n"], fun stream/0}; quit -> {<<>>, done} end. Streaming Responses Thursday, July 11, 13
63. %% Create a process group ok = pg2:create(chat). %% Get members of the process group Members = pg2:get_members(chat). %% Join the process group pg2:join(chat, self()). %% Send a message to all members [Member ! {chat, Msg} || Member <- Members]. OTP: Process Groups Thursday, July 11, 13
64. •Find the bug in the streaming response and fix it. *Hint: http://www.erlang.org/doc/man/erlang.html •Add a new streaming response that uses HTML5 text/event-stream instead of multipart/mixed. Exercises Thursday, July 11, 13
65. Caching and Preconditions $ git checkout -f etag-tweets Thursday, July 11, 13
66. HTTP Caching •Expiration: Cache-Control + max-age (TTL), Expires (Date) •Validation: ETag, Last-Modified, If-* •304 Not Modified •Computing ETag and Last-Modified should be cheap Thursday, July 11, 13
67. %% Default is undefined, i.e. no ETag %% Compute some hash, convert it to a hex string generate_etag(ReqData, Context) -> ETag = mochihex:to_hex(erlang:phash2(Context)), {ETag, ReqData, Context}. %% Default is undefined, i.e. no timestamp %% Return some {{Y,M,D},{H,M,S}} tuple: last_modified(ReqData, Context) -> {ok, #file_info{mtime=Date}} = file:read_file_info("somefile"), {Date, ReqData, Context}. generate_etag & last_modified Thursday, July 11, 13
68. •Fetch the tweets with curl, copy the ETag from response, fetch again with If-None-Match header. •Add a tweet via the UI, send same curl request as last step. •Add a last_modified callback, using ID of the latest tweet as the timestamp. Exercises Thursday, July 11, 13
69. Templating $ git checkout -f template Thursday, July 11, 13
70. erlydtl •Resembles Django’s template language •Compiles the template into an Erlang module •templates/foo.dtl -> foo_dtl module Thursday, July 11, 13
71. Exercises •Edit the template to change some text, recompile and refresh the browser. Thursday, July 11, 13
72. Authorization & CSRF $ git checkout -f csrf Thursday, July 11, 13
73. Authorization • 401 Unauthorized Authorization WWW-Authenticate • 403 Forbidden - GTFO Thursday, July 11, 13
74. %% Defaults to true %% If unauthorized, return a challenge string for 401 is_authorized(ReqData, Context) -> Auth = wrq:get_req_header("authorization", ReqData), case check_auth(Auth) of true -> {true, ReqData, Context}; false -> {"Basic realm=\"Webmachine\"", ReqData, Context} end. %% Defaults to false, return true for 403 forbidden(ReqData, Context) -> {true, ReqData, Context}. is_authorized & forbidden Thursday, July 11, 13
75. •Modify the CSRF protection to protect DELETE requests. •Attempt to launch a CSRF attack to create a tweet! Exercises Thursday, July 11, 13
76. Dialyzer $ git checkout -f dialyzer Thursday, July 11, 13
77. Dialyzer •Erlang is dynamically-typed, but most functions have specific parameter and return types. •Many bugs can be found by static analysis using Dialyzer. •Annotations are also documentation. Thursday, July 11, 13
78. Exercises •Run dialyzer using the make target. •Break a function’s types, compile, and see if dialyzer will catch it. Thursday, July 11, 13
79. Visual Debugger $ git checkout -f debugger Thursday, July 11, 13
80. Debugger It’s nice to know where your errors are. Thursday, July 11, 13
81. %% Initialize the resource, but enable tracing. init([]) -> wmtrace_resource:add_dispatch_rule("wmtrace", "/tmp"), {{trace, "/tmp"}, #context{}}. Enabling tracing trace storage enable the trace resource Thursday, July 11, 13
82. Exercises •Open the visual debugger at localhost:8080/wmtrace •Refresh the root URL, find the bug in the resource and fix it! Thursday, July 11, 13
83. Asset Resource $ git checkout -f assets-final Thursday, July 11, 13
84. Asset Resource •Catch-all dispatch rule •Renders erlydtl template •ETag & Last-Modified •Checks file existence •Infers media type from file •Adds CSRF token/cookie Thursday, July 11, 13
85. {<<“class”>>, done} Thursday, July 11, 13