Web servers - and how I created my own one

Coffe and Code Donetsk Jan 2012

Transcript

1. Web servers Igor Afonov @iafonov (And how I created my own one)
2. Basic principles Web server
3. Basic principles Web server HTTP Request HTTP Response
4. HTTP • Text based • RFC 2616
5. HTTP POST /files/new/isubjkzdaeqtfhmycrnwlpvgxo HTTP/1.1 Host: 127.0.0.1:31337 Connection: keep-alive Content-Length: 219 Origin: http://127.0.0.1:31337 Accept-Encoding: gzip,deflate Accept-Language: en-US Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 ------WebKitFormBoundaryVreBnvR3qKu3B5Iv Content-Disposition: form-data; name="file"; filename="smally" Content-Type: application/octet-stream Very small file. ------WebKitFormBoundaryVreBnvR3qKu3B5Iv--
6. Lifecycle • Setup networking & signal handlers • Accept connection • Parse & process request • Serialize response • Send it back
7. Networking • socket • bind • listen • accept • send/recv/sendfile* • close do the job POSIX
8. Blocking IO • select • Forking (Pre-forking) • Threading • Bad idea for web sockets (Apache)
9. Non-blocking IO • kqueue/poll/epoll • libevent/libev/libuv • Reactor/Notifier/Dispatcher pattern • Works good for web sockets & persistent connections (Nginx, node.js, tornado, event machine...)
10. The Big Picture Your Application Rack Web server Operating System Rails Request Response
11. Cosmonaut • Blocking (At least for now) • Simple & straightforward • Made from tiny isolated modules • Fast, not so slow (Implemented in C) • 3 weeks old
12. Features • Serves static content (via sendfile) • Parses multipart data • Basic routing • Provides simple API • Tiny memory footprint
13. Architecture • Main process forks workers for each request • Forking is fast & reliable with copy-on-write • ~10K ram per request • Context switching is expensive :( • ~600 concurrent processes max on Xeon
14. C • Surprisingly simple • Dangerous by default, safe by demand • Still modern
15. C & OOP module.h typedef struct module module; typedef struct module_state module_state; struct module { void *public_data; polymorfic *polymorfic; module_state *_s; }; module *module_init(...); void module_free(module *m); int module_do_something(module *m);
16. Memory Management • Allocate & free at the same level • Allocate & free at the same module • Use Valgrind • Be careful
17. Parsing HTTP • Server is set of parsers • Callback based • Must work with chunks of data • Stateful • No buffering • No bugs
18. API #include <cosmonaut/cosmonaut.h> void action_index(http_request *request, http_response *response) { render_text(response, "Hello world"); } void configure() { mount("/", action_index); } int main(int argc, char *argv[]) { return cosmonaut_start(argc, argv, configure); } Hello World
19. API struct http_request { url *url; headers_map *headers; params_map *params; configuration *configuration; route *route; void *data; char *uid; }; char *c_type = headers_map_get(request->headers, "Content-Type"); Request
20. API typedef struct http_response { int code; char *header_summary; char *file_path; char *content_type; int content_length; headers_map* headers; char* raw_response; } http_response; render_file(http_response *response, const char *path); render_text(http_response *response, const char *text); render_json(http_response *response, const char *json); Response
21. API mount("/", action_index); mount("/upload_file", action_upload); Routing pt.1
22. API mount("/photos/(:id)", action); void action(http_request *request, http_response *response) { int photo_id = params_map_get(request->params, "id")->val; ... } Routing pt.2
23. API mount("/patients/(:id)/(:action)", rest_application); void rest_application(http_request *request, http_response *response) { char *action = params_map_get(request->params, "action")->val; int patient_id = params_map_get(request->params, "id")->val; process_data(patient, action, response); } /patients/1/new // params: {id => 1, action => "new"} /patients/1/show // params: {id => 1, action => "show"} /patients/1/edit // params: {id => 1, action => "edit"} /patients/1/delete // params: {id => 1, action => "delete"} Routing pt.3
24. API route *rt = mount("/files/new/(:upload_id)", action_upload); rt->before_filter = check_upload_size; rt->after_filter = clear_db_connections; Filters
25. Testing • Cucumber • Rest-client • Travis CI
26. Testing Feature: Routing and params processing Scenario Outline: Requesting action mounted to '/photos/(:id)' When I request "<path>" Then I should get "<expected_response>" Examples: | path | expected_response | | /photos/1 | [id:1] | | /photos/word | [id:word] | | /photos/1/somth | Not Found | | /photos/1/ | Not Found | | /photos1/1 | Not Found |
27. Testing When /^I request "([^"]*)"$/ do |path| begin @response = RestClient.get("http://127.0.0.1:31337#{path}") rescue Exception => e @client_exception = e end end Then /^I should get "([^"]*)"$/ do |body| response_body = @client_exception.nil? ? @response.to_s : @client_exception.http_body response_body.should == body end
28. Sample Application • Supports file uploads/downloads • Supports tracking of upload progress • Uses redis for persistence & IPC https://github.com/iafonov/uploader_app
29. Don't reinvent the wheel Do it!
30. https://github.com/iafonov/cosmonaut https://github.com/iafonov/cosmonaut/wiki/Links http://iafonov.github.com/ @iafonov