Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical nginx module development〜C and Lua〜

Practical nginx module development〜C and Lua〜

Avatar for Tatsuhiko Kubo

Tatsuhiko Kubo

August 22, 2015
Tweet

More Decks by Tatsuhiko Kubo

Other Decks in Technology

Transcript

  1. @cubicdaiya / Tatsuhiko Kubo • Software Engineer @ Mercari, Inc.

    • Infrastructure Engineering • Skills • C, Go, Lua, nginx, … • OSS developer • ngx_small_light, ngx_dynamic_upstream, nginx-build, slackboard, cachectl, gaurun, …
  2. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  3. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  4. nginx • Open Source Software • HTTP server for various

    purpose • Static content delivery • Reverse proxy ( HTTP / FastCGI / uWSGI / … ) • Load balancer • Cache • TLS Termination / SPDY Gateway • etc… • Mail proxy ( SMTP / POP3 / IMAP ) • TCP proxy
  5. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  6. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  7. nginx is module-oriented • most features are designed as module

    • ngx_http_core_module • ngx_http_access_module • ngx_http_log_module • ngx_http_proxy_module • ngx_http_upstream_module • etc…
  8. 3rd party module for nginx • nginx-user can develop an

    original module • It is not required to modify source code of nginx • embed original module with ‘./configure’ $ cd nginx-1.9.x $ ./configure —-add-module=${ngx_module_path}
  9. Example • ngx_http_empty_gif_module (Official) • return 1x1 GIF location /gif

    { empty_gif; } $ curl -I http://127.0.0.1/gif HTTP/1.1 200 OK Content-Length: 43 Content-Type: image/gif ... $
  10. Example • ngx_small_light • Dynamic Image Transformation for nginx small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; }
  11. ngx_small_light • Dynamic Image Transformation for nginx • https://github.com/cubicdaiya/ngx_small_light •

    Provides various • Image Manipulations ( Resize / Crop / Composite / …) • Image Processing Libraries (Image Magick / Imlib2 / GD) • Image Formats (JPEG / GIF / PNG / WEBP)
  12. Transform image on S3 proxy from nginx to S3 small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location /img { proxy_pass http://<s3_bucket_url>; }
  13. Example • ngx_dynamic_upstream • Dynamic Upstream for nginx location /

    { allow xxx.xxx.xxx.xxx deny all; dynamic_upstream; }
  14. ngx_dynamic_upstream • Dynamic Upstream for nginx • https://github.com/cubicdaiya/ngx_dynamic_upstream • Provides

    HTTP APIs for manipulating upstream for nginx • It is useful for • zero-down-time deployment • blue green deployment
  15. ɾɾɾ ɾɾɾ "1* "QBDIF NPE@QIQ OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY 

    OHY@EZOBNJD@VQTUSFBN HTTPS or SPDY OHJOY  OHY@EZOBNJD@VQTUSFBN HTTP Mercari frontend "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ ɾɾɾ ɾɾɾ
  16. "1* "QBDIF NPE@QIQ Previous deployment "1* "QBDIF NPE@QIQ "1* "QBDIF

    NPE@QIQ ChatOps with Slack yes EFQMPZCPU rsync rsync rsync
  17. Pros / Cons • Pros • Very Very Very simple

    • Cons • many 500 error happens at deployment!!!!!
  18. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ
  19. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  20. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  21. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ
  22. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  23. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ
  24. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  25. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  26. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  27. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  28. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  29. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  30. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  31. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  32. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  33. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  34. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; # original directive } $ curl http://127.0.0.1/hello Hello, World! $
  35. Directive declaration struct ngx_command_s { ngx_str_t name; ngx_uint_t type; char

    *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_uint_t conf; ngx_uint_t offset; void *post; };
  36. Module context typedef struct { ngx_int_t (*preconfiguration)(ngx_conf_t *cf); ngx_int_t (*postconfiguration)(ngx_conf_t

    *cf); void *(*create_main_conf)(ngx_conf_t *cf); char *(*init_main_conf)(ngx_conf_t *cf, void *conf); void *(*create_srv_conf)(ngx_conf_t *cf); char *(*merge_srv_conf)(ngx_conf_t *cf, void *prev, void *conf); void *(*create_loc_conf)(ngx_conf_t *cf); char *(*merge_loc_conf)(ngx_conf_t *cf, void *prev, void *conf); } ngx_http_module_t;
  37. Module context static ngx_http_module_t ngx_http_hello_world_module_ctx = { NULL, /* preconfiguration

    */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; In this case, all is NULL.
  38. Module definition ngx_module_t ngx_http_hello_world_module = { NGX_MODULE_V1, &ngx_http_hello_world_module_ctx, /* module

    context */ ngx_http_hello_world_commands, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING };
  39. Hook original handler static char *ngx_http_hello_world(ngx_conf_t *cf, ngx_command_t *cmd, void

    *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_hello_world_handler; return NGX_CONF_OK; }
  40. ngx_http_hello_world_handler() #define NGX_HTTP_HELLO_WORLD “Hello, World!” static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r) {

    ngx_int_t rc; ngx_chain_t out; ngx_buf_t *b; ngx_str_t body = ngx_string(NGX_HTTP_HELLO_WORLD); if (r->method != NGX_HTTP_GET && r->method != NGX_HTTP_HEAD) { return NGX_HTTP_NOT_ALLOWED; } if (r->headers_in.if_modified_since) { return NGX_HTTP_NOT_MODIFIED; } b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } b->pos = body.data; b->last = b->pos + body.len; b->memory = 1; b->last_buf = 1; out.buf = b; out.next = NULL; ngx_str_set(&r->headers_out.content_type, "text/plain"); r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = body.len; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; }
  41. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location
  42. Phase definitions typedef enum { NGX_HTTP_POST_READ_PHASE = 0, NGX_HTTP_SERVER_REWRITE_PHASE, NGX_HTTP_FIND_CONFIG_PHASE,

    NGX_HTTP_REWRITE_PHASE, NGX_HTTP_POST_REWRITE_PHASE, NGX_HTTP_PREACCESS_PHASE, NGX_HTTP_ACCESS_PHASE, NGX_HTTP_POST_ACCESS_PHASE, NGX_HTTP_TRY_FILES_PHASE, NGX_HTTP_CONTENT_PHASE, NGX_HTTP_LOG_PHASE } ngx_http_phases;
  43. Hook handler to specified phases ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cmcf; cmcf

    = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cmcf->phases[NGX_HTTP_ACCESS_PHASE].handlers); if (h == NULL) { return NGX_ERROR; } *h = ngx_http_xxx_handler;
  44. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  45. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  46. nginx API • memory-pool • string • array • hash-table

    • time • list • regular-expression • temporary-file • shared-memory • etc…
  47. essence of nginx API programming • understand nginx architecture •

    event-driven • non-blocking • asynchronous • single-threaded • multi-process
  48. essence of nginx API programming • memory management strategies •

    memory-pool • slab ( for shared-memory) • string with length • string is not always NULL-terminated
  49. essence of nginx API programming • structs for HTTP •

    ngx_http_request_t • ngx_http_headers_in • ngx_http_headers_out • etc… ( See nginx/src/http/ngx_http_request.h )
  50. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  51. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  52. Test::Nginx • Testing framework for nginx • https://github.com/openresty/test-nginx • We

    are able to test nginx’s behavior with CLI • Install from CPAN $ cpanm Test::Nginx
  53. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200
  54. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests
  55. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests define behavior ɾnginx.conf ɾHTTP request ɾHTTP response
  56. By the way, • C programming is difficult and complicated

    • productivity is not high If scripting language is available on nginx, productivity would improve dramatically.
  57. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  58. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  59. ngx_lua • embed Lua/LuaJIT into nginx • Lua scripting in

    nginx.conf • Hooking request processing phase with Lua • wrap nginx API with Lua • non-blocking APIs (e.g. ngx.location.capture())
  60. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file Hook directives for request processing phases
  61. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-string Hook directives for request processing phases
  62. Hook directives for request processing phases • set_by_lua / set_by_lua_file

    • rewrite_by_lua / rewrite_by_lua_file • access_by_lua / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-file
  63. set_by_lua $result <lua-script-string> location /yapc { set $conference YAPC::Asia; set

    $location Tokyo; set $year 2015; set_by_lua $result ‘return ngx.var.conference .. “ ” .. ngx.var.location .. “ ” .. ngx.var.year ‘; content_by_lua ‘ngx.say(ngx.var.result)’; } ▪ nginx.conf $ curl -s http://127.0.0.1/hello YAPC::Asia Tokyo 2015 $ ▪ run
  64. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location set_by_lua*
 rewrite_by_lua* access_by_lua* content_by_lua* log_by_lua*
  65. Other hook directives and phases IPPLOBNF QIBTF JOJU@CZ@MVB JOJU@CZ@MVB@pMF MPBEJOHOHJOYDPOG

    JOJU@XPSLFS@CZ@MVB JOJU@XPSLFS@CZ@MVB@pMF TUBSUJOHXPSLFSQSPDFTT IFBEFS@pMUFS@CZ@MVB IFBEFS@pMUFS@CZ@MVB@pMF pMUFSJOHIFBEFS CPEZ@pMUFS@CZ@MVB CPEZ@pMUFS@CZ@MVB@pMF pMUFSJOHCPEZ
  66. ngx_lua APIs • ngx.say() / ngx.exit() / ngx.log / …

    • ngx.var.VARIABLE • ngx.shared.DICT • ngx.header.HEADER • etc…
  67. HTTP status code in ngx_lua • ngx.HTTP_OK (200) • ngx.HTTP_CREATED

    (201) • ngx.HTTP_MOVED_PERMANENTLY (301) • ngx.HTTP_MOVED_TEMPORARILY (302) • ngx.HTTP_NOT_MODIFIED (304) • ngx.HTTP_BAD_REQUEST (400) • ngx.HTTP_INTERNAL_SERVER_ERROR (500) • etc…
  68. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG
  69. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG log-level High Low
  70. ngx.var.VARIABLE • read / overwrite nginx variables location / {

    set $fn “Tatsuhiko Kubo”; set $hn “@cubicdaiya”; content_by_lua ‘ local body = ngx.var.fn .. ngx.var.hn ngx.say(body) —- Tatsuhiko Kubo@cubicdaiya ‘; }
  71. ngx.var.VARIABLE • creating new variable is not allowed location /

    { content_by_lua ‘ ngx.var.fn = “Tatsuhiko Kubo” ngx.var.hn = “@cubicdaiya” ‘; } runtime error: … variable “fn” not found for writing; maybe it is a built-in variable that is not changeable or … ɾ error.log
  72. ngx.shared.DICT • dictionary with shared memory http { lua_shared_dict yapc

    1m; init_by_lua ‘ ngx.shared.yapc:set(“presentator”, “@cubicdaiya”) ’; … server { … location / { content_by_lua ‘ ngx.say(ngx.shared.yapc:get(“presentator”)) ’; } } }
  73. ngx.header.HEADER • response header reference and manipulation location / {

    content_by_lua ‘ local body = “{ \\”presentator\\” : \\”@cubicdaiya\\” }” —- assign application/json to Content-Type ngx.header.content_type = “application/json” ngx.say(body) ‘; }
  74. Header reference and manipulation • ngx.req.get_headers() • ngx.req.set_header() • ngx.req.clear_header()

    • … • ngx.resp.get_headers() • ngx.header.HEADER • … ▪ For request-header ▪ For response-header
  75. ngx_lua time APIs • ngx.time() • ngx.today() • ngx.now() •

    ngx.update_time() • ngx.parse_http_time() • ngx.localtime() • ngx.utctime() • ngx.cookie_time() • ngx.http_time()
  76. Use ngx.time() instead of os.time() • os.time() always calls gettimeofday()

    • ngx.time() uses timer-cache of nginx • avoid system-call’s overhead • ngx.time() is more efficient than os.time()
  77. ngx.re.match() • Regular Expression matching • powered by PCRE local

    matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”) -— matches[0] -> image/webp -— matches[1] -> image -— matches[2] -> webp
  78. • Use PCRE options(e.g. o, j) • o -> enable

    compile-once • j -> enable PCRE-JIT Optimize ngx.re.* local matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”, “oj”)
  79. • ngx.re is based PCRE • Powerful & Fat •

    Lua builtin-re(regular expression) • Simple & Light ngx.re.* VS Lua builtin-re If ‘re’ is simple, Lua builtin-re is faster.
  80. ngx.location.capture() • send non-blocking sub-request to other location location /external

    { internal; proxy_pass http://external; } location / { content_by_lua ‘ —— non-blocking behavior local res = ngx.location.capture(“/external”) … ‘; }
  81. By the way, • ngx_lua is very powerful • But

    setup and deployment is a little painful • Lua / LuaJIT instalation • Lua packages management
  82. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  83. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  84. OpenResty • Web application framework bundled • nginx and 3rd

    party modules (e.g. ngx_lua) • Lua, LuaJIT • Resty modules(lua-resty-xxx…) • All-in-one superset of nginx
  85. lua-resty-core • pure Lua partial-re-implementation of ngx_lua APIs • accelerate

    ngx_lua APIs with LuaJIT • because ngx_lua APIs with C is not JIT compiled http { # accelerate ngx_lua APIs init_by_lua ‘require resty.core’; … }
  86. lua-resty-cli • command-line utility for OpenResty • run OpenResty with

    CLI • all APIs are not always enabled $ resty -e ‘ngx.say(“Hello, World!”)’ Hello, World! $
  87. lua-resty-mysql / redis / memcached • client drivers for mysql

    / redis / memcached • non-blocking behavior • Powered by ngx.socket
  88. PascalʙMercari app-event analysis baseʙ 0QFO3FTUZ 0QFO3FTUZ 0QFO3FTUZ (PPHMF#JH2VFSZ Developer Data

    Sientist Analyze by SQL send events send events send events Powered by cookpad/puree-(ios|android) utilize events transfer utilize events utilize events transfer
  89. OpenResty’s role on Pascal • High performance event logger •

    receive events as JSON from devices • parse JSON • utilize events for BigQuery • log events into file every event-channel
  90. References • The Architecture of Open Source Applications (Volume 2):

    nginx • http://www.aosabook.org/en/nginx.html • Nginx Internals • http://www.slideshare.net/joshzhu/nginx-internals • Emiller’s Guide To Nginx Module Development • http://www.evanmiller.org/nginx-modules-guide.html
  91. References • openresty/lua-nginx-module • https://github.com/openresty/lua-nginx-module • Using ngx_lua / lua-nginx-module

    in pixiv • http://www.slideshare.net/harukayon/ngx-lua-public • ngx_mruby and ngx_lua • http://qiita.com/cubicdaiya/items/3913d3dddf6999984c5b