Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical nginx module development〜C and Lua〜

Practical nginx module development〜C and Lua〜

Tatsuhiko Kubo

August 22, 2015
Tweet

More Decks by Tatsuhiko Kubo

Other Decks in Technology

Transcript

  1. @cubicdaiya / Tatsuhiko Kubo • Software Engineer @ Mercari, Inc.

    • Infrastructure Engineering • Skills • C, Go, Lua, nginx, … • OSS developer • ngx_small_light, ngx_dynamic_upstream, nginx-build, slackboard, cachectl, gaurun, …
  2. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  3. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  4. nginx • Open Source Software • HTTP server for various

    purpose • Static content delivery • Reverse proxy ( HTTP / FastCGI / uWSGI / … ) • Load balancer • Cache • TLS Termination / SPDY Gateway • etc… • Mail proxy ( SMTP / POP3 / IMAP ) • TCP proxy
  5. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  6. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  7. nginx is module-oriented • most features are designed as module

    • ngx_http_core_module • ngx_http_access_module • ngx_http_log_module • ngx_http_proxy_module • ngx_http_upstream_module • etc…
  8. 3rd party module for nginx • nginx-user can develop an

    original module • It is not required to modify source code of nginx • embed original module with ‘./configure’ $ cd nginx-1.9.x $ ./configure —-add-module=${ngx_module_path}
  9. Example • ngx_http_empty_gif_module (Official) • return 1x1 GIF location /gif

    { empty_gif; } $ curl -I http://127.0.0.1/gif HTTP/1.1 200 OK Content-Length: 43 Content-Type: image/gif ... $
  10. Example • ngx_small_light • Dynamic Image Transformation for nginx small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; }
  11. ngx_small_light • Dynamic Image Transformation for nginx • https://github.com/cubicdaiya/ngx_small_light •

    Provides various • Image Manipulations ( Resize / Crop / Composite / …) • Image Processing Libraries (Image Magick / Imlib2 / GD) • Image Formats (JPEG / GIF / PNG / WEBP)
  12. Transform image on S3 proxy from nginx to S3 small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location /img { proxy_pass http://<s3_bucket_url>; }
  13. Example • ngx_dynamic_upstream • Dynamic Upstream for nginx location /

    { allow xxx.xxx.xxx.xxx deny all; dynamic_upstream; }
  14. ngx_dynamic_upstream • Dynamic Upstream for nginx • https://github.com/cubicdaiya/ngx_dynamic_upstream • Provides

    HTTP APIs for manipulating upstream for nginx • It is useful for • zero-down-time deployment • blue green deployment
  15. ɾɾɾ ɾɾɾ "1* "QBDIF NPE@QIQ OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY 

    OHY@EZOBNJD@VQTUSFBN HTTPS or SPDY OHJOY  OHY@EZOBNJD@VQTUSFBN HTTP Mercari frontend "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ ɾɾɾ ɾɾɾ
  16. "1* "QBDIF NPE@QIQ Previous deployment "1* "QBDIF NPE@QIQ "1* "QBDIF

    NPE@QIQ ChatOps with Slack yes EFQMPZCPU rsync rsync rsync
  17. Pros / Cons • Pros • Very Very Very simple

    • Cons • many 500 error happens at deployment!!!!!
  18. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ
  19. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  20. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  21. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ
  22. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  23. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ
  24. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  25. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  26. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  27. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  28. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  29. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  30. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  31. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  32. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  33. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  34. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; # original directive } $ curl http://127.0.0.1/hello Hello, World! $
  35. Directive declaration struct ngx_command_s { ngx_str_t name; ngx_uint_t type; char

    *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_uint_t conf; ngx_uint_t offset; void *post; };
  36. Module context typedef struct { ngx_int_t (*preconfiguration)(ngx_conf_t *cf); ngx_int_t (*postconfiguration)(ngx_conf_t

    *cf); void *(*create_main_conf)(ngx_conf_t *cf); char *(*init_main_conf)(ngx_conf_t *cf, void *conf); void *(*create_srv_conf)(ngx_conf_t *cf); char *(*merge_srv_conf)(ngx_conf_t *cf, void *prev, void *conf); void *(*create_loc_conf)(ngx_conf_t *cf); char *(*merge_loc_conf)(ngx_conf_t *cf, void *prev, void *conf); } ngx_http_module_t;
  37. Module context static ngx_http_module_t ngx_http_hello_world_module_ctx = { NULL, /* preconfiguration

    */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; In this case, all is NULL.
  38. Module definition ngx_module_t ngx_http_hello_world_module = { NGX_MODULE_V1, &ngx_http_hello_world_module_ctx, /* module

    context */ ngx_http_hello_world_commands, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING };
  39. Hook original handler static char *ngx_http_hello_world(ngx_conf_t *cf, ngx_command_t *cmd, void

    *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_hello_world_handler; return NGX_CONF_OK; }
  40. ngx_http_hello_world_handler() #define NGX_HTTP_HELLO_WORLD “Hello, World!” static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r) {

    ngx_int_t rc; ngx_chain_t out; ngx_buf_t *b; ngx_str_t body = ngx_string(NGX_HTTP_HELLO_WORLD); if (r->method != NGX_HTTP_GET && r->method != NGX_HTTP_HEAD) { return NGX_HTTP_NOT_ALLOWED; } if (r->headers_in.if_modified_since) { return NGX_HTTP_NOT_MODIFIED; } b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } b->pos = body.data; b->last = b->pos + body.len; b->memory = 1; b->last_buf = 1; out.buf = b; out.next = NULL; ngx_str_set(&r->headers_out.content_type, "text/plain"); r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = body.len; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; }
  41. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location
  42. Phase definitions typedef enum { NGX_HTTP_POST_READ_PHASE = 0, NGX_HTTP_SERVER_REWRITE_PHASE, NGX_HTTP_FIND_CONFIG_PHASE,

    NGX_HTTP_REWRITE_PHASE, NGX_HTTP_POST_REWRITE_PHASE, NGX_HTTP_PREACCESS_PHASE, NGX_HTTP_ACCESS_PHASE, NGX_HTTP_POST_ACCESS_PHASE, NGX_HTTP_TRY_FILES_PHASE, NGX_HTTP_CONTENT_PHASE, NGX_HTTP_LOG_PHASE } ngx_http_phases;
  43. Hook handler to specified phases ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cmcf; cmcf

    = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cmcf->phases[NGX_HTTP_ACCESS_PHASE].handlers); if (h == NULL) { return NGX_ERROR; } *h = ngx_http_xxx_handler;
  44. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  45. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  46. nginx API • memory-pool • string • array • hash-table

    • time • list • regular-expression • temporary-file • shared-memory • etc…
  47. essence of nginx API programming • understand nginx architecture •

    event-driven • non-blocking • asynchronous • single-threaded • multi-process
  48. essence of nginx API programming • memory management strategies •

    memory-pool • slab ( for shared-memory) • string with length • string is not always NULL-terminated
  49. essence of nginx API programming • structs for HTTP •

    ngx_http_request_t • ngx_http_headers_in • ngx_http_headers_out • etc… ( See nginx/src/http/ngx_http_request.h )
  50. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  51. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  52. Test::Nginx • Testing framework for nginx • https://github.com/openresty/test-nginx • We

    are able to test nginx’s behavior with CLI • Install from CPAN $ cpanm Test::Nginx
  53. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200
  54. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests
  55. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests define behavior ɾnginx.conf ɾHTTP request ɾHTTP response
  56. By the way, • C programming is difficult and complicated

    • productivity is not high If scripting language is available on nginx, productivity would improve dramatically.
  57. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  58. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  59. ngx_lua • embed Lua/LuaJIT into nginx • Lua scripting in

    nginx.conf • Hooking request processing phase with Lua • wrap nginx API with Lua • non-blocking APIs (e.g. ngx.location.capture())
  60. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file Hook directives for request processing phases
  61. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-string Hook directives for request processing phases
  62. Hook directives for request processing phases • set_by_lua / set_by_lua_file

    • rewrite_by_lua / rewrite_by_lua_file • access_by_lua / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-file
  63. set_by_lua $result <lua-script-string> location /yapc { set $conference YAPC::Asia; set

    $location Tokyo; set $year 2015; set_by_lua $result ‘return ngx.var.conference .. “ ” .. ngx.var.location .. “ ” .. ngx.var.year ‘; content_by_lua ‘ngx.say(ngx.var.result)’; } ▪ nginx.conf $ curl -s http://127.0.0.1/hello YAPC::Asia Tokyo 2015 $ ▪ run
  64. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location set_by_lua*
 rewrite_by_lua* access_by_lua* content_by_lua* log_by_lua*
  65. Other hook directives and phases IPPLOBNF QIBTF JOJU@CZ@MVB JOJU@CZ@MVB@pMF MPBEJOHOHJOYDPOG

    JOJU@XPSLFS@CZ@MVB JOJU@XPSLFS@CZ@MVB@pMF TUBSUJOHXPSLFSQSPDFTT IFBEFS@pMUFS@CZ@MVB IFBEFS@pMUFS@CZ@MVB@pMF pMUFSJOHIFBEFS CPEZ@pMUFS@CZ@MVB CPEZ@pMUFS@CZ@MVB@pMF pMUFSJOHCPEZ
  66. ngx_lua APIs • ngx.say() / ngx.exit() / ngx.log / …

    • ngx.var.VARIABLE • ngx.shared.DICT • ngx.header.HEADER • etc…
  67. HTTP status code in ngx_lua • ngx.HTTP_OK (200) • ngx.HTTP_CREATED

    (201) • ngx.HTTP_MOVED_PERMANENTLY (301) • ngx.HTTP_MOVED_TEMPORARILY (302) • ngx.HTTP_NOT_MODIFIED (304) • ngx.HTTP_BAD_REQUEST (400) • ngx.HTTP_INTERNAL_SERVER_ERROR (500) • etc…
  68. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG
  69. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG log-level High Low
  70. ngx.var.VARIABLE • read / overwrite nginx variables location / {

    set $fn “Tatsuhiko Kubo”; set $hn “@cubicdaiya”; content_by_lua ‘ local body = ngx.var.fn .. ngx.var.hn ngx.say(body) —- Tatsuhiko Kubo@cubicdaiya ‘; }
  71. ngx.var.VARIABLE • creating new variable is not allowed location /

    { content_by_lua ‘ ngx.var.fn = “Tatsuhiko Kubo” ngx.var.hn = “@cubicdaiya” ‘; } runtime error: … variable “fn” not found for writing; maybe it is a built-in variable that is not changeable or … ɾ error.log
  72. ngx.shared.DICT • dictionary with shared memory http { lua_shared_dict yapc

    1m; init_by_lua ‘ ngx.shared.yapc:set(“presentator”, “@cubicdaiya”) ’; … server { … location / { content_by_lua ‘ ngx.say(ngx.shared.yapc:get(“presentator”)) ’; } } }
  73. ngx.header.HEADER • response header reference and manipulation location / {

    content_by_lua ‘ local body = “{ \\”presentator\\” : \\”@cubicdaiya\\” }” —- assign application/json to Content-Type ngx.header.content_type = “application/json” ngx.say(body) ‘; }
  74. Header reference and manipulation • ngx.req.get_headers() • ngx.req.set_header() • ngx.req.clear_header()

    • … • ngx.resp.get_headers() • ngx.header.HEADER • … ▪ For request-header ▪ For response-header
  75. ngx_lua time APIs • ngx.time() • ngx.today() • ngx.now() •

    ngx.update_time() • ngx.parse_http_time() • ngx.localtime() • ngx.utctime() • ngx.cookie_time() • ngx.http_time()
  76. Use ngx.time() instead of os.time() • os.time() always calls gettimeofday()

    • ngx.time() uses timer-cache of nginx • avoid system-call’s overhead • ngx.time() is more efficient than os.time()
  77. ngx.re.match() • Regular Expression matching • powered by PCRE local

    matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”) -— matches[0] -> image/webp -— matches[1] -> image -— matches[2] -> webp
  78. • Use PCRE options(e.g. o, j) • o -> enable

    compile-once • j -> enable PCRE-JIT Optimize ngx.re.* local matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”, “oj”)
  79. • ngx.re is based PCRE • Powerful & Fat •

    Lua builtin-re(regular expression) • Simple & Light ngx.re.* VS Lua builtin-re If ‘re’ is simple, Lua builtin-re is faster.
  80. ngx.location.capture() • send non-blocking sub-request to other location location /external

    { internal; proxy_pass http://external; } location / { content_by_lua ‘ —— non-blocking behavior local res = ngx.location.capture(“/external”) … ‘; }
  81. By the way, • ngx_lua is very powerful • But

    setup and deployment is a little painful • Lua / LuaJIT instalation • Lua packages management
  82. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  83. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  84. OpenResty • Web application framework bundled • nginx and 3rd

    party modules (e.g. ngx_lua) • Lua, LuaJIT • Resty modules(lua-resty-xxx…) • All-in-one superset of nginx
  85. lua-resty-core • pure Lua partial-re-implementation of ngx_lua APIs • accelerate

    ngx_lua APIs with LuaJIT • because ngx_lua APIs with C is not JIT compiled http { # accelerate ngx_lua APIs init_by_lua ‘require resty.core’; … }
  86. lua-resty-cli • command-line utility for OpenResty • run OpenResty with

    CLI • all APIs are not always enabled $ resty -e ‘ngx.say(“Hello, World!”)’ Hello, World! $
  87. lua-resty-mysql / redis / memcached • client drivers for mysql

    / redis / memcached • non-blocking behavior • Powered by ngx.socket
  88. PascalʙMercari app-event analysis baseʙ 0QFO3FTUZ 0QFO3FTUZ 0QFO3FTUZ (PPHMF#JH2VFSZ Developer Data

    Sientist Analyze by SQL send events send events send events Powered by cookpad/puree-(ios|android) utilize events transfer utilize events utilize events transfer
  89. OpenResty’s role on Pascal • High performance event logger •

    receive events as JSON from devices • parse JSON • utilize events for BigQuery • log events into file every event-channel
  90. References • The Architecture of Open Source Applications (Volume 2):

    nginx • http://www.aosabook.org/en/nginx.html • Nginx Internals • http://www.slideshare.net/joshzhu/nginx-internals • Emiller’s Guide To Nginx Module Development • http://www.evanmiller.org/nginx-modules-guide.html
  91. References • openresty/lua-nginx-module • https://github.com/openresty/lua-nginx-module • Using ngx_lua / lua-nginx-module

    in pixiv • http://www.slideshare.net/harukayon/ngx-lua-public • ngx_mruby and ngx_lua • http://qiita.com/cubicdaiya/items/3913d3dddf6999984c5b