lua-nginx-module 0,0,2,6,0,6,11,5 travis-ci

Embed the Power of Lua into NGINX HTTP servers

2 years after

Name

ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers.

This module is not distributed with the Nginx source. See the installation instructions.

Table of Contents

Status

Production ready.

Version

This document describes ngx_lua v0.10.8 released on 8 April 2017.

Synopsis


 # set search paths for pure Lua external libraries (';;' is the default path):
 lua_package_path '/foo/bar/?.lua;/blah/?.lua;;';

 # set search paths for Lua external libraries written in C (can also use ';;'):
 lua_package_cpath '/bar/baz/?.so;/blah/blah/?.so;;';

 server {
     location /lua_content {
         # MIME type determined by default_type:
         default_type 'text/plain';

         content_by_lua_block {
             ngx.say('Hello,world!')
         }
     }

     location /nginx_var {
         # MIME type determined by default_type:
         default_type 'text/plain';

         # try access /nginx_var?a=hello,world
         content_by_lua_block {
             ngx.say(ngx.var.arg_a)
         }
     }

     location = /request_body {
         client_max_body_size 50k;
         client_body_buffer_size 50k;

         content_by_lua_block {
             ngx.req.read_body()  -- explicitly read the req body
             local data = ngx.req.get_body_data()
             if data then
                 ngx.say("body data:")
                 ngx.print(data)
                 return
             end

             -- body may get buffered in a temp file:
             local file = ngx.req.get_body_file()
             if file then
                 ngx.say("body is in file ", file)
             else
                 ngx.say("no body found")
             end
         }
     }

     # transparent non-blocking I/O in Lua via subrequests
     # (well, a better way is to use cosockets)
     location = /lua {
         # MIME type determined by default_type:
         default_type 'text/plain';

         content_by_lua_block {
             local res = ngx.location.capture("/some_other_location")
             if res then
                 ngx.say("status: ", res.status)
                 ngx.say("body:")
                 ngx.print(res.body)
             end
         }
     }

     location = /foo {
         rewrite_by_lua_block {
             res = ngx.location.capture("/memc",
                 { args = { cmd = "incr", key = ngx.var.uri } }
             )
         }

         proxy_pass http://blah.blah.com;
     }

     location = /mixed {
         rewrite_by_lua_file /path/to/rewrite.lua;
         access_by_lua_file /path/to/access.lua;
         content_by_lua_file /path/to/content.lua;
     }

     # use nginx var in code path
     # CAUTION: contents in nginx var must be carefully filtered,
     # otherwise there'll be great security risk!
     location ~ ^/app/([-_a-zA-Z0-9/]+) {
         set $path $1;
         content_by_lua_file /path/to/lua/app/root/$path.lua;
     }

     location / {
        client_max_body_size 100k;
        client_body_buffer_size 100k;

        access_by_lua_block {
            -- check the client IP address is in our black list
            if ngx.var.remote_addr == "132.5.72.3" then
                ngx.exit(ngx.HTTP_FORBIDDEN)
            end

            -- check if the URI contains bad words
            if ngx.var.uri and
                   string.match(ngx.var.request_body, "evil")
            then
                return ngx.redirect("/terms_of_use.html")
            end

            -- tests passed
        }

        # proxy_pass/fastcgi_pass/etc settings
     }
 }

Back to TOC

Description

This module embeds Lua, via the standard Lua 5.1 interpreter or LuaJIT 2.0/2.1, into Nginx and by leveraging Nginx's subrequests, allows the integration of the powerful Lua threads (Lua coroutines) into the Nginx event model.

Unlike Apache's mod_lua and Lighttpd's mod_magnet, Lua code executed using this module can be 100% non-blocking on network traffic as long as the Nginx API for Lua provided by this module is used to handle requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services.

At least the following Lua libraries and Nginx modules can be used with this ngx_lua module:

Almost all the Nginx modules can be used with this ngx_lua module by means of ngx.location.capture or ngx.location.capture_multi but it is recommended to use those lua-resty-* libraries instead of creating subrequests to access the Nginx upstream modules because the former is usually much more flexible and memory-efficient.

The Lua interpreter or LuaJIT instance is shared across all the requests in a single nginx worker process but request contexts are segregated using lightweight Lua coroutines.

Loaded Lua modules persist in the nginx worker process level resulting in a small memory footprint in Lua even when under heavy loads.

This module is plugged into NGINX's "http" subsystem so it can only speaks downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, WebSockets, and etc). If you want to do generic TCP communications with the downstream clients, then you should use the ngx_stream_lua module instead which has a compatible Lua API.

Back to TOC

Typical Uses

Just to name a few:

  • Mashup'ing and processing outputs of various nginx upstream outputs (proxy, drizzle, postgres, redis, memcached, and etc) in Lua,
  • doing arbitrarily complex access control and security checks in Lua before requests actually reach the upstream backends,
  • manipulating response headers in an arbitrary way (by Lua)
  • fetching backend information from external storage backends (like redis, memcached, mysql, postgresql) and use that information to choose which upstream backend to access on-the-fly,
  • coding up arbitrarily complex web applications in a content handler using synchronous but still non-blocking access to the database backends and other storage,
  • doing very complex URL dispatch in Lua at rewrite phase,
  • using Lua to implement advanced caching mechanism for Nginx's subrequests and arbitrary locations.

The possibilities are unlimited as the module allows bringing together various elements within Nginx as well as exposing the power of the Lua language to the user. The module provides the full flexibility of scripting while offering performance levels comparable with native C language programs both in terms of CPU time as well as memory footprint. This is particularly the case when LuaJIT 2.x is enabled.

Other scripting language implementations typically struggle to match this performance level.

The Lua state (Lua VM instance) is shared across all the requests handled by a single nginx worker process to minimize memory use.

Back to TOC

Nginx Compatibility

The latest version of this module is compatible with the following versions of Nginx:

  • 1.11.x (last tested: 1.11.2)
  • 1.10.x
  • 1.9.x (last tested: 1.9.15)
  • 1.8.x
  • 1.7.x (last tested: 1.7.10)
  • 1.6.x

Nginx cores older than 1.6.0 (exclusive) are not supported.

Back to TOC

Installation

It is highly recommended to use OpenResty releases which integrate Nginx, ngx_lua, LuaJIT 2.1, as well as other powerful companion Nginx modules and Lua libraries. It is discouraged to build this module with nginx yourself since it is tricky to set up exactly right. Also, the stock nginx cores have various limitations and long standing bugs that can make some of this modules' features become disabled, not work properly, or run slower.

Alternatively, ngx_lua can be manually compiled into Nginx:

  1. Install LuaJIT 2.0 or 2.1 (recommended) or Lua 5.1 (Lua 5.2 is not supported yet). LuaJIT can be downloaded from the LuaJIT project website and Lua 5.1, from the Lua project website. Some distribution package managers also distribute LuaJIT and/or Lua.
  2. Download the latest version of the ngx_devel_kit (NDK) module HERE.
  3. Download the latest version of ngx_lua HERE.
  4. Download the latest version of Nginx HERE (See Nginx Compatibility)

Build the source with this module:


 wget 'http://nginx.org/download/nginx-1.11.2.tar.gz'
 tar -xzvf nginx-1.11.2.tar.gz
 cd nginx-1.11.2/

 # tell nginx's build system where to find LuaJIT 2.0:
 export LUAJIT_LIB=/path/to/luajit/lib
 export LUAJIT_INC=/path/to/luajit/include/luajit-2.0

 # tell nginx's build system where to find LuaJIT 2.1:
 export LUAJIT_LIB=/path/to/luajit/lib
 export LUAJIT_INC=/path/to/luajit/include/luajit-2.1

 # or tell where to find Lua if using Lua instead:
 #export LUA_LIB=/path/to/lua/lib
 #export LUA_INC=/path/to/lua/include

 # Here we assume Nginx is to be installed under /opt/nginx/.
 ./configure --prefix=/opt/nginx \
         --with-ld-opt="-Wl,-rpath,/path/to/luajit-or-lua/lib" \
         --add-module=/path/to/ngx_devel_kit \
         --add-module=/path/to/lua-nginx-module

 # Note that you may also want to add `./configure` options which are used in your
 # current nginx build.
 # You can get usually those options using command nginx -V

 # you can change the parallism number 2 below to fit the number of spare CPU cores in your
 # machine.
 make -j2
 make install

Back to TOC

Building as a dynamic module

Starting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the --add-dynamic-module=PATH option instead of --add-module=PATH on the ./configure command line above. And then you can explicitly load the module in your nginx.conf via the load_module directive, for example,


 load_module /path/to/modules/ndk_http_module.so;  # assuming NDK is built as a dynamic module too
 load_module /path/to/modules/ngx_http_lua_module.so;

Back to TOC

C Macro Configurations

While building this module either via OpenResty or with the NGINX core, you can define the following C macros via the C compiler options:

  • NGX_LUA_USE_ASSERT When defined, will enable assertions in the ngx_lua C code base. Recommended for debugging or testing builds. It can introduce some (small) runtime overhead when enabled. This macro was first introduced in the v0.9.10 release.
  • NGX_LUA_ABORT_AT_PANIC When the Lua/LuaJIT VM panics, ngx_lua will instruct the current nginx worker process to quit gracefully by default. By specifying this C macro, ngx_lua will abort the current nginx worker process (which usually result in a core dump file) immediately. This option is useful for debugging VM panics. This option was first introduced in the v0.9.8 release.
  • NGX_LUA_NO_FFI_API Excludes pure C API functions for FFI-based Lua API for NGINX (as required by lua-resty-core, for example). Enabling this macro can make the resulting binary code size smaller.

To enable one or more of these macros, just pass extra C compiler options to the ./configure script of either NGINX or OpenResty. For instance,

./configure --with-cc-opt="-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC"

Back to TOC

Installation on Ubuntu 11.10

Note that it is recommended to use LuaJIT 2.0 or LuaJIT 2.1 instead of the standard Lua 5.1 interpreter wherever possible.

If the standard Lua 5.1 interpreter is required however, run the following command to install it from the Ubuntu repository:


 apt-get install -y lua5.1 liblua5.1-0 liblua5.1-0-dev

Everything should be installed correctly, except for one small tweak.

Library name liblua.so has been changed in liblua5.1 package, it only comes with liblua5.1.so, which needs to be symlinked to /usr/lib so it could be found during the configuration process.


 ln -s /usr/lib/x86_64-linux-gnu/liblua5.1.so /usr/lib/liblua.so

Back to TOC

Community

Back to TOC

English Mailing List

The openresty-en mailing list is for English speakers.

Back to TOC

Chinese Mailing List

The openresty mailing list is for Chinese speakers.

Back to TOC

Code Repository

The code repository of this project is hosted on github at openresty/lua-nginx-module.

Back to TOC

Bugs and Patches

Please submit bug reports, wishlists, or patches by

  1. creating a ticket on the GitHub Issue Tracker,
  2. or posting to the OpenResty community.

Back to TOC

Lua/LuaJIT bytecode support

As from the v0.5.0rc32 release, all *_by_lua_file configure directives (such as content_by_lua_file) support loading Lua 5.1 and LuaJIT 2.0/2.1 raw bytecode files directly.

Please note that the bytecode format used by LuaJIT 2.0/2.1 is not compatible with that used by the standard Lua 5.1 interpreter. So if using LuaJIT 2.0/2.1 with ngx_lua, LuaJIT compatible bytecode files must be generated as shown:


 /path/to/luajit/bin/luajit -b /path/to/input_file.lua /path/to/output_file.ljbc

The -bg option can be used to include debug information in the LuaJIT bytecode file:


 /path/to/luajit/bin/luajit -bg /path/to/input_file.lua /path/to/output_file.ljbc

Please refer to the official LuaJIT documentation on the -b option for more details:

http://luajit.org/running.html#opt_b

Also, the bytecode files generated by LuaJIT 2.1 is not compatible with LuaJIT 2.0, and vice versa. The support for LuaJIT 2.1 bytecode was first added in ngx_lua v0.9.3.

Similarly, if using the standard Lua 5.1 interpreter with ngx_lua, Lua compatible bytecode files must be generated using the luac commandline utility as shown:


 luac -o /path/to/output_file.luac /path/to/input_file.lua

Unlike as with LuaJIT, debug information is included in standard Lua 5.1 bytecode files by default. This can be striped out by specifying the -s option as shown:


 luac -s -o /path/to/output_file.luac /path/to/input_file.lua

Attempts to load standard Lua 5.1 bytecode files into ngx_lua instances linked to LuaJIT 2.0/2.1 or vice versa, will result in an error message, such as that below, being logged into the Nginx error.log file:

[error] 13909#0: *1 failed to load Lua inlined code: bad byte-code header in /path/to/test_file.luac

Loading bytecode files via the Lua primitives like require and dofile should always work as expected.

Back to TOC

System Environment Variable Support

If you want to access the system environment variable, say, foo, in Lua via the standard Lua API os.getenv, then you should also list this environment variable name in your nginx.conf file via the env directive. For example,


 env foo;

Back to TOC

HTTP 1.0 support

The HTTP 1.0 protocol does not support chunked output and requires an explicit Content-Length header when the response body is not empty in order to support the HTTP 1.0 keep-alive. So when a HTTP 1.0 request is made and the lua_http10_buffering directive is turned on, ngx_lua will buffer the output of ngx.say and ngx.print calls and also postpone sending response headers until all the response body output is received. At that time ngx_lua can calculate the total length of the body and construct a proper Content-Length header to return to the HTTP 1.0 client. If the Content-Length response header is set in the running Lua code, however, this buffering will be disabled even if the lua_http10_buffering directive is turned on.

For large streaming output responses, it is important to disable the lua_http10_buffering directive to minimise memory usage.

Note that common HTTP benchmark tools such as ab and http_load issue HTTP 1.0 requests by default. To force curl to send HTTP 1.0 requests, use the -0 option.

Back to TOC

Statically Linking Pure Lua Modules

When LuaJIT 2.x is used, it is possible to statically link the bytecode of pure Lua modules into the Nginx executable.

Basically you use the luajit executable to compile .lua Lua module files to .o object files containing the exported bytecode data, and then link the .o files directly in your Nginx build.

Below is a trivial example to demonstrate this. Consider that we have the following .lua file named foo.lua:


 -- foo.lua
 local _M = {}

 function _M.go()
     print("Hello from foo")
 end

 return _M

And then we compile this .lua file to foo.o file:

/path/to/luajit/bin/luajit -bg foo.lua foo.o

What matters here is the name of the .lua file, which determines how you use this module later on the Lua land. The file name foo.o does not matter at all except the .o file extension (which tells luajit what output format is used). If you want to strip the Lua debug information from the resulting bytecode, you can just specify the -b option above instead of -bg.

Then when building Nginx or OpenResty, pass the --with-ld-opt="foo.o" option to the ./configure script:


 ./configure --with-ld-opt="/path/to/foo.o" ...

Finally, you can just do the following in any Lua code run by ngx_lua:


 local foo = require "foo"
 foo.go()

And this piece of code no longer depends on the external foo.lua file any more because it has already been compiled into the nginx executable.

If you want to use dot in the Lua module name when calling require, as in


 local foo = require "resty.foo"

then you need to rename the foo.lua file to resty_foo.lua before compiling it down to a .o file with the luajit command-line utility.

It is important to use exactly the same version of LuaJIT when compiling .lua files to .o files as building nginx + ngx_lua. This is because the LuaJIT bytecode format may be incompatible between different LuaJIT versions. When the bytecode format is incompatible, you will see a Lua runtime error saying that the Lua module is not found.

When you have multiple .lua files to compile and link, then just specify their .o files at the same time in the value of the --with-ld-opt option. For instance,


 ./configure --with-ld-opt="/path/to/foo.o /path/to/bar.o" ...

If you have just too many .o files, then it might not be feasible to name them all in a single command. In this case, you can build a static library (or archive) for your .o files, as in


 ar rcus libmyluafiles.a *.o

then you can link the myluafiles archive as a whole to your nginx executable:


 ./configure \
     --with-ld-opt="-L/path/to/lib -Wl,--whole-archive -lmyluafiles -Wl,--no-whole-archive"

where /path/to/lib is the path of the directory containing the libmyluafiles.a file. It should be noted that the linker option --whole-archive is required here because otherwise our archive will be skipped because no symbols in our archive are mentioned in the main parts of the nginx executable.

Back to TOC

Data Sharing within an Nginx Worker

To globally share data among all the requests handled by the same nginx worker process, encapsulate the shared data into a Lua module, use the Lua require builtin to import the module, and then manipulate the shared data in Lua. This works because required Lua modules are loaded only once and all coroutines will share the same copy of the module (both its code and data). Note however that Lua global variables (note, not module-level variables) WILL NOT persist between requests because of the one-coroutine-per-request isolation design.

Here is a complete small example:


 -- mydata.lua
 local _M = {}

 local data = {
     dog = 3,
     cat = 4,
     pig = 5,
 }

 function _M.get_age(name)
     return data[name]
 end

 return _M

and then accessing it from nginx.conf:


 location /lua {
     content_by_lua_block {
         local mydata = require "mydata"
         ngx.say(mydata.get_age("dog"))
     }
 }

The mydata module in this example will only be loaded and run on the first request to the location /lua, and all subsequent requests to the same nginx worker process will use the reloaded instance of the module as well as the same copy of the data in it, until a HUP signal is sent to the Nginx master process to force a reload. This data sharing technique is essential for high performance Lua applications based on this module.

Note that this data sharing is on a per-worker basis and not on a per-server basis. That is, when there are multiple nginx worker processes under an Nginx master, data sharing cannot cross the process boundary between these workers.

It is usually recommended to share read-only data this way. You can also share changeable data among all the concurrent requests of each nginx worker process as long as there is no nonblocking I/O operations (including ngx.sleep) in the middle of your calculations. As long as you do not give the control back to the nginx event loop and ngx_lua's light thread scheduler (even implicitly), there can never be any race conditions in between. For this reason, always be very careful when you want to share changeable data on the worker level. Buggy optimizations can easily lead to hard-to-debug race conditions under load.

If server-wide data sharing is required, then use one or more of the following approaches:

  1. Use the ngx.shared.DICT API provided by this module.
  2. Use only a single nginx worker and a single server (this is however not recommended when there is a multi core CPU or multiple CPUs in a single machine).
  3. Use data storage mechanisms such as memcached, redis, MySQL or PostgreSQL. The OpenResty bundle associated with this module comes with a set of companion Nginx modules and Lua libraries that provide interfaces with these data storage mechanisms.

Back to TOC

Known Issues

Back to TOC

TCP socket connect operation issues

The tcpsock:connect method may indicate success despite connection failures such as with Connection Refused errors.

However, later attempts to manipulate the cosocket object will fail and return the actual error status message generated by the failed connect operation.

This issue is due to limitations in the Nginx event model and only appears to affect Mac OS X.

Back to TOC

Lua Coroutine Yielding/Resuming

  • Because Lua's dofile and require builtins are currently implemented as C functions in both Lua 5.1 and LuaJIT 2.0/2.1, if the Lua file being loaded by dofile or require invokes ngx.location.capture*, ngx.exec, ngx.exit, or other API functions requiring yielding in the top-level scope of the Lua file, then the Lua error "attempt to yield across C-call boundary" will be raised. To avoid this, put these calls requiring yielding into your own Lua functions in the Lua file instead of the top-level scope of the file.
  • As the standard Lua 5.1 interpreter's VM is not fully resumable, the methods ngx.location.capture, ngx.location.capture_multi, ngx.redirect, ngx.exec, and ngx.exit cannot be used within the context of a Lua pcall() or xpcall() or even the first line of the for ... in ... statement when the standard Lua 5.1 interpreter is used and the attempt to yield across metamethod/C-call boundary error will be produced. Please use LuaJIT 2.x, which supports a fully resumable VM, to avoid this.

Back to TOC

Lua Variable Scope

Care must be taken when importing modules and this form should be used:


 local xxx = require('xxx')

instead of the old deprecated form:


 require('xxx')

Here is the reason: by design, the global environment has exactly the same lifetime as the Nginx request handler associated with it. Each request handler has its own set of Lua global variables and that is the idea of request isolation. The Lua module is actually loaded by the first Nginx request handler and is cached by the require() built-in in the package.loaded table for later reference, and the module() builtin used by some Lua modules has the side effect of setting a global variable to the loaded module table. But this global variable will be cleared at the end of the request handler, and every subsequent request handler all has its own (clean) global environment. So one will get Lua exception for accessing the nil value.

The use of Lua global variables is a generally inadvisable in the ngx_lua context as:

  1. the misuse of Lua globals has detrimental side effects on concurrent requests when such variables should instead be local in scope,
  2. Lua global variables require Lua table look-ups in the global environment which is computationally expensive, and
  3. some Lua global variable references may include typing errors which make such difficult to debug.

It is therefore highly recommended to always declare such within an appropriate local scope instead.


 -- Avoid
 foo = 123
 -- Recommended
 local foo = 123

 -- Avoid
 function foo() return 123 end
 -- Recommended
 local function foo() return 123 end

To find all instances of Lua global variables in your Lua code, run the lua-releng tool across all .lua source files:

$ lua-releng
Checking use of Lua global variables in file lib/foo/bar.lua ...
        1       [1489]  SETGLOBAL       7 -1    ; contains
        55      [1506]  GETGLOBAL       7 -3    ; setvar
        3       [1545]  GETGLOBAL       3 -4    ; varexpand

The output says that the line 1489 of file lib/foo/bar.lua writes to a global variable named contains, the line 1506 reads from the global variable setvar, and line 1545 reads the global varexpand.

This tool will guarantee that local variables in the Lua module functions are all declared with the local keyword, otherwise a runtime exception will be thrown. It prevents undesirable race conditions while accessing such variables. See Data Sharing within an Nginx Worker for the reasons behind this.

Back to TOC

Locations Configured by Subrequest Directives of Other Modules

The ngx.location.capture and ngx.location.capture_multi directives cannot capture locations that include the add_before_body, add_after_body, auth_request, echo_location, echo_location_async, echo_subrequest, or echo_subrequest_async directives.


 location /foo {
     content_by_lua_block {
         res = ngx.location.capture("/bar")
     }
 }
 location /bar {
     echo_location /blah;
 }
 location /blah {
     echo "Success!";
 }

 $ curl -i http://example.com/foo

will not work as expected.

Back to TOC

Cosockets Not Available Everywhere

Due to internal limitations in the nginx core, the cosocket API is disabled in the following contexts: set_by_lua*, log_by_lua*, header_filter_by_lua*, and body_filter_by_lua.

The cosockets are currently also disabled in the init_by_lua* and init_worker_by_lua* directive contexts but we may add support for these contexts in the future because there is no limitation in the nginx core (or the limitation might be worked around).

There exists a work-around, however, when the original context does not need to wait for the cosocket results. That is, creating a zero-delay timer via the ngx.timer.at API and do the cosocket results in the timer handler, which runs asynchronously as to the original context creating the timer.

Back to TOC

Special Escaping Sequences

NOTE Following the v0.9.17 release, this pitfall can be avoided by using the *_by_lua_block {} configuration directives.

PCRE sequences such as \d, \s, or \w, require special attention because in string literals, the backslash character, \, is stripped out by both the Lua language parser and by the nginx config file parser before processing if not within a *_by_lua_block {} directive. So the following snippet will not work as expected:


 # nginx.conf
 ? location /test {
 ?     content_by_lua '
 ?         local regex = "\d+"  -- THIS IS WRONG OUTSIDE OF A *_by_lua_block DIRECTIVE
 ?         local m = ngx.re.match("hello, 1234", regex)
 ?         if m then ngx.say(m[0]) else ngx.say("not matched!") end
 ?     ';
 ? }
 # evaluates to "not matched!"

To avoid this, double escape the backslash:


 # nginx.conf
 location /test {
     content_by_lua '
         local regex = "\\\\d+"
         local m = ngx.re.match("hello, 1234", regex)
         if m then ngx.say(m[0]) else ngx.say("not matched!") end
     ';
 }
 # evaluates to "1234"

Here, \\\\d+ is stripped down to \\d+ by the Nginx config file parser and this is further stripped down to \d+ by the Lua language parser before running.

Alternatively, the regex pattern can be presented as a long-bracketed Lua string literal by encasing it in "long brackets", [[...]], in which case backslashes have to only be escaped once for the Nginx config file parser.


 # nginx.conf
 location /test {
     content_by_lua '
         local regex = [[\\d+]]
         local m = ngx.re.match("hello, 1234", regex)
         if m then ngx.say(m[0]) else ngx.say("not matched!") end
     ';
 }
 # evaluates to "1234"

Here, [[\\d+]] is stripped down to [[\d+]] by the Nginx config file parser and this is processed correctly.

Note that a longer from of the long bracket, [=[...]=], may be required if the regex pattern contains [...] sequences. The [=[...]=] form may be used as the default form if desired.


 # nginx.conf
 location /test {
     content_by_lua '
         local regex = [=[[0-9]+]=]
         local m = ngx.re.match("hello, 1234", regex)
         if m then ngx.say(m[0]) else ngx.say("not matched!") end
     ';
 }
 # evaluates to "1234"

An alternative approach to escaping PCRE sequences is to ensure that Lua code is placed in external script files and executed using the various *_by_lua_file directives. With this approach, the backslashes are only stripped by the Lua language parser and therefore only need to be escaped once each.


 -- test.lua
 local regex = "\\d+"
 local m = ngx.re.match("hello, 1234", regex)
 if m then ngx.say(m[0]) else ngx.say("not matched!") end
 -- evaluates to "1234"

Within external script files, PCRE sequences presented as long-bracketed Lua string literals do not require modification.


 -- test.lua
 local regex = [[\d+]]
 local m = ngx.re.match("hello, 1234", regex)
 if m then ngx.say(m[0]) else ngx.say("not matched!") end
 -- evaluates to "1234"

As noted earlier, PCRE sequences presented within *_by_lua_block {} directives (available following the v0.9.17 release) do not require modification.


 # nginx.conf
 location /test {
     content_by_lua_block {
         local regex = "\d+"
         local m = ngx.re.match("hello, 1234", regex)
         if m then ngx.say(m[0]) else ngx.say("not matched!") end
     }
 }
 # evaluates to "1234"

Back to TOC

Mixing with SSI Not Supported

Mixing SSI with ngx_lua in the same Nginx request is not supported at all. Just use ngx_lua exclusively. Everything you can do with SSI can be done atop ngx_lua anyway and it can be more efficient when using ngx_lua.

Back to TOC

SPDY Mode Not Fully Supported

Certain Lua APIs provided by ngx_lua do not work in Nginx's SPDY mode yet: ngx.location.capture, ngx.location.capture_multi, and ngx.req.socket.

Back to TOC

Missing data on short circuited requests

Nginx may terminate a request early with (at least):

  • 400 (Bad Request)
  • 405 (Not Allowed)
  • 408 (Request Timeout)
  • 414 (Request URI Too Large)
  • 494 (Request Headers Too Large)
  • 499 (Client Closed Request)
  • 500 (Internal Server Error)
  • 501 (Not Implemented)

This means that phases that normally run are skipped, such as the rewrite or access phase. This also means that later phases that are run regardless, e.g. log_by_lua, will not have access to information that is normally set in those phases.

Back to TOC

TODO

  • cosocket: implement LuaSocket's unconnected UDP API.
  • port this module to the "datagram" subsystem of NGINX for implementing general UDP servers instead of HTTP servers in Lua. For example,

    
    datagram {
     server {
         listen 1953;
         handler_by_lua_block {
             -- custom Lua code implementing the special UDP server...
         }
     }
    }
  • shm: implement a "shared queue API" to complement the existing shared dict API.
  • cosocket: add support in the context of init_by_lua*.
  • cosocket: implement the bind() method for stream-typed cosockets.
  • cosocket: pool-based backend concurrency level control: implement automatic connect queueing when the backend concurrency exceeds its connection pool limit.
  • cosocket: review and merge aviramc's patch for adding the bsdrecv method.
  • add new API function ngx.resp.add_header to emulate the standard add_header config directive.
  • review and apply vadim-pavlov's patch for ngx.location.capture's extra_headers option
  • use ngx_hash_t to optimize the built-in header look-up process for ngx.req.set_header, ngx.header.HEADER, and etc.
  • add configure options for different strategies of handling the cosocket connection exceeding in the pools.
  • add directives to run Lua codes when nginx stops.
  • add ignore_resp_headers, ignore_resp_body, and ignore_resp options to ngx.location.capture and ngx.location.capture_multi methods, to allow micro performance tuning on the user side.
  • add automatic Lua code time slicing support by yielding and resuming the Lua VM actively via Lua's debug hooks.
  • add stat mode similar to mod_lua.
  • cosocket: add client SSL certificate support.

Back to TOC

Changes

The changes made in every release of this module are listed in the change logs of the OpenResty bundle:

http://openresty.org/#Changes

Back to TOC

Test Suite

The following dependencies are required to run the test suite:

The order in which these modules are added during configuration is important because the position of any filter module in the filtering chain determines the final output, for example. The correct adding order is shown above.

  • 3rd-party Lua libraries:

  • Applications:

    • mysql: create database 'ngx_test', grant all privileges to user 'ngx_test', password is 'ngx_test'
    • memcached: listening on the default port, 11211.
    • redis: listening on the default port, 6379.

See also the developer build script for more details on setting up the testing environment.

To run the whole test suite in the default testing mode:

cd /path/to/lua-nginx-module
export PATH=/path/to/your/nginx/sbin:$PATH
prove -I/path/to/test-nginx/lib -r t

To run specific test files:

cd /path/to/lua-nginx-module
export PATH=/path/to/your/nginx/sbin:$PATH
prove -I/path/to/test-nginx/lib t/002-content.t t/003-errors.t

To run a specific test block in a particular test file, add the line --- ONLY to the test block you want to run, and then use the prove utility to run that .t file.

There are also various testing modes based on mockeagain, valgrind, and etc. Refer to the Test::Nginx documentation for more details for various advanced testing modes. See also the test reports for the Nginx test cluster running on Amazon EC2: http://qa.openresty.org.

Back to TOC

Copyright and License

This module is licensed under the BSD license.

Copyright (C) 2009-2016, by Xiaozhe Wang (chaoslawful) [email protected].

Copyright (C) 2009-2017, by Yichun "agentzh" Zhang (章亦春) [email protected], OpenResty Inc.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Back to TOC

See Also

Back to TOC

Directives

The basic building blocks of scripting Nginx with Lua are directives. Directives are used to specify when the user Lua code is run and how the result will be used. Below is a diagram showing the order in which directives are executed.

Lua Nginx Modules Directives

Back to TOC

lua_capture_error_log

syntax: lua_capture_error_log size

default: none

context: http

Enables a buffer of the specified size for capturing all the nginx error log message data (not just those produced by this module or the nginx http subsystem, but everything) without touching files or disks.

You can use units like k and m in the size value, as in


 lua_capture_error_log 100k;

As a rule of thumb, a 4KB buffer can usually hold about 20 typical error log messages. So do the maths!

This buffer never grows. If it is full, new error log messages will replace the oldest ones in the buffer.

The size of the buffer must be bigger than the maximum length of a single error log message (which is 4K in OpenResty and 2K in stock NGINX).

You can read the messages in the buffer on the Lua land via the get_logs() function of the ngx.errlog module of the lua-resty-core library. This Lua API function will return the captured error log messages and also remove these already read from the global capturing buffer, making room for any new error log data. For this reason, the user should not configure this buffer to be too big if the user read the buffered error log data fast enough.

Note that the log level specified in the standard error_log directive does have effect on this capturing facility. It only captures log messages of a level no lower than the specified log level in the error_log directive. The user can still choose to set an even higher filtering log level on the fly via the Lua API function errlog.set_filter_level. So it is more flexible than the static error_log directive.

It is worth noting that there is no way to capture the debugging logs without building OpenResty or NGINX with the ./configure option --with-debug. And enabling debugging logs is strongly discouraged in production builds due to high overhead.

This directive was first introduced in the v0.10.9 release.

Back to TOC

lua_use_default_type

syntax: lua_use_default_type on | off

default: lua_use_default_type on

context: http, server, location, location if

Specifies whether to use the MIME type specified by the default_type directive for the default value of the Content-Type response header. Deactivate this directive if a default Content-Type response header for Lua request handlers is not desired.

This directive is turned on by default.

This directive was first introduced in the v0.9.1 release.

Back to TOC

lua_malloc_trim

syntax: lua_malloc_trim <request-count>

default: lua_malloc_trim 1000

context: http

Asks the underlying libc runtime library to release its cached free memory back to the operating system every N requests processed by the NGINX core. By default, N is 1000. You can configure the request count by using your own numbers. Smaller numbers mean more frequent releases, which may introduce higher CPU time consumption and smaller memory footprint while larger numbers usually lead to less CPU time overhead and relatively larger memory footprint. Just tune the number for your own use cases.

Configuring the argument to 0 essentially turns off the periodical memory trimming altogether.


 lua_malloc_trim 0;  # turn off trimming completely

The current implementation uses an NGINX log phase handler to do the request counting. So the appearance of the log_subrequest on directives in nginx.conf may make the counting faster when subrequests are involved. By default, only "main requests" count.

Note that this directive does not affect the memory allocated by LuaJIT's own allocator based on the mmap system call.

This directive was first introduced in the v0.10.7 release.

Back to TOC

lua_code_cache

syntax: lua_code_cache on | off

default: lua_code_cache on

context: http, server, location, location if

Enables or disables the Lua code cache for Lua code in *_by_lua_file directives (like set_by_lua_file and content_by_lua_file) and Lua modules.

When turning off, every request served by ngx_lua will run in a separate Lua VM instance, starting from the 0.9.3 release. So the Lua files referenced in set_by_lua_file, content_by_lua_file, access_by_lua_file, and etc will not be cached and all Lua modules used will be loaded from scratch. With this in place, developers can adopt an edit-and-refresh approach.

Please note however, that Lua code written inlined within nginx.conf such as those specified by set_by_lua, content_by_lua, access_by_lua, and rewrite_by_lua will not be updated when you edit the inlined Lua code in your nginx.conf file because only the Nginx config file parser can correctly parse the nginx.conf file and the only way is to reload the config file by sending a HUP signal or just to restart Nginx.

Even when the code cache is enabled, Lua files which are loaded by dofile or loadfile in *_by_lua_file cannot be cached (unless you cache the results yourself). Usually you can either use the init_by_lua or init_by_lua_file directives to load all such files or just make these Lua files true Lua modules and load them via require.

The ngx_lua module does not support the stat mode available with the Apache mod_lua module (yet).

Disabling the Lua code cache is strongly discouraged for production use and should only be used during development as it has a significant negative impact on overall performance. For example, the performance of a "hello world" Lua example can drop by an order of magnitude after disabling the Lua code cache.

Back to TOC

lua_regex_cache_max_entries

syntax: lua_regex_cache_max_entries <num>

default: lua_regex_cache_max_entries 1024

context: http

Specifies the maximum number of entries allowed in the worker process level compiled regex cache.

The regular expressions used in ngx.re.match, ngx.re.gmatch, ngx.re.sub, and ngx.re.gsub will be cached within this cache if the regex option o (i.e., compile-once flag) is specified.

The default number of entries allowed is 1024 and when this limit is reached, new regular expressions will not be cached (as if the o option was not specified) and there will be one, and only one, warning in the error.log file:

2011/08/27 23:18:26 [warn] 31997#0: *1 lua exceeding regex cache max entries (1024), ...

If you are using the ngx.re.* implementation of lua-resty-core by loading the resty.core.regex module (or just the resty.core module), then an LRU cache is used for the regex cache being used here.

Do not activate the o option for regular expressions (and/or replace string arguments for ngx.re.sub and ngx.re.gsub) that are generated on the fly and give rise to infinite variations to avoid hitting the specified limit.

Back to TOC

lua_regex_match_limit

syntax: lua_regex_match_limit <num>

default: lua_regex_match_limit 0

context: http

Specifies the "match limit" used by the PCRE library when executing the ngx.re API. To quote the PCRE manpage, "the limit ... has the effect of limiting the amount of backtracking that can take place."

When the limit is hit, the error string "pcre_exec() failed: -8" will be returned by the ngx.re API functions on the Lua land.

When setting the limit to 0, the default "match limit" when compiling the PCRE library is used. And this is the default value of this directive.

This directive was first introduced in the v0.8.5 release.

Back to TOC

lua_package_path

syntax: lua_package_path <lua-style-path-str>

default: The content of LUA_PATH environment variable or Lua's compiled-in defaults.

context: http

Sets the Lua module search path used by scripts specified by set_by_lua, content_by_lua and others. The path string is in standard Lua path form, and ;; can be used to stand for the original search paths.

As from the v0.5.0rc29 release, the special notation $prefix or ${prefix} can be used in the search path string to indicate the path of the server prefix usually determined by the -p PATH command-line option while starting the Nginx server.

Back to TOC

lua_package_cpath

syntax: lua_package_cpath <lua-style-cpath-str>

default: The content of LUA_CPATH environment variable or Lua's compiled-in defaults.

context: http

Sets the Lua C-module search path used by scripts specified by set_by_lua, content_by_lua and others. The cpath string is in standard Lua cpath form, and ;; can be used to stand for the original cpath.

As from the v0.5.0rc29 release, the special notation $prefix or ${prefix} can be used in the search path string to indicate the path of the server prefix usually determined by the -p PATH command-line option while starting the Nginx server.

Back to TOC

init_by_lua

syntax: init_by_lua <lua-script-str>

context: http

phase: loading-config

NOTE Use of this directive is discouraged following the v0.9.17 release. Use the init_by_lua_block directive instead.

Runs the Lua code specified by the argument <lua-script-str> on the global Lua VM level when the Nginx master process (if any) is loading the Nginx config file.

When Nginx receives the HUP signal and starts reloading the config file, the Lua VM will also be re-created and init_by_lua will run again on the new Lua VM. In case that the lua_code_cache directive is turned off (default on), the init_by_lua handler will run upon every request because in this special mode a standalone Lua VM is always created for each request.

Usually you can register (true) Lua global variables or pre-load Lua modules at server start-up by means of this hook. Here is an example for pre-loading Lua modules:


 init_by_lua 'cjson = require "cjson"';

 server {
     location = /api {
         content_by_lua_block {
             ngx.say(cjson.encode({dog = 5, cat = 6}))
         }
     }
 }

You can also initialize the lua_shared_dict shm storage at this phase. Here is an example for this:


 lua_shared_dict dogs 1m;

 init_by_lua '
     local dogs = ngx.shared.dogs;
     dogs:set("Tom", 56)
 ';

 server {
     location = /api {
         content_by_lua_block {
             local dogs = ngx.shared.dogs;
             ngx.say(dogs:get("Tom"))
         }
     }
 }

But note that, the lua_shared_dict's shm storage will not be cleared through a config reload (via the HUP signal, for example). So if you do not want to re-initialize the shm storage in your init_by_lua code in this case, then you just need to set a custom flag in the shm storage and always check the flag in your init_by_lua code.

Because the Lua code in this context runs before Nginx forks its worker processes (if any), data or code loaded here will enjoy the Copy-on-write (COW) feature provided by many operating systems among all the worker processes, thus saving a lot of memory.

Do not initialize your own Lua global variables in this context because use of Lua global variables have performance penalties and can lead to global namespace pollution (see the Lua Variable Scope section for more details). The recommended way is to use proper Lua module files (but do not use the standard Lua function module() to define Lua modules because it pollutes the global namespace as well) and call require() to load your own module files in init_by_lua or other contexts (require() does cache the loaded Lua modules in the global package.loaded table in the Lua registry so your modules will only loaded once for the whole Lua VM instance).

Only a small set of the Nginx API for Lua is supported in this context:

More Nginx APIs for Lua may be supported in this context upon future user requests.

Basically you can safely use Lua libraries that do blocking I/O in this very context because blocking the master process during server start-up is completely okay. Even the Nginx core does blocking I/O (at least on resolving upstream's host names) at the configure-loading phase.

You should be very careful about potential security vulnerabilities in your Lua code registered in this context because the Nginx master process is often run under the root account.

This directive was first introduced in the v0.5.5 release.

Back to TOC

init_by_lua_block

syntax: init_by_lua_block { lua-script }

context: http

phase: loading-config

Similar to the init_by_lua directive except that this directive inlines the Lua source directly inside a pair of curly braces ({}) instead of in an NGINX string literal (which requires special character escaping).

For instance,


 init_by_lua_block {
     print("I need no extra escaping here, for example: \r\nblah")
 }

This directive was first introduced in the v0.9.17 release.

Back to TOC

init_by_lua_file

syntax: init_by_lua_file <path-to-lua-script-file>

context: http

phase: loading-config

Equivalent to init_by_lua, except that the file specified by <path-to-lua-script-file>

Related Repositories

nginx-lua-module-zh-wiki

nginx-lua-module-zh-wiki

https://github.com/openresty/lua-nginx-module chinese version ...

stream-lua-nginx-module

stream-lua-nginx-module

Embed the power of Lua into NGINX TCP servers ...

lua-resty-core

lua-resty-core

New FFI-based API for lua-nginx-module ...

lua-nginx-module-digest

lua-nginx-module-digest

lua-nginx-module source code learning, parsing and summary ...

stream-lua-nginx-module

stream-lua-nginx-module

Embed the power of Lua into NGINX TCP servers ...


Top Contributors

agentzh chaoslawful shrimp jinglong zhuzhaoyuan liseen doujiang24 bakins PiotrSikora cubicdaiya chipitsine ghedo thibaultcha kindy maage dirkfeytons pintsized cuiweixie hamishforbes ctrochalakis mizzy typhonius jdesgats pushrax lution client9 GUI ottok monadbobo tatsuyafw

Releases

-   v0.10.6 zip tar
-   v0.10.6rc2 zip tar
-   v0.10.6rc1 zip tar
-   v0.10.5 zip tar
-   v0.10.4 zip tar
-   v0.10.4rc1 zip tar
-   v0.10.3 zip tar
-   v0.10.2 zip tar
-   v0.10.1 zip tar
-   v0.10.1rc1 zip tar
-   v0.10.1rc0 zip tar
-   v0.10.0 zip tar
-   v0.10.0rc0 zip tar
-   v0.9.20 zip tar
-   v0.9.20rc3 zip tar
-   v0.9.20rc2 zip tar
-   v0.9.20rc1 zip tar
-   v0.9.19 zip tar
-   v0.9.18 zip tar
-   v0.9.18rc1 zip tar
-   v0.9.17 zip tar
-   v0.9.17rc1 zip tar
-   v0.9.16 zip tar
-   v0.9.16rc3 zip tar
-   v0.9.16rc2 zip tar
-   v0.9.16rc1 zip tar
-   v0.9.15 zip tar
-   v0.9.14 zip tar
-   v0.9.13 zip tar
-   v0.9.13rc1 zip tar