wiki.nth.ro

We want to use Nginx's Lua scripting capabilities, but this module is neither distributed with the source code, nor available in the official binary package. Because Nginx doesn't support run-time module loading, we have to build our own version.

To make the matter more interesting, we need to use SHA256 inside our Lua scripts, but the Nginx module provides only SHA1, so we need to use a 3rd party Lua module. For performance reasons (and since everything else is already built as such), we want both the Lua interpreter and this module statically linked into our binary.

• The following steps apply to CentOS and should be followed on a build machine with wget, tar, unzip, patch and rpmbuild commands available.
• LuaJIT, a just-in-time compiler for Lua that is a lot faster than the reference interpreter
• LuaCrypto, a Lua frontend to OpenSSL, to enable SHA256 support
• Nginx development kit, needed by ngx_lua
• Nginx Lua module
• Nginx source package

We start by downloading all the required packages:

wget http://luajit.org/download/LuaJIT-2.0.4.tar.gz
wget -O luacrypto-master.zip https://github.com/mkottman/luacrypto/archive/master.zip
wget -O ngx_devel_kit-0.2.19.tar.gz https://github.com/simpl/ngx_devel_kit/archive/v0.2.19.tar.gz
wget -O lua-nginx-module-0.9.16.tar.gz https://github.com/openresty/lua-nginx-module/archive/v0.9.16.tar.gz
wget http://nginx.org/packages/centos/6/SRPMS/nginx-1.8.0-1.el6.ngx.src.rpm

These were the latest versions available at the time of writing, and please note that we went for the LuaCrypto master branch instead of the last release, which is more than two years old.

We then extract the two Lua archives and copy the LuaCrypto code into the LuaJIT source:

unzip luacrypto-master.zip
tar xzvf LuaJIT-2.0.4.tar.gz
cp luacrypto-master/src/lcrypto.* LuaJIT-2.0.4/src/

We do this because we want the LuaCrypto built into the LuaJIT interpreter, which itself will in turn be statically linked into Nginx. But copying the files isn't enough. We need to let LuaJIT know this module is there. In order to do so, we apply the following patch to the interpreter:

luajit.diff

diff -rupN src/lib_init.c src.new/lib_init.c
--- src/lib_init.c	2015-05-14 18:30:00.000000000 +0000
+++ src.new/lib_init.c	2015-09-03 09:09:54.247183905 +0000
@@ -25,6 +25,7 @@ static const luaL_Reg lj_lib_load[] = {
   { LUA_MATHLIBNAME,	luaopen_math },
   { LUA_DBLIBNAME,	luaopen_debug },
   { LUA_BITLIBNAME,	luaopen_bit },
+  { LUACRYPTO_CORENAME,	luaopen_crypto }
   { LUA_JITLIBNAME,	luaopen_jit },
   { NULL,		NULL }
 };
diff -rupN src/ljamalg.c src.new/ljamalg.c
--- src/ljamalg.c	2015-05-14 18:30:00.000000000 +0000
+++ src.new/ljamalg.c	2015-09-03 09:08:23.183309866 +0000
@@ -89,5 +89,6 @@
 #include "lib_bit.c"
 #include "lib_jit.c"
 #include "lib_ffi.c"
+#include "lcrypto.c"
 #include "lib_init.c"
 

by running:

cd LuaJIT-2.0.4
patch < luajit.diff

The LuaJIT amalgamation is a single code file, including all source files, which enables the compiler to perform additional optimizations on the code, as it it contained in a single unit. The above patch adds LuaCrypto to the amalgamation and registers it into the environment upon loading. To then build LuaJIT, we run:

make amalg PREFIX=/usr CFLAGS='-pipe -O2' LDFLAGS='-lssl -lcrypto'

where amalg is the amalgamation target, PREFIX is the installation path (default is /usr/local, but CentOS uses /usr) and CFLAGS specifies some sane compilation flags. LDFLAGS instructs the linker to add OpenSSL libraries to the list of files to be linked. These are needed by the LuaCrypto module.

When compilation is done we install LuaJIT to a temporary location, but discard the shared libraries, as we're not interested in them (by default the linker will first search for shared objects and then for archives):

make install PREFIX=/usr DESTDIR=/tmp/buildroot
rm -rf /tmp/buildroot/usr/lib/libluajit-5.1.so*

We can now proceed to the next step, preparing to build Nginx itself and package it as a rpm.

We start by decompressing the remaining archives:

tar xzvf ngx_devel_kit-0.2.19.tar.gz
tar xzvf lua-nginx-module-0.9.16.tar.gz
rpm -i nginx-1.8.0-1.el6.ngx.src.rpm

The NDK and ngx_lua paths will be required later, while the source RPM will be extracted in the rpmbuild folder of the home directory. We'll cd into it and make a copy of the spec file, for editing:

cd ~/rpmbuild/SPECS/
cp nginx.spec nginx-lua.spec

We do this mainly because we want to rename the package from nginx to something else, to avoid confusion and being overwritten by a newer version of the vanilla package. The following patch renames the package to nginx-lua and adds the two required modules. You need to modify it (or the result file), pointing the add-module directives to the proper paths of the Nginx developer kit and Lua module we just extracted:

nginx-lua.spec.diff

--- nginx-lua.spec.orig	2015-04-21 15:34:33.000000000 +0000
+++ nginx-lua.spec	2015-09-03 09:29:25.648410139 +0000
@@ -56,14 +56,16 @@
 
 # end of distribution specific definitions
 
+%define cname nginx
+
 Summary: High performance web server
-Name: nginx
+Name: %{cname}-lua
 Version: 1.8.0
-Release: 1%{?dist}.ngx
+Release: 1%{?dist}.rdemo
 Vendor: nginx inc.
 URL: http://nginx.org/
 
-Source0: http://nginx.org/download/%{name}-%{version}.tar.gz
+Source0: http://nginx.org/download/%{cname}-%{version}.tar.gz
 Source1: logrotate
 Source2: nginx.init
 Source3: nginx.sysconf
@@ -77,7 +79,7 @@
 
 License: 2-clause BSD-like license
 
-BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
+BuildRoot: %{_tmppath}/%{cname}-%{version}-%{release}-root
 BuildRequires: zlib-devel
 BuildRequires: pcre-devel
 
@@ -99,7 +101,7 @@
 %endif
 
 %prep
-%setup -q
+%setup -q -n %{cname}-%{version}
 
 %build
 ./configure \
@@ -134,13 +136,15 @@
         --with-mail_ssl_module \
         --with-file-aio \
         --with-ipv6 \
+        --add-module=/ngx_devel_kit-0.2.19 \
+        --add-module=/lua-nginx-module-0.9.16 \
         --with-debug \
         %{?with_spdy:--with-http_spdy_module} \
         --with-cc-opt="%{optflags} $(pcre-config --cflags)" \
         $*
 make %{?_smp_mflags}
-%{__mv} %{_builddir}/%{name}-%{version}/objs/nginx \
-        %{_builddir}/%{name}-%{version}/objs/nginx.debug
+%{__mv} %{_builddir}/%{cname}-%{version}/objs/nginx \
+        %{_builddir}/%{cname}-%{version}/objs/nginx.debug
 ./configure \
         --prefix=%{_sysconfdir}/nginx \
         --sbin-path=%{_sbindir}/nginx \
@@ -173,6 +177,8 @@
         --with-mail_ssl_module \
         --with-file-aio \
         --with-ipv6 \
+        --add-module=/ngx_devel_kit-0.2.19 \
+        --add-module=/lua-nginx-module-0.9.16 \
         %{?with_spdy:--with-http_spdy_module} \
         --with-cc-opt="%{optflags} $(pcre-config --cflags)" \
         $*
@@ -235,7 +241,7 @@
    $RPM_BUILD_ROOT%{_sysconfdir}/logrotate.d/nginx
 %endif
 
-%{__install} -m644 %{_builddir}/%{name}-%{version}/objs/nginx.debug \
+%{__install} -m644 %{_builddir}/%{cname}-%{version}/objs/nginx.debug \
    $RPM_BUILD_ROOT%{_sbindir}/nginx.debug
 
 %clean

Because we changed the package name, but everything else remained the same, we needed to keep the original name in a different variable and use it everywhere in the spec file, instead of the package name. A notable use is in the %prep stage setup, as the source archive will extract to a folder corresponding to the original name.

We apply the patch:

patch < nginx-lua.spec.diff

and are now finally ready to build Nginx itself:

CFLAGS='-pipe -O2 -lssl -lcrypto' LUAJIT_LIB='/tmp/buildroot/usr/lib/' LUAJIT_INC='/tmp/buildroot/usr/include/' rpmbuild -ba nginx-lua.spec

As before, we pass the compiler safe defaults and give the linker instructions to use the OpenSSL libraries when constructing the end result, but this time in a single FLAGS variable.

We also provide the location of the LuaJIT library previously built. We do this by setting LUAJIT_LIB to the folder where the library itself is located and LUAJIT_INC to the folder containing LuaJIT's header files.

When rpmbuild finishes, our new packages should be ready for use:

ls -al rpmbuild/RPMS/x86_64/nginx-lua-*
 
yum localinstall nginx-lua-1.8.0-1.el6.rdemo.x86_64.rpm 

To test out the new functionality, we modify the server block to call a Lua script on any request:

default.conf

server {
    listen       80;
    server_name  localhost;
 
    location / {
        default_type 'text/plain';
 
        content_by_lua_file lua/reqhash.lua;
    }
}

Our test file is called reqhash.lua and is located in the lua folder of the server path (default location is /etc/nginx). When the first request comes in, Nginx will load the script from disk and pass it to LuaJIT, that will compile parts of it to native code as they are invoked. Every subsequent request will run the cached machine code.

The code is pretty straightforward:

reqhash.lua

local crypto = require('crypto')
 
local print = ngx.say
local hmacdigest = crypto.hmac.digest
local base64 = ngx.encode_base64
local headers = ngx.req.get_headers()
 
ngx.header.content_type = 'text/plain';
for k, v in pairs(headers) do
    print(string.format("%s\t%s: %s", base64(hmacdigest('sha256', v, k, true)), tostring(k), tostring(v)))
end

It iterates over all the request headers and for each of them outputs the header's name, it's value, and a SHA256 digest of the latter using the former as a key. You should see something similar to:

s1NWjQzxUCKGu9DVIDhFSJoO3GMR3I0JwTdfuyYX3eU=	accept-language: en-US,en;q=0.5
R+MVTzKo4XPghzDOhgzktAHGXYP/+c0TECzYTSsnUnQ=	cache-control: no-cache
JZ67jT4dL9yytOhKQajC/bU3T/FiSzBX7Ozf/65/pWI=	connection: keep-alive
9VP7chF8gtYxEY4Qk66ILRCqnhoX/9O5OQ4ladc3lmY=	accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
XDUFYfDkl5KlTKX7MaFArPRmTZp4I9xNgjdABzeFhYc=	pragma: no-cache
bz2AY1CDnVT7o+Ncjsp955XQ/UEc1/Bs0Fyu8Ab9WME=	accept-encoding: gzip, deflate

You are now ready to start taking advantage of Lua scripting in Nginx.

As we built Nginx by hand, we are now stuck on this version unless another manual run is done. This introduces some overhead when upgrading, either taking up human resources, by having someone manually updating the package whenever necessary, or computing resouces, by automating the build task.