At CloudFlare, Nginx is at the core of what we do. It is part of the
underlying foundation of our reverse proxy service. In addition to the
built-in Nginx functionalities, we use an array of custom C modules that
are specific to our infrastructure including load balancing, monitoring,
and caching. Recently, we’ve been adding more simple services. And they
are almost exclusively written in Lua.

I wanted to share more about how we are augmenting Nginx with new
capabilities using Lua and provide some examples so you can do the
same.

What is Lua?

Lua is a scripting language. Specifically, it is a full-featured
multi-paradigm language with a simple syntax and semantics that resemble
JavaScript or Scheme. Lua also has an interesting story to it, as it is
one of the only languages from an emerging country that has had
worldwide impact.

Lua has always meant to be embedded with larger systems written in other
languages (like C and C++), and has thrived at staying very minimal and
easy to integrate. As a result, Lua is popular within video
games, security oriented software, and,
more
recently, Wikipedia.

Benefits of Nginx+Lua

Nginx+Lua is a self-contained web server embedding the scripting
language Lua. Powerful applications can be written directly inside Nginx
without using cgi, fastcgi, or uwsgi. By adding a little Lua code to an
existing Nginx configuration file, it is easy to add small features. To
see it yourself, at the end of this post I’ve included some logging code
that can be added to any existing configuration.

One of the core benefits of Nginx+Lua is that it is fully
asynchronous. Nginx+Lua inherits the same event loop model that has
made Nginx a popular choice of webserver. “Asynchronous” simply means
that Nginx can interrupt your code when it is waiting on a blocking
operation, such as an outgoing connection or reading a file, and run the
code of another incoming HTTP Request.

All the Lua code is written in a sequential fashion. The asynchronous
logic is hidden to the Nginx+Lua programmer. If you are familiar with
other event-driven webservers, that means no callbacks. In addition,
Nginx+Lua is blazingly fast,
leveraging the LuaJIT interpreter.

Getting Nginx+Lua installed

You can install it from source by compiling the lua-nginx-module with
your existing Nginx. If you chose that path you will also need a Lua
interpreter. LuaJIT-2.0.0 is
recommended.

Or, you can use the
tested ngx_openresty bundle.
ngx_openresty comes loaded with Nginx, 3rd party modules, Lua
libraries and other goodies. If you
already use Nginx without 3rd party modules, from your Linux
distribution for instance, you can safely swap it out with
ngx_openresty. (Quick shout-out to my CloudFlare colleague Yichun Zhang
who wrote ngx_openresty. Thanks, Yichun!)

Limitations

What makes Nginx, and therefore Nginx+Lua, really fast is the
asychronous model and the event loop that Nginx relies on. To stay
within that model, outgoing communication that is outside of Nginx has
to be treated carefully. It is not recommended that you use classic
LuaSocket, and instead it is recommended that you rely on the
built-in ngx_lua
sockets.

However, with a multitude of openresty libraries to
“speak” SQL, memcached,
and Redis, as well as
the DNS built on top of
ngx_lua sockets, this isn’t really a problem in practice.

An example to try: Nginx Log Aggregation

Here is an example of how
to build and run a simple log aggregator for Nginx. You can add it to
any of your own existing configuration. This is the output once the
aggregated logs are funneled to a time series system:

This particular example graph shows the average number of requests per
second on certain nodes of the CloudFlare infrastructure.

Show me the code already!

Let’s assume you already have a working webapp in Nginx, or that you use
the proxy_pass directives to upstream to an Apache server.

First, add some lines in the Nginx conf to look at .lua files, and use a
1MB space of shared memory between your Nginx workers. ($prefix is
relative to your Nginx install).

Next, add a little Lua snippet to calculate request_time for each
request, and aggregate it into shared memory using a logging library
available. Here is a simple logging library that I
built.

This snippet can be used directly inline in your Nginx conf, using
the log_by_lua directive.

Displaying/collecting aggregated logs

The last step to complete the example is a system to collect and/or
display logs. In the full example,
we set the aggregation as a separate server listening on a different
port.

You should now have a functioning Nginx+Lua modification running in your
environment.

Using Lua instead of custom C modules

This example showed how Lua found its way into our system at CloudFlare,
but we soon realized that it wasn’t limited to aggregating and printing
logs. Using the same phases that Nginx has laid out
for processing HTTP requests, it is becoming possible to add interesting
new capabilities to Nginx, with almost as much control as a custom C
module, while being pleasant and easy to write.

For instance,
the access phase
can be seen as a programmatic .htaccess, and
even more. Whereas
the content phase
is where your web application would go.

Nginx+Lua has become a foundation for the work that I do at CloudFlare.
As a long-time C developer, I am constantly struck by how powerful and
extremely expressive Lua can be, while being simple and approachable as
well.

Sometimes, simple is beautiful.

PS – We’re hiring Lua programmers who are interested in working at
extreme scale. Check out the Systems Engineer listing on our careers
page if you’re interested.

Via Cloudflare.com