Outdated files being served by nginx

by on
  • linux
  • nginx
  • technical

While updating one of my websites I noticed that a CSS file wasn’t being updated.

As I’m using Cloudflare to cache my website my first guess was that the cache on their side wasn’t being cleared. Pushed the clear cache button multiple times without any effect. It’s clearly broken! A direct request to my webserver cleared any blame on Cloudflare though: my server was replying with an old version of my CSS file.

Then it must be rsync! I’m using rsync to push the new versions of my website from my development machine to the server. A few retries later the funny looking webpages suggested the CSS was still wrong. Checking the file on my server confirmed that rsync had actually replaced the CSS, retrying the upload was useless.

Why was nginx still serving the old CSS? Turns out it was my configuration.

Sendfile

As recommended by most optimization guides my nginx configuration allows using sendfile. This option is especially useful for static websites like mine, as every single page can be served right from the filesystem.

http {
  sendfile on;
  tcp_nopush on;
}

But what does it do exactly?

The nginx documentation explains the sendfile option as:

Enables or disables the use of sendfile().

That’s exactly what I would expect from a boolean option! Sendfile is actually a Linux system call executed by nginx, for more information we have to resort to the man pages.

sendfile() copies data between one file descriptor and another. Because this copying is done within the kernel, sendfile() is more efficient than the combination of read(2) and write(2), which would require transferring data to and from user space.

So basically nginx just asks the kernel to copy the contents of a file to the connection with the visitor’s browser. This makes serving static files a lot faster, as it eliminates the step where nginx copies the file into its own buffers. Yay!

Open file cache

Another optimization I’ve been using is the open_file_cache options:

http {
  open_file_cache max=1000 inactive=20s;
  open_file_cache_errors on;
  open_file_cache_min_uses 2;
  open_file_cache_valid 30s;
}

When we turn to the nginx documentation again, what we should know about the open file cache is that it contains information about files being served from the filesystem:

This cache is quite useful as the modification time allows nginx to reply to If-Modified-Since requests without actually opening the file. Another situation that happens quite frequently is a non-existing file being requested multiple times. The best known example is the favicon.ico that most browsers request whenever you visit a website. As nginx already knows that the file is not actually there, it can simply reply with a 404 response right away.

Great, huh?

Remember how sendfile copies data from one file descriptor to another? If the file on disk was replaced (not just modified) then this causes nginx to keep sending data from the old file instead of the new one. This can be very confusing if you also have a web cache (Varnish, or a service like Cloudflare for example) as you’ll be blaming the cache first.

Looks like there is no way to clear the open file cache. As I don’t want to restart nginx every time I update my website I have modified my configuration to no longer cache files for 30s.

Rsync

As an extra: rsync is the reason why my files were replaced.

After the temp-file has been completed, its ownership and permissions and modification time are set. It is then renamed to replace the basis file.

This means the existing file is not modified. It is actually replaced by a new file, to make sure we don’t end up with only half of the contents in case transfer fails.