Varnish and WordPress

The following is a guest post by Austin Gunter of WPEngine.com.

If you’re looking to boost performance for a content-heavy WordPress installation, adding a cache like Varnish is a great way to boost your site’s performance.

NB – This is an advanced topic, and only relevant if you have full control over your server (e.g. you’re on a VPS). It does not apply to regular webhosting.

What is Varnish?

Varnish Cache is a web app accelerator, or a caching HTTP reverse proxy. Install it in front of any server that speaks HTTP and configure it to cache the contents, and the Varnish community claims that it speeds up delivery by a factor of 300 – 1000x. 

In a nutshell, caching a webpage means storing a copy of that website’s content for future visitors to see. Varnish can cache pages of your WordPress site so that your server doesn’t need to call the database each time a visitor visits your site. This reduces server load because the stored copy means the webserver doesn’t have to go find the same images and content for each visitor.

Varnish caches page data in virtual memory, so your site will load much faster, providing a killer SEO boost. Google found that every additional 0.5s load time meant 20% fewer site visitors (Source). Reducing the page load time can dramatically increase your visitors and boost your search rankings at the same time.

Varnish themselves have put together a great video for explaining what it does, as simply as possible:

Installing Varnish

Varnish is free software that you can install this afternoon. It runs on Linux, and primarily FreeBSD, but can work on other platforms as well. Once installed, you can customize how incoming requests will be handled with the Varnish Configuration Language (VCL).

This flexibility means that each Varnish install can and should be designed with a particular site in mind.

You’ll want to begin with a basic configuration of Varnish, and then begin testing small changes as you get the hang of how it works with your particular site, and how the functions work. There are several different subroutines that tell Varnish how to respond to requests going in and out, to errors, and so on.

I’ll start with a basic set up, and then cover basic VCL functions that you’ll be tweaking as you go.

Step by Step

Setting up Varnish is pretty simple. I’m going to assume you’re using Apache on a Debian-based system. It does work on other systems too though.

Start in the command line with this:

apt-get install varnish

First, you want to configure Apache to listen on port 8080 of the localhost interface. Varnish can then listen on port 80 (Where your visitors connect). In /etc/apache2/ports.conf, edit the following settings:

NameVirtualHost 127.0.0.1:8080
Listen 127.0.0.1:8080

To get Varnish to start (it won’t by default), edit the following in /etc/default/varnish

START=yes
DAEMON_OPTS=”-a EXTERNAL_IP_ADDRESS:80 \
	-T localhost:6082 \
	-f /etc/varnish/default.vcl \
	-S /etc/varnish/secret \
	-s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G”

Replace EXTERNAL_IP_ADDRESS with the IP of your external IP address. It can also be an internal address if your server is setup behind a load balancer, or something like NGINX. This setting controls what IP address and port you want Varnish to bind to and listen on.

Now edit /etc/varnish/default.vcl, which should already exist with lots of it commented out. Start by changing the default backend.

backend default {
	.host = “127.0.0.1”;</p>
	.port = “8080”;</p>
}

Now Varnish knows that Apache is listening on port 8080 of the localhost interface, and you can start using the functions. The majority of the work will be with vcl_recv and vcl_fetch, and if you don’t call an action in this subroutine and Varnish reaches the end, it will execute the built-in code from default.vcl.

Note: you never want to cache wp_admin, wp_login, or similar pages.

Here’s how it works – the 4 basic subroutines in your Varnish config which you need to handle requests are:

sub vcl_recv

This is called at the beginning of a request and tells varnish what to do with that particular request: whether or not to serve it, how to serve it, and which backend to use.

Varnish receives a request from your browser, vcl_recv and chooses to do one of 3 things with it: vcl_hash, vcl_pass, and vcl_pipe (More details on those in a minute!). You can change the request if you like, altering cookies or removing the request header.

sub vcl_fetch

vcl_fetch is called after a document has been successfully retrieved from the backend. Use this to alter the response headers, trigger ESI processing, or try alternate backend servers if the request failed.

The request object, req, is still available, and there is also a backend response, beresp, which contains the HTTP headers from the backend.

sub vcl_hash

You can call hash_data on the data you want to add to the hash. This subroutine may terminate with calling return() with one of the keywords, hash or proceed.

sub vcl_deliver

Call this before a cached object is delivered to the client. This may terminate with deliver, error code, or restart. Deliver, delivers the object to the client. Error will return the specified error code to the client and abandon the request. Restart will restart the transaction and increase the restart counter. If the number of restarts exceed the max_restarts, varnish emits a guru meditation error.

Actions

There are a number of actions you can take inside each subroutine as you customize Varnish:

pass

Pass the request and subsequent response to and from the backend server, not cached. pass can be called in both vcl_recv and vcl_fetch.

lookup

Called from vcl_recv to deliver content from cache even if the request indicates that the request should be passed. You can’t call lookup from vcl_fetch.

pipe

From vcl_recv, pipe short-circuits client and backend connections and Varnish will just sits there passing the data back and forth, logging the data, so logs will be incomplete. Beware that with HTTP 1.1 a client can send several requests on the same connection and so you should instruct Varnish to add a “Connection: close” header before actually calling pipe.

deliver

Deliver the cached object to the client. Usually called in vcl_fetch.

esi

ESI-process the fetched document.

The Varnish community has a very detailed tutorial on using VCL, and the functions that you can perform on your site, which I won’t go into here.

Example Configurations

Hopefully this post has given you a good introduction to Varnish, but the best way to truly start playing with it is to see some sample configuration files.

The Varnish website has a great collection sample configurations, which can make a perfect starting point for building your own.

More specifically, Mattias Geniar has put up sample fetch and receive configs for WordPress on Github.

Suffice to say that Varnish is very customizable and can do wonders for WordPress. A great example would be a Membership site. You can customize Varnish to cache content based on a user’s permissions, on their membership level, or even if they’re just visiting your site.

Facebook uses Varnish to great avail, and no doubt they have customized it to hell and back, but it goes to show you how efficient and flexible Varnish is.

Plugins

Update (13 Apr 2012): Thanks to Alexander and Pothi for letting us know about an updated Varnish plugin for WP 3.0+ and one on WP.org also for purging your cache (Also, check out both of these guy’s blogs for more Varnish and WP performance tips!)

The varnish plugin will purge the cache when content is added or edited, and it works for multisite. The plugin does include a sample config file for your Varnish backend, however, keep in mind that when you’ve customized it for one particular site, those customizations may or may not play nicely with a different site.

Have you had any luck installing Varnish for your WordPress? What works and doesn’t? Feel free to share some of your ideas in the comments!

Update (24 Jul 2012): Jason Ormand has put together a cool Bash script to make configuring an NGINX server with Varnish and WordPress even faster.

Enjoy this post? You should follow me on Twitter!