How to Install and Customize Varnish for WordPress
32The following is a guest post by Austin Gunter of WPEngine.com.
If you’re looking to boost performance for a content-heavy WordPress installation, adding a cache like Varnish is a great way to boost your site’s performance.
NB – This is an advanced topic, and only relevant if you have full control over your server (e.g. you’re on a VPS). It does not apply to regular webhosting.
What is Varnish?
Varnish Cache is a web app accelerator, or a caching HTTP reverse proxy. Install it in front of any server that speaks HTTP and configure it to cache the contents, and the Varnish community claims that it speeds up delivery by a factor of 300 – 1000x.
In a nutshell, caching a webpage means storing a copy of that website’s content for future visitors to see. Varnish can cache pages of your WordPress site so that your server doesn’t need to call the database each time a visitor visits your site. This reduces server load because the stored copy means the webserver doesn’t have to go find the same images and content for each visitor.
Varnish caches page data in virtual memory, so your site will load much faster, providing a killer SEO boost. Google found that every additional 0.5s load time meant 20% fewer site visitors (Source). Reducing the page load time can dramatically increase your visitors and boost your search rankings at the same time.
Varnish themselves have put together a great video for explaining what it does, as simply as possible:
Installing Varnish
Varnish is free software that you can install this afternoon. It runs on Linux, and primarily FreeBSD, but can work on other platforms as well. Once installed, you can customize how incoming requests will be handled with the Varnish Configuration Language (VCL).
This flexibility means that each Varnish install can and should be designed with a particular site in mind.
You’ll want to begin with a basic configuration of Varnish, and then begin testing small changes as you get the hang of how it works with your particular site, and how the functions work. There are several different subroutines that tell Varnish how to respond to requests going in and out, to errors, and so on.
I’ll start with a basic set up, and then cover basic VCL functions that you’ll be tweaking as you go.
Step by Step
Setting up Varnish is pretty simple. I’m going to assume you’re using Apache on a Debian-based system. It does work on other systems too though.
Start in the command line with this:
apt-get install varnish |
First, you want to configure Apache to listen on port 8080 of the localhost interface. Varnish can then listen on port 80 (Where your visitors connect). In /etc/apache2/ports.conf, edit the following settings:
NameVirtualHost 127.0.0.1:8080 Listen 127.0.0.1:8080 |
To get Varnish to start (it won’t by default), edit the following in /etc/default/varnish
START=yes DAEMON_OPTS=”-a EXTERNAL_IP_ADDRESS:80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G” |
Replace EXTERNAL_IP_ADDRESS with the IP of your external IP address. It can also be an internal address if your server is setup behind a load balancer, or something like NGINX. This setting controls what IP address and port you want Varnish to bind to and listen on.
Now edit /etc/varnish/default.vcl, which should already exist with lots of it commented out. Start by changing the default backend.
backend default { .host = “127.0.0.1”;</p> .port = “8080”;</p> } |
Now Varnish knows that Apache is listening on port 8080 of the localhost interface, and you can start using the functions. The majority of the work will be with vcl_recv and vcl_fetch, and if you don’t call an action in this subroutine and Varnish reaches the end, it will execute the built-in code from default.vcl.
Note: you never want to cache wp_admin, wp_login, or similar pages.
Here’s how it works – the 4 basic subroutines in your Varnish config which you need to handle requests are:
sub vcl_recv |
This is called at the beginning of a request and tells varnish what to do with that particular request: whether or not to serve it, how to serve it, and which backend to use.
Varnish receives a request from your browser, vcl_recv and chooses to do one of 3 things with it: vcl_hash, vcl_pass, and vcl_pipe (More details on those in a minute!). You can change the request if you like, altering cookies or removing the request header.
sub vcl_fetch |
vcl_fetch is called after a document has been successfully retrieved from the backend. Use this to alter the response headers, trigger ESI processing, or try alternate backend servers if the request failed.
The request object, req, is still available, and there is also a backend response, beresp, which contains the HTTP headers from the backend.
sub vcl_hash |
You can call hash_data on the data you want to add to the hash. This subroutine may terminate with calling return() with one of the keywords, hash or proceed.
sub vcl_deliver |
Call this before a cached object is delivered to the client. This may terminate with deliver, error code, or restart. Deliver, delivers the object to the client. Error will return the specified error code to the client and abandon the request. Restart will restart the transaction and increase the restart counter. If the number of restarts exceed the max_restarts, varnish emits a guru meditation error.
Actions
There are a number of actions you can take inside each subroutine as you customize Varnish:
pass
Pass the request and subsequent response to and from the backend server, not cached. pass can be called in both vcl_recv and vcl_fetch.
lookup
Called from vcl_recv to deliver content from cache even if the request indicates that the request should be passed. You can’t call lookup from vcl_fetch.
pipe
From vcl_recv, pipe short-circuits client and backend connections and Varnish will just sits there passing the data back and forth, logging the data, so logs will be incomplete. Beware that with HTTP 1.1 a client can send several requests on the same connection and so you should instruct Varnish to add a “Connection: close” header before actually calling pipe.
deliver
Deliver the cached object to the client. Usually called in vcl_fetch.
esi
ESI-process the fetched document.
The Varnish community has a very detailed tutorial on using VCL, and the functions that you can perform on your site, which I won’t go into here.
Example Configurations
Hopefully this post has given you a good introduction to Varnish, but the best way to truly start playing with it is to see some sample configuration files.
The Varnish website has a great collection sample configurations, which can make a perfect starting point for building your own.
More specifically, Mattias Geniar has put up sample fetch and receive configs for WordPress on Github.
Suffice to say that Varnish is very customizable and can do wonders for WordPress. A great example would be a Membership site. You can customize Varnish to cache content based on a user’s permissions, on their membership level, or even if they’re just visiting your site.
Facebook uses Varnish to great avail, and no doubt they have customized it to hell and back, but it goes to show you how efficient and flexible Varnish is.
Plugins
Update (13 Apr 2012): Thanks to Alexander and Pothi for letting us know about an updated Varnish plugin for WP 3.0+ and one on WP.org also for purging your cache (Also, check out both of these guy’s blogs for more Varnish and WP performance tips!)
The varnish plugin will purge the cache when content is added or edited, and it works for multisite. The plugin does include a sample config file for your Varnish backend, however, keep in mind that when you’ve customized it for one particular site, those customizations may or may not play nicely with a different site.
Have you had any luck installing Varnish for your WordPress? What works and doesn’t? Feel free to share some of your ideas in the comments!
Update (24 Jul 2012): Jason Ormand has put together a cool Bash script to make configuring an NGINX server with Varnish and WordPress even faster.
Enjoy this post? You should follow me on Twitter!
Wow, this was an awesome post. This was probably one of the most detailed step-by-step guides on how to do anything in regards to Word Press. I will definitely be referring back to this post, an many others for that matter, as I am trying to become a master affiliate marketer, and I believe that with posts like these I can definitely achieve that. Thanks again.
Thanks for this tut. Test with my site now :)
Best of luck! :)
Please note, that plugin archive on WordPress is outdated. You should grab latest version(with fixes,secret key support and Varnish 3.x compatibility) from GitHub https://github.com/pkhamre/wp-varnish
Thanks very much for the heads up Alexander, I’ve updated the post now!
Hi Austin,
Great article.
On a production environment, I’d usually recommend Nginx in front so that if I had to tweak anything with Varnish, it’s easy to transfer the traffic to Apache temporarily. :)
To purge Varnish cache, there is an alternative plugin available too… http://wordpress.org/extend/plugins/varnish-http-purge/
[off-topic] The link to your twitter handle is broken.
Thank you Pothi! I’ve added the plugin link and fixed the broken Twitter handle now. And thanked you of course :)
I like the sound of your approach with Nginx, its always good to have a safe fallback.
Thanks Michael.
Thanks for all the feedback on the post, guys. I’m glad that you’re finding it useful. Varnish is a really awesome way to get a lot of speed out of your site, especially for a DIY-er. It’s the technology that Facebook uses, along with the company I work for, WP Engine.
Pothi, you’re right about nginx and Varnish. At WP Engine, we use a combination of nginx and apache, and throw Varnish into the mix to make it extra fast (and extra complicated). I know my SysAdmin is proud of all the tweaks we’ve made to the hosting. Traffic hits nginx first, can go to apache if necessary, and we’ve tweaked varnish on every box to make it all work. This is a setup I don’t recommend folks set up for themselves…it takes a lot of upkeep to stay online. If you need this kind of speed, that’s what WP Engine is for.
The last time install several times all not line, expect this to succeed.
Is it possible to hide the default rss feed in wordpress and at the same time without getting error when the tags are accessed in search engine?
Thank you Pothi! I’ve included the tool weblink and set the damaged Tweets manage now. And thanked you of course.
Really best tutorial i have ever read. Thanks for sharing this information about how to install and customize varnish-for wordpress
I am not get it… for what is Varnish. Can somebody explain me… pls
Thanks you for this article. It’s a very usefull information.
My all installing and managing work is done by a guy who is Database manager and working with me whenever i need him.
anyhow its a great useful information and I also learned much about installing process.
Great plugin!!! Makes my website faster!! Thanks a lot
it takes a lot of upkeep to stay online. If you need this kind of speed, that’s what WP Engine is for.
So, that’s actually super true about Varnish taking a lot of upkeep. Caching is something that has to constantly be tweaked. You should see the hours that our sysadmins work sometimes to keep everything working.
Installing varnish on your self-hosted blog will add to your workload, but I know that a lot of DIY-ers love that, so I won’t try and talk anyone out of it. If it’s working, don’t try and fix it.
That said, we DO take care of this for you ;-)
It looks excellent, I think in this way could greatly reduce the download time for visitors, try to install it
Some great tips! Thanks very much for explaining it. By the way, I think a few of the other comments might have been translated to English. Does that happen a lot?
I ended up writing an open source script to set up a LEMP+Varnish+Wordpress server on a fresh Ubuntu 12.04 install. I wrote a short summary of the project: http://bit.ly/ND2duG
Dude, that’s awesome Jason. Nice script. Are you using Nginx as well?
Sounds great Jason. I’ve added a link to your post to the end of this one now! Thanks for sharing it here! :)
Yep, it uses Nginx.
this is wonderful subject .. i read it 3 times and get a fantastic results
The links to the fetch and receive returns 404.
These are working links:
Fetch:
https://github.com/mattiasgeniar/varnish-3.0-configuration-templates/blob/master/conf.d/fetch/wordpress.vcl
Receive:
https://github.com/mattiasgeniar/varnish-3.0-configuration-templates/blob/master/conf.d/receive/wordpress.vcl
Thanks for letting me know, fixed those now!
this is very useful!
Thanks very much for explaining it. By the way, I think a few of the other comments might have been translated to English.
It looks very interesting plugin set it up at one of my websites to see how it goes
Thank you very much for taking the time to share this valuable plugin!
greetings and a big hug!
Great article! Thank you for posting.
I’m wondering how Varnish works with membership software like amember4. We’re using wordpress and have many pages where members get additional content using the am4show tag. Has anyone tried Varnish with a worpress/amember4 configuration?