Varnish

From wiki.mikejung.biz
Revision as of 21:33, 10 October 2014 by Admin (talk | contribs) (Install Varnish 4.0.2 On CentOS 6.5)
Jump to: navigation, search

Poweredby.gif

Overview

Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture. A high level overview of what Varnish does can be seen in the video attached to this web page.

Install Varnish 3.0.5 On CentOS 6.5

Download and Install the Repo from Varnish

rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-3.0.el6.rpm

Install Varnish

yum install varnish

Enable Varnish on Reboot

chkconfig varnish on

Change Apache Listen Port to 8080

vim /etc/httpd/conf/httpd.conf

Listen 80

##change to

Listen 8080

Configure Varnish

vim /etc/sysconfig/varnish

VARNISH_LISTEN_PORT=6081

##change to

VARNISH_LISTEN_PORT=80

Configure Varnish Backend

vim /etc/varnish/default.vcl

backend default {
  .host = "127.0.0.1";
  .port = "80";
}

##change to

backend default {
  .host = "127.0.0.1";
  .port = "8080";
}

Restart Apache and Varnish to enable changes made to configuration

service httpd start
service varnish start

Make sure Varnish and Apache are listening on the correct ports

netstat -plan | grep 80

Should look like:

tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      1868/varnishd
tcp        0      0 :::80                       :::*                        LISTEN      1868/varnishd
tcp        0      0 :::8080                     :::*                        LISTEN      1825/httpd

Create a test file to make sure Varnish is working correctly

touch /var/www/html/test.txt
echo "test" > /var/www/html/test.txt
  • You should be able to view the file in your browser:

http://$IP/test.txt

  • You should also be able to verify that Varnish is serving the file:
curl -I http://$IP/test.txt

HTTP/1.1 200 OK
Server: Apache/2.2.15 (CentOS)
Last-Modified: Tue, 01 Jul 2014 16:33:52 GMT
ETag: "41cf0-5-4fd2457515364"
Content-Type: text/plain; charset=UTF-8
Content-Length: 5
Accept-Ranges: bytes
Date: Tue, 01 Jul 2014 16:36:35 GMT
X-Varnish: 677655568 677655567
Age: 53
Via: 1.1 varnish
Connection: keep-alive

Install Varnish 4.0.2 On CentOS 6.5

Download and Install the Repo from Varnish

rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.0.el6.rpm

Grab jemalloc since CentOS includes nothing useful by default for anything ever....

wget https://dl.fedoraproject.org/pub/epel/6/x86_64/jemalloc-3.6.0-1.el6.x86_64.rpm
rpm -i jemalloc-3.6.0-1.el6.x86_64.rpm

Install Varnish

yum install varnish

Enable Varnish on Reboot

chkconfig varnish on

Change Apache Listen Port to 8080

vim /etc/httpd/conf/httpd.conf

Listen 80

##change to

Listen 8080

Configure Varnish

vim /etc/sysconfig/varnish

VARNISH_LISTEN_PORT=6081

##change to

VARNISH_LISTEN_PORT=80

Configure Varnish Backend

vim /etc/varnish/default.vcl

backend default {
  .host = "127.0.0.1";
  .port = "80";
}

##change to

backend default {
  .host = "127.0.0.1";
  .port = "8080";
}

Restart Apache and Varnish to enable changes made to configuration

service httpd start
service varnish start

Make sure Varnish and Apache are listening on the correct ports

netstat -plan | grep 80

Should look like:

tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      1868/varnishd
tcp        0      0 :::80                       :::*                        LISTEN      1868/varnishd
tcp        0      0 :::8080                     :::*                        LISTEN      1825/httpd

Create a test file to make sure Varnish is working correctly

touch /var/www/html/test.txt
echo "test" > /var/www/html/test.txt
  • You should be able to view the file in your browser:

http://$IP/test.txt

  • You should also be able to verify that Varnish is serving the file:
curl -I http://$IP/test.txt

HTTP/1.1 200 OK
Server: Apache/2.2.15 (CentOS)
Last-Modified: Tue, 01 Jul 2014 16:33:52 GMT
ETag: "41cf0-5-4fd2457515364"
Content-Type: text/plain; charset=UTF-8
Content-Length: 5
Accept-Ranges: bytes
Date: Tue, 01 Jul 2014 16:36:35 GMT
X-Varnish: 677655568 677655567
Age: 53
Via: 1.1 varnish
Connection: keep-alive

Upgrade Varnish 3.0.5 to Varnish 4.0.2 CentOS 6.5

This is not exactly seamless, or at least as simple as it sounds. All you should really have to do is grab the new repo, install, remove varnish 3, install varnish 4, however if you don't have a few things in place your site is going to be down.

Install jemalloc because CentOS never contains anything useful in its repos. Luckily this is easy once you head over to Fedora land. Install this sucker because Varnish 4 needs it. Varnish 3 didn't require this, so you just try and do a quick upgrade, you are going to get pissed without this.

wget https://dl.fedoraproject.org/pub/epel/6/x86_64/jemalloc-3.6.0-1.el6.x86_64.rpm
rpm -i jemalloc-3.6.0-1.el6.x86_64.rpm


Backup existing config files. A backup is made when you remove Varnish 3, but do it anyway!

cp /etc/sysconfig/varnish /etc/sysconfig/varnish.3bak
cp /etc/varnish/default.vcl /etc/varnish/default.vcl.3back

Configuration Files

  • Used for parameters and command line arguments. When you change this, you need to run service varnish restart for the changes to take effect.

Debian based

/etc/default/varnish 

Redhat / CentOS

/etc/sysconfig/varnish


  • This contains your VCL and backend-definitions. After changing this, you can run either service varnish reload, which will not restart Varnish, or you can run service varnish restart, which empties the cache.
/etc/varnish/default.vcl
backend default {
   .host = "localhost";
   .port = "8080";
}
  • A backend server is the server providing the content Varnish will accelerate.

.host = The host that varnish will talk to get content. If this is installed on the same server that Apache is using, this would be localhost or 127.0.0.1. If the Apache server is on another server, you would put in the IP of that server, private network is preferable, but a public IP will work as well.

.port = The port that Varnish will connect to to get content to cache. If Apache is listening on port 80, then you would configure this to listen on port 80. If Apache is listening on port 8080, then you would set this to port 8080.

Main Configuration Options

# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=80

#### I've added this to explain these options a bit more thoroughly. 

VARNISH_LISTEN_ADDRESS= Leaving this blank means that Varnish will listen on all interfaces. You can specify whether or not to listen on a public IP, or private, 
if this is the front end server then you will want this to listen on the public IP.

VARNISH_LISTEN_PORT= The port that Varnish will listen on, you will more than likely want to set this to port 80.

Varnish Storage Backends

malloc

Varnish will request the entire size of the cache with a malloc call. Every object stored by Varnish will be stored in RAM. Keep in mind that if you do not limit RAM usage Varnish might use up all available RAM, forcing the OS to start to swap, at this point performance may degrade. Be sure to size Varnish Malloc allocation accordingly.

Each object that is stored in Varnish will consume about 1KB of RAM, regardless of the size. So many small files may start to bloat memory usage in a hurry. It's best to set a limit when using Malloc.

Malloc is the fastest option to use with Varnish

You can configure Varnish to use Malloc as backend storage by modifying /etc/sysconfig/varnish. You can either define the variable, and pass to the DAEMON_OPTS, or just define DAEMON_OPTS

vim /etc/sysconfig/varnish
VARNISH_STORAGE="malloc,100M"

*OR*

DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,100M \
             -u www-data \
             -g www-data"

Then restart Varnish

service varnish restart

file

Varnish creates a file on the filesystem to contain the entire cache. It then tells the OS via mmap() to map the entire file into memory if possible. Keep in mind that if you are still using spinning disks, you may see lower performance than using malloc due to write latency. If you are using SSDs, or have a large RAID that can handle some extra IO, the File option may be a good option. You can modify the file location to whatever you want, so if you have a spare disk, create the file and update VARNISH_STORAGE_FILE with the full path to the file.

  • Choose file if you have a large cache that will not fit entirely into RAM AND have fast disks (ssds)


You can configure Varnish to use File as backend storage by modifying /etc/sysconfig/varnish. You can either define the variable, and pass to the DAEMON_OPTS, or just define DAEMON_OPTS

vim /etc/sysconfig/varnish
VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin
VARNISH_STORAGE="file,100M,${VARNISH_STORAGE_FILE}"

*OR*

DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s file,100M,/var/lib/varnish/varnish_storage.bin \
             -u www-data \
             -g www-data"

Then restart Varnish

service varnish restart


Shared memory log -- Not much needs to be done with this besides making sure the log is stored in RAM (/dev/shm)

Varnish 3 Configuration

VARNISH_MIN_THREADS

For Varnish 3.0 the default value is 2 (according to man page). If you have a very busy server, you may want to raise this value. Keep in mind that this is only the minimum value, and really only matters if you get frequent bursts of traffic. This can be found and configured in /etc/sysconfig/varnish

VARNISH_MIN_THREADS=2

VARNISH_MAX_THREADS

For Varnish 3.0 the default value is 500 (according to man page). If you have a very busy server, you may want to raise this value. Raising this to very high levels may cause some issues, so you want to watch the server after you change this value to make sure varnish doesn't go all crazy on you. This can be found and configured in /etc/sysconfig/varnish

VARNISH_MAX_THREADS=500

VARNISH_THREAD_TIMEOUT

For Varnish 3.0 the default value is 300 seconds (according to man page). If you prefer to kill your threads while they are young, you can lower the value. If you want your threads to have a long, fulfilling life and experience all the static files they can, raise this value. This can be found and configured in /etc/sysconfig/varnish

VARNISH_THREAD_TIMEOUT=300

VARNISH_TTL

For Varnish 3.0 the default value is 120 seconds (according to man page). This is the default TTL that is assigned to new objects that no one cared about, or specified a TTL for. This is a minimum, hard value to live in the cache. If you want to clear out the cache more frequently, and don't like to specify ttls elsewhere, then lowering or raising this value is your best bet. This can be found and configured in /etc/sysconfig/varnish

VARNISH_TTL=120

Varnish backend Misc section

To configure Varnish using the file backend, do nothing, it's the default. However, you can change the size of RAM to use by changing the "1GB" value to whatever is appropriate.

vim /etc/sysconfig/varnish

# # Cache file location
VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=1GB
#
# # Backend storage specification
VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"

To configure Varnish to use the malloc backend, you will need to comment out the original VARNISH_STORAGE setting, create a new one, specify "malloc,$size_of_cache"

# # Backend storage specification
#VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
VARNISH_STORAGE="malloc,256MB"

Commands and Arguments

Restart Varnish and clear it's cache

service varnish restart

Reloads the vcl file, the cache is not affected

service varnish reload

Shows the cache status and hit ratio

varnishstat

Command line options.

Listen address
-a <[hostname]:port>

Specifies the vcl file location
-f <filename>

Set the tunable parameters
-p <parameter=value>

Authentication secret for management
-S <secretfile>

Management interface
-T <hostname:port>

Where and how to store objects
-s <storagetype,options>

Troubleshooting

nf_conntrack

If you notice some odd issues with Varnish, check dmesg, if you see this you should raise the limit.

# dmesg
nf_conntrack: table full, dropping packet.
nf_conntrack: table full, dropping packet.
nf_conntrack: table full, dropping packet.

To raise the limit:

#See what the current limit is and note it:
cat /proc/sys/net/nf_conntrack_max

vim /etc/sysctl.conf

#Add the following
net.nf_conntrack_max = 100000

#Once that is added, run this to make the change permanent.
sysctl -p

Benchmarks

Setup

  • Running on CentOS 6
  • Apache 2.2 is being used behind Varnish on the same server.
  • Varnish 3.0 using the following settings: VARNISH_STORAGE="malloc,256MB"
  • This is on a 1GB instance

Test files:

http:///index.html (static, basic html page)
http:///index.php  (phpinfo page)
http:///perl.php   (grabs perl version)

Varnish

10,000 requests, 10 concurrency

ab -n 10000 -c 10 http:///index.html
Document Length:        169 bytes
Time taken for tests:   1.029 seconds
Requests per second:    9716.42 [#/sec] (mean)
Transfer rate:          4725.37 [Kbytes/sec] received

ab -n 10000 -c 10 http:///index.php
Document Length:        44303 bytes
Time taken for tests:   1.202 seconds
Requests per second:    8316.46 [#/sec] (mean)
Transfer rate:          362058.58 [Kbytes/sec] received

ab -n 10000 -c 10 http:///perl.php
Document Length:        237 bytes
Time taken for tests:   1.033 seconds
Requests per second:    9678.93 [#/sec] (mean)
Transfer rate:          4839.45 [Kbytes/sec] received

50,000 requests, 20 concurrency

ab -n 50000 -c 20 http:///index.html
Document Length:        169 bytes
Time taken for tests:   5.176 seconds
Requests per second:    9659.51 [#/sec] (mean)
Transfer rate:          4688.26 [Kbytes/sec] received

ab -n 50000 -c 20 http:///index.php
Document Length:        44303 bytes
Time taken for tests:   6.064 seconds
Requests per second:    8245.66 [#/sec] (mean)
Transfer rate:          358976.27 [Kbytes/sec] received

ab -n 50000 -c 20 http:///perl.php
Document Length:        237 bytes
Time taken for tests:   5.190 seconds
Requests per second:    9633.36 [#/sec] (mean)
Transfer rate:          4816.68 [Kbytes/sec] received

Apache

10,000 requests, 10 concurrency

ab -n 10000 -c 10 http:///index.html
Document Length:        169 bytes
Time taken for tests:   1.416 seconds
Requests per second:    7063.33 [#/sec] (mean)
Transfer rate:          3015.24 [Kbytes/sec] received

ab -n 10000 -c 10 http:///index.php
Document Length:        43540 bytes
Time taken for tests:   6.479 seconds
Requests per second:    1543.56 [#/sec] (mean)
Transfer rate:          65943.43 [Kbytes/sec] received

ab -n 10000 -c 10 http:///perl.php
Document Length:        237 bytes
Time taken for tests:   1.792 seconds
Requests per second:    5578.88 [#/sec] (mean)
Transfer rate:          2344.57 [Kbytes/sec] received

50,000 requests, 20 concurrency

ab -n 50000 -c 20 http:///index.html
Document Length:        169 bytes
Time taken for tests:   7.119 seconds
Requests per second:    7023.53 [#/sec] (mean)
Transfer rate:          2997.59 [Kbytes/sec] received

ab -n 50000 -c 20 http:///index.php
Document Length:        43540 bytes
Time taken for tests:   34.014 seconds
Requests per second:    1470.00 [#/sec] (mean)
Transfer rate:          62750.60 [Kbytes/sec] received

ab -n 50000 -c 20 http:///perl.php
Document Length:        237 bytes
Time taken for tests:   9.256 seconds
Requests per second:    5402.16 [#/sec] (mean)
Transfer rate:          2268.62 [Kbytes/sec] received