varnish_cache

What is Varnish ?

Varnish is an HTTP cache server. It considered as an HTTP accelerator or sometimes a reversy-proxy between application servers and clients. It is designed for content-heavy dynamic web sites and highly consumed APIs. According to the rules which are defined by system administrator and site developers, it allows alleviating server load and serving queries faster by caching data.

Examples of web sites using Varnish: Wikipedia, Facebook, Twitter, Vimeo, Tumblr, The New York Times, The Guardian, The Hindu, Corriere della Sera…

In 2012, 5% of the top 10000 sites in the web used the software.

How it works ?

 

As mentionned in the figure, Varnish is between the client and the web server. Let’s consider that we are using Apache as web server:

  • Varnsih will listen on port 80 and Apache will move to another port (8080)
  • Apache will be considered as a backend to Varnish

 

Installation


[root@server varnish]# yum install varnish

Related packages are:


[root@server varnish]# rpm -qa | grep varnish
varnish-libs-4.0.3-3.el7.x86_64 varnish-4.0.3-3.el7.x86_64

Related files are:


[root@server varnish]# tree /etc/varnish/
 /etc/varnish/
 |___ default.vcl
 |___ secret
 |___ varnish.params 0 directories, 3 files

 

Configuration

The file /etc/varnish/default.vcl is written in VCL (Varnish Cache Language) and contains blocks defining how varnish is working when receiving, fechting/asking the backend and delivering responses.
Each block begins with a little description explaining it’s role.
Edit /etc/varnish/default.vcl and modify/add this lines


# Default backend definition. Set this to point to your content server.
backend default {
.host = "Apache server IP";
.port = "8080";
}


sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove Optimizely Cookies
set req.http.Cookie = regsuball(req.http.Cookie, "optim.=[^;]+(; )?", "");
# Remove Gauges Cookies
set req.http.Cookie = regsuball(req.http.Cookie, "_gau.=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
if (req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.method != "GET" && req.method != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
return (hash);
}


sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
set beresp.ttl = 10s;
set beresp.grace = 1h;
}


sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
# You can do accounting or modifying the final object here.
if (obj.hit's > 0) {
# Add debug header to see if it's a HIT/MISS and the number of
hit's, disable when not needed
set resp.http.X-Cache = "HIT";
} else {
set resp.http.X-Cache = "MISS";
}
}

Change Varnish parameters in /etc/varnish/varnish.params like below:


VARNISH_LISTEN_ADDRESS=0.0.0.0
VARNISH_LISTEN_PORT=80

Tests

After changing Apache port to 8080, restart varnish and apache. Verify all is OK:


[root@server varnish]# netstat -nlp | grep '80'
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 11156/varnishd
tcp6 0 0 :::8080 :::* LISTEN 9352/httpd


[root@server varnish]# apachectl configtest
Syntax OK

Test with curl:


[20:50:50] [VPS1-centos7] root:~ # curl -I http://your_domain/

# curl -I http://your_domain/
HTTP/1.1 200 OK
Date: Sun, 06 Nov 2016 18:05:23 GMT
Server: Apache
Vary: Accept-Encoding
Last-Modified: Wed, 26 Oct 2016 08:11:04 GMT
Content-Type: text/html
X-Varnish: 132182 34762
Age: 69
Via: 1.1 varnish-v4
ETag: W/"14e1ff7-5ef3-53fc02a9391ed"
X-Cache: HIT
Connection: keep-alive

X-Cache field is set to HIT, Varnish is responding
X-Cache field is set to MISS, Varnish fetched it’s backend because it can not handle the request or the requested page is not in the cache.

I hope that this blog helped you. Please visit our blog website for other interesting blogs and feel free to leave your feedbacks and comments. Till next time

Post Views: 5,449
The following two tabs change content below.
My Facebook profile

ZIADI Mohamed Ali

I am a hands-on, competent Linux system engineer with 9 years’ experience. I have a strong performance background in wide variety of professional Linux system support including monitoring, configuration, troubleshooting and maintenance. I have worked on numerous projects from concept to completion. A specialist in LAMP platforms, I take pride in administrating Linux systems and regularly refresh my skills to ensure I keep up with ongoing developments and new technologies.