Amasty research: Full Page Cache vs Varnish Cache
After the release of Full Page Cache Magento extension many clients started asking us what makes our extension better than Varnish Cache and whether it is better at all. At first we didn’t consider conducting such research, but the amount of our clients interested in this issue encouraged us to knock these two together to learn which of them provides the best page load time. Get inside the review to know who has become the winner!
Indeed, if you need to speed up your Magento store, just ‘faster’ is not enough to describe the performance of your website. A fair contest of two Magento cache extensions with detailed speed results is at your service.
Since a new version of Magento 18.104.22.168 has been released we have changed the test configuration a little bit in comparison to our previous Full Page Cache test.
- CPU: Intel Xeon L5520 2.27GHz (16 core)
- RAM: 16 GB
- Storage: SSD INTEL SSDSC2BB480G4
- Magento Community 22.214.171.124
- Magento Sample Data 126.96.36.199
- Apache 2.2.22
- Varnish 3.0.2
- PHP 5.4.4 (mod_php + Zend OpCache 7.0.3)
- MySQL 5.5.35
Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. It is installed in front of any server that speaks HTTP.
Varnish Cache itself is not designed for working with Magento directly, so we took Nexcess Turpentine module for Magento and Varnish integration for the speed test. Basing on the recommendations of Turpentine module producer, we added «-p esi_syntax=0×2 -p cli_buffer=16384» to Varnish startup options. As a result Varnish startup options looked as follows:
DAEMON_OPTS=”-a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m -p esi_syntax=0x2 -p cli_buffer=16384″
To prevent Apache and Varnish conflict Apache was switched to port 8080.
For testing we used the list of 14 URLs of Category and Product pages: http://our-test-site/
Before every test launch Magento cache was cleared and a single visit to all URLs was performed to “warm the cache up” before taking the test results. After the warm-up test for 10, 20, 30, 40 and 50 simultaneous visitors was launched.
Comparison test results (in ms) are in the table as follows:
|10 visitors||20 visitors||30 visitors||40 visitors||50 visitors|
|Full Page Cache||Varnish & Turpentine||Full Page Cache||Varnish & Turpentine||Full Page Cache||Varnish & Turpentine||Full Page Cache||Varnish & Turpentine||Full Page Cache||Varnish & Turpentine|
|Home / Women||7||46||9||69||14||58||13||63||16||81|
|Home / Women / New Arrivals||7||44||11||71||8||89||16||80||15||51|
|Home / Women / New Arrivals / Tori Tank||7||43||8||43||12||46||14||43||21||42|
|Home / Women / New Arrivals / Elizabeth Knit Top||7||40||8||40||10||40||15||40||18||34|
|Home / Women / New Arrivals / Lafayette Convertible Dress||7||40||8||40||10||39||15||38||17||34|
|Home / Women / Tops & Blouses||7||41||9||41||13||40||11||40||16||37|
|Home / Women / Tops & Blouses / NoLIta Cami||7||40||8||40||15||39||15||39||14||34|
|Home / Women / Tops & Blouses / Black Nolita Cami||7||40||9||40||8||39||13||39||14||33|
|Home / Women / Pants & Denim||7||40||9||40||8||40||9||42||18||35|
|Home / Women / Pants & Denim / DUMBO Boyfriend Jean||7||40||8||48||16||39||11||38||17||34|
|Home / Women / Dresses & Skirts||7||40||8||41||8||41||11||39||16||37|
|Home / Women / Dresses & Skirts / Essex Pencil Skirt||7||40||9||40||12||40||13||38||17||34|
|Home / Women / Dresses & Skirts / Ludlow Sheath Dress||7||40||8||40||8||39||9||41||11||34|
To make the results easier to comprehend and let you vividly see the page load speed each Magento extension provides we have built some diagrams.
As you can see, the speed of Full Page Cache exceeds combination of Varnish Cache and Turpentine in 2-4 times. The biggest difference can be seen during the first visit of the website – at this moment both caches create a session for the user and have to initialize Magento. Since Full Page Cache initializes at the early stage, it can give the cached result quickly and Turpentine uses a lot of time in vain on extra initialization. This is an important point, as its impact deals with first visits and users’ impression of your website.
According to Kissmetrics, a 1 second delay in website load speed can decrease conversions by 7%. In fact, you might even want to count the money you lose using a Magento cache extension providing worse page load time.
This way Amasty Team recommends you using Full Page Cache Magento extension for providing the minimal page load time possible.