Amasty Research: How fast can Magento be with Full Page Cache?

Amasty Team has recently issued a new extension Full Page Cache, aimed to speed up Magento store pages, known for their huge weight. Full Page Cache is a great success, it considerably reduces page load time and offer a number of other cool features. But actions speak louder than words. And statistics speaks even louder. Check Amasty new research on how fast can Magento be with Full Page Cache?

Test configuration:

Hardware

  • CPU: Intel Xeon L5520 2.27GHz (16 core)
  • RAM: 16 GB
  • Storage: SSD INTEL SSDSC2BB480G4

Software

  • Magento Community 1.8.1.0
  • Magento Sample Data 1.2.0
  • Apache 2.2.22
  • PHP 5.4.4 (mod_php + Zend OpCache 7.0.3)
  • MySQL 5.5.35

For testing Apache jMeter with the following scenario was used:

  1. A visitor goes to website home page;
  2. Searches for “shirt” in the search field;
  3. The result is filtered by price ($0.00 — $99.99);
  4. After that the result is filtered by color (white color);
  5. As a result only 1 Product is left, the user opens its page;
  6. The Product is added to the cart at the Product Page;
  7. After that the user goes to the Category Page «Electronics / Cameras / Digital Cameras»;
  8. The user reloads the Category Page «Electronics / Cameras / Digital Cameras»;
  9. At the Category Page camera Canon PowerShot A630 is added for comparison;
  10. At the Category Page camera Kodak EasyShare C530 is added for comparison;
  11. The Products Comparison Page is opened;
  12. From the Products Comparison Page camera Canon PowerShot A630 is added to the cart.

The test was performed for 10, 20, 30, 40 and 50 simultaneous visitors in the variants “Only Magento cache on” and “Magento cache and FPC on”. Before every launch the Magento cache was cleared and a test for 10 simultaneous visitors was launched, but it results were not taken into account (this is done in order to prevent difference in the results connected with the increasing amount of cache during the successive test launches).

In order to compare all possible variants we have additionally tested a variant when Magento has caching turned off. The most impatient readers can see its result in the diagram below (the result for the Product Page with 50 simultaneous visitors).

FPC on (sec) FPC off (sec) No caching (sec)
Product Page 0.035 0.295 3.625

Full Page Cache - Magento research

In order to prevent the influence of the data transfer speed in the Internet the test was launched in the mode “Remote start”. In this case in the server where the tested Magento is installed Apache jMeter is launched in the server mode, which receives the test from the managing client, performs it and sends the results back to the client. The client part of Apache jMeter is launched on the desktop, it manages only the server part and shows the results. This way we have prevented the influence of the Internet channel load on the test results.

Every test was performed 10 times in order to provide correct calculation of the average value of the test results. This was for 50 simultaneous users jMeter will perform the test scenario 500 times. The test scenario contains 12 requests to the server (download of CSS page resources/JS/images are not taken into account in the test results), this way during the test with 50 simultaneous visitors 6000 requests are sent to the server.

The test results in ms come in the table as follows:

Full Page Cache - Magento research

You have probably noticed that Category Page opens twice. This can be put to the fact that when  product is added to the cart the status of the dynamic blocks, the content of which should be reloaded during the next request to the them, is changed. In this case the first Page impression from the cache is slower that the successive ones. To see the difference in the load speed (when the dynamic blocks are already reloaded) the test goes to the Category page one more time. And the page is downloaded completely from the cache.

According to the table data the following diagrams have been created for better visual assessment of the difference in the server reply speed.

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

Full Page Cache - Magento research

The last diagram shows the average results on all the mentioned categories. Usage of the Full Page Cache Magento extension lets us in average increase the speed by 5 times. In some tests the difference is even bigger.