Caching in magento
Magento Performance and caching issues....
MySQL comes with a query cache, which can save the result of SELECT queries for short periods of time – that is, until some change is made to the database. How much benefit you get from the query cache depends strongly on the kind of application you use – this is why the query cache is not turned on by default. When it comes to Magento, tests have shown again and again just how much extra scalability you get if it is enabled. To enable query caching, go to your my.cnf and set the following options in the [mysqld] section: query_cache_type=1 query_cache_size=64M Save your changes and restart the MySQL server – getting a third more requests per second is not uncommon after the change!
----------------------------------------------------------------------
Enable Expires Headers Browsers use caching extensively, and can save a lot of the elements included in a web site locally so that they can be served from the browser’s cache rather than the web server on the next request. This can help quite a bit in shortening load times. The problem is for the browser to know when a file can be served from the cache, and when not – because the local copy is outdated. To solve this issue, browsers rely on two HTTP headers, Expires and Cache-Control. Magento’s default .htaccess file already configures these according to Yahoo’s performance recommendations (more on them below), but does not enable them by default. To enable them, all you need to do is add the following lines to your Apache server configuration (usually found in /etc/apache2/apache.conf): <IfModule mod_expires.c> ExpiresActive On </IfModule> If you don’t have access to the server configuration, you can also add those lines to Magento’s .htaccess file – but then you need to be careful when updating Magento that your changes do not get lost.
---------------------------------------------------------------------------
Enable Block Caching where it makes sense Out of the box, Magento can cache the Block output of your pages. When the next user requests the same block, the output that was previously calculated can be returned directly – without going through all the database queries and model calls again. This is really helpful for parts of pages that don’t change too often but are somewhat expensive to calculate – such as category pages. Clearly, it makes a lot more sense to calculate output once and then cache it for, say, five minutes, than to continuously go through the same model calls with the same results again and again.
---------------------------------------------------------------------------
Make fewer HTTP Requests in your Theme As YSlow alerts you, having many HTTP requests to load a single web page is a performance killer. Even if you enable keep-alive or use a content delivery network, it is still much slower to load many small images than a single bigger file that contains all images. Reducing the number of HTTP requests is largely a performance measure, but also helps scalability a bit.
Using image sprites, you can put all the icons, buttons etc. that you have in your theme into one file that can be downloaded quickly and then just show parts of this large image where you need it. All this involves is a little bit of CSS wizardry. A List Apart has a nice tutorial to get you started at http://www.alistapart.com/articles/sprites/.
-------------------------------------------------------------------------------
Use FastCGI to run PHP If you are using Apache as your web server, there are two ways you can sensibly set up PHP. The first way is to use the mod_php module, which is easiest to use and hence the default with many hosting providers. In this case, a PHP interpreter is run with each process of the web server and on stand-by until a script is executed. If you are wondering why your Apache needs a lot of memory, probably the fact that you are using mod_php is a big part of the answer. On a large site, there may well be hundreds of Apache processes running, and each has its own PHP interpreter. However, only very few of them – often less than one in ten – actually need to run PHP. The rest serve static files or simply wait for new requests. Because PHP uses a lot of memory, it is a good idea to see if you can avoid the overhead generated by having dozens and dozens idle PHP processes running.
The way to avoid the overhead is to use FastCGI instead of mod_php. With FastCGI, a separate internal daemon is run on your web server that is contacted by the web server only when execution of PHP required. Thus you do not need to carry the PHP baggage for all requests.
Setting up FastCGI requires you to make some changes to your server configuration, but the benefits will be large. A good starting point is this article: http://2bits.com/articles/apache-fcgid-acceptable-performance-and-better-resource-utilization.html. For more details, check the Apache web site and the documentation of your distribution.
-------------------------------------------------------------------------------
Use a Content Delivery Network Another way to relieve stress from your servers is to get an external party to serve static files for you. These services, called Content Delivery Networks (CDNs), are getting very popular and prices have fallen a lot over the last year or so. A CDN can really improve the user experience on your site, and is another way around the problems created by mod_php.
Setting up a CDN is quite easy with the free One Pica CDN extension from Magento Connect.
------------------------------------------------------------------------------------
Going Further
You have followed all the steps of this guide, and still your Magento does not scale to the level you need or give you the performance you crave? First, check again if you have really implemented all the steps. The configuration I described above is known to work very well with large Magento stores, serving hundreds of thousands of page impressions per day. Check that your hardware works well and that you are not limited by the bandwidth of your server.
If your site is truly huge, you will want to move to a cluster infrastructure that also provides high availability to insure you against software or hardware failures. You need to set up MySQL replication for your database and balance load between web servers. In practice, this will usually mean that you move to a specialised managed infrastructure provider or get outside consultants to set up the infrastructure for you.
Magento is very well suited for such deployments. You can set different database connections for read and write queries, which is important when you use MySQL replication, use memcache as a distributed cache backend, and the like. So Magento will not limit you there – the problems you may encounter will be related to the operating system and other software stack.
-----------------------------------------------------------------------------------------
How To Check If Query Caching Is Enabled
To check if your host has query caching enabled in the MySQL server, you can issue the following command from your MySQL command prompt:
mysql> SHOW VARIABLES LIKE 'have_query_cache';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| have_query_cache | YES |
+------------------+-------+
----------------------------------------------------------------------------------------
To verify that query cache is actually operational, you can issue the following command to the MySQL server:
mysql> SHOW VARIABLES LIKE 'query_cache_size';
+------------------+----------+
| Variable_name | Value |
+------------------+----------+
| query_cache_size | 67108864 |
+------------------+----------+
This shows that we have 64 MB available to our query cache size, a very respectable amount of memory.
The following demonstrates a server that has MySQL query cache disabled by setting the value to zero:
mysql> SHOW VARIABLES LIKE 'query_cache_size';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| query_cache_size | 0 |
+------------------+-------+
------------------------------------------------------------------------------------
Magento Performance and caching issues....
MySQL comes with a query cache, which can save the result of SELECT queries for short periods of time – that is, until some change is made to the database. How much benefit you get from the query cache depends strongly on the kind of application you use – this is why the query cache is not turned on by default. When it comes to Magento, tests have shown again and again just how much extra scalability you get if it is enabled. To enable query caching, go to your my.cnf and set the following options in the [mysqld] section: query_cache_type=1 query_cache_size=64M Save your changes and restart the MySQL server – getting a third more requests per second is not uncommon after the change!
----------------------------------------------------------------------
Enable Expires Headers Browsers use caching extensively, and can save a lot of the elements included in a web site locally so that they can be served from the browser’s cache rather than the web server on the next request. This can help quite a bit in shortening load times. The problem is for the browser to know when a file can be served from the cache, and when not – because the local copy is outdated. To solve this issue, browsers rely on two HTTP headers, Expires and Cache-Control. Magento’s default .htaccess file already configures these according to Yahoo’s performance recommendations (more on them below), but does not enable them by default. To enable them, all you need to do is add the following lines to your Apache server configuration (usually found in /etc/apache2/apache.conf): <IfModule mod_expires.c> ExpiresActive On </IfModule> If you don’t have access to the server configuration, you can also add those lines to Magento’s .htaccess file – but then you need to be careful when updating Magento that your changes do not get lost.
---------------------------------------------------------------------------
Enable Block Caching where it makes sense Out of the box, Magento can cache the Block output of your pages. When the next user requests the same block, the output that was previously calculated can be returned directly – without going through all the database queries and model calls again. This is really helpful for parts of pages that don’t change too often but are somewhat expensive to calculate – such as category pages. Clearly, it makes a lot more sense to calculate output once and then cache it for, say, five minutes, than to continuously go through the same model calls with the same results again and again.
---------------------------------------------------------------------------
Make fewer HTTP Requests in your Theme As YSlow alerts you, having many HTTP requests to load a single web page is a performance killer. Even if you enable keep-alive or use a content delivery network, it is still much slower to load many small images than a single bigger file that contains all images. Reducing the number of HTTP requests is largely a performance measure, but also helps scalability a bit.
Using image sprites, you can put all the icons, buttons etc. that you have in your theme into one file that can be downloaded quickly and then just show parts of this large image where you need it. All this involves is a little bit of CSS wizardry. A List Apart has a nice tutorial to get you started at http://www.alistapart.com/articles/sprites/.
-------------------------------------------------------------------------------
Use FastCGI to run PHP If you are using Apache as your web server, there are two ways you can sensibly set up PHP. The first way is to use the mod_php module, which is easiest to use and hence the default with many hosting providers. In this case, a PHP interpreter is run with each process of the web server and on stand-by until a script is executed. If you are wondering why your Apache needs a lot of memory, probably the fact that you are using mod_php is a big part of the answer. On a large site, there may well be hundreds of Apache processes running, and each has its own PHP interpreter. However, only very few of them – often less than one in ten – actually need to run PHP. The rest serve static files or simply wait for new requests. Because PHP uses a lot of memory, it is a good idea to see if you can avoid the overhead generated by having dozens and dozens idle PHP processes running.
The way to avoid the overhead is to use FastCGI instead of mod_php. With FastCGI, a separate internal daemon is run on your web server that is contacted by the web server only when execution of PHP required. Thus you do not need to carry the PHP baggage for all requests.
Setting up FastCGI requires you to make some changes to your server configuration, but the benefits will be large. A good starting point is this article: http://2bits.com/articles/apache-fcgid-acceptable-performance-and-better-resource-utilization.html. For more details, check the Apache web site and the documentation of your distribution.
-------------------------------------------------------------------------------
Use a Content Delivery Network Another way to relieve stress from your servers is to get an external party to serve static files for you. These services, called Content Delivery Networks (CDNs), are getting very popular and prices have fallen a lot over the last year or so. A CDN can really improve the user experience on your site, and is another way around the problems created by mod_php.
Setting up a CDN is quite easy with the free One Pica CDN extension from Magento Connect.
------------------------------------------------------------------------------------
Going Further
You have followed all the steps of this guide, and still your Magento does not scale to the level you need or give you the performance you crave? First, check again if you have really implemented all the steps. The configuration I described above is known to work very well with large Magento stores, serving hundreds of thousands of page impressions per day. Check that your hardware works well and that you are not limited by the bandwidth of your server.
If your site is truly huge, you will want to move to a cluster infrastructure that also provides high availability to insure you against software or hardware failures. You need to set up MySQL replication for your database and balance load between web servers. In practice, this will usually mean that you move to a specialised managed infrastructure provider or get outside consultants to set up the infrastructure for you.
Magento is very well suited for such deployments. You can set different database connections for read and write queries, which is important when you use MySQL replication, use memcache as a distributed cache backend, and the like. So Magento will not limit you there – the problems you may encounter will be related to the operating system and other software stack.
-----------------------------------------------------------------------------------------
How To Check If Query Caching Is Enabled
To check if your host has query caching enabled in the MySQL server, you can issue the following command from your MySQL command prompt:
mysql> SHOW VARIABLES LIKE 'have_query_cache';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| have_query_cache | YES |
+------------------+-------+
----------------------------------------------------------------------------------------
To verify that query cache is actually operational, you can issue the following command to the MySQL server:
mysql> SHOW VARIABLES LIKE 'query_cache_size';
+------------------+----------+
| Variable_name | Value |
+------------------+----------+
| query_cache_size | 67108864 |
+------------------+----------+
This shows that we have 64 MB available to our query cache size, a very respectable amount of memory.
The following demonstrates a server that has MySQL query cache disabled by setting the value to zero:
mysql> SHOW VARIABLES LIKE 'query_cache_size';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| query_cache_size | 0 |
+------------------+-------+
------------------------------------------------------------------------------------
