Difference between revisions of "GooglePageSpeed"

From wiki.mikejung.biz
Jump to navigation Jump to search
(In-Memory LRU Cache)
(Mod_pagespeed Caches)
Line 98: Line 98:
ModPagespeedLRUCacheKbPerProcess  8192 (8MB Per Apache Process)
ModPagespeedLRUCacheKbPerProcess  8192 (8MB Per Apache Process)
ModPagespeedLRUCacheByteLimit      16384 (16KB size limit per object in LRU)
ModPagespeedLRUCacheByteLimit      16384 (16KB size limit per object in LRU)
===SharedMemory MetadataCache===
In addition to the FileCache and per process LRU caches, you can customize yet another cache option for mod_pagespeed. This cache is the awesome cache because it uses shared memory which means that all the Apache processes can access the same cache, unlike the per process LRU caches (which are still good to have). Since there is only one cache, we can set the size limit to 50MB or higher without having to worry about too many processes using memory.
To configure ModPagespeedCreateSharedMemoryMetadataCache you should specify the same path you used for "ModPagespeedFileCachePath" and as a second option, specify a size for the cache, the value here is in KB. 
ModPagespeedCreateSharedMemoryMetadataCache "/var/cache/pagespeed/" 51200 (50MB)

Revision as of 16:11, 10 March 2015

Liquidweb 728x90.jpg

mod_pagespeed overview

You can use mod_pagespeed to automatically optimize websites that use Nginx or Apache. Installing the module is pretty easy, regardless of what webserver you use. Mod_pagespeed can significantly improve the page load time of most websites and the default filters and optimizations used are totally safe for like 99.99% of websites out there.

mod_pagespeed will parse html before the user's browser recieves anything, pagespeed with apply compression, rewrite images, and perform other optimizations that reduce the amount of requests it takes to load a page and to reduce the page size. Not only does mod_pagespeed handle all the best practices for your website, it also utilizes a file system cache that it can pull from so Apache isn't always optimizing the same files over and over again.

The filesystem cache is enabled by default and must remain enabled for mod_pagespeed to work correctly. In addition to the file system cache you can also configure pagespeed to use a Shared Memory Cache for metadata which can be used by all Apache processes (and thread if you use Event). If two caches aren't good enough for you then you can also enable the PER Apache Process LRU cache, which gives each Apache process it's own cache to store small files in.

On top of all those caches, you can also enable the use of Memcached for even more caching. Confused? I was too at first, with so many caches and possible configuration options, it seems overwhelming at first, but I will cover all the caches in their own section below to hopefully explain how they all work together to make your site fast.

Enabling mod_pagespeed Console and Admin UI

To enable the mod_pagespeed Console, Admin UI and statistics you should make sure that you configure pagespeed.conf to look similar to what is shown below. You should enable logging and statistics if you want to have more information on what mod_pagespeed is doing. All of this information can be viewable from http://domain.com/pagespeed_admin assuming you allow the correct IPs. I strongly suggest you use the "allow,deny" statement, if you do not do that then anyone could view your pagespeed stats.

ModPagespeedEnableFilters add_instrumentation
ModPagespeedMessageBufferSize 100000
ModPagespeedStatistics on
ModPagespeedStatisticsLogging on
ModPagespeedLogDir /var/log/pagespeed

<Location /mod_pagespeed_beacon>
      SetHandler mod_pagespeed_beacon

<Location /mod_pagespeed_statistics>
    Order allow,deny
    Allow from localhost
    Allow from $IP_1
    Allow from $IP_2
    SetHandler mod_pagespeed_statistics

<Location /pagespeed_admin>
  Order allow,deny
  Allow from localhost
  Allow from
  Allow from $IP_1
  Allow from $IP_2
  SetHandler pagespeed_admin

Mod_pagespeed Caches


You can and should enable the FileCache for mod_pagespeed. This should be enabled by default, but if you notice that pagespeed isn't using the file cache or that Apache is complaining about something related to the FileCachePath you should view the main conf file and make sure the path is correctly set. You can use the syntax below to customize where the cache directory is / should be located. The path must be writable by the Apache user.

ModPagespeedFileCachePath  "$path_to_apache_writable_dir" 
ModPagespeedFileCacheSizeKb          102400 (100MB)
ModPagespeedFileCacheCleanIntervalMs 3600000 (3600 seconds)
ModPagespeedFileCacheInodeLimit      500000

For example, on a cPanel server you might find that the cache is located under /var/

ModPagespeedFileCachePath   "/var/mod_pagespeed/cache/"

If you have many vhosts on the same server and notice that Apache complains about "ModPagespeedFileCachePath must not be empty". You can try adding the ModPagespeedInheritVHostConfig option in the main pagespeed.conf file. This allows all vhosts to inherit some globabl settings without having to specify the same settings for each vhost.

ModPagespeedInheritVHostConfig          on
ModPagespeedFileCachePath               "/var/mod_pagespeed/cache/"

There are a few settings for the FileCache which can be adjusted based on your needs. These settings are only a suggestion to mod_pagespeed and are not a hard cap on the amount of space that the file cache can use. mod_pagespeed will scan the FileCache directory based on the value of ModPagespeedFileCacheCleanIntervalMs, if the cache has exceeded the size, or inode limit then the oldest cached files will be removed to make way for new items. If neither of the limits have been reached, nothing will happen and the FileCache will continue to grow until the next scan interval comes along. This means that it's possible for the FileCache to grow quickly in size and take up free space, so be careful when configuring the settings below.

  • ModPagespeedFileCacheSizeKb - It appears that the default value for the cache is 102400 KB, or 100 MB. If you have fast storage and plenty of storage space then you can raise this value to whatever makes sense. If you only have 50MB of content then a value of 100MB should be plenty, however if you have lots of images and content, raising this to a value of 1GB or more might make sense.
  • ModPagespeedFileCacheCleanIntervalMs - The values for this setting are in milliseconds, because Google wants to make things complicated. A value of 3600000 ms or 3600s or 1 hour is the default option. This means that once an hour the FileCache directory will be scanned and if it has reached the size limit, or inode limit the old cache entries will get removed to make way for new entries.
  • ModPagespeedFileCacheInodeLimit - The default limit for inodes in the FileCache is 500000 by default. Most of the time you should never have to mess around with this option unless you would rather think about inodes instead of file sizes.

So, let's say we have a server that has lots of storage space to spare, and hosts a single website with 1GB of files and content and we want to configure the FileCache to hold all the things. This is what the configuration for the FileCache would look like.

ModPagespeedFileCachePath  "/var/mod_pagespeed/cache/"
ModPagespeedFileCacheSizeKb          1024000 (1GB)
ModPagespeedFileCacheCleanIntervalMs 3600000 (3600 seconds)
ModPagespeedFileCacheInodeLimit      500000

Pretty simple right? Just make sure that the device that's storing the cache has enough free space. Now that the FileCache has been configured, we can move on to the next mod_pagespeed cache, which is even more awesome than the FileCache!

In-Memory LRU Cache

In addition to the FileCache, mod_pagespeed can be configured to allow for a small in-memory write-through LRU cache which runs in each Apache process. If you use Prefork, this could mean many processes, which could mean a lot of memory usage per process. You shouldn't use Prefork anyway because Event is way better, so I won't cover how to configure the LRU cache for Prefork, only for Event.

To figure out the best size of the LRUCache you should look at how you've configured Apache. Most of the time the Event MPM will have a ServerLimit of 16, which means that up to 16 Apache processes could run at the same time. If we set ModPagespeedLRUCacheKbPerProcess to 1024 (1MB) then under the worst case ontario the LRU cache would use 16MB total, (16 Apache Procs x 1MB). Each LRU Cache is limited to serving only the Apache process it was created in, the threads in that process can use the same cache, but threads from other processes will have to stick with their own cache (unlike the shared metadata cache which is covered next).

So, if we have a server with 1GB of memory, and we plan on keeping Apache's ServerLimit under 10, we could set the LRU cache size to 10240 (10MB) and if we had 10 apache processes running at all times the total LRU cache size would be 100MB between all 10 processes. Obviously if we set this to something crazy, like 100MB and we started up 10 Apache procs, the server could run out of memory quickly so be careful with these values.

The ModPagespeedLRUCacheByteLimit is the limit on how large a cache entry the LRU cache will accept. Unlike the ModPagespeedLRUCacheKbPerProcess value, this setting is in BYTES, not KB. A value of 16384 is 0.015 MB. Generally you want to keep the max size down low to avoid wasted space in the LRU cache. If you set this to say, 1MB then all the LRU caches might be constantly removing items like images because nothing else can fit in the LRU, this cache should be for small stuff, not large stuff.

If you are using Apache Event and aren't running tons of Apache processes, the settings below should be a good starting point. Usually even on an idle server there will be 4 - 5 apache processes running, so if you set each LRU cache to use 8MB you will be looking at an extra 40MB of memory usage, which is totally worth it imo.

ModPagespeedLRUCacheKbPerProcess   8192 (8MB Per Apache Process)
ModPagespeedLRUCacheByteLimit      16384 (16KB size limit per object in LRU)

SharedMemory MetadataCache

In addition to the FileCache and per process LRU caches, you can customize yet another cache option for mod_pagespeed. This cache is the awesome cache because it uses shared memory which means that all the Apache processes can access the same cache, unlike the per process LRU caches (which are still good to have). Since there is only one cache, we can set the size limit to 50MB or higher without having to worry about too many processes using memory.

To configure ModPagespeedCreateSharedMemoryMetadataCache you should specify the same path you used for "ModPagespeedFileCachePath" and as a second option, specify a size for the cache, the value here is in KB.

ModPagespeedCreateSharedMemoryMetadataCache "/var/cache/pagespeed/" 51200 (50MB)

How to install mod_pagespeed Apache module on Ubuntu 14.10

This is for Ubuntu 14.10, however the same applies for CentOS 7, just use replace .deb with .rpm and use rpm to install the package instead of dpkg.

wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_amd64.deb
dpkg -i mod-pagespeed-*.deb
apt-get -f install

After the module is installed, Apache should already have been restarted and the module loaded, to make sure this is the case you can run

apache2 -M

Ubuntu 14.10 mod_pagespeed configuration

To modify the default settings for mod pagespeed you can edit the main configuration file, which is called pagespeed.conf. If you are using Ubuntu, this can be found under the mods-enabled directory for apache. If you make any changes to pagespeed.conf you must restart apache or else the new settings will not take effect.

vim /etc/apache2/mods-enabled/pagespeed.conf

mod_pagespeed default CoreFilter list

By default, pagespeed will enable and use this list of filters, also know as the "core filters". Google considers all of these filters to be "safe" meaning that they shouldn't cause issues with most websites and CMS. I've found that the core filters work perfectly fine with WordPress and MediaWiki. If you wish to enable more filters you certainly can, but the list below contains the already enabled filters.


How to configure mod_pagespeed to use Memcached

To configure mod_pagespeed to use Memcached simply add the line below, and optionally the lines below it to enable moar caches that can be shared by all the apache processes. Obviously this assumes you have Memcached installed on the server and that it is listening on an open port that is accessible via localhost. If you are using cpanel you will want to edit the pagespeed.conf file located at /usr/local/apache/conf/pagespeed.conf, if you are using Ubuntu the file will be located at /etc/apache2/mods-enabled/pagespeed.conf


ModPagespeedMemcachedServers localhost:11211

##File Cache, Shared Memory Cache, and LRU Cache## 

ModPagespeedFileCachePath            "/var/mod_pagespeed/cache/"
ModPagespeedCreateSharedMemoryMetadataCache  "/var/mod_pagespeed/cache/" 51200
ModPagespeedFileCacheSizeKb          102400
ModPagespeedLRUCacheKbPerProcess   8192
ModPagespeedLRUCacheByteLimit      65536

You can view memcached stats by echoing stats and piping that into netcat.

echo stats | nc 11211

To view pagespeed specific memcached statistics you can simply curl the url from localhost if you have access to the server. This will display all of the pagespeed and memcached specific information

curl http://localhost/mod_pagespeed_statistics?memcached

How to bypass mod_pagespeed filters

By default, the module uses "CoreFilters", which is a safe set of rules that most sites benefit from, if you want to disable everything and only enable certain items, you can do this globally here, however it's recommended to just enable / disable things per vhost using .htaccess or vhost directive. You would just uncomment the line below

# ModPagespeedRewriteLevel PassThrough

How to include extra filters for mod_pagespeed

If you want to globally enable more filters than the core rules offer you can uncomment these lines and just add the filters you want to enable.

# ModPagespeedEnableFilters rewrite_javascript,rewrite_css
# ModPagespeedEnableFilters collapse_whitespace,elide_attributes

mod_pagespeed .htaccess directives

You can specify additional filters in .htaccess, or in the main apache2.conf, or on the vhost level using a directive like this:

<IfModule pagespeed_module>
ModPagespeed on
ModPagespeedEnableFilters remove_comments,rewrite_javascript,rewrite_css,rewrite_images
ModPagespeedEnableFilters elide_attributes,defer_javascript,move_css_to_head
ModPagespeedJpegRecompressionQuality -1

Install PageSpeed Module (mod_pagespeed) on cPanel Server

The easiest way to install mod_pagespeed on a cPanel server is to use git to clone the cpanel / pagespeed project from github. I've tested this out on CentOS 6.5 and CentOS 6.6 with WHM 11.48, and have never noticed any issues with the plugin, or with this installation method.

/usr/local/cpanel/3rdparty/bin/git clone https://github.com/pagespeed/cpanel.git /tmp/pagespeed/

After you've cloned the project using git, cd into /tmp/pagespeed/Easy and create a tarball that EasyApache can use to install the module when you run future EasyApache.

cd /tmp/pagespeed/Easy
tar -zcvf Speed.pm.tar.gz pagespeed
mkdir -p /var/cpanel/easy/apache/custom_opt_mods/Cpanel/Easy
mv Speed.pm Speed.pm.tar.gz -t /var/cpanel/easy/apache/custom_opt_mods/Cpanel/Easy/
cd && rm -rf /tmp/pagespeed

At this point you should be able to run EasyApache, and when you go to customize profile, there should be a "mod_pagespeed" entry under the Apache module section. Make sure this is selected / checked and then tell EasyApache to save and build with the new profile. This can take up to 20 minutes to complete, but all your websites should remain online while this process occurs.


Now you can check to make sure the EA finished successfully and the module is enabled:

httpd -M

##You should see
 pagespeed_module (shared)

Link: https://github.com/pagespeed/cpanel

How to use Memcached with ModPageSpeed on a cPanel Server

If you are using cPanel and have mod_pagespeed installed you can modify the configuration file so that pagespeed uses memcached. This is not the base by default and if you want to speed up the site and use memory instead of just using disk you can edit the file listed below and enter in the IP and port of Memcached. In this case I have Memcached running on the same host. You will need to start apache after you are done editing the file.

vim /usr/local/apache/conf/pagespeed.conf

##Add These Lines
  ModPagespeedMemcachedThreads 1
  ModPagespeedCreateSharedMemoryMetadataCache "/var/mod_pagespeed/cache" 51200

Additional Links

  • PageSpeed can be added to any server runtime and applied dynamically to any application. This is available as a module for Apache and Nginx. This helps to optimize resources based on a lot of "web optimization filters".