GooglePageSpeed

From wiki.mikejung.biz
Jump to: navigation, search

Liquidweb high performance banner.gif

mod_pagespeed overview

You can use mod_pagespeed to automatically optimize websites that run on a server that uses Nginx or Apache. Installing the pagespeed module is pretty easy, regardless of what web server you use. This page mainly focuses on installing and configuring mod_pagespeed with Apache, but most of the configuration settings should also work with Nginx.

Mod_pagespeed can significantly improve the page load time of most websites and the default filters and optimizations used are totally safe for like 99.99% of websites out there. Mod_pagespeed is able to reduce time to first byte by utilizing one of it's internal caches. After pagespeed rewrites images or html it will save the optimized version in a cache and the next time the content is requested it will serve the content from the cache, assuming nothing has changed since the last request.

mod_pagespeed will parse html before the user's browser receives anything, pagespeed with apply compression, rewrite images, and perform other optimizations that reduce the amount of requests it takes to load a page and to reduce the page size. If you have mod_pagespeed enabled, you don't have to worry about enabling browser caching in the .htaccess file since pagespeed automatically enables browser caching for any resource it can.

Not only does mod_pagespeed handle all the best practices for your website, it also utilizes a file system cache that it can pull data from, so Apache isn't always optimizing the same files over and over again. It can take some time for the mod_pagespeed caches to warm up, so wait at least a few hours after restarting services to judge if performance improved or not. By utilizing Memcached you can avoid losing a lot of the cache when you restart apache.

The file system cache is enabled by default and must remain enabled for mod_pagespeed to work correctly. In addition to the file system cache you can also configure pagespeed to use a Shared Memory Cache for metadata which can be used by all Apache processes (and thread if you use Event). If two caches aren't good enough for you then you can also enable the PER Apache Process LRU cache, which gives each Apache process it's own cache to store small files in.

On top of all those caches, you can also enable the use of Memcached for even more caching. Confused? I was too at first, with so many caches and possible configuration options, it seems overwhelming at first, but I will cover all the caches in their own section below to hopefully explain how they all work together to make your site fast.

If you want the best page load time performance for your site then using something like Google's mod_pagespeed is the best place to start.

How to Install and Configure mod_pagespeed for EasyApache 4

The first thing to do is

CELEBRATE! We can haz cPanel, PHP 7, PHP FPM and mod_pagespeed?? Wow. Cpanel is doing something right, great job!

Cpanel php7 php-fpm its happening.gif

Ok, now that we're done celebrating this event we can go ahead and run the wget command to get the latest version of mod_pagespeed from some GitHub page that has cpanel mod pagespeed, I think Google maintains this, so it probably isn't a virus.

Seriously though, do you know how long I've waited for this time to come? Literally years, for years I've been waiting for this moment, for all my liiiiife, hold ooonnn, hold onnnn. You can choose to just clone this instead of grabbing the .zip, but I choose to do it this way, because 'Merica.

wget https://github.com/pagespeed/cpanel/archive/master.zip
unzip master.zip

The next step is to create a new file that will get around the RPM build errors you would get if you didn't do this step. Create the new file and add in the text below.

vim /etc/rpm/macros.apache2

%_httpd_mmn 20120211x8664
%_httpd_apxs /usr/bin/apxs
%_httpd_dir /etc/apache2
%_httpd_bindir %{_httpd_dir}/bin
%_httpd_modconfdir %{_httpd_dir}/conf.modules.d
%_httpd_confdir %{_httpd_dir}/conf.d
%_httpd_contentdir /usr/share/apache2
%_httpd_moddir /usr/lib64/apache2/modules

Now we need to cd into the directory that contains the source rpm for mod_pagespeed and rebuild it, then install it.

cd /root/cpanel-master/EA4
rpmbuild --rebuild ea-apache24-mod_pagespeed-latest-stable.src.rpm
rpm -ivh /root/rpmbuild/RPMS/x86_64/ea-apache24-mod_pagespeed-latest-stable.x86_64.rpm

Assuming the rebuild went well, we can simply restart Apache and the module should be loaded

/etc/init.d/httpd restart

To check if mod_pagespeed is loaded, use httpd -M and look for "pagespeed_module (shared)"

httpd -M

EasyApache 4 uses different locations for config files, if you want to modify the pagespeed configuration, modify this file

vim /etc/apache2/conf.modules.d/456_pagespeed.conf

Congratulations, you are now rocking'. Totally unrelated...I'm from America, which means I operate and own this wiki, and I can do whatever I want.

After drinking a few beers, I've decided that the song /video below should be the end of this section. Personally I find it fitting, and amusing. No offense meant to any non Americans (I love all people just the same) but seriously, if you didn't know how to get pagespeed working with EA4, I just saved your motherfucking day! If anything, I just want to make it clear that Americans can sometimes be useful ;)

Enabling mod_pagespeed Console and Admin UI

To enable the mod_pagespeed Console, Admin UI and statistics you can configure the pagespeed.conf file to look similar to what is shown below. You should enable logging and statistics if you want to have more information on what mod_pagespeed is doing, without these statistics it is hard to know if pagespeed is as effective as it can be. All of this information can be view able from http://domain.com/pagespeed_admin assuming you allow the correct IPs. I strongly suggest you use the "allow,deny" statement, if you do not do that then anyone could view your pagespeed stats.

ModPagespeedEnableFilters add_instrumentation
ModPagespeedMessageBufferSize 100000
ModPagespeedStatistics on
ModPagespeedStatisticsLogging on
ModPagespeedLogDir /var/log/pagespeed

<Location /mod_pagespeed_beacon>
      SetHandler mod_pagespeed_beacon
</Location>

<Location /mod_pagespeed_statistics>
    Order allow,deny
    Allow from localhost
    Allow from $IP_1
    Allow from $IP_2
    SetHandler mod_pagespeed_statistics
</Location>

<Location /pagespeed_admin>
  Order allow,deny
  Allow from localhost
  Allow from 127.0.0.1
  Allow from $IP_1
  Allow from $IP_2
  SetHandler pagespeed_admin
</Location>

Save the configuration file once you've added the content above, then restart apache. At this point you should be able to load the pagespeed_admin URL in your browser.

Mod_pagespeed Caches

There are a few different caches that mod_pagespeed can use to reduce your website's response time, these caches will have the most impact on your site's First Byte Response Time, which is the amount of time it takes your server to handle the request, process things like PHP and MySQL queries, generate HTML and send it back to the client. By utilizing in memory caches you can significantly reduce the amount of time it takes for your server to handle requests.

  • ModPagespeedFileCache -- The File Cache is a directory on disk that mod_pagespeed will use to store cached entries. If you have the LRU and SharedMetaData Cached enabled, the FileCache won't get used too often. Mainly it will be used to store large images that have been optimized by mod_pagespeed, anything over 1MB cannot be stored by Memcached.
  • ModPagespeedLRUCache -- This cache resides in each Apache or Nginx process. Each Process will have it's own small cache of Last Recently Used objects. These cached objects are typically very small and having them cached inside of the process means less latency per request. If something is not in the LRU cache, then mod_pagespeed will try to use the Shared Memory cache, check memcached, or check the File Cache before serving up a non cached request.
  • SharedMemoryMetadataCache -- This is a shared memory cache which can be used by all Apache or Nginx processes. As mod_pagespeed rewrites and optimizes things, it stores instructions on how it did this in the shared memory metadata cache. This cache can be used in addition to the LRU and File Cache.


FileCache

You can and should enable the FileCache for mod_pagespeed. This should be enabled by default, but if you notice that pagespeed isn't using the file cache or that Apache is complaining about something related to the FileCachePath you should view the main conf file and make sure the path is correctly set. You can use the syntax below to customize where the cache directory is / should be located. The path must be writable by the Apache user.

ModPagespeedFileCachePath  "$path_to_apache_writable_dir" 
ModPagespeedFileCacheSizeKb          102400 (100MB)
ModPagespeedFileCacheCleanIntervalMs 3600000 (3600 seconds)
ModPagespeedFileCacheInodeLimit      500000

For example, on a cPanel server you might find that the cache is located under /var/

ModPagespeedFileCachePath   "/var/mod_pagespeed/cache/"

If you have many vhosts on the same server and notice that Apache complains about "ModPagespeedFileCachePath must not be empty". You can try adding the ModPagespeedInheritVHostConfig option in the main pagespeed.conf file. This allows all vhosts to inherit some global settings without having to specify the same settings for each vhost.

ModPagespeedInheritVHostConfig          on
ModPagespeedFileCachePath               "/var/mod_pagespeed/cache/"

There are a few settings for the FileCache which can be adjusted based on your needs. These settings are only a suggestion to mod_pagespeed and are not a hard cap on the amount of space that the file cache can use. mod_pagespeed will scan the FileCache directory based on the value of ModPagespeedFileCacheCleanIntervalMs, if the cache has exceeded the size, or inode limit then the oldest cached files will be removed to make way for new items. If neither of the limits have been reached, nothing will happen and the FileCache will continue to grow until the next scan interval comes along. This means that it's possible for the FileCache to grow quickly in size and take up free space, so be careful when configuring the settings below.

  • ModPagespeedFileCacheSizeKb - It appears that the default value for the cache is 102400 KB, or 100 MB. If you have fast storage and plenty of storage space then you can raise this value to whatever makes sense. If you only have 50MB of content then a value of 100MB should be plenty, however if you have lots of images and content, raising this to a value of 1GB or more might make sense.
  • ModPagespeedFileCacheCleanIntervalMs - The values for this setting are in milliseconds, because Google wants to make things complicated. A value of 3600000 ms or 3600s or 1 hour is the default option. This means that once an hour the FileCache directory will be scanned and if it has reached the size limit, or inode limit the old cache entries will get removed to make way for new entries.
  • ModPagespeedFileCacheInodeLimit - The default limit for inodes in the FileCache is 500000 by default. Most of the time you should never have to mess around with this option unless you would rather think about inodes instead of file sizes.

So, let's say we have a server that has lots of storage space to spare, and hosts a single website with 1GB of files and content and we want to configure the FileCache to hold all the things. This is what the configuration for the FileCache would look like.

ModPagespeedFileCachePath  "/var/mod_pagespeed/cache/"
ModPagespeedFileCacheSizeKb          1024000 (1GB)
ModPagespeedFileCacheCleanIntervalMs 3600000 (3600 seconds)
ModPagespeedFileCacheInodeLimit      500000

Pretty simple right? Just make sure that the device that's storing the cache has enough free space. Now that the FileCache has been configured, we can move on to the next mod_pagespeed cache, which is even more awesome than the FileCache!

In-Memory LRU Cache

In addition to the FileCache, mod_pagespeed can be configured to allow for a small in-memory write-through LRU cache which runs in each Apache process. If you use Prefork, this could mean many processes, which could mean a lot of memory usage per process. You shouldn't use Prefork anyway because Event is way better, so I won't cover how to configure the LRU cache for Prefork, only for Event.

To figure out the best size of the LRUCache you should look at how you've configured Apache. Most of the time the Event MPM will have a ServerLimit of 16, which means that up to 16 Apache processes could run at the same time. If we set ModPagespeedLRUCacheKbPerProcess to 1024 (1MB) then under the worst case ontario the LRU cache would use 16MB total, (16 Apache Procs x 1MB). Each LRU Cache is limited to serving only the Apache process it was created in, the threads in that process can use the same cache, but threads from other processes will have to stick with their own cache (unlike the shared metadata cache which is covered next).

So, if we have a server with 1GB of memory, and we plan on keeping Apache's ServerLimit under 10, we could set the LRU cache size to 10240 (10MB) and if we had 10 apache processes running at all times the total LRU cache size would be 100MB between all 10 processes. Obviously if we set this to something crazy, like 100MB and we started up 10 Apache procs, the server could run out of memory quickly so be careful with these values.

The ModPagespeedLRUCacheByteLimit is the limit on how large a cache entry the LRU cache will accept. Unlike the ModPagespeedLRUCacheKbPerProcess value, this setting is in BYTES, not KB. A value of 16384 is 0.015 MB. Generally you want to keep the max size down low to avoid wasted space in the LRU cache. If you set this to say, 1MB then all the LRU caches might be constantly removing items like images because nothing else can fit in the LRU, this cache should be for small stuff, not large stuff.

If you are using Apache Event and aren't running tons of Apache processes, the settings below should be a good starting point. Usually even on an idle server there will be 4 - 5 apache processes running, so if you set each LRU cache to use 8MB you will be looking at an extra 40MB of memory usage, which is totally worth it imo.

ModPagespeedLRUCacheKbPerProcess   8192 (8MB Per Apache Process)
ModPagespeedLRUCacheByteLimit      16384 (16KB size limit per object in LRU)

If you are using cPanel and have statistics software such as awstats, analog, or webalizer, please make sure that you are not processing log files frequently or else the LRU caches will constantly be dropped because cPanel has to restart Apache to process the logs.

I noticed this by viewing the graphs that pagespeed admin provides. I suggest configuring statistics to run once or twice a day. You can modify "configure statistics process time schedule" under "statistics software configuration" in WHM. I suggest checking almost all of the boxes / times so that you are not processing logs during busy hours.

SharedMemory MetadataCache

In addition to the FileCache and per process LRU caches, you can customize yet another cache option for mod_pagespeed. This cache is the awesome cache because it uses shared memory which means that all the Apache processes can access the same cache, unlike the per process LRU caches (which are still good to have). When mod_pagespeed optimizes a resource, it will cache the information / steps it took to optimize that resource so that the next time the resource needs optimizing it already knows what to do. Since there is only one cache, we can set the size limit to 50MB or higher without having to worry about too many processes using memory.

To configure ModPagespeedCreateSharedMemoryMetadataCache you should specify the same path you used for "ModPagespeedFileCachePath" and as a second option, specify a size for the cache, the value here is in KB.

ModPagespeedCreateSharedMemoryMetadataCache "/var/cache/pagespeed/" 51200 (50MB)

How to install mod_pagespeed Apache module on Ubuntu 14.10

This is for Ubuntu 14.10, however the same applies for CentOS 7, just use replace .deb with .rpm and use rpm to install the package instead of dpkg.

wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_amd64.deb
dpkg -i mod-pagespeed-*.deb
apt-get -f install

After the module is installed, Apache should already have been restarted and the module loaded, to make sure this is the case you can run

apache2 -M


Ubuntu 14.10 mod_pagespeed configuration

To modify the default settings for mod pagespeed you can edit the main configuration file, which is called pagespeed.conf. If you are using Ubuntu, this can be found under the mods-enabled directory for apache. If you make any changes to pagespeed.conf you must restart apache or else the new settings will not take effect.

vim /etc/apache2/mods-enabled/pagespeed.conf

mod_pagespeed default CoreFilter list

By default, pagespeed will enable and use this list of filters, also know as the "core filters". Google considers all of these filters to be "safe" meaning that they shouldn't cause issues with most websites and CMS. I've found that the core filters work perfectly fine with WordPress and MediaWiki. If you wish to enable more filters you certainly can, but the list below contains the already enabled filters.

   add_head
   combine_css
   combine_javascript
   convert_meta_tags
   extend_cache
   fallback_rewrite_css_urls
   flatten_css_imports
   inline_css
   inline_import_to_link
   inline_javascript
   rewrite_css
   rewrite_images
   rewrite_javascript
   rewrite_style_attributes_with_url

How to configure mod_pagespeed to use Memcached

To configure mod_pagespeed to use Memcached simply add the line below, and optionally the lines below it to enable moar caches that can be shared by all the apache processes. Obviously this assumes you have Memcached installed on the server and that it is listening on an open port that is accessible via localhost. If you are using cpanel you will want to edit the pagespeed.conf file located at /usr/local/apache/conf/pagespeed.conf, if you are using Ubuntu the file will be located at /etc/apache2/mods-enabled/pagespeed.conf

##Memcached##

ModPagespeedMemcachedServers localhost:11211

##File Cache, Shared Memory Cache, and LRU Cache## 

ModPagespeedFileCachePath            "/var/mod_pagespeed/cache/"
ModPagespeedCreateSharedMemoryMetadataCache  "/var/mod_pagespeed/cache/" 51200
ModPagespeedFileCacheSizeKb          102400
ModPagespeedLRUCacheKbPerProcess   8192
ModPagespeedLRUCacheByteLimit      16384

You can view memcached stats by echoing stats and piping that into netcat.

echo stats | nc 127.0.0.1 11211

To view pagespeed specific memcached statistics you can simply curl the url from localhost if you have access to the server. This will display all of the pagespeed and memcached specific information

curl http://localhost/mod_pagespeed_statistics?memcached

How to bypass mod_pagespeed filters

By default, the module uses "CoreFilters", which is a safe set of rules that most sites benefit from, if you want to disable everything and only enable certain items, you can do this globally here, however it's recommended to just enable / disable things per vhost using .htaccess or vhost directive. You would just uncomment the line below

# ModPagespeedRewriteLevel PassThrough

How to include extra filters for mod_pagespeed

If you want to globally enable more filters than the core rules offer you can uncomment these lines and just add the filters you want to enable.

# ModPagespeedEnableFilters rewrite_javascript,rewrite_css
# ModPagespeedEnableFilters collapse_whitespace,elide_attributes

mod_pagespeed .htaccess directives

You can specify additional filters in .htaccess, or in the main apache2.conf, or on the vhost level using a directive like this:

<IfModule pagespeed_module>
ModPagespeed on
ModPagespeedEnableFilters remove_comments,rewrite_javascript,rewrite_css,rewrite_images
ModPagespeedEnableFilters elide_attributes,defer_javascript,move_css_to_head
ModPagespeedJpegRecompressionQuality -1
</IfModule>


Install PageSpeed Module (mod_pagespeed) on cPanel Server

The easiest way to install mod_pagespeed on a cPanel server is to use git to clone the CPanel / pagespeed project from github. I've tested this out on CentOS 6.5 and CentOS 6.6 with WHM 11.48, and have never noticed any issues with the plugin, or with this installation method.

/usr/local/cpanel/3rdparty/bin/git clone https://github.com/pagespeed/cpanel.git /tmp/pagespeed/

After you've cloned the project using git, cd into /tmp/pagespeed/Easy and create a tarball that EasyApache can use to install the module when you run future EasyApache.

cd /tmp/pagespeed/Easy
tar -zcvf Speed.pm.tar.gz pagespeed
mkdir -p /var/cpanel/easy/apache/custom_opt_mods/Cpanel/Easy
mv Speed.pm Speed.pm.tar.gz -t /var/cpanel/easy/apache/custom_opt_mods/Cpanel/Easy/
cd && rm -rf /tmp/pagespeed

At this point you should be able to run EasyApache, and when you go to customize profile, there should be a "mod_pagespeed" entry under the Apache module section. Make sure this is selected / checked and then tell EasyApache to save and build with the new profile. This can take up to 20 minutes to complete, but all your websites should remain online while this process occurs.

/scripts/easyapache

Now you can check to make sure the EA finished successfully and the module is enabled:

httpd -M

##You should see
 pagespeed_module (shared)

Link: https://github.com/pagespeed/cpanel

How to use Memcached with ModPageSpeed on a cPanel Server

If you are using cPanel and have mod_pagespeed installed you can modify the configuration file so that mod_pagespeed uses memcached to store all kinds of awesome things. In this case I have Memcached running on the same host. You will need to start Apache after you are done editing the file.

vim /usr/local/apache/conf/pagespeed.conf

##Add These Lines
  ModPagespeedMemcachedServers 127.0.0.1:11211

mod_pagespeed issue: Unicode value 0x1a is not valid for interchange

If you notice broken CSS / webpages not loading correctly, specifically with MediaWiki CMS, check your apache or nginx error logs for a message that looks like:

[Tue Jan 26 13:49:44.435131 2016] [pagespeed:warn] [pid 7158:tid 140195164837632] [mod_pagespeed 1.10.33.2-7599 @7158] [0126/134944:WARNING:unicodetext.cc(360)] Unicode value 0x1a is not valid for interchange

I've encountered this issue a few times with mod_pagespeed versions 1.10.33.2-beta and 1.9.32.11-stable. The issue randomly showed up on certain pages, but not all pages, sometimes the page CSS looked fine, other times not.

Currently I use this pagespeed configuration, which seems to alleviate the issue, but doesn't entirely resolve the problem. I've disabled most of the CSS filters for the time being, as the pagespeed_admin messages seemed to indicate that parsing / combining CSS is not possible. You may need to enable the pagespeed_admin section and allow access to it, the admin section provides a ton of useful graphs, stats and displays more detailed messages than what you find in the apache error logs.

ModPagespeedFetchWithGzip on
SetOutputFilter DEFLATE
ModPagespeedFetchHttps enable
ModPagespeedDisableFilters combine_css,flatten_css_imports,inline_css,rewrite_css

The issues doesn't appear to be related to any specific version of apache or nginx, and the issue also seems to be in multiple versions of pagespeed, even the beta version. I'll continue to update this section as I troubleshoot!

For more info on this specific issue, check out these related pagespeed issues:

Additional Links


  • PageSpeed can be added to any server runtime and applied dynamically to any application. This is available as a module for Apache and Nginx. This helps to optimize resources based on a lot of "web optimization filters".

.