Mar 012013
 

Today, we’ll be setting up a base CentOS 5.9 server with nginx, php-fpm, mySQL, and extracting a cPanel account into it. This is useful if you have a single website you need to migrate from one provider into a dedicated environment, and you need it to run as predictably as possible with as little RAM as possible. Let’s get started by updating the operating system. Then we’ll proceed to setting up each individual service before extracting the backup into them all at once.

1. Update the Operating System

The first thing to do is make sure you’re running your latest-and-greatest before you open yourself up to the public.

yum update -y

Review the packages you just installed. If you see a kernel, reboot the system before continuing.

Next, let’s add the nginx repo. That just makes things easier, we probably don’t need to manually compile nginx.

rpm -Uvh http://nginx.org/packages/centos/5/noarch/RPMS/nginx-release-centos-5-0.el5.ngx.noarch.rpm

We also need the EPEL and IUS repositories in order to get some more updated PHP binaries as well as php-fpm.

rpm -Uvh http://mirrors.servercentral.net/fedora/epel/5/i386/epel-release-5-4.noarch.rpm
rpm -Uvh http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/ius-release-1.0-10.ius.el5.noarch.rpm

2. Install and Configure nginx

And now, let’s install (and start) nginx.

yum -y install nginx
/etc/init.d/nginx start

At this point, you should be able to go to your server’s main IP address and see the nginx default page.

3. Install mySQL

Let’s install mySQL next, so we have the client and server set up before we try to install PHP to connect to it.

yum install mysql-server

Once you’ve installed the package, you still need to configure the mysql server:

/etc/init.d/mysqld start
mysql_secure_installation

Answer some questions, and then create the file /root/.my.cnf in your favorite text editor. Add the following lines:

[client]
user=root
pass=Your new root password

This will allow you to connect to mySQL from the root user command line without specifying the mySQL root password.

4. Install PHP/php-fpm

Time to install PHP! At a minimum, you need the following packages:

yum install php53u php53u-mysql php53u-fpm

A more complete installation that would more closely mirror a typical cPanel server will look something like this:

yum install php53u php53u-fpm php53u-mysql php53u-gd php53u-mbstring php53u-devel php53u-mcrypt php53u-imap php53u-pear php53u-pdo php53u-xml php53u-soap php53u-ioncube-loader

And finally, start the php-fpm listener:

/etc/init.d/php-fpm start

5. Configure nginx to use php-fpm

Now it’s time to configure your nginx configuration file to (securely!) pass PHP files to nginx. This information comes via this blog entry, which contains some nifty information on why other blog entries on how to set up nginx and php-fpm leave you open for a really nasty attack. I’ve made a lot of modifications to the code though, so don’t copy directly from that blog, as it may not work. I found that I needed to set “root” under the PHP location as well to get the path info working.
First, open your /etc/nginx/conf.d/default.conf file in your favorite text editor. You’ll likely see the following:

# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}

I like to leave the default configuration example/comments intact in these files so I have something to refer back to if I break my server, so just add some empty lines below those comments and add the following:

# Pass all .php files onto a php-fpm/php-fcgi server.
location ~ \.php$ {
root /home/user/public_html;

fastcgi_split_path_info ^(.+\.php)(/.+)$;
include fastcgi_params;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;
# fastcgi_intercept_errors on;
add_header request $request_filename;
if (-f $request_filename) {
fastcgi_pass 127.0.0.1:9000;
}
}

You’re also going to want to uncomment the lines below that in the default file:

location ~ /\.ht {
deny all;
}

At this point you can make any other changes you want, including changing the “root” location (your document root; I recommend setting the same location you had in cPanel), your listen IP/port, your server_name, your error_pages, your “index” (you probably want to add “index.php” here), etc. At this point, you should also review /etc/php-fpm.conf and /etc/php-fpm.d/www.conf. The latter file especially impacts your permissions, as you will set the user PHP runs as on this. It might be a good idea to create a system user named the same thing as your cPanel user, and make sure your files are chowned and chmoded appropriately.

6. Extract your cPanel backup

You’re halfway there! It’s time to start picking apart the elements of a cPanel backup file. Let’s look at an extracted cpmove backup file.

addons homedir.tar mysql/ resellerpackages/ userconfig/
bandwidth/ httpfiles/ mysql.sql sds userdata/
counters/ interchange/ mysql-timestamps/ sds2 va/
cp/ locale/ nobodyfiles shadow vad/
cron/ logaholic/ pds shell version
dnszones/ logs/ proftpdpasswd sslcerts/ vf/
domainkeys/ meta/ psql/ ssldomain/
fp/ mm/ quota sslkeys/
homedir/ mma/ resellerconfig/ suspended/
homedir_paths mms/ resellerfeatures/ suspendinfo/

Ouch! Okay, let’s just mess with what’s absolutely essential to get this up and running.

mv homedir.tar homedir/
cd homedir/
tar -xf homedir.tar
ls

Now this looks a little more familiar, right?

bin/ cpmove.psql/ homedir.tar php/ tmp/
cpanel3-skel/ cpmove.psql.1339066964/ mail/ public_ftp/ www/
cpbackup-exclude.conf etc/ perl5/ public_html/

Your directory structure may vary somewhat. The most important thing to us here is that public_html directory. Move it to wherever you want your document_root to be and set the ownership properly.

mv public_html /home/user/
chown -R user:user /home/user/public_html

Now it’s time to import the mySQL databases and users. Users are stored in the “mysql.sql” file at the root of the backup; individual databases are stored in the mysql directory.

cd ../
mysql < mysql.sql
for i in `ls | grep .sql | grep -v horde | grep -v roundcube | cut -d\. -f1`; do mysql < $i.create; mysql $i < $i.sql;done

6. Manual configs

Okay, now we're into some more dicey stuff this tutorial won't cover. You should check your backup file for things like PostgreSQL, cron jobs, SSL certificates, restoring log files/webalizer information, DNS, and e-mail. We might cover that in a later tutorial. At this point, however, you should have a fully-functional website! Just host DNS elsewhere, point it at your new server, and go!

Aug 032012
 

Apache’s mod_security is a system that scans URL data and data POSTed to the server in forms for malicious attacks such as XSS, SQL injections, etc. It comes with a default ruleset that always trips up a number of standard AJAX functions in applications like WordPress and Joomla, necessitating that rules be removed from the mod_security configuration to allow the site to function properly. The method of removing these rules under Apache 2 is by whitelisting the individual rules in the VHost entry for the domain. Here’s how to do it in cPanel.

Is mod_security enabled?

root@server [~]# httpd -M | grep sec
security2_module (shared)

Yep.

A typical mod_security error message

[Sun Jul 29 18:29:47 2012] [error] [client 1.2.3.4] ModSecurity: Access denied with code 403 (phase 2). Match of "rx (?:/event\\\\.ng/|horde/services/go\\\\.php|tiki-view_cache\\\\.php|^/\\\\?out=http://|homecounter\\\\.php\\\\?offerid=.*ureferrer=http|__utm\\\\.gif\\\\?|/plugins/wpeditimage/editimage\\\\.html|/spc\\\\.php)" against "REQUEST_URI" required. [file "/usr/local/apache/conf/modsec/50_asl_rootkits.conf"] [line "53"] [id "390145"] [rev "10"] [msg "Atomicorp.com UNSUPPORTED DELAYED Rules: Rootkit attack: Generic Attempt to install rootkit"] [data "=http://www.hackers-web-site.com/what/e.txt?"] [severity "CRITICAL"] [hostname "www.domain.com"] [uri "/index.php"] [unique_id "UBVB02B-nBoACs9n0p4AAAAH"]

The error log for mod_security is /usr/local/apache/logs/error_log. There are 2 important pieces of information contained in this error:

  • [id "390145"]
    This is the mod_security rule ID number. We will use this to tell mod_security which rule we want to whitelist.

  • [hostname "www.domain.com"]
    This is the name of the VirtualHost under which the error was generated. This is important because whitelisting a mod_security rule for a domain doesn’t whitelist it for the whole account; subdomains, addon domains, etc all have separate whitelists.

Creating the userdata includes

mkdir -p /usr/local/apache/conf/userdata/std/2/user/domain.com

root@server [~]# mkdir -p /usr/local/apache/conf/userdata/std/2/user/domain.com

In the above example, we are setting up an Inside VHost include. The italic std indicates this is being done in a port 80 (standard/non-secure) VirtualHost, the bold user is the cPanel username of the account, and the bold and italic domain.com is the hostname from the error message above (we discard www because that’s just a ServerAlias under the same VHost). Once we create the userdata include directory, it’s time to set up the configuration file to include. We’ll use nano because it’s the best text editor.

nano -w /usr/local/apache/conf/userdata/std/2/user/domain.com/domain.com.conf

root@server [~]# /usr/local/apache/conf/userdata/std/2/user/domain.com/domain.com.conf

You can name the file anything as long as it ends in .conf; I prefer to use domain.com.conf for clarity and simplicity. Now let’s whitelist some mod_security rules.

SecRuleRemoveById 390145

Notice that we used 390145 from the error message above. If this site was generating errors from multiple IDs, we’d put in multiple SecRuleRemoveById lines.

Enabling the userdata includes in Apache conf

/scripts/ensure_vhost_includes --user=user

This uncomments a line in the VirtualHost entry in httpd.conf for this domain to include any .conf files in /usr/local/apache/conf/userdata/std/2/user/domain.com/

/usr/local/cpanel/bin/apache_conf_distiller --update
/usr/local/cpanel/bin/build_apache_conf

These compile and distill the changes into Apache’s configuration.

Testing

The first thing to check is to ensure that the following line in the VirtualHost entry for domain.com is uncommented:

Include "/usr/local/apache/conf/userdata/std/2/user/domain.com/*.conf"

The only other way to test this is to reproduce the issue and tail the Apache error log. Make sure you have reproduction instructions available; many scripts will trigger more than one SecRule, but you’ll only see the first one due to the request being blocked.

Apr 282012
 

It’s been a long time coming, but here’s my guide to install the latest ffmpeg and associated packages for a video upload site on base CentOS, no cPanel required (if Google got you here and you have cPanel, try the cPanel ffmpeg guide). This article assumes you’ve already set up your HTTP daemon (Apache, nginx, lighttpd, litespeed, etc), scripting agent (PHP), and database provider (mySQL, PostgreSQL, etc). The commands in this guide assume you are running the latest CentOS 5.8 64-bit. Some modifications may be necessary for other architectures and versions of CentOS.

  1. Make sure you’re up to date
  2. Make sure you’re on the latest CentOS branch. We’re installing the latest version of ffmpeg, and if you’re not running the latest version of your CentOS branch, you may run into unexpected issues.

    yum update

    Install whatever is required, and compatible with your other software. If you update your kernel, reboot before continuing.

  3. Extra Repositories
  4. RPMForge is a useful, fully-compatible extra repository. We will use it for a number of packages that don’t need to be bleeding-edge. For more information, see the CentOS wiki.

    wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
    rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
    rpm -K rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
    rpm -i rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm

  5. Development Tools
  6. You must have the required packages to download and compile the source code.

    yum groupinstall "Development Tools"
    yum install subversion git

  7. Install RPM libraries
  8. Some libraries don’t need to be bleeding-edge. We’ll install those now.

    yum install gettext-devel expat-devel curl-devel zlib-devel openssl-devel libXext libXext-devel flvtool2

  9. ldconfig
  10. We need to make sure ldconfig is checking the correct directories for libraries.

    echo /usr/local/lib >> /etc/ld.so.conf; ldconfig

  11. Download libraries
  12. Okay, here’s the big one. It’s probably a good idea to go ahead and copy+paste this whole code block into your terminal at once to save time. One note; the official distribution release of ffmpeg-php no longer compiles. The SVN version available here works, but the SVN tarball link here can’t easily be downloaded in a terminal session, so I’ve rehosted the file here on this server. This is not my file, I haven’t modified it, and I take no responsibility for it – it’s merely a copy of the file available from the above link.

    cd /usr/local/src
    wget http://iweb.dl.sourceforge.net/project/lame/lame/3.99/lame-3.99.5.tar.gz
    tar zxf lame-3.99.5.tar.gz
    wget http://hasaninter.net/ffmpeg-php.tar.gz
    tar xzf ffmpeg-php.tar.gz
    wget http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.3.tar.gz
    tar xzf libvorbis-1.3.3.tar.gz
    wget http://downloads.xiph.org/releases/ogg/libogg-1.3.0.tar.gz
    tar xzf libogg-1.3.0.tar.gz
    wget http://iweb.dl.sourceforge.net/project/opencore-amr/opencore-amr/opencore-amr-0.1.3.tar.gz
    tar xzf opencore-amr-0.1.3.tar.gz
    wget http://downloads.xiph.org/releases/theora/libtheora-1.1.1.tar.bz2
    tar xjf libtheora-1.1.1.tar.bz2
    wget http://downloads.xvid.org/downloads/xvidcore-1.3.2.tar.gz
    tar zxf xvidcore-1.3.2.tar.gz
    wget http://downloads.sourceforge.net/faac/faad2-2.7.tar.gz
    tar zxf faad2-2.7.tar.gz
    wget http://downloads.sourceforge.net/faac/faac-1.28.tar.gz
    tar zxf faac-1.28.tar.gz
    mkdir /usr/local/lib/codecs
    wget ftp://ftp.mplayerhq.hu/MPlayer/releases/codecs/all-20110131.tar.bz2
    tar -jxf all-20110131.tar.bz2
    wget http://www.tortall.net/projects/yasm/releases/yasm-1.2.0.tar.gz
    tar zxf yasm-1.2.0.tar.gz
    cp all-20110131/* /usr/local/lib/codecs/
    chmod -R 755 /usr/local/lib/codecs/
    mkdir /usr/local/src/tmp
    chmod 777 /usr/local/src/tmp
    export TMPDIR=/usr/local/src/tmp
    svn co https://gpac.svn.sourceforge.net/svnroot/gpac/trunk/gpac gpac
    git clone git://git.videolan.org/x264.git
    git clone git://git.videolan.org/ffmpeg.git ffmpeg
    svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer

  13. Compile everything
  14. Here we go! This is the hard part, where we compile bleeding-edge libraries. At the time this was posted, this all worked. If you run into any errors, try Googling them. If you still can’t figure it out (or more importantly, if you do!) please comment here and I’ll edit this post with the latest fixes.

    1. Lame

    2. cd /usr/local/src/lame-3.99.5/
      ./configure
      make
      make install

    3. Libogg

    4. cd /usr/local/src/libogg-1.3.0
      ./configure
      make
      make install

    5. Libvorbis

    6. cd /usr/local/src/libvorbis-1.3.3
      ./configure
      make
      make install

    7. yasm

    8. cd /usr/local/src/yasm-1.2.0
      ./configure
      make
      make install

    9. Libxvid

    10. cd /usr/local/src/xvidcore/build/generic
      ./configure
      make
      make install

    11. Libx264

    12. cd /usr/local/src/x264
      ./configure --enable-shared
      make
      make install

    13. Opencore-amr

    14. cd /usr/local/src/opencore-amr-0.1.3
      ./configure
      make
      make install

    15. Libtheora

    16. cd /usr/local/src/libtheora-1.1.1
      ./configure
      make
      make install

    17. Fadd2

    18. cd /usr/local/src/faad2-2.7
      ./configure
      make
      make install

    19. Faac

    20. cd /usr/local/src/faac-1.28
      ./configure
      make
      make install

    21. MP4Box

    22. cd /usr/local/src/gpac
      ./configure
      make
      make install

    23. mplayer

    24. cd /usr/local/src/mplayer/
      ./configure --enable-jpeg
      make
      make install

    25. ffmpeg

    26. cd /usr/local/src/ffmpeg
      ./configure --enable-libmp3lame --enable-libvorbis --enable-shared --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-nonfree --enable-libtheora --enable-version3 --enable-gpl --enable-libxvid
      make
      make install

    27. ffpmeg-php

    28. cd /usr/local/src/ffmpeg-php
      phpize
      ./configure
      make
      make install

    29. php.ini
    30. If you’re adding in the ffmpeg module to PHP (which this guide assumes you are), you need to add the extension to PHP. Again, this guide assumes you’re using PHP on Apache with CentOS 5.

      echo "extension=ffmpeg.so" > /etc/php.d/ffmpeg.ini
      /etc/init.d/httpd restart

  15. ldconfig
  16. To ensure that all of the libraries are loaded and linkable by ffmpeg, etc, run:

    ldconfig

That’s it! There are a few tests you can run to ensure that this is actually installed properly. Here are the two most important.


php -m | grep ffmpeg
ffmpeg

Apr 132012
 

Need to run 2 versions of memcached? I have no idea why you’d need to do this, but I had a request for it tonight, and all of the Google results for it are terrible. Here’s what you need to know.

  1. /etc/sysconfig/memcached2

  2. PORT="11212"
    USER="memcached"
    MAXCONN="8000"
    CACHESIZE="1024"
    OPTIONS=""

  3. /etc/init.d/memcached2

  4. #! /bin/sh
    #
    # chkconfig: - 55 45
    # description: The memcached daemon is a network memory cache service.
    # processname: memcached
    # config: /etc/sysconfig/memcached
    # pidfile: /var/run/memcached/memcached.pid

    # Standard LSB functions
    #. /lib/lsb/init-functions

    # Source function library.
    . /etc/init.d/functions

    PORT=11212
    USER=memcached
    MAXCONN=1024
    CACHESIZE=64
    OPTIONS=""

    if [ -f /etc/sysconfig/memcached2 ];then
    . /etc/sysconfig/memcached2
    fi

    # Check that networking is up.
    . /etc/sysconfig/network

    if [ "$NETWORKING" = "no" ]
    then
    exit 0
    fi

    RETVAL=0
    prog="memcached"

    start () {
    echo -n $"Starting $prog: "
    # insure that /var/run/memcached has proper permissions
    if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then
    chown $USER /var/run/memcached
    fi

    daemon --pidfile /var/run/memcached/memcached2.pid memcached -d -p $PORT -u $USER -m $CACHESIZE -c $MAXCONN -P /var/run/memcached/memcached2.pid $OPTIONS
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/memcached2
    }
    stop () {
    echo -n $"Stopping $prog: "
    killproc -p /var/run/memcached/memcached2.pid /usr/bin/memcached
    RETVAL=$?
    echo
    if [ $RETVAL -eq 0 ] ; then
    rm -f /var/lock/subsys/memcached2
    rm -f /var/run/memcached2.pid
    fi
    }

    restart () {
    stop
    start
    }

    # See how we were called.
    case "$1" in
    start)
    start
    ;;
    stop)
    stop
    ;;
    status)
    status memcached
    ;;
    restart|reload|force-reload)
    restart
    ;;
    condrestart)
    [ -f /var/lock/subsys/memcached2 ] && restart || :
    ;;
    *)
    echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
    exit 1
    esac

    exit $?

  5. Configure the service to run on start-up

  6. chmod +x /etc/init.d/memcached2
    chkconfig memcached2 on
    /etc/init.d/memcached2

    That’s it!

Apr 132012
 

Google’s mod_pagespeed is an Apache module for performance optimization. It can be used to increase the speed of pages served through Apache. This module is compatible with cPanel and WHM’s Apache setup, and it’s pretty easy to install.

  1. Download the correct mod_pagespeed RPM for your architecture.
  2. mod_pagespeed has RPMs available for 32-bit and 64-bit systems. First, let’s download the RPM files.
    For 32-bit:

    cd /usr/local/src
    mkdir mod_pagespeed
    cd mod_pagespeed
    wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-beta_current_i386.rpm

    For 64-bit:

    cd /usr/local/src/
    mkdir mod_pagespeed
    wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-beta_current_x86_64.rpm

  3. Extract the necessary files.
  4. I’m going to assume a 64-bit system from here on out. If you’re on a 32-bit system, the extracted files exist in slightly different paths; adjust accordingly.

    rpm2cpio mod-pagespeed-beta_current_x86_64.rpm | cpio -idmv
    cp usr/lib64/httpd/modules/mod_pagespeed.so /usr/local/apache/modules/
    chmod 755 /usr/local/apache/modules/mod_pagespeed.so

  5. Create pagespeed directories.

  6. mkdir -p /var/mod_pagespeed/{cache,files}
    chown nobody:nobody /var/mod_pagespeed/*

  7. Enable mod_deflate (required for mod_pagespeed).
  8. The location of your httpd source directory will vary depending on the version of Apache you have installed. The quickest way to find the correct location is tab completion.

    /usr/local/apache/bin/apxs -c -i /home/cpeasyapache/src/httpd-[tab]/modules/filters/mod_deflate.c

  9. Set up your mod_pagespeed configuration.
  10. Add the following to the Apache configuration files. The simplest way to do this is to create a new configuration file called /usr/local/apache/conf/pagespeed.conf and then include that using the cPanel include files. Place the following in pagespeed.conf:

    LoadModule pagespeed_module modules/mod_pagespeed.so

    # Only attempt to load mod_deflate if it hasn't been loaded already.
    <IfModule !mod_deflate.c>
    LoadModule deflate_module modules/mod_deflate.so
    </IfModule>
    <IfModule pagespeed_module>
    # Turn on mod_pagespeed. To completely disable mod_pagespeed, you
    # can set this to "off".
    ModPagespeed on

    # Direct Apache to send all HTML output to the mod_pagespeed
    # output handler.
    AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html

    # The ModPagespeedFileCachePath and
    # ModPagespeedGeneratedFilePrefix directories must exist and be
    # writable by the apache user (as specified by the User
    # directive).
    ModPagespeedFileCachePath "/var/mod_pagespeed/cache/"
    ModPagespeedGeneratedFilePrefix "/var/mod_pagespeed/files/"

    # Override the mod_pagespeed 'rewrite level'. The default level
    # "CoreFilters" uses a set of rewrite filters that are generally
    # safe for most web pages. Most sites should not need to change
    # this value and can instead fine-tune the configuration using the
    # ModPagespeedDisableFilters and ModPagespeedEnableFilters
    # directives, below. Valid values for ModPagespeedRewriteLevel are
    # PassThrough, CoreFilters and TestingCoreFilters.
    #
    # ModPagespeedRewriteLevel PassThrough

    # Explicitly disables specific filters. This is useful in
    # conjuction with ModPagespeedRewriteLevel. For instance, if one
    # of the filters in the CoreFilters needs to be disabled for a
    # site, that filter can be added to
    # ModPagespeedDisableFilters. This directive contains a
    # comma-separated list of filter names, and can be repeated.
    #
    # ModPagespeedDisableFilters rewrite_images

    # Explicitly enables specific filters. This is useful in
    # conjuction with ModPagespeedRewriteLevel. For instance, filters
    # not included in the CoreFilters may be enabled using this
    # directive. This directive contains a comma-separated list of
    # filter names, and can be repeated.
    #
    # ModPagespeedEnableFilters rewrite_javascript,rewrite_css
    # ModPagespeedEnableFilters collapse_whitespace,elide_attributes

    # ModPagespeedDomain
    # authorizes rewriting of JS, CSS, and Image files found in this
    # domain. By default only resources with the same origin as the
    # HTML file are rewritten. For example:
    #
    # ModPagespeedDomain cdn.myhost.com
    #
    # This will allow resources found on http://cdn.myhost.com to be
    # rewritten in addition to those in the same domain as the HTML.
    #
    # Wildcards (* and ?) are allowed in the domain specification. Be
    # careful when using them as if you rewrite domains that do not
    # send you traffic, then the site receiving the traffic will not
    # know how to serve the rewritten content.

    # Other defaults (cache sizes and thresholds):
    #
    # ModPagespeedFileCacheSizeKb 102400
    # ModPagespeedFileCacheCleanIntervalMs 3600000
    # ModPagespeedLRUCacheKbPerProcess 1024
    # ModPagespeedLRUCacheByteLimit 16384
    # ModPagespeedCssInlineMaxBytes 2048
    # ModPagespeedImageInlineMaxBytes 2048
    # ModPagespeedCssImageInlineMaxBytes 2048
    # ModPagespeedJsInlineMaxBytes 2048
    # ModPagespeedCssOutlineMinBytes 3000
    # ModPagespeedJsOutlineMinBytes 3000

    # Bound the number of images that can be rewritten at any one time; this
    # avoids overloading the CPU. Set this to 0 to remove the bound.
    #
    # ModPagespeedImageMaxRewritesAtOnce 8

    # Settings for image optimization:
    #
    # Jpeg recompression quality (0 to 100, -1 strips metadata):
    # ModPagespeedJpegRecompressionQuality -1
    #
    # Percent of original image size below which optimized images are retained:
    # ModPagespeedImageLimitOptimizedPercent 100
    #
    # Percent of original image area below which image resizing will be
    # attempted:
    # ModPagespeedImageLimitResizeAreaPercent 100

    # When Apache is set up as a browser proxy, mod_pagespeed can record
    # web-sites as they are requested, so that an image of the web is built up
    # in the directory of the proxy administrator's choosing. When ReadOnly is
    # on, only files already present in the SlurpDirectory are served by the
    # proxy.
    #
    # ModPagespeedSlurpDirectory ...
    # ModPagespeedSlurpReadOnly on

    # The maximum URL size is generally limited to about 2k characters
    # due to IE: See http://support.microsoft.com/kb/208427/EN-US.
    # Apache servers by default impose a further limitation of about
    # 250 characters per URL segment (text between slashes).
    # mod_pagespeed circumvents this limitation, but if you employ
    # proxy servers in your path you may need to re-impose it by
    # overriding the setting here. The default setting is 1024
    # characters.
    #
    # ModPagespeedMaxSegmentLength 250

    # Uncomment this if you want to prevent mod_pagespeed from combining files
    # (e.g. CSS files) across paths
    #
    # ModPagespeedCombineAcrossPaths off

    # Explicitly tell mod_pagespeed to load some resources from disk.
    # This will speed up load time and update frequency.
    #
    # This should only be used for static resources which do not need
    # specific headers set or other processing by Apache.
    #
    # Both URL and filesystem path should specify directories and
    # filesystem path must be absolute (for now).
    #
    # ModPagespeedLoadFromFile "http://example.com/static/" "/var/www/static/"

    # Enables server-side instrumentation and statistics. If this rewriter is
    # enabled, then each rewritten HTML page will have instrumentation javacript
    # added that sends latency beacons to /mod_pagespeed_beacon. These
    # statistics can be accessed at /mod_pagespeed_statistics. You must also
    # enable the mod_pagespeed_statistics and mod_pagespeed_beacon handlers
    # below.
    #
    # ModPagespeedEnableFilters add_instrumentation

    # Uncomment the following line so that ModPagespeed will not cache or
    # rewrite resources with Vary: in the header, e.g. Vary: User-Agent.
    # ModPagespeedRespectVary on

    # This handles the client-side instrumentation callbacks which are injected
    # by the add_instrumentation filter.
    # You can use a different location by adding the ModPagespeedBeaconUrl
    # directive; see the documentation on add_instrumentation.
    <Location /mod_pagespeed_beacon>
    SetHandler mod_pagespeed_beacon
    </Location>

    # Uncomment the following line if you want to disable statistics entirely.
    #
    # ModPagespeedStatistics off

    # This page lets you view statistics about the mod_pagespeed module.
    <Location /mod_pagespeed_statistics>
    Order allow,deny
    # You may insert other "Allow from" lines to add hosts you want to
    # allow to look at generated statistics. Another possibility is
    # to comment out the "Order" and "Allow" options from the config
    # file, to allow any client that can reach your server to examine
    # statistics. This might be appropriate in an experimental setup or
    # if the Apache server is protected by a reverse proxy that will
    # filter URLs in some fashion.
    Allow from localhost
    Allow from 127.0.0.1
    SetHandler mod_pagespeed_statistics
    </Location>

    # Page /mod_pagespeed_message lets you view the latest messages from
    # mod_pagespeed, regardless of log-level in your httpd.conf
    # ModPagespeedMessageBufferSize is the maximum number of bytes you would
    # like to dump to your /mod_pagespeed_message page at one time,
    # its default value is 100k bytes.
    # Set it to 0 if you want to disable this feature.

    ModPagespeedMessageBufferSize 100000

    # mod_pagespeed has the ability to collect statistics about page visits as
    # well as page, resource, and div location (see div_structure_filter)
    # referrals. These will eventually be used to speed up pages with at least
    # resource pre-fetch, if not Chrome's new pre-render, technology. See
    # googlewebmastercentral.blogspot.com/2011/06/announcing-instant-pages.html
    # We recommend enabling the div_structure filter if turning on statistics
    # collection below. Enabling the div_structure filter will increase the
    # effectiveness of pre-rendering prediction, because it will take into
    # account both URLs and page locations when aggregating user click through
    # behavior. To enable the div_structure filter, uncomment the appropriate
    # line below or add div_structure to the enabled filters at the top of this
    # configuration file.
    # Page /mod_pagespeed_referer_statistics lets you view the accumulated
    # referral statistics.
    # ModPagespeedCollectRefererStatistics on enables collection (default off)
    # ModPagespeedHashRefererStatistics obscures collected info (default off)
    # ModPagespeedRefererStatisticsOutputLevel can be changed if the page
    # /mod_pagespeed_referer_statistics is slow to load:
    # - Organized (default) is the most readable and ordered logically, but
    # involves computation
    # - Simple is readable but unordered
    # - Fast is the fastest and contains all necessary information, but is
    # fairly unreadable

    # ModPagespeedCollectRefererStatistics on
    # ModPagespeedHashRefererStatistics on
    # ModPagespeedRefererStatisticsOutputLevel Simple
    # ModPagespeedEnableFilters div_structure

    <Location /mod_pagespeed_message>
    Allow from localhost
    Allow from 127.0.0.1
    SetHandler mod_pagespeed_message
    </Location>
    <Location /mod_pagespeed_referer_statistics>
    Allow from localhost
    Allow from 127.0.0.1
    SetHandler mod_pagespeed_referer_statistics
    </Location>
    </IfModule>

    And then open /usr/local/apache/conf/includes/pre_main_global.conf and add:

    Include conf/pagespeed.conf

  11. Rebuild Apache config and restart apache.

  12. /scripts/buildhttpdconf
    /etc/init.d/httpd restart

That’s it!

May 072011
 

By default, cPanel sets up a paltry <512MB /tmp disk. This isn't nearly large enough for large file uploads or other disk-intensive purposes. While it's better to use RAM for temporary storage, sometimes you need a place to dump huge files (such as uploads). Luckily, raising the /tmp disk size has been fairly simple. cPanel's script to secure the /tmp partition against drop-in hacks by making it noexec can also resize the /tmp directory.

The file we'll be modifying is:


/scripts/securetmp

Until recently, modifying one variable was enough to change this, but it seems like there was a change recently that has caused that method to no longer remount /tmp properly. Luckily, the fix for this is two additional small changes to the file.

Let's open up /scripts/securetmp in your favorite editor:


nano /scripts/securetmp

First, we're going to modify line 49:


my $auto = 1

If this isn't already set to 1, set it. Just makes things easier. Next, let's set the /tmp size, line 148:


my $tmpdsksize = 2097152;

This size is in KB - 2GB aught to do it. Now, to fix the issue of mounting /tmp, line 289:


system 'mount', '-o', $mountkeyword . ',loop,noexec,nosuid', $tmpmnt, '/tmp';

We're adding "loop," to the options passed to the mount command to ensure that the system understands /tmp is a loopback device being created on /usr/tmpDSK. Save and exit your file.

Next, we need to shut off anything using /tmp:


/etc/init.d/mysql stop
/etc/init.d/httpd stop

And unmount it and /var/tmp:


umount /tmp
umount /var/tmp

If you get errors, retry a few times, it'll usually unmount after the 2nd or 3rd try. If you're still getting errors, make sure nothing is open in /tmp:


lsof | grep /tmp

Shut it down or delete it. Next, we need to remove the existing /tmp partition file:


rm -f /usr/tmpDSK

And finally, create the new device:


/scripts/securetmp

Depending on the size of your partition, this may take up to 15-20 minutes. After you're done, start everything back up and ensure /tmp is mounted and the right size with a simple:


df -h

Apr 082011
 

mySQL is the database backbone behind a wide variety of software programs and scripts, but most people don’t know how to code for it properly. As such, mySQL-related overloads and optimization of sub-par mySQL databases can take a lot of time. The easiest way to track and troubleshoot troublesome threads is with mtop – live mySQL query querying.

Prerequisites

mtop requires curses. In CentOS, this is as simple as a few yum commands. There are a few extra steps for cPanel compatibility.

  1. Install curses and curses-devel:

    yum install curses curses-devel
  2. Install curses-Perl:

    yum install curses-Perl

    1. If you’re on cPanel, you’ll first need to remove the perl* exclude from yum.conf.

      nano /etc/yum.conf

      In the exclude= line, remove “perl*” – save (CTRL+O) and exit (CTRL+X). Be sure to add it back after installing curses-Perl as per the above command.

Install mtop

Now it’s time to fetch the latest version of mtop and compile it. The homepage is here if you wish to check for the latest version. As of this post, we will be using 0.6.6, the latest version.

  1. Grab the mtop source:

    wget http://softlayer.dl.sourceforge.net/project/mtop/mtop/v0.6.6/mtop-0.6.6.tar.gz

  2. Configure, make, and install:

    tar -zxf mtop-0.6.6.tar.gz
    cd mtop-0.6.6
    perl Makefile.PL
    make
    make install

Configure and run

By default, mtop doesn’t connect as the root user. For most server installations, we want to connect as root. Here’s the quickest and easiest way to run mtop.

  1. Find your current mySQL root password:

    cat /root/.my.cnf

  2. Copy the root password (and hostname, if different than localhost) and run the command as follows:

    mtop --host=YOURHOSTNAME --dbuser=root --password=YOURPASSWORD --seconds=1

Apr 082011
 

So you’ve migrated to a new database server, or a new web server, or you’ve added a new database server to your cluster. The problem is, all of your mySQL users only have grants on the old server. Luckily, the guys at cPanel know your pain and have added an easy way to make sure all of your users have access to their databases.

Add the new hostname

  1. Back up the “mysql” database before making ANY changes.

  2. Log in to WHM as root
  3. Navigate to SQL Services->Additional SQL Access Hosts
  4. Put the new IP address that you’ll be accessing FROM in the box

If you’ve got a deadline, you can add the host % here – this will allow access to mySQL from all hosts. SQL authentication is still required, so you’re not exposing any passwords, but if your clients use insecure mySQL usernames, passwords, and database prefixes, this could still leave you open to a huge SQL injection vulnerability risk, so it is highly not recommended.

Add the grants

  1. Navigate back to the Additional SQL Access Hosts page, and at the bottom of the page, click the link labeled “click here.”
  2. Wait for the process to finish and confirm access.

This quick-and-dirty method will ensure your users will still be able to authenticate using their same passwords, and saves a lot of nasty mySQL batch work.

Apr 012011
 

There are many tutorials on how to install ffmpeg for cPanel. I will be combining one with my own modifications. Please submit modifications or corrections when necessary and I will update this post. The last update to this post was December 7th, 2012.

Some Prerequisites

The ffmpeg-php extension (that allows direct function calls from PHP to ffmpeg, rather than using the exec() function) is NOT compatible with CentOS 6. You will unfortunately receive compilation errors due to updated system libraries. This is not recoverable; unfortunately the ffmpeg-php project seems to be dead, and as of now there are no alternatives. If you cannot reinstall your server with CentOS 5.x, you will need to use other methods through PHP to call ffmpeg.
There are a few things you’ll need to prepare if you’re on a fresh installation.

  1. Subversion
  2. Subversion is required to download and install ffmpeg and mplayer. First, you need to allow yum to install Perl packages.

    nano -w /etc/yum.conf

    In the exclude= line, remove “perl*” – save (CTRL+O) and exit (CTRL+X).

    yum install subversion

    Be sure to add the perl* line back to /etc/yum.conf after you install subversion! This is to prevent yum from accidentally overwriting cPanel’s Perl modules.

  3. Git
  4. Git is required because for some reason, when you install mplayer, it forces you to download a Git repository of ffmpeg. Don’t use this repository for installing ffmpeg – it breaks with a strange error message. How you install Git will vary across OSes. On CentOS 5.5 x64, there are several dependencies as well.

    yum install gettext-devel expat-devel curl-devel zlib-devel openssl-devel
    cd /usr/local/src
    wget https://git-core.googlecode.com/files/git-1.8.0.1.tar.gz
    tar -zxf git-1.8.0.1.tar.gz
    cd git-1.8.0.1
    make prefix=/usr/local all
    make prefix=/usr/local install

  5. Ruby
  6. Ruby is required for flvtool. You can install this using the cPanel script:

    /scripts/installruby

    If you’re not using cPanel, you can probably install this using your package manager.

  7. libXext
  8. libXext is required for MP4Box. It’s in the yum repositories.

    yum install libXext libXext-devel

  9. ldconfig
  10. To make sure mplayer and ffmpeg install using the correct libs, we need to make sure ld knows where to find our libraries. First, check:

    nano /etc/ld.so.conf

    Make sure that this file contains the line:

    /usr/local/lib

    If you’re on a 64-bit system you should also add:

    /usr/local/lib64

    Save (CTRL+O), exit (CTRL+W), and then update:

    ldconfig

    If you run in to any “file not found” errors, it’s probably ldconfig.

  11. Remove any other installations
  12. You might already have ffmpeg installed. If it was compiled in, that’s fine, this will overwrite it. But if you have RPM binaries installed, this will cause issues. Run the following:

    rpm -qa | grep ffmpeg
    rpm -qa | grep mplayer
    rpm -qa | grep mencoder
    rpm -qa | grep 264

    Remove with rpm -e any libraries, common files, or binary installations of these packages to prevent conflicts.

Download All The Things

One note; the official distribution release of ffmpeg-php no longer compiles. The SVN version available here works, but the SVN tarball link here can’t easily be downloaded in a terminal session, so I’ve rehosted the file here on this server. This is not my file, I haven’t modified it, and I take no responsibility for it – it’s merely a copy of the file available from the above link.

  1. Let’s get started! This step can and probably should be copy/pasted into your terminal all in one block.

  2. cd /usr/local/src
    wget http://rubyforge.org/frs/download.php/17497/flvtool2-1.0.6.tgz
    tar zxf flvtool2-1.0.6.tgz
    wget http://iweb.dl.sourceforge.net/project/lame/lame/3.99/lame-3.99.5.tar.gz
    tar zxf lame-3.99.5.tar.gz
    wget http://hasaninter.net/ffmpeg-php.tar.gz
    tar xzf ffmpeg-php.tar.gz
    wget http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.3.tar.gz
    tar xzf libvorbis-1.3.3.tar.gz
    wget http://downloads.xiph.org/releases/ogg/libogg-1.3.0.tar.gz
    tar xzf libogg-1.3.0.tar.gz
    wget http://iweb.dl.sourceforge.net/project/opencore-amr/opencore-amr/opencore-amr-0.1.3.tar.gz
    tar xzf opencore-amr-0.1.3.tar.gz
    wget http://downloads.xiph.org/releases/theora/libtheora-1.1.1.tar.bz2
    tar xjf libtheora-1.1.1.tar.bz2
    wget http://downloads.xvid.org/downloads/xvidcore-1.3.2.tar.gz
    tar zxf xvidcore-1.3.2.tar.gz
    wget http://downloads.sourceforge.net/faac/faad2-2.7.tar.gz
    tar zxf faad2-2.7.tar.gz
    wget http://downloads.sourceforge.net/faac/faac-1.28.tar.gz
    tar zxf faac-1.28.tar.gz
    mkdir /usr/local/lib/codecs
    wget ftp://ftp.mplayerhq.hu/MPlayer/releases/codecs/essential-20071007.tar.bz2
    tar xjf essential-20071007.tar.bz2
    wget http://www.tortall.net/projects/yasm/releases/yasm-1.2.0.tar.gz
    tar zxf yasm-1.2.0.tar.gz
    cp /usr/local/src/essential-20071007/* /usr/local/lib/codecs/
    chmod -R 755 /usr/local/lib/codecs/
    mkdir /usr/local/src/tmp
    chmod 777 /usr/local/src/tmp
    export TMPDIR=/usr/local/src/tmp
    svn co https://gpac.svn.sourceforge.net/svnroot/gpac/trunk/gpac gpac
    git clone git://git.videolan.org/x264.git

  3. Now it’s time to grab mplayer and ffmpeg, the latest versions possible.

  4. git clone git://source.ffmpeg.org/ffmpeg.git ffmpeg
    svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer

Let’s Compile This Shit!

Alright, now we’re down to the good stuff – compilation! Hopefully if the above steps went correctly this will just fly by. However I do not recommend that you copy and paste the entire command tree into the terminal at this point. Run everything one step at a time to catch any errors if they do occur. Google is your friend, but feel free to leave a comment here if you run in to any problems.

  1. Lame

  2. cd /usr/local/src/lame-3.99.5/
    ./configure
    make
    make install

  3. Libogg

  4. cd /usr/local/src/libogg-1.3.0
    ./configure
    make
    make install

  5. Libvorbis

  6. cd /usr/local/src/libvorbis-1.3.3
    ./configure
    make
    make install

  7. yasm

  8. cd /usr/local/src/yasm-1.2.0
    ./configure
    make
    make install

  9. Libxvid

  10. cd /usr/local/src/xvidcore/build/generic
    ./configure
    make
    make install

  11. Libx264

  12. cd /usr/local/src/x264
    ./configure --enable-shared
    make
    make install

  13. Flvtool

  14. cd /usr/local/src/flvtool2-1.0.6
    ruby setup.rb config
    ruby setup.rb setup
    ruby setup.rb install

  15. Opencore-amr

  16. cd /usr/local/src/opencore-amr-0.1.3
    ./configure
    make
    make install

  17. Libtheora

  18. cd /usr/local/src/libtheora-1.1.1
    ./configure
    make
    make install

  19. faad2

  20. cd /usr/local/src/faad2-2.7
    ./configure
    make
    make install

  21. faac

  22. cd /usr/local/src/faac-1.28
    ./configure
    make
    make install

  23. MP4Box

  24. cd /usr/local/src/gpac
    ./configure
    make
    make install

  25. Mplayer – remember, this will download a new version of ffmpeg for some crazy reason. When it asks, just hit enter and allow it to download. It’s only used for compiling mplayer. We must use –yasm=” because we’ve switched from yasm to nasm and mplayer can’t deal with that.

  26. cd /usr/local/src/mplayer/
    ./configure --enable-jpeg
    make
    make install

  27. FFMpeg

  28. cd /usr/local/src/ffmpeg
    ./configure --enable-libmp3lame --enable-libvorbis --enable-shared --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-nonfree --enable-libtheora --enable-version3 --enable-gpl --enable-libxvid
    make
    make install

  29. FFMpeg-PHP – at this point you may receive an error message. To fix this error refer to the second code block.

  30. cd /usr/local/src/ffmpeg-php
    phpize
    ./configure
    make
    make install


    nano -w ffmpeg_frame.c
    CTRL+W CTRL+R PIX_FMT_RGBA32 [enter] PIX_FMT_RGB32
    A
    CTRL+O CTRL+X

    What this does is replaces all instances of PIX_FMT_RGBA32 with PIX_FMT_RGB32, saves, and exits. This will happen after the “make” step – you will need to run “make” again, then “make install.”

  31. php.ini – All that’s left is to modify your php.ini file! Add the following near the top of the file /usr/local/lib/php.ini (to avoid conflicts)

  32. extension=ffmpeg.so

  33. Restart Apache
  34. /etc/init.d/httpd restart

That’s It!

At this point you should have a fully functional ffmpeg installation compiled in to PHP. You can test this with the following:


php -m | grep ffmpeg

You should get the output:

ffmpeg

Thank you very much to serverhostingsecrets.com for providing a very stable base for me to build this tutorial on!