brettbrewer.com

Ever been annoyed when your custom error handlers or exceptions don't catch fatal PHP errors and leave you with the dreaded "white screen of death" on your web pages instead of a useful stack trace? Me too. So I was happy to find this helpful article that shows a trick you can use to catch fatal errors such as "out of memory" errors. 

<?php
set_error_handler('myErrorHandler'); 
register_shutdown_function('fatalErrorShutdownHandler');
function myErrorHandler($code, $message, $file, $line) {   
     //handle the fatal error however you want here.  
}  
function fatalErrorShutdownHandler() {
     $last_error = error_get_last();   
     if ($last_error['type'] === E_ERROR) {
          // fatal error     
          myErrorHandler(E_ERROR, $last_error['message'], $last_error['file'], $last_error['line']);  
     } 
}
?>

You may also need to put this in your script to get fatal errors onto the error stack...

<?php
error_reporting(0);
?>

Also keep in mind, if you have any other functions registered as shutdown functions and one of them throws an error, your custom fatalErrorShutdownHandler will not work.

I recently had the unpleasant experience of weathering a rather nasty distributed denial of service (DDoS) attack on one of the sites I manage. It's a high traffic site that does many tens of thousands of dollars of business each and every day. Downtime on the site costs a lot of money and makes us look really bad. Earlier this week we received a short, poorly worded email from an anonymous email address informing us that our site was under attack and demanding an unspecified ransom to cease the attack. The attack took the site down early Monday morning and I, along with a few other people, scrambled to find a solution, working with our dedicated server company and Akamai to block the deluge of traffic that had maxxed out the connections on our firewall. This particular type of attack is known as a "SYN Flood Attack".  It is very hard to defend against.  We tried blocking IP addresses with our firewall and some POS software from Cisco called "Cisco Guard" which proved utterly useless. Akamai tried some other things that took hours to implement and also proved fruitless. In the end, both the dedicated server company and Akamai advised us there was little we could do other than "wait it out". That was simply not an option. By late in the day, we had lost thousands in revenue, not to mention the sinking feeling of utter helplessness at the hands of some asshole "hackers" who we could not hope to track down or identify. We suspected they were in eastern Europe, possibly in Hungary, but that was about all we could deduce. We called the FBI and got a recording and a message about submitting an incident report on their web site. I called CERT's 1-800 number to see if they had any advice or knew who I could report this to and they nearly laughed at me when I told them I'd tried to call the FBI to report it. CERT also directed me to a web form where I could fill out a report, but advised me that most likely nothing would be done. Apparently the WWW in internet addresses really does stand for "Wild Wild West". When it comes to dealing with a DDoS attack, you really are on your own. So what were we to do? Would we pay some unknown stranger an unspecified ransom? Would we wait another day and lose thousands more? Would we hire our own hackers to fight back? Finally as the day was quckly turning to evening and financial losses were piling up, we called one of our business associates who runs an even bigger and more prominent web site and told them what was happening to us and asked if they had any suggestions. They had two words for us....."Call Dosarrest.com", they said.  We'd already done some research online looking for solutions and came across dosarrest's web site, but we were hesitant to try them at first. There were dozens of companies out there claiming to be experts in stopping DoS attacks. Some were big, some were small, some had proprietary hardware they would want to sell us to install at our datacenter, some were just services with no explanation of how they would help. But after the recommendation from our business associates, we decided to give dosarrest.com a call. Our associates assured us that when they had gone through a similar situation, they used dosarrest's services and though it wasn't particularly cheap,  dosarrest got them back online within an hour or two. We had nothing to lose. Our mounting lost revenue had already cost us far more than dosarrest wanted to get us set up with their service. We made contact with dosarrest's sales rep who told us that depending on how quick we could make the required config changes to our site and DNS, they could have us back online within minutes or at most an hour or two. The sale rep wasn't joking.  Within a few minutes of agreeing to the terms of their contract and making our initial paypal payment, their support team had sent us all the required configuration information, set up a proxy server to filter our traffic and were waiting on us to make the needed changes to our web server and dns config. Within the next hour we had completed the DNS and web server updates and our site was back in business.  Throughout the process, I emailed their support with questions and had answers within minutes. When I called them, I was quickly connected to a tech-savvy support person who had all the answers I needed. Each time I have contacted them via email since, I have had a response within minutes. I think the longest response time from them was about 20 minutes for a non-emergency email, but usually it's much quicker. In fact, this evening, after several days of running with their service, I made a major update to the web site. About 100 files were changed and out of that 100 files I made 1 mistake in a config file that caused an endless redirect loop on the site. Because I had updated so many files at once, it took about 15 minutes to find my mistake and correct it. When I finished fixing the redirect problem, I checked my inbox to find an email already waiting from dosalert.com support informing me that they had detected that our site was down, had determined that it was not the result of an attack, had added some additional traffic filters as a precaution and were continuing to monitor the situation. It was not an automated message, it was from an actual support person who was actually on top of the situation. I immediately emailed them back to let them know the problem was caused by a stupid mistake on my part and to tell them how impressed I was that they were so diligent in monitoring our site. And then I sat down to write this post. I'm just not used to service this good. We pay a lot of money to our dedicated server company for 24/7 support and though they are often good, they don't come close to the level of service we've gotten from dosarrest so far. In fact, our web server company is supposed to be monitoring our server for downtime too, but I didn't get any messages from them tonight when I took our server down for 15 minutes. Dosarrest was on top of it within a few minutes. My only complaint with dosarrest is that they don't offer dedicated server hosting because I'd probably bite the bullet and go through all hassles to switch to them for that too.

So, in summary, if you are dealing with a DDoS attack on your servers and don't know what to do, call dosarrest.com. Tell 'em Brett Brewer sent you. I won't say they are cheap, but I will say that if your business lives and dies by its web site, there is no substitute for the kind of service they provide. They are the real deal. 

I've been getting cozy with Github lately and I thought I should share my experiences of getting a post-receive URL hook working so that when someone does a git push to my main development repository it will force my dev site to do a git pull to deploy all the latest changes from the repository to the staging server. In my case, I have a fairly secure development server running on a domain that has no public DNS records. This means that to view the site, someone must have the ip-hostname mapping in their computer's host file. This also means that Github has no way of finding the site to call any post-recieve url scripts to trigger the git pull on the remote site. What follows is a list of everything that needed to be done to make this work.

First off, if you're using PHP like me, and your server has the slightest modicum of security, there's no way you're going to be able to successfully issue a shell_exec('git pull') command via php and have it work because php will only have the priviliges of your web server and chances are your repos and site files are owned by a different user. Enter a nice little apache module called "suphp". I can't get into the details of how to prperly set up suphp because I had my dedicated server experts do that for me, but assuming you can get that set up so that your staging server serves your development site under the same user that owns the site files and git repos, you can then set up a php script that can successfully do a shell_exec('git pull') and have it work. So, assuming you got suphp set up, or you're running a really insecure setup that allows your webserver user to run shell commands, you can create a php script on your server such as this:

<?php
$payload = json_decode(stripslashes(@$_POST['payload']));
$message = print_r(@$payload,true);
$message .= shell_exec('/usr/bin/git pull');
mail('me@mydomain.com','gihub post receive hook fired',$message);
echo $message;
exit;
?>

That was my first post-recieve URL hook script. I saved it on my staging domain's webroot folder and then in the Github admin for my repos, I just entered something like http://my.staging.server.com/Github.php. This worked great so long as my staging domain had a public dns record that allowed Github to find it, but for security reasons I didn't really want most people having access to the staging server, so I removed the public dns records and just added a hostfile mapping for my domain to my computers so I could browse it like it had a normal dns record. Of course, this broke the Github post-receive url hook. The first thing I thought to do was to ask Github support for a way to map my ip to my staging server in their system so I fired off an email to them to see if they would comply and then immediately thought of a rather simple solution. Github can't find a site with no DNS records, but it could easily find my server via its ip address. Since my staging server serves some other domains, I couldn't just give Github the server ip and expect it to find the right domain to call the script on, so I created a new apache config file to handle all requests that don't map to a specific server. For example if your staging server is on ip address 123.123.123.123, you could create an apache VirtualHost container such as this:

<VirtualHost 123.123.123.123:80>
        ServerAdmin me@mydomain.com
        ServerName 123.123.123.123
        ErrorLog /var/log/default.error
        CustomLog /var/log/default.access combined
        DocumentRoot /var/www/html
 
       #turn on suphp for this site
        suPHP_Engine on
        AddHandler x-httpd-php .php .php3 .php4 .php5
        suPHP_AddHandler x-httpd-php
 
</VirtualHost>

This makes any requests to your main ip address look for files in the defaut apache docroot at /var/www/html. The suPHP_Engine directives force the requests for php files to be handled by suPHP so it uses whatever user you've got configured for suPHP. There are ways to get suPHP to use different users based on directive you put in the VirtualHost containers, but that's another story. We only needed one user so we've got suPHP configured to always use that user and restricted to only work in certain directories. Securing your system is up to you. So anyway, I threw my post-receive URL php script in /var/www/html and add some things to allow it to update the proper dev site....

<?php 
 //put your own email address here if you want to receive email updates
 //when commits happen otherwise leave it blank or set it to false
 $email = "gitadmin@mydomain.com";
 
 //set $output_message to true if you want to be able to
 //view the output of the script in a browser
 $output_message = true;
 
 $message = "Github script called on staging server<br/>";
 
 //look for a request var named 'site'. This way you could theoretically
 //have the same script update different sites depending on the site
 //you pass in when you call the script.
 $site = isset($_GET['site'])?$_GET['site']:isset($_POST['site'])?$_POST['site']:"";
 
 if($site=='my.staging.domain.com'){
   $payload = json_decode(stripslashes(@$_POST['payload']));
   $message .= print_r(@$payload,true);
   $message .= shell_exec('cd /path/to/staging/site/dir;/usr/bin/git pull');
 }else{
   $message .= "No valid site specified";
 }
 
 if(!empty($email)){
   mail($email,'gihub post receive hook fired',$message);
 }
if($output_message){
   echo $message;
 }
exit;
 ?>

Then for the post-recieve-url hook in Github I used something like this:

http://123.123.123.123/?site=my.staging.domain.com

There is an apparently undocumented feature in the Github post-receive-url hook that converts any $_GET vars passed in the url into $_POST vars, so in addition to the "payload" post var you will get any other vars you passed as $_GET vars in your $_POST array. I just included the check for a $_GET['site'] var in my script so that I can also call the script from a normal request in my browser. The above script obviously provides very little security, but since the domain I'm using it for has no public dns records, there's not much chance of someone stumbling across it without knowing the site's IP. Since the script restricts the "git pull" command to running only on the domains specified in the $_GET['site'] request var, the worst someone could do is update my staging server with the latest code in my repos if they called it with the right site in the request, which is sort of the whole point of the script anyway so that's not much of a worry. Having the script email me the results every time it runs keeps me informed of what is going on, so if anyone starts messing around with the script it will be hard to miss in my inbox.

Now all is well in my Github world. 

IBM's Jeopardy-playing supercomputer named "Watson" just played and nearly beat both of the top winning Jeopardy champs of all time. I missed the show, but then found this video of the first half of it on YouTube. I had hoped to find the second half posted by the time I got done watching it, but no such luck. I did a little searching and apparently Watson played one competitor to a tie, while handily trouncing Ken Jennings, the winningest player in Jeopardy's history. So marks a milestone in computing history. The technology behind Watson can be used to answer questions of all kinds in a variety of industries, perhaps even solving problems that have heretofore been too complex for humans to reliably answer themselves. Could we be on the verge of a sci-fi like future where some giant supercomputer finds the solutions to all our problems and we simply carry out its instructions? Probably not, but I can dream.

UPDATE: Both parts are now on YouTube:

If you've ever delved much into web site performance optimization, you're probably familiar with Google's PageSpeed and Yahoo's YSlow Firefox plugins and the typical list of suggestions they provide for improving page load times. One of the more common and least implemented suggestions (or most often wrongly implemented) is the idea of splitting request for resources across multiple hostnames. The idea is that since the early days of browsers there has been a limit on the number of resources that can be downloaded in parallel per hostname. The early browsers had a limit of 2 parallel downloads per hostname and I'm really not sure if that limit has increased or by how much, but the point is, if you split your requests across a few different hostnames, your browser can download more resources in parallel without blocking loading of other elements. Unfortunately, there are very few good explanations of how to actually achieve this without destroying most of the inherent benefits of browser caching. You might be tempted just to set up some new CNAMEs for the new hostnames in your DNS and then write a simple function that will randomly choose a hostname to serve each of your static resources from, but this is actually a bad idea because you have no guarantee that the resources will be served from the same URL on subsequent requests, so you will lose the benefits of browser caching. So then you might think, okay, I'll just use a static variable in a function to serve content sequentially in a round-robin fashion, so each resource would be served from the next host in the list and you just cycle through them repeatedly. As long as your pages never change and your resources are always in the exact same sequence on your page, then this would work fine, but if you add an element somewhere, you'll throw off the sequence for the other page elements and you'll end up busting your cache again. So what is an aspiring site optimization wizard to do? The answer turns out to be quite simple - use something called "consistent hashing".  Consistent hashing is the same thing used by popular backend technologies such as Memcached to determine which server in a pool of multiple Memcached servers to pull a particular resource from. Basically you create an algorithm that allows you to hash your filenames in such a way that they will always map to a particular server. This can get a little complicated when used for actual caching where you may want to have a file mapped to multiple servers for failover purposes, but for something as simple as spreading requests across multiple hostnames, all you really need is an algorithm that you can use to consistently map a particular filename to a single hostname. Fortunately for all of us PHP developers, there's a neat little class calledFlexihash that is suited for both simple and more complex uses of consistent hashing.

But enought of the useless background info, let's see how this would work in real life. For the sake of arguement, let's say you're running a fairly big ecommerce site and you're already serving your static content from a Content Delivery Network (CDN) such as Akamai. You serve your web site requests from www.mydomain.com and your images are all served from static.mydomain.com. This mean you most likely have a CNAME record set up in your DNS corresponding to static.mydomain.com. You now decide you want to add 3 more hostnames that all point to the same static content server as static.mydomain.com. You choose static1.mydomain.com, static2.mydomain.com and static3.mydomain.com and add CNAME records for them to your DNS zone, wherever your DNS is hosted. So now you can serve the same image files from any of the 4 domains you set up as CNAMEs. So, how do we set up our image file requests so that the requests are somewhat randomly served by these different host names, but ensure that every image is requested using the same hostname every time? We use a little library called Flexihash from a nice coder named Paul Annesley. So without further ado, here is a very simple example of how you'd use Flexihash to generate your image urls.

<?php
 //There are a couple of ways to include the required Flexihash
 //library files and we'll just assume you figured that part out
 //and included them already.
 
 //So, assuming your Flexihash lib is already included....
 //Instantiate our Flexihash object, to use the defalult hashing
 //algorithm (CRC32) and to hash each filename to just 1
 //target in our list of servers
 $flexiHash = new Flexihash(null,1);
 
 //Now set up our list of servers, each with a weight of 1,
 //so that Flexihash knows what to map the input filenames to.
 $flexiHash->addTarget("static.mydomain.com",1);
 $flexiHash->addTarget("static1.mydomain.com",1);
 $flexiHash->addTarget("static2.mydomain.com",1);
 $flexiHash->addTarget("static2.mydomain.com",1);
 
 //set up some test filenames...
 $filename1 = "somefilename.jpg";
 $filename2 = "someotherfilename.jpg";
 $filename3 = "yetanotherfilename.jpg";
 
 //spit out a message showing how some test filenames
 //will map to specific servers...
 echo "<br/>$filename1 maps to ".$flexiHash->lookup($filename1);
 echo "<br/>$filename2 maps to ".$flexiHash->lookup($filename2);
 echo "<br/>$filename3 maps to ".$flexiHash->lookup($filename3);
 ?>

• Flexihash project on GitHub
  
• Old Flexihash page on Google Code
• Java.net article on consistent hashing
• Another article on consistent hashing
  

For anyone that's spent much time trying to implement the Facebook Javascript API on their web site, you may have come across some odd behavior or unexplained javascript errors in Microsoft Internet Explorer. One of the main causes of problems with the FB Javascript API in IE is a missing 'channelUrl' parameter in your FB.init() function call. To remedy such problems, you create a file such as "fbchannel.html" and in it you place the following contents.

<script src="http://connect.facebook.net/en_US/all.js"></script>

window.fbAsyncInit = function() {
        FB.init({
            appId: '1234567890',
            status: true, cookie: true,
            xfbml: true,
            channelUrl: document.location.protocol + '//www.mydomain.com/fbchannel.html'
        });
};
(function() {
    var e = document.createElement('script'); e.async = true;
    e.src = document.location.protocol +
    '//connect.facebook.net/en_US/all.js';
    document.getElementById('fb-root').appendChild(e);
}());

The email marketing world is about to be shaken up a bit by a new player with some disruptive pricing plans that may finally force the big players to start lowering their prices. PHPList has been the defacto open source standard for managing mailing lists for as long as I can remember, but as a self-hosted solution, it is often difficult to maintain deliverability. Commercial hosted solutions tend to have much better deliverability than self-hosted solutions because commercial providers maintain strict anti-spam compliance and have relationships with the major email providers and ISPs to ensure that their messages get through spam filters so long as senders comply with anti-spam regulations. Email marketers almost ALWAYs want to break the rules, so if you run a self-hosted solution, it's often impossible to comply with anti spam regulations because the people calling the shots tell the people controlling the software to break the rules. Virtually every person I've ever worked for has wanted to break the email marketing rules/laws at some point. So going with a commercially hosted solution has the secondary benefit of taking that decision of anti-spam compliance out of the hands of your potentially irresponsible clients. Unfortunately, most of the major players in the commercial email marketing space have priced themselves right out of many people's range. They charge ridiculous rates for anything over a few thousand users. Enter PHPList's new hosted solution .

The pricing is less than half of what the nearest commercial players are charging. They claim to have relationships with the major email providers and ISPs and maintain a very strict anti-spam compliance, in fact you have to go through a trial period where they monitor your sending behavior and then slowly grant you increased sending priviliges. Their most expensive pricing tier is just  $180 for 100,000 messages per month. Still not cheap, but it's way cheaper than the alternatives. I have yet to find any commercial service that even advertises a pricing tier with a monthly sending limit over 100,000, which seems odd because it's easy to build a list with over 100,000 users these days. One site I work on just got 40,000 new signups from a 1-week promotional campaign. If we send out two messages per month we'll go over the limit for most commercially hosted programs. So I'm looking forward to seeing how well the hosted PHPList service works and whether the prices stay at their current levels over time.

If you're looking for some alternatives to PHPList, you might want to check out this article, which lists "The Top 8 PHPList Alternatives ", though I'm not sure if they are really THE top 8 alternatives, it might give you some good ideas.