Resolved - Apache Alias or Symlinks Not Working with Unexpected 403 Forbidden Error

Apache Forbidden Access Issues

Linking files to Apache on a new install of Cent OS 7 with Apache 2.4 was not quite as smooth as I imagined.

The Setup

I'm working a new server and I want to link to another location so I can use Dropbox to work on files locally and have them automatically updated to my development system.  Since it is easiest to install in the root directory, the Dropbox files automatically get setup in the root user's home directory.  That is not a good place to link the document root to in Apache.  The most logical thing to do was to copy those files out to another better location where they can be served by Apache.  These files were still owned by root and I didn't want to change that, so I creating a symlink to get it to work.

As root (note: my document root was /var/html)

cd /var/html
ln -s /usr/demo/html/ demo

When trying to run in Apache, I would still get the Forbidden error message.

NOTE: When Apache follows symlinks, the path must be accessible all the way down by the calling user (this means you need execute access in the folder you are linking and the parent folders above it).  To make sure this folder is accessible by others, I would use the following command:
chmod o+x /usr /usr/demo /usr/demo/html 

That didn't work for me, but it should work.  I just didn't realize the underlying problem I was experiencing which I will get to in a minute.  So now, I'm thinking I'll try to use an alias and edited and saved the new config file.  

Opening Apache config, I edited it as follows:

sudo nano /etc/httpd/conf/httpd.conf

Alias /demo /usr/demo/html

<Directory "/usr/demo/html">
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    allow from all
</Directory>

All the online literature was pointing me in this direction.
Since I am using the new CentOS7 I need to restart the service using the system control program.

sudo systemctl restart httpd.service
#but, on most servers this is: 
#sudo /etc/init.d/httpd restart

The server restarted properly but I am still not able to access the page and still the Forbidden error pops up in my browser.  Looking in the /var/log/httpd/error_log was somewhat helpful:

[Thu Jan 15 14:37:07.549412 2015] [authz_core:error] [pid 30582] [client x.x.x.x:yyyy] AH01630: client denied by server configuration: /usr/demo/html/test.php

This was telling me that I didn't have a linux permission error accessing the file, but that I had an Apache configuration file error.  Back into the httpd.conf file.

The Solution

After a little digging, I found that Apache 2.4 (that I had on the new server) handles permissions differently that the previous version 2.2 that I was used to using.

Finally modifying my httpd.conf file resulted in:

Alias /demo /usr/demo/html

<Directory "/usr/demo/html">
    Options FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

Now everything works.  I just hadn't been aware that setting aliases in new Apache 2.4 installtion requires a couple changes in the httpd.conf file to get things working properly.  All this time it was just I was using: 

Order allow,deny
allow from all

when I should have been using:

Require all granted

Hopefully this helps someone else save some time.

What are static bindings in PHP - Using self:: versus static::

[1] Why an abstract horse?  See below.

What are static bindings?

Static bindings are functions or variables that can be called on a class that don't need an object created to use them first.  For example, you don't need to use the new operator to call a static method or access a static variable.  These methods exist the first time the class is loaded by the php process and are accessed using the '::' accessor.

How and when to use late static binding in PHP

Late static bindings should be used anytime you are likely to want to redefine or override static objects in children class.  In those cases you will use the static keyword which is a signal to php to check the child class for the appropriate overridden function or variable before it goes up the chain in checking the parent classes.

If you are sure the static function in the base class isn't going to change, or you don't want it to change, then you should consider using the self keyword to access the item.

<?php

abstract class booger {
    abstract function b();
    public static function a() { echo("base class function<br />"); }
    public function c() { self::a();}
    public function d() { static::a();}
}

class goober extends booger{
    public function b() { echo("extended class function<br />"); }
    public static function a() { echo("late static binding<br />"); }
}

$goob = new goober();
$goob::a();
$goob->b();
$goob->c();
$goob->d(); //this is the line that actually uses the late static binding
goober::a();

?>

OUTPUT:

late static binding
extended class function
base class function
late static binding
late static binding


The first line $goob::a() is just a regular overridden function call and does not use the static keyword for the access.  Similarly, $goob->b() is regular method call to a function that was required to be created because of the abstract keyword.

$goob->c() makes use of the self keyword.  This means that the whichever class the keyword is found in is the one that is searched for the requested method.  That is why it shows the output from the base class function.
 
$goob->d() makes use of the static keyword which shows how although the same function a() is called, that since the static keyword was used php knows to check not the base class but the calling class first for the requested function.  That was the desired behavior we were looking for in this case.

Summary

Generally, in most cases static will be the correct keyword to use since the programmer often expects the class to prefer to use the functions in the children classes over those in the parent class.  There are exceptions in situations where self is more appropriate such as when you don't ever plan on calling a particular function from a child class and only the parent class.

Some Gotchas

Please note incorrect order or positioning of the classes in your code can affect the interpreter and can cause a Fatal error:
Class 'YourClass' not found 

This can happen when there are multiple levels of abstraction and the base classes are out of order in the source code.  

For example:
 

<?php

abstract class horse extends animal {
    public function get_breed() { return "Jersey"; }
}

class cart extends horse {
    public function get_breed() { return "Wood"; }
}
 
abstract class animal {
    public abstract function get_breed();
}

$cart = new cart();
print($cart->get_breed());
?>

this outputs:
Wood
 
However, if you put the cart before the abstract horse (literally):
 

<?php
//same code, just in a different order
class cart extends horse {
    public function get_breed() { return "Wood"; }
}

abstract class horse extends animal {
    public function get_breed() { return "Jersey"; }
}
 
abstract class animal {
    public abstract function get_breed();
}

$cart = new cart();
print($cart->get_breed());

?>

this throws an error:
Fatal error: Class 'horse' not found
So, when using multiple levels of abstraction, be careful of the positioning of the classes within the source code - and don't put the cart before the abstract horse.

[1] The abstract horse image was provided by pptbackgroundstemplates.

Four Pitfalls of PHP's Comparison Operators: When to Use == Versus ===

Tips for choosing between '==' (equals or loose comparison) and '===' (identical) operators in php.

Equals or == for loose comparisons.

Identical or === for exact matches.

When choosing the correct operator you want to use in PHP you will want to consider whether you want PHP to try different ways to match your data or if a more strict or exact match is needed. If your data is from a secure location or already validated you may not need the strict identical === operator, but if your data is from an outside source (HTTP GET/POST or user entered) you may want to be more precise depending on the situation. Below are some example cases to keep in mind, starting with a look at the equals == operator.

0. If you don't have time for the entire article, please at least look at this code:

$input = 0;
if ($input == "any string") {
    print("Matches.");
} else {
    print("No match.");
}

The code above outputs "Matches.", which is probably not what you want or expected (see loose evaluation item #2).  This comparison should either make sure $input is casted to a string (i.e. string($input)) or use the '===' operator.

1. Zero, false, empty strings, empty arrays, null are 'loosely' equal in PHP

Below are some base cases to keep in mind. The equals operator uses type juggling, and 0 can become false or an empty string.

Comparisons to Zero
0 == ""                => True
"0" == false           => True
"0" == 0               => True
0 == false             => True


2. Watch out for string comparisons to 0 in PHP

Perhaps the strangest one is that any string with an equals comparison to 0 (as an integer) is True. If you always are expecting to compare both sides with a string, you should strongly consider using the identical operator '==='. Perhaps the strangest one is that any string with an equals comparison to 0.

String Comparisons 

"str" == 0 => True (This one is definitely weird) 

"str" == 1 => False 

"str" == "0" => False 

"strA" == "strB" => False

3. Type juggling converts strings to numbers and vice-versa

When using the equals operator, the strings are converted to a numeric value when the string comparison match fails.

Numeric Comparisons
3 == 4                 => False
"3" == 3.0             => True
1 == "1"               => True




4. Blank strings are 'loosely' evaluated null

This example shows that you need to be careful if there is a potential for blank strings to be passed along as data. Also, you may have expected that the string zero "0" would loosely evaluate to null since "0" loosely evaluated to false, but that is not the case.

Null Usage
"" == null             => True
1 == null              => False
null == null           => True
null == 0              => True
null == "0"            => False

null == false          => True

The Identical Operator removes the loose evaluations

The identical operator is great whenever you expect both sides of the comparison to have the same type and it will help keep you out of trouble. In this case, it helps remove the guesswork from the comparison and your output is more likely to match what you expect.

Comparisons to Zero
0 === ""               => False
"0" === false          => False
"0" === 0              => False
0 === false            => False

String Comparisons
"str" === 0            => False
"str" === 1            => False
"str" === "0"          => False
"strA" === "strB"      => False

Numeric Comparisons
3 === 4                => False
"3" === 3.0            => False
1 === "1"              => False

Null Usage
"" === null            => False
1 === null             => False
null === null          => True
null === 0             => False
null === "0"           => False

null === false         => False

The Code

The PHP documentation for the comparison operators can be found here, and the chart that displays what the value of the comparisons should be is here.

Here is the sample code that generated the tests above:

Echo "Comparisons to Zero<br />";
eval_string('0 == ""');
eval_string('"0" == false');
eval_string('"0" == 0');
eval_string('0 == false');
echo "<br />String Comparisons<br />";
eval_string('"str" == 0');
eval_string('"str" == 1');
eval_string('"str" == "0"');
eval_string('"strA" == "strB"');
echo "<br />Numeric Comparisons<br />";
eval_string('3 == 4');
eval_string('"3" == 3.0');
eval_string('1 == "1"');
echo "<br />Null Usage<br />";
eval_string('"" == null');
eval_string('1 == null');
eval_string('null == null');
eval_string('null == 0');
eval_string('null == "0"');

eval_string('null == false');

echo ("<br />Now Check as the === operator which removes type juggling:<br />");
Echo "Comparisons to Zero<br />";
eval_string('0 === ""');
eval_string('"0" === false');
eval_string('"0" === 0');
eval_string('0 === false');
echo "<br />String Comparisons<br />";
eval_string('"str" === 0');
eval_string('"str" === 1');
eval_string('"str" === "0"');
eval_string('"strA" === "strB"');
echo "<br />Numeric Comparisons<br />";
eval_string('3 === 4');
eval_string('"3" === 3.0');
eval_string('1 === "1"');
echo "<br />Null Usage<br />";
eval_string('"" === null');
eval_string('1 === null');
eval_string('null === null');
eval_string('null === 0');
eval_string('null === "0"');

eval_string('null === false');

function eval_string($str) {
  $disp_str = str_pad($str,20," ");
  echo "$disp_str \t\t=> ".get_boolean_output(eval("return {$str};"))."<br />";
}

function get_boolean_output($val) {
    if ($val == false)
        return "False";
    else 
        return "True";
}

PHPUnit - Migrate from PEAR Install to PHAR (Windows & Linux)

Migrate PHPUnit from PEAR toPHAR

You have installed PHPUnit with PEAR, but the installation method is migrating and you keep getting the error message:

You have installed PHPUnit via PEAR. This installation method is no longer supported and http://pear.phpunit.de/ will be shut down no later than December, 31 2014.

Please read http://phpunit.de/manual/current/en/installation.html and learn how to use PHPUnit from a PHAR or install it via Composer. 

It lets you know that you haven't installed it the right way, but now how do you correct the problem and remove the annoying error message? 

First remove it (Windows & Linux):

pear uninstall phpunit/PHPUnit

Get and install PHPUnit using the phar, Linux (or Amazon EC2) version:

wget https://phar.phpunit.de/phpunit.phar
chmod +x phpunit.phar
mv phpunit.phar /usr/local/bin/phpunit
#and in my case, the phpunit executable also needed to be placed here
cp /usr/local/bin/phpunit /usr/bin/phpunit

Get and Install PHPUnit using the phar, Windows version:

Download the .phar file to somewhere you can run it with php.

1. Open this address in a browser and save to your disk: https://phar.phpunit.de/phpunit.phar 
2. Open the command line and go to the directory (cd {savepath}) you have saved the file in. 
3. Execute tests with: php phpunit.phar testfile.php  
4. Copy phpunit.phar to overwrite your existing phpunit file (replace c:\php\phpunit with your installed location).  Back up the original phpunit file just in case.
   copy c:\php\phpunit c:\php\phpunit.bakcopy phpunit.phar c:\php\phpunit

NOTE:

If you are getting the error below when running your tests:
Failed opening required 'PHPUnit/Autoload.php'
You may need to remove or comment out the Autoload include line .  This file has already been included in the .phar PHP Archive.

//require_once 'PHPUnit/Autoload.php';

Now, use this testing framework to check your code and help you feel confident when deploying your applications.

There are a great number of resources on the official PHPUnit page that you should check out if you want to learn more about PHPUnit and how to use it to your advantage - PHPUnit Presentations.

How to use Amazon SES to Send Email from PHP

CodeProject

Sending mail using Amazon's SES (Simple Email Service)

I couldn't find too many good examples for this online and the Amazon AWS PHP SDK had incomplete documentation for a SendEmail function when I was researching this topic.

NOTE: One pitfall with this is using the SMTP username and password instead of your AWS credentials.  Use your AWS credentials when sending emails using the SDK.

Otherwise, you may get this error:
SignatureDoesNotMatch, Status Code: 403, AWS Request ID: xxxxx, AWS Error Type: client, AWS Error Message: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details. The Canonical String for this request should have been

 

Setup:

It's easy to install the Amazon SDK using PEAR (per Amazon documentation): 
sudo pear -D auto_discover=1 install pear.amazonwebservices.com/sdk

1. Install the SDK
2. Know if your account is sandboxed or not - If you account is sandboxed, you will only be able to send emails to the email addresses in the verified senders list.
3. Make sure your source email's sending address is listed as a verified sender - Verify an email address that you own and use that in your sample code.  Check this email and the ReturnPath address in your AWS console if you are getting the "Email address is not verified" error.
4. If you use the ReturnPath parameter (not shown here) to receive bounced emails, then that email address or domain must be verified as well

When you have the right credentials, sending Email using the SDK is very easy.

 

Sample Code:

require 'AWSSDKforPHP/aws.phar';

use Aws\Ses\SesClient;
$client = SesClient::factory(array(
    'key'    => 'aws_key',
    'secret' => 'aws_secret',
    'region' => 'us-east-1'
));

//Now that you have the client ready, you can build the message
$msg = array();
$msg['Source'] = "authorized_aws_email@somewhere.com";
//ToAddresses must be an array
$msg['Destination']['ToAddresses'][] = "someone@somwhere.com";

$msg['Message']['Subject']['Data'] = "Text only subject";
$msg['Message']['Subject']['Charset'] = "UTF-8";

$msg['Message']['Body']['Text']['Data'] ="Text data of email";
$msg['Message']['Body']['Text']['Charset'] = "UTF-8";
$msg['Message']['Body']['Html']['Data'] ="HTML Data of email<br />";
$msg['Message']['Body']['Html']['Charset'] = "UTF-8";

try{
     $result = $client->sendEmail($msg);

     //save the MessageId which can be used to track the request
     $msg_id = $result->get('MessageId');
     echo("MessageId: $msg_id");

     //view sample output
     print_r($result);
} catch (Exception $e) {
     //An error happened and the email did not get sent
     echo($e->getMessage());
}
//view the original message passed to the SDK 
print_r($msg);

 

Result:

Run the above code using the correct information and your emails should be on their way.  Make sure to set the SenderID, DKIM, and SPF on your domain and Amazon properly to prevent your emails getting marked as spam.

Update:

Because of a request from Mohit Singh, I've updated the code to allow for adding a single attachment to the email.  See below for the details.

Using Attachments:

Use this example code in your program to send an email with an attachment using Amazon SES.

include_once("SESUtils.php");

$subject_str = "Some Subject";
$body_str = "<strong>Some email body</strong>";
$attachment_str = get_file_contents("/htdocs/test/sample.pdf");

//send the email
$result = SESUtils::deliver_mail_with_attachment(
    array('email1@gmail.com', 'email2@lutz-engr.com'),       
    $subject_str, $body_str, 'sender@verifiedbyaws', 
    $attachment_str);

//now handle the result if you wish
print_r($result);

Complete Source for PHP Solution for sending mail using SES

Update #2 - (2015-01-27) Michael Deal was kind enough to provide additional features and enhancements in this new version

Update #3 - (2015-03-03) Code has been updated to properly handle plaintext with HTML and multiple attachments.  It's not handled quite the way you would think.  Thank you RFC-2046!

<?php

require_once('AWSSDKforPHP/aws.phar');

use Aws\Ses\SesClient;

/**
 * SESUtils is a tool to make it easier to work with Amazon Simple Email Service
 * Features:
 * A client to prepare emails for use with sending attachments or not
 * 
 * There is no warranty - use this code at your own risk.  
 * @author sbossen 
 * http://righthandedmonkey.com
 *
 * Update: Error checking and new params input array provided by Michael Deal
 * Update2: Corrected for allowing to send multiple attachments and plain text/html body
 *   Ref: Http://stackoverflow.com/questions/3902455/smtp-multipart-alternative-vs-multipart-mixed/
 */
class SESUtils {

    const version = "1.0";
    const AWS_KEY = "YOUR-KEY";
    const AWS_SEC = "YOUR-SECRET";
    const AWS_REGION = "us-east-1";
    const MAX_ATTACHMENT_NAME_LEN = 60;

    /**
     * Usage:
        $params = array(
          "to" => "email1@gmail.com",
          "subject" => "Some subject",
          "message" => "<strong>Some email body</strong>",
          "from" => "sender@verifiedbyaws",
          //OPTIONAL
          "replyTo" => "reply_to@gmail.com",
          //OPTIONAL
          "files" => array(
            1 => array(
               "name" => "filename1", 
              "filepath" => "/path/to/file1.txt", 
              "mime" => "application/octet-stream"
            ),
            2 => array(
               "name" => "filename2", 
              "filepath" => "/path/to/file2.txt", 
              "mime" => "application/octet-stream"
            ),
          )
        );
      
      $res = SESUtils::sendMail($params);
      
     * NOTE: When sending a single file, omit the key (ie. the '1 =>') 
     * or use 0 => array(...) - otherwise the file will come out garbled
     * ie. use:
     *    "files" => array(
     *        0 => array( "name" => "filename", "filepath" => "path/to/file.txt",
     *        "mime" => "application/octet-stream")
     * 
     * For the 'to' parameter, you can send multiple recipiants with an array
     *    "to" => array("email1@gmail.com", "other@msn.com")
     * use $res->success to check if it was successful
     * use $res->message_id to check later with Amazon for further processing
     * use $res->result_text to look for error text if the task was not successful
     * 
     * @param array $params - array of parameters for the email
     * @return \ResultHelper
     */
    public static function sendMail($params) {

        $to = self::getParam($params, 'to', true);
        $subject = self::getParam($params, 'subject', true);
        $body = self::getParam($params, 'message', true);
        $from = self::getParam($params, 'from', true);
        $replyTo = self::getParam($params, 'replyTo');
        $files = self::getParam($params, 'files');

        $res = new ResultHelper();

        // get the client ready
        $client = SesClient::factory(array(
                    'key' => self::AWS_KEY,
                    'secret' => self::AWS_SEC,
                    'region' => self::AWS_REGION
        ));

        // build the message
        if (is_array($to)) {
            $to_str = rtrim(implode(',', $to), ',');
        } else {
            $to_str = $to;
        }

        $msg = "To: $to_str\n";
        $msg .= "From: $from\n";

        if ($replyTo) {
            $msg .= "Reply-To: $replyTo\n";
        }

        // in case you have funny characters in the subject
        $subject = mb_encode_mimeheader($subject, 'UTF-8');
        $msg .= "Subject: $subject\n";
        $msg .= "MIME-Version: 1.0\n";
        $msg .= "Content-Type: multipart/mixed;\n";
        $boundary = uniqid("_Part_".time(), true); //random unique string
        $boundary2 = uniqid("_Part2_".time(), true); //random unique string
        $msg .= " boundary=\"$boundary\"\n";
        $msg .= "\n";

        // now the actual body
        $msg .= "--$boundary\n";

        //since we are sending text and html emails with multiple attachments
        //we must use a combination of mixed and alternative boundaries
        //hence the use of boundary and boundary2
        $msg .= "Content-Type: multipart/alternative;\n";
        $msg .= " boundary=\"$boundary2\"\n";
        $msg .= "\n";
        $msg .= "--$boundary2\n";

        // first, the plain text
        $msg .= "Content-Type: text/plain; charset=utf-8\n";
        $msg .= "Content-Transfer-Encoding: 7bit\n";
        $msg .= "\n";
        $msg .= strip_tags($body); //remove any HTML tags
        $msg .= "\n";

        // now, the html text
        $msg .= "--$boundary2\n";
        $msg .= "Content-Type: text/html; charset=utf-8\n";
        $msg .= "Content-Transfer-Encoding: 7bit\n";
        $msg .= "\n";
        $msg .= $body; 
        $msg .= "\n";
        $msg .= "--$boundary2--\n";

        // add attachments
        if (is_array($files)) {
            $count = count($files);
            foreach ($files as $file) {
                $msg .= "\n";
                $msg .= "--$boundary\n";
                $msg .= "Content-Transfer-Encoding: base64\n";
                $clean_filename = self::clean_filename($file["name"], self::MAX_ATTACHMENT_NAME_LEN);
                $msg .= "Content-Type: {$file['mime']}; name=$clean_filename;\n";
                $msg .= "Content-Disposition: attachment; filename=$clean_filename;\n";
                $msg .= "\n";
                $msg .= base64_encode(file_get_contents($file['filepath']));
                $msg .= "\n--$boundary";
            }
            // close email
            $msg .= "--\n";
        }

        // now send the email out
        try {
            $ses_result = $client->sendRawEmail(
                    array(
                'RawMessage' => array(
                    'Data' => base64_encode($msg)
                )
                    ), array(
                'Source' => $from,
                'Destinations' => $to_str
                    )
            );
            if ($ses_result) {
                $res->message_id = $ses_result->get('MessageId');
            } else {
                $res->success = false;
                $res->result_text = "Amazon SES did not return a MessageId";
            }
        } catch (Exception $e) {
            $res->success = false;
            $res->result_text = $e->getMessage().
                    " - To: $to_str, Sender: $from, Subject: $subject";
        }
        return $res;
    }

    private static function getParam($params, $param, $required = false) {
        $value = isset($params[$param]) ? $params[$param] : null;
        if ($required && empty($value)) {
            throw new Exception('"'.$param.'" parameter is required.');
        } else {
            return $value;
        }
    }

    /**
    Clean filename function - to be mail friendly 
    **/
    public static function clean_filename($str, $limit = 0, $replace=array(), $delimiter='-') {
        if( !empty($replace) ) {
            $str = str_replace((array)$replace, ' ', $str);
        }

        $clean = iconv('UTF-8', 'ASCII//TRANSLIT', $str);
        $clean = preg_replace("/[^a-zA-Z0-9\.\/_| -]/", '', $clean);
        $clean = preg_replace("/[\/| -]+/", '-', $clean);
        
        if ($limit > 0) {
            //don't truncate file extension
            $arr = explode(".", $clean);
            $size = count($arr);
            $base = "";
            $ext = "";
            if ($size > 0) {
                for ($i = 0; $i < $size; $i++) {
                    if ($i < $size - 1) { //if it's not the last item, add to $bn
                        $base .= $arr[$i];
                        //if next one isn't last, add a dot
                        if ($i < $size - 2)
                            $base .= ".";
                    } else {
                        if ($i > 0)
                            $ext = ".";
                        $ext .= $arr[$i];
                    }
                }
            }
            $bn_size = mb_strlen($base);
            $ex_size = mb_strlen($ext);
            $bn_new = mb_substr($base, 0, $limit - $ex_size);
            // doing again in case extension is long
            $clean = mb_substr($bn_new.$ext, 0, $limit); 
        }
        return $clean;
    }
    
}

class ResultHelper {

    public $success = true;
    public $result_text = "";
    public $message_id = "";

}

?>

Sending multiple attachments using SES - Completed!

The above is a more complete and robust version of the sending email attachments with Amazon SES.  The one above now lets you have more than one attachment to send from the previous version.  Hope you enjoyed this and thanks to all for participating! 

How to Handle Spammers or Spambots (on Contact Forms)

The Form

If you have a contact form on your website for the purpose of allowing users to request additional information or forward contact information, then you've probably come across spambots.

Here is the simple form we use on our website to notify us by email of simple requests.

Contact Us Your Name:   * Phone:   Email:   * Purpose:                            General                         Proposal Request                         Sales Inquiry                         Request Company Info                         Customer Service                       Comments:

The Spammer

In addition to legitimate content, we've also been getting these comments on filled out forms that are just spam.

Actual received form data: 
Name: Arianna 
Email: pitfighter@hotmail.com 
Phone: 41684792535 
Purpose: Proposal Request 
Comment: Could I have , please? <a href=" http://www.digitrak.com ">accutane prescription requirements</a>  Hepatic Issues, Atrial Fibrillation

I'm not sure who makes money off sending this type of spam.  I'm even less sure of someone who would be foolish enough to click on the link except by curiosity or accident.

Exposing the Spammer

It would be interesting to know where this spam is coming from, luckily finding out who is sending it is easy, being sure is not (because the IP address may be spoofed).  I choose a nice free geolocation plugin for PHP  which can be found at http://www.geoplugin.com

Capture the sender's IP thanks to a StackOverflow snippet:

function getClientIP() {
    if (!empty($_SERVER['HTTP_CLIENT_IP'])) {   //check ip
        $ip = $_SERVER['HTTP_CLIENT_IP'];
    } elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {   //to check if ip is passed from a proxy
        $ip = $_SERVER['HTTP_X_FORWARDED_FOR'];
    } elseif (!empty($_SERVER['REMOTE_ADDR'])) {
        $ip = $_SERVER['REMOTE_ADDR'];
    } else {
        $ip = "localhost";
    }   
    return $ip;
}

If you're willing to send me spam I am perfectly willing to share your IP address and the spammer in this case sent from: 188.143.232.31.  The PHP function returns the IP address, which you can then pass to the geoplugin PHP API site.

/** returns array of geolocation data
**/
function getGeolocation($ip) {
    $geo_str = "http://www.geoplugin.net/php.gp?ip=$ip";
    $arr = unserialize(file_get_contents($geo_str));
    //$country = $arr["geoplugin_countryName"];
    //$code = $arr["geoplugin_countryCode"];
    //$region = $arr['geoplugin_regionName'];
    //$regionCode = $arr['geoplugin_regionCode'];
    //$city = $arr['geoplugin_city'];
    return $arr;
}

Running the IP address through the geolocation service returns the following data:

Array
(
    [geoplugin_request] => 188.143.232.31
    [geoplugin_status] => 200
    [geoplugin_credit] => Some of the returned data includes GeoLite data created by MaxMind, available from http://www.maxmind.com.
    [geoplugin_city] => Saint Petersburg
    [geoplugin_region] => Sankt-Peterburg
    [geoplugin_areaCode] => 0
    [geoplugin_dmaCode] => 0
    [geoplugin_countryCode] => RU
    [geoplugin_countryName] => Russian Federation
    [geoplugin_continentCode] => EU
    [geoplugin_latitude] => 59.894402
    [geoplugin_longitude] => 30.2642
    [geoplugin_regionCode] => 66
    [geoplugin_regionName] => Sankt-Peterburg
    [geoplugin_currencyCode] => RUB
    [geoplugin_currencySymbol] => руб
    [geoplugin_currencySymbol_UTF8] => руб
    [geoplugin_currencyConverter] => 32.4624
)

So now we have information that points to a spammer in Saint Petersburg, Russia.  This makes sense, a lot of the spam in the world comes from Russia (of course many sources indicate most comes out of the good-ole USA).

Blocking Spam - Preventing spam from getting successfully sent

1. Block by country code or region.  Once you know where people are sending data from, your website might only make sense to traffic from certain regions.  You can easily check the returned values from the geoplugin function to see if the sender is on your blocked list.  For us, customers outside the United States don't apply to our business so we block them
   if ($arr['geoplugin_countryCode'] != 'US') {
   //silently block the request
   }
2. The comments typically have links in them when sent by spammers.  You can submit this text to another plugin called Akismet (found here) which will give you an idea if the content is detected as spam.  This plugin is originally intended for WordPress, but don't worry - you don't need to be running WordPress to use it, but you do need to signup for a free API key (here). 
   require_once("Akismet.class.php");
   public static function is_spam($name, $email, $comment) {     $WordPressAPIKey = 'xxxxxx';
   $MyBlogURL = "http://mysite.com";
   $akismet = new Akismet($MyBlogURL, $WordPressAPIKey); $akismet->setCommentAuthor($name); $akismet->setCommentAuthorEmail($email); $akismet->setCommentContent($comment); $akismet->setPermalink("http://lutz-engr.com/index.php");    return $akismet->isCommentSpam();
   }
3. The above steps have been enough to block the vast majority of the spam for our sites.  If you are really serious about blocking more, use a CAPTCHA.  Personally, I hate these and don't recommend their use unless absolutely necessary, but they are definitely needed in certain applications.

Even when spam is detected using these methods, I still send it along, but mark it as spam.  It's fun to peek evey once in a while to see what silly things these spam bots are trying to do to promote a certain product or web link that I'll never buy.

Do spammers/spam bots run client side javascript?  

UPDATE: After running the code below, I have found that javascript is NOT being run on the forms.  This is based on the very same IP address identified earlier.  Since our website requires javascript anyway, it might make sense to block submission of the form unless javascript is run on the client machine.  The offending machine appears to be running an Apache 2.2.15 server on CentOS and it may be worth exploring more to see what else I can find. 

I have been wondered this and thought it would be easier and safer for a spambot to not run any client side scripting.  After all, doesn't it just crawl a page and fill out forms?  Maybe I could require users to be running javascript to submit the form.  To test for this, I've added a simple line that just changes the value if javascript is running:

<head>
...

$(document).ready (function() { $('#timedayjs').val('set by js'); }); 

...

 </head>

...
<input type="hidden" id="timedayjs" name="timedayjs" value="nojs (unused)"></input>

I called it 'timedayjs' just to be inconspicuous, but really this value just notifies me if the client has run the onready javascript function on the page.  I'll either get the value 'nojs (unsed)' or "set by js" passed into my contact form. 

I'm awaiting results from this experiment now.

UPDATE: Now that the results are in, see the next post for how you can fight contact form spam by requiring the use of javascript. 

Debugging and Profiling MySQL Performance with PHP

We already talked about PHP performance profiling by using the handy built-in xdebug profiler and passing the XDEBUG_PROFILE parameter.  You can find the original blog article here.  But, that method does not give you and easy way to view the SQL calls you are making or the individual performance of those calls.  To add a SQL performance profile to your database calls we will create a wrapper class that will intercept each query and store the timing information and call that is being made.  This wrapper is a drop in replacement for the original mysqli calls.

<?php

/** - Database debug helper

 * This is a wrapper class for FirePHP or other logging system
 *
 * USAGE: use debug_mysql in place of mysqli:
 * global $is_dev;
 * $is_dev = true;
 * global $debug_sql_flag;
 * $debug_sql_flag = true;
 * $dbHandle = new debug_mysql("host_ip", "user", "pw", "dbname");
 * $dbHandle->set_charset("utf8"); //optional
 * $sql ="SELECT * FROM tablename";
 * $query_result = $dbHandle->query($sql, __FILE__, __METHOD__, __LINE__);
 *
 * @author sbossen
 */

class debug_mysql extends mysqli {

    const log_file = "/var/log/firephp.log";

    public function query($sql, $file = "", $method = "", $line = "") {

        global $is_dev; //indicates if this is a development system
        global $debug_sql_flag; //indicates if we are processing debug log events

        if ($is_dev) {
            $query_start = microtime(true);
            $result = parent::query($sql);
            $query_end = microtime(true);
            $query_time = $query_end - $query_start;

            if ($debug_sql_flag) {
                if ($query_time > 10) {
                    $speed = "slow";
                } else {
                    $speed = "normal";
                }
                if (!$result) {
                    $err = "ERROR: $this->error";
                } else {
                    $err = "Ok";
                }
                $db = $this->get_database_name();
                self::log_to_file("QUERY: $query_time ms, $speed, $db, $sql, $err", $file,$method,$line);
            }
            return $result;
        } else {
            //pass original query through if this is a production system
            return parent::query($sql);
        }
    }

    /**
     * Get name of the database that is currently selected
     * @return string - database name
     */
    public function get_database_name() {
        $str = "{unknown db}";
        if ($result = parent::query("SELECT DATABASE()")) {
            $row = $result->fetch_row();
            $str = $row[0];
            $result->close();
        }
        return $str;
    }

    /**
     * @param type $str - message to output
     * @param type $file - file that the method was called from
     * @param type $method - method that was called
     * @param type $line - line method was called from
     */
    public static function log_to_file($str, $file = "", $method = "", $line = "") {
        file_put_contents(self::log_file, $file."::".$method."[".$line."] - \"".$str."\"\n", FILE_APPEND);
    }

}
?>

This gives you a log file that stores all of your SQL commands and can be imported as a CSV file into Excel for sorting or other analysis.  SQL error messages are captured along with the location in the code the call was made from (provided you pass them along

If you are running the server on linux you can simply open a shell session and run:

tail -f /var/log/firephp.log

You can watch this terminal window show all your queries along with the time it takes to execute each query.  Any SQL errors that are found will display along with the error information.  This helps for quick debugging if you just want to find out if your queries are running properly.

Instead of displaying to a log file, it can be useful to output the log to FirePHP.  FirePHP allows you to display debug information to your Firebug console through the Firefox web browser.  It can certainly be handy if you don't want to or can't open up a shell connection.  Install FirePHP using pear as described on the FirePHP website and install the FirePHP Firefox extension to your browser.

To display to FirePHP instead of a file, replace the log_to_file() function with the code below.  This way you can have SQL code show right in your browser.

public static function log_to_file($str, $file = "", $method = "", $line = "") {
    $firephp = FirePHP::getInstance(true);
    $firephp->log($file."::".$method."[".$line."] - \"".$str."\"");
}

Always be careful not to show debug information on a production system.  At best it is a sloppy practice and at worst it can reveal sensitive system data.

Installing PHPUnit with XAMPP

Install PHPUnit with XAMPP

Update:

This method is now deprecated per Sebastian Bergmann.  Use the phar or composer method to install PHPUnit.

XAMPP is a great utility to get you started quickly with Apache, MySQL, and PHP.  In fact I wasn't even able to get these programs working together properly on my new Windows 7 machine until I used XAMPP to install all the components I needed.  But do you how to setup PHPUnit with XAMPP?  To get PHPUnit working, the steps aren't quite as obvious.  As you will see the PEAR installation tool is used, but it needs some additional work as explained below.

1. First get and install XAMPP: http://www.apachefriends.org/en/xampp.html

Now after the install, usually you'll have your files under something like c:\xampp\php, etc.  This works pretty well, but to get PHPUnit, you need PEAR which needs some additional install.

2. Get the latest PEAR update and run the PEAR installer:
download the file at http://pear.php.net/go-pear.phar and place it in your C:\xampp\php folder

cd \xampp\php
php go-pear.phar

3. Change the location of your pear.ini (item#11 or 12 - the menu varies according to your system) to C:\xampp\php - This is especially important on Windows 7 that protects the windows folder. If this is not changed your PEAR install will not be able to save its configuration files.

4. Run the C:\xampp\php\PEAR_ENV.reg file created to import your PEAR environment variable settings.

5. Log out of Windows and back in - This will load the new environment variables into memory (otherwise the old location for pear.ini will be used).  Confirm your settings by running:

pear config-show

 It should now read: C:\xampp\php\pear.ini

6. Install PHPUnit & Skeleton Generator through PEAR

pear config-set auto_discover 1 
pear install pear.phpunit.de/PHPUnit
pear install phpunit/PHPUnit_SkeletonGenerator

7. Check your version of PHPUnit - It should be at least version 3.6.x

phpunit --version

You should also now have phpunit-skelgen.bat in the C:\xampp\php folder.

If this blog was helpful to you, please thank me by clicking on the google +1 link.  Every time you click +1 a unicorn gets its wings... and what's cooler than a flying unicorn?

Netbeans NOTE:
First of all, if you are trying to use Netbeans to do unit testing in PHP, then you should probably start at the great tutorial at: http://netbeans.org/kb/docs/php/phpunit.html#installing-phpunit

Update:
The PEAR install method is now at end of life and can no longer be used according to Sebastian Bergmann's github page.

Try this method for using the phar install method: Phar install method.