Introduction to MongoDB [PHPMaster]

This is kinda old news for some of you. My 2nd PHPMaster article was published like a month ago but I forgot to write about it here. This time it’s a step by step introduction to using MongoDB in PHP.

In this article I’ll introduce you to NoSQL with MongoDB. You’ll learn how to install the MongoDB extension for PHP, and how to add, update, and retrieve document objects. If you’re used to working with RDBMSs like MySQL or PostgreSQL, you’ll find some of the concepts of working with MongoDB a bit strange, but you’ll soon grow to love the flexibility and power that MongoDB gives you!

You can read the article here and wait for the 2nd part which is scheduled to be published on jan 20, 2012. it will discuss more advanced topics that you will need in real life projects.

 

New article on PhpMaster.com about Zend_Pdf

I wrote a new article about using Zend_Pdf , which was published on PhpMaster.com days ago.

The PDF format is currently the most used format to exchange documents. If you provide your website users with printable versions of invoices, event tickets and other similar documents, you’ll most likely want to generate them as PDFs on the fly. In this article you will see how you can use Zend_Pdf to auto-generate PDF invoices.

I’m glad to write for the first time for PhpMaster, Sitepoint.com ‘s PHP blog. I have been reading their online content and books since 2005 and having the chance to write for them now is an honer.

check the article there.

The story of version 1.1

Once upon a time, a software project took around 7 months to get to version 1.0 . We didn’t have any problem all that time with our SVN (Subversion) repository as we didn’t have to release anytime before version 1.0 . we didn’t have any SVN strategy till that moment and all of us were happily committing to the trunk and enjoying a branching-less life.

The last 2 months before releasing version 1.0 were meanly focused on enhancing performance and bug fixing. we couldn’t stop at that point as we had many developers with nothing to do and more features waiting to be add to the next version. the work on the new features had to go in parallel with the code enhancements being applied to version 1.0 to get it released and ready to be deployed to production. we did create another branch for the 1.1 version.

Some of our developers started committing to the 1.1 branch while some others continued to commit to the trunk. then after 2 months of work, we wanted to release the 1.1 version. the plan was to merge the 1.1 branch into trunk, test it, fix bugs and then deploy that to production. things didn’t go that simple as we had very big code base and there were many differences between the 1.1 branch and the trunk. merging took more than a week and lots of bugs that were fixed before in the trunk, came back to life with more new bugs that had to be fixed as fast as possible to get the 1.1 version released and deployed to production.

The bug fixing phase took more than what was expected and our product team (who were playing the rule of the owner in our little software development game) had a plan with lots of features that had to be completed before the end of year and they didn’t want to stop the development of new features till we finish fixing all the bugs.

They gave us a plan for the rest of the year and they wanted to change our release strategy so we release and deploy to production at the end of every week. this was a kinda good strategy for our kind of business. the problem was that we still didn’t have a stable code base that we were sure that we can build on. we had lots of fatal bugs waiting to be fixed and we didn’t yet release version 1.1 .

Anyway, we started working on the features of the first week of the plan. while the amount of features was more than what can be finished in one week, we couldn’t finish the features in less than 10 days. this left us behind the plan by some days and we didn’t have enough time yet to fix the bugs of version 1.1 . we were pushed more to start working on the planned features for the second week. that week had more complicated features and it took us 2 weeks to finish it.

At that moment we had all that code in the trunk with lots of new implemented features and more bugs. we didn’t release anything in the previous 3 months (since version 1.0) . the project didn’t benefit of the new features. every team is blaming the other for making things go like that. the product team had more plans and features to be implemented till the end of year. the development team can’t make a new release with all those bugs.

that’s not the end of the story, but I have to stop here and start analyzing what happened. mainly, I see 2 problems.

First, the development team was committing all the code directly into the trunk. we would have created a branch for each feature. this way each feature will stay isolated with all its bugs and then when it’s stable, we can merge the branch into the trunk. this will keep the trunk always ready for deployment at any moment.

like what Mr.Radwan say, “your main code base is gold, you can’t mix it with rubbish code”. this way we can keep our gold in the trunk always ready for deployment and we can keep history of each release in a tag with the right label.

branching in source control systems

Second, the product team shouldn’t have continued requesting new features while there were many bugs not fixed and while they can’t release the finished features. they should have stopped till the development team can get all the bugs fixed and then they can meet with the development team to see how they can stop the problem from appearing again in the future.

software development isn’t that simple and not respecting this craft can turn a project into a big failure. problems in projects like that doesn’t appear just over night, people are the main reason of the problems. if you can’t control all the participants and keep them focused on what they should do, they will start behaving the way they like and will start fighting each other instead of collaborating. if they were focused, they would have noticed that the project is shifting from the right way and they would have started early to fix this.

Soap Web Services in PHP & hint on consuming from .Net

last week, I was working on implementing a SOAP server for one of our projects at work, this web service was built to be consumed by some other software that is built using .Net . It wasn’t that easy and I faced some problems that made me go crazy for some hours and I wanted to share them here with the solutions. let me first show you how I built the web service in PHP.

by the way, This project was built using CakePHP, but the web service I made doesn’t depend that much on that framework except that I use its MVC implementation.

If we want to build a SOAP web service, we will need first to create a WSDL file and make it accessible from a defined URL. I made a new controller with a new action named [index] that will return the WSDL file when requested using GET, and will also handle SOAP requests when requested using POST.

<?php

class SoapController extends AppController
{

    /**
     * handles SOAP requests
     */
    public function inedx()
    {

        if ( $this->RequestHandler->isGet() ) { //just show the wsdl
            $this->RequestHandler->respondAs('xml');
            echo self::_getWsdlContent();
            return;
        }

        $soapServer = new SoapServer( Configure::read('soap.wsdlFilePath') );

        $soapHandler = new Shreef_WebService_SoapHandler( /* Inject Dependencies here */ );

        $soapServer->setObject($soapHandler);

        $soapServer->handle();

    }

    /**
     * gets WSDL content for the SOAP service
     */
    protected static function _getWsdlContent()
    {
        $fileName = Configure::read('soap.wsdlFilePath');

        if (file_exists($fileName) ) {
            return file_get_contents($fileName);
        }

        $soapAutodiscover = new Zend_Soap_AutoDiscover( );

        $wsdlContent = $soapAutodiscover
                        ->setClass('Shreef_WebService_SoapHandler')
                        ->toXml();

        file_put_contents($fileName, $wsdlContent);
        return $wsdlContent;
    }

}

as you see, I’m dynamically generating the WSDL file using Zend_Soap_AutoDiscover. and caching it into a file. Zend_Soap_AutoDiscover will read the DocBlocks defined in Shreef_WebService_SoapHandler to build the WSDL file.

Notice that I gave SoapServer an instance of Shreef_WebService_SoapHandler instead of giving it the name of the class. this will give me the chance to inject any needed dependencies into the constructor.

here is my SoapHandler class with a simple method [getCurrentTime()] that will return a String, and another method [doSomething()] that returns Array.

<?php

class Shreef_WebService_SoapHandler
{

    /**
     * construct
     */
    public function __construct()
    {
        /** Dependency Injection happens here **/
    }

    /**
     * returns the current server time formated like YYYYMMDDhhmmss
     *
     * @return String
     */
    public function getCurrentTime()
    {
        return date('Ymdhis');
    }

    /**
     * do something
     *
     * @param String $something
     * @return Array
     */
    public function doSomething($something)
    {
        return array(1, 2, 3);
    }

    /** other methods **/
}

and that’s it, now if I open http://localhost/soap in my browser, I will see the WSDL. then I can use that WSDL to make requests to the web service. this was working fine on my machine and on the testing server, but it started to go crazy when we deployed that to the staging server. all methods returned null.

DocBlocks and OpCode caching

After some hours I noticed that the generated WSDL on the staging server was different than what was generated on my machine and on the testing server. there were no Out messages and the type of all method parameters was set to “anyType”. at first, I thought that this is a bug in Zend_Soap_AutoDiscover, but later I commented out the lines that cache the generated WSDL to a file and started to change things around.

I noticed that whenever I make a change to the file that contains the SoapHandler, the WSDL gets generated like expected for one time, but if I open the URL of the WSDL again, it will go back to generate the wrong WSDL. and here we have another “Aha!!” moment.

We have eAccelerator installed on that server. it is doing opcode caching for all the PHP code. this means that after every modification to the file that contains the SoapHandler class, the class is read from the file with all the DocBlocks, but in the next request, SoapHandler classs will get read from the opcode cache with all DockBlocks removed. so Zend_Soap_AutoDiscover won’t find DocBlocks to read and will guess that all methods are void and all method parameters are of type “anyType”.

As stopping eAccelerator doesn’t make sense, I turned on the WSDL file caching again and ran the “touch” command on all files that contain code that should get described in the WSDL ( the soap handler and any other complex types used ). this will make the WSDL get generated well the first time and cached.

I have to automate this later using a shell script or something.

.Net and Soap Arrays

The second problem was that Visual Studio.net was refusing to generate the required proxy classes using that WSDL file. It didn’t understand the type “Array”. I searched for this problem, and it looked like that were no way to make VS.net understand it. I solved that by instead of returning values of type Array, I changed my code a bit to return a complex type.

Also, I had to tell Zend_Soap_AutoDiscover that I want to use another WSDL strategy by passing the string ‘Zend_Soap_Wsdl_Strategy_ArrayOfTypeComplex’ to its constructor.

iconv misunderstands UTF-16 strings with no BOM

I had a problem last week with converting UTF-16 encoded strings to UTF-8 using PHP’s iconv library on a Linux server. my code worked fine on my machine but the same code resulted in a rubbish unreadable characters on our production server.

Let me take you to the beginning of the problem. I had a Hexadecimal representation of a UTF-16 string like that

0635 0628 0627 062D 0020 0627 0644 062E 064A 0631

this is the equivalent of “Good morning” in Arabic “صباح الخير” . I had some lines of code that will convert this into a normal stream of UTF-16 bytes so I can be able to use iconv to convert the string to UTF-8. maybe you noticed that there is no BOM at the beginning of Hexadecimal representation of the string. so let me quote what is written on Unicode’s BOM FAQ page about this.

Q: Why do some of the UTFs have a BE or LE in their label, such as UTF-16LE?

A: UTF-16 and UTF-32 use code units that are two and four bytes long respectively. For these UTFs, there are three sub-flavors: BE, LE and unmarked. The BE form uses big-endian byte serialization (most significant byte first), the LE form uses little-endian byte serialization (least significant byte first) and the unmarked form uses big-endian byte serialization by default, but may include a byte order mark at the beginning to indicate the actual byte serialization used.

so when there is no BOM, the string should be treated as big-endian. libiconv has a different opinion about this and will try to guess if it should use big-endian or little-endian depending on the operating system. so you will get different results on different machines.

The simple solution to this problem (after a long time trying to identify it), is to just tell iconv that I’m converting from UTF-16BE (big-endian) so it won’t try to guess the endianess of the bytes. so in php it will be like that

$result = iconv('UTF-16BE', 'UTF-8', $str);

or better, I can check the BOM before converting the Hexadecimal codes to a stream of bytes and taking the decision of converting from UTF-16BE or UTF-16LE depending on if it begins with FEFF or FFFE.