How we decided to use Mongo instead of MySQL

I got a question recently from Sadek my new colleague at Eventtus about why we did decide to use MongoDB instead of something like MySQL, especially that dealing with Mongo is hard if you’re used to the use of things like Joins in other relational DBs. Luckily we had a 2 years old internal email thread where I listed why I think using Mongo would be a good idea for our case. Eventtus is 2+ years old startup, a social platform for events where you use the mobile app as a guide during events, and as a way to find interesting events happening near you. I will put the important parts of the email thread here, and sure don’t take this as a guarantee that MongoDB would work for all kinds of problems. it just was a good fit for our case, but maybe won’t be good solution for your problem and some of our future problems.

….

(read the rest on Eventtus’s Engineering Blog)

Introduction to MongoDB [PHPMaster]

This is kinda old news for some of you. My 2nd PHPMaster article was published like a month ago but I forgot to write about it here. This time it’s a step by step introduction to using MongoDB in PHP.

In this article I’ll introduce you to NoSQL with MongoDB. You’ll learn how to install the MongoDB extension for PHP, and how to add, update, and retrieve document objects. If you’re used to working with RDBMSs like MySQL or PostgreSQL, you’ll find some of the concepts of working with MongoDB a bit strange, but you’ll soon grow to love the flexibility and power that MongoDB gives you!

You can read the article here and wait for the 2nd part which is scheduled to be published on jan 20, 2012 (published: MongoDB Revisited). it will discuss more advanced topics that you will need in real life projects.

 

New article on PhpMaster.com about Zend_Pdf

I wrote a new article about using Zend_Pdf , which was published on PhpMaster.com days ago.

The PDF format is currently the most used format to exchange documents. If you provide your website users with printable versions of invoices, event tickets and other similar documents, you’ll most likely want to generate them as PDFs on the fly. In this article you will see how you can use Zend_Pdf to auto-generate PDF invoices.

I’m glad to write for the first time for PhpMaster, Sitepoint.com ‘s PHP blog. I have been reading their online content and books since 2005 and having the chance to write for them now is an honer.

check the article there.

The story of version 1.1

Once upon a time, a software project took around 7 months to get to version 1.0 . We didn’t have any problem all that time with our SVN (Subversion) repository as we didn’t have to release anytime before version 1.0 . we didn’t have any SVN strategy till that moment and all of us were happily committing to the trunk and enjoying a branching-less life.

The last 2 months before releasing version 1.0 were mainly focused on enhancing performance and bug fixing. we couldn’t stop at that point as we had many developers with nothing to do and more features waiting to be added to the next version. the work on the new features had to go in parallel with the code enhancements being applied to version 1.0 to get it released and ready to be deployed to production. we did create another branch for the 1.1 version.

Some of our developers started committing to the 1.1 branch while some others continued to commit to the trunk. then after 2 months of work, we wanted to release the 1.1 version. the plan was to merge the 1.1 branch into trunk, test it, fix bugs and then deploy that to production. things didn’t go that simple as we had very big code base and there were many differences between the 1.1 branch and the trunk. merging took more than a week and lots of bugs that were fixed before in the trunk, came back to life with more new bugs that had to be fixed as fast as possible to get the 1.1 version released and deployed to production.

The bug fixing phase took more than what was expected and our product team (who were playing the rule of the owner in our little software development game) had a plan with lots of features that had to be completed before the end of year and they didn’t want to stop the development of new features till we finish fixing all the bugs.

They gave us a plan for the rest of the year and they wanted to change our release strategy so we release and deploy to production at the end of every week. this was a kinda good strategy for our kind of business. the problem was that we still didn’t have a stable code base that we were sure that we can build on. we had lots of fatal bugs waiting to be fixed and we didn’t yet release version 1.1 .

Anyway, we started working on the features of the first week of the plan. while the amount of features was more than what can be finished in one week, we couldn’t finish the features in less than 10 days. this left us behind the plan by some days and we didn’t have enough time yet to fix the bugs of version 1.1 . we were pushed more to start working on the planned features for the second week. that week had more complicated features and it took us 2 weeks to finish it.

At that moment we had all that code in the trunk with lots of new implemented features and more bugs. we didn’t release anything in the previous 3 months (since version 1.0) . the project didn’t benefit of the new features. every team is blaming the other for making things go like that. the product team had more plans and features to be implemented till the end of year. the development team can’t make a new release with all those bugs.

that’s not the end of the story, but I have to stop here and start analyzing what happened. mainly, I see 2 problems.

First, the development team was committing all the code directly into the trunk. we would have created a branch for each feature. this way each feature will stay isolated with all its bugs and then when it’s stable, we can merge the branch into the trunk. this will keep the trunk always ready for deployment at any moment.

like what Mr.Radwan say, “your main code base is gold, you can’t mix it with rubbish code”. this way we can keep our gold in the trunk always ready for deployment and we can keep history of each release in a tag with the right label.

branching in source control systems

Second, the product team shouldn’t have continued requesting new features while there were many bugs not fixed and while they can’t release the finished features. they should have stopped till the development team can get all the bugs fixed and then they can meet with the development team to see how they can stop the problem from appearing again in the future.

software development isn’t that simple and not respecting this craft can turn a project into a big failure. problems in projects like that doesn’t appear just over night, people are the main reason of the problems. if you can’t control all the participants and keep them focused on what they should do, they will start behaving the way they like and will start fighting each other instead of collaborating. if they were focused, they would have noticed that the project is shifting from the right way and they would have started early to fix this.

Soap Web Services in PHP & hint on consuming from .Net

last week, I was working on implementing a SOAP server for one of our projects at work, this web service was built to be consumed by some other software that is built using .Net . It wasn’t that easy and I faced some problems that made me go crazy for some hours and I wanted to share them here with the solutions. let me first show you how I built the web service in PHP.

by the way, This project was built using CakePHP, but the web service I made doesn’t depend that much on that framework except that I use its MVC implementation.

If we want to build a SOAP web service, we will need first to create a WSDL file and make it accessible from a defined URL. I made a new controller with a new action named [index] that will return the WSDL file when requested using GET, and will also handle SOAP requests when requested using POST.

<?php

class SoapController extends AppController
{

    /**
     * handles SOAP requests
     */
    public function inedx()
    {

        if ( $this->RequestHandler->isGet() ) { //just show the wsdl
            $this->RequestHandler->respondAs('xml');
            echo self::_getWsdlContent();
            return;
        }

        $soapServer = new SoapServer( Configure::read('soap.wsdlFilePath') );

        $soapHandler = new Shreef_WebService_SoapHandler( /* Inject Dependencies here */ );

        $soapServer->setObject($soapHandler);

        $soapServer->handle();

    }

    /**
     * gets WSDL content for the SOAP service
     */
    protected static function _getWsdlContent()
    {
        $fileName = Configure::read('soap.wsdlFilePath');

        if (file_exists($fileName) ) {
            return file_get_contents($fileName);
        }

        $soapAutodiscover = new Zend_Soap_AutoDiscover( );

        $wsdlContent = $soapAutodiscover
                        ->setClass('Shreef_WebService_SoapHandler')
                        ->toXml();

        file_put_contents($fileName, $wsdlContent);
        return $wsdlContent;
    }

}

as you see, I’m dynamically generating the WSDL file using Zend_Soap_AutoDiscover. and caching it into a file. Zend_Soap_AutoDiscover will read the DocBlocks defined in Shreef_WebService_SoapHandler to build the WSDL file.

Notice that I gave SoapServer an instance of Shreef_WebService_SoapHandler instead of giving it the name of the class. this will give me the chance to inject any needed dependencies into the constructor.

here is my SoapHandler class with a simple method [getCurrentTime()] that will return a String, and another method [doSomething()] that returns Array.

<?php

class Shreef_WebService_SoapHandler
{

    /**
     * construct
     */
    public function __construct()
    {
        /** Dependency Injection happens here **/
    }

    /**
     * returns the current server time formated like YYYYMMDDhhmmss
     *
     * @return String
     */
    public function getCurrentTime()
    {
        return date('Ymdhis');
    }

 
    /**
     * do something
     *
     * @param String $something
     * @return Array
     */
    public function doSomething($something)
    {
        return array(1, 2, 3);
    }
  
    /** other methods **/
}

and that’s it, now if I open http://localhost/soap in my browser, I will see the WSDL. then I can use that WSDL to make requests to the web service. this was working fine on my machine and on the testing server, but it started to go crazy when we deployed that to the staging server. all methods returned null.

DocBlocks and OpCode caching

After some hours I noticed that the generated WSDL on the staging server was different than what was generated on my machine and on the testing server. there were no Out messages and the type of all method parameters was set to “anyType”. at first, I thought that this is a bug in Zend_Soap_AutoDiscover, but later I commented out the lines that cache the generated WSDL to a file and started to change things around.

I noticed that whenever I make a change to the file that contains the SoapHandler, the WSDL gets generated like expected for one time, but if I open the URL of the WSDL again, it will go back to generate the wrong WSDL. and here we have another “Aha!!” moment.

We have eAccelerator installed on that server. it is doing opcode caching for all the PHP code. this means that after every modification to the file that contains the SoapHandler class, the class is read from the file with all the DocBlocks, but in the next request, SoapHandler classs will get read from the opcode cache with all DockBlocks removed. so Zend_Soap_AutoDiscover won’t find DocBlocks to read and will guess that all methods are void and all method parameters are of type “anyType”.

As stopping eAccelerator doesn’t make sense, I turned on the WSDL file caching again and ran the “touch” command on all files that contain code that should get described in the WSDL ( the soap handler and any other complex types used ). this will make the WSDL get generated well the first time and cached.

I have to automate this later using a shell script or something.

.Net and Soap Arrays

The second problem was that Visual Studio.net was refusing to generate the required proxy classes using that WSDL file. It didn’t understand the type “Array”. I searched for this problem, and it looked like that were no way to make VS.net understand it. I solved that by instead of returning values of type Array, I changed my code a bit to return a complex type.

Also, I had to tell Zend_Soap_AutoDiscover that I want to use another WSDL strategy by passing the string ‘Zend_Soap_Wsdl_Strategy_ArrayOfTypeComplex’ to its constructor.