Learning to Swim in a Sea of Code

drowning in code

Quite often projects tend to become much larger than you anticipate going in.  Most of us have dealt with “feature creep” before, but it isn”t always so simple to spot up front.  It’s very easy to get lost in the energy of developing a new system and let your eyes get bigger than your stomach.  This is not a cautionary tale however, but a story of jumping into the deep end and swimming to the finish line.

When I first started working as a contractor, my first job was to integrate an e-commerce system with an existing website.  The scope of the project was fairly simple as they were using a Drupal framework already and I had experience with configuring the well documented module Ubercart with my own site.  Progress was quick and mostly painless as I setup the store front and added the handful of required products.

It bears mentioning at this point that the company I was working with is in the convention business and as such puts on a yearly event for which they sell badges.  They had commissioned a system to facilitate the tracking and purchasing of said badges several years prior and though it worked it had never been capable of fully serving their needs.  Most of the staff had settled on the assumption that they would never really get the improvements they wanted in this system or from the original team of developers that maintained it.

Once they saw the simplicity of the cart system I had installed for them it started their wheels turning.  They began to ask if the system could be used for other things as well, reporting, access control and of course badge purchasing.  While some of their specific use cases would require extra modules, none of it was too far-fetched.  This delighted them, as their previous development team had long since taken up the motto of “NO”.  Little did I know at the time that I was feeding a monster.

At this point there were a flurry of emails back and forth regarding possibilities of the new system.  I often found myself having to reassure them that radical changes to their established business processes were not only possible, but would yield them significant improvements over their old methods.  One of the departments had still been maintaining all of their registration of booths for vendors in a spreadsheet because they were terrified that the old system would somehow eat them.  Mind you there were good foundations for that fear.  In my attempts to calm their fears I had talked myself into a project now growing with exponential speed.

Soon after, we negotiated a new contract, and work on their new system started in earnest.  This is when things started to get interesting.  On top of the fairly large list of feature requests which seemed to grow weekly, if not daily, at times, I was told that I would have to develop the entire system in a somewhat clandestine fashion because the original team was still under contract and there was a real fear that they might simply torch the database and its backups if they knew they were being replaced.  Thankfully in the end this was not the case and they were truly happy to be rid of the job as they had been in over their heads.  None the less this added to the stress on a project that was already getting a little out of control.

Development marched forward with most of the milestones being met on time or very shortly afterward.  However, with each new feature completed their appetite seemed to grow and there was a real sense of urgency starting.  Since this was all work relating to an event that had been pre-scheduled for over a year there was no possibility of pushing back the final deadline.  There were roughly 20,000 people depending on me getting everything finished in time.  I realized that doubling my efforts would simply not be enough and so I had to work smarter.  A tough task especially given that I was in completely uncharted territory for myself, not having done a whole lot of PHP or Drupal programming before.

This is when I took some time to reevaluate my skills and my tools.  I had learned to do web programming in the early days when notepad was the only “acceptable” choice, and I had at least upgraded myself to Notepad++ which gave me syntax highlighting.  But it was during this self evaluation that I realized I had been doing it all wrong.  I had settled for tools that seemed simple but made me do all of the work.  I started to think about the problem from more of a C++ angle and realized that I missed having a full IDE with debugging capabilities, not to mention a development server.  Yes, up until this point everything had been done on the live server as the IT dept could not provide me with any alternative.

So with this fresh perspective I set forth on creating a better environment for myself to work in.  This is also when I found the true benefits of source control, but that is another article entirely.  It took a fair bit of time for me to setup my own Linux box, running Apache and MySQL, and get that to communicate with my debugger.  Not to mention establishing a whole new workflow using Netbeans and checking in my code.  This lead to a whole new level of productivity for me as I was now able to quickly iterate through new features.  Work was getting done much faster and with less bug squashing.

I was now also capable of creating more sweeping changes to existing functionality thanks to the development server.  Building and testing full featured subsystems before moving them into production.  My day to day workflow changed drastically from one of putting out fires constantly as new changes broke old functionality, to a much more relaxed launch of trusted code.

Even with all of the new improvements in my development environment there were hard choices to be made.  Some of the requested features would simply not be possible in the timeframe that I had left.  This is when it became supremely important to keep an open line of communication with the staff.  I was able to give them daily reports of progress and include them on decisions of development priority.  This had been of special importance as this is where things had gone sour with their old developers and is also the key point of failure in too many project post mortems.

In the end I finished all of the requested features, save one or two, and did it just in time.  If I had not stopped to re-evaluate my process and tools I know it would have been a different story and I think that learning to “swim” a different way is what saved me from drowning in the deep end.