Doomed Before You Start

I've had some blog posts floating around in my head for a while about various things I come across in environments that are good indicators that down the road in a development project you are going to run into some severe problems.  By severe I mean failure.  By failure I mean way over cost / exceedingly late.  I have spent quite a bit of time over the years working with executive teams to understand why their IT initiatives fail.

The project is late... again, and over budget... again.  Releases are filled with bugs, the users have lost faith, morale on the development team is low.  Management is furious and dismayed, they don't understand how all these highly paid "professionals" can put out such a low quality work product, especially when they started practicing that hot new "Agile Development".  Hopefully at this point they reach out for help.

 "So how many Agile projects have you completed?", is usually the first question I ask.  Usually the answer to that is "well this is our first one".  "So this project you're so concerned about, it's not really that important?"  At this point I tend to get looks of disbelief, "Of course it's important, it's key to X".

The Virginal Methodology

Here's big huge mistake number one.  You decided to change your process at the outset of a critical, time sensitive project.  Implementing any process takes time, the team has to grow accustomed to it, and there is always a period of inefficiency as they feel their way through it.  By implementing a new process on a critical, time sensitive project, you have already sacrificed a good chunk of your productivity.  So we shouldn't be surprised when the schedule begins to slip.

Loss of Trust

Big mistake number two is continuing onward, without stopping, doing a retrospective, and repairing the damage that has been done.  Feeling pressure to meet the deadlines, the team forges onward even though every member knows in their heart that things are not going well.  Unfortunately, over time this leads to an even bigger organizational problem, the various team members begin to stop trusting each other.

Trust is a critical component of any methodology.  When trust starts to fail, the signs are pretty easy to see.  Teams will begin circling the wagons, scapegoating other team members, former team members, just about anything to deflect blame.  Oftentimes as a defensive measure, teams will start insisting on unreasonable levels of detailed specification before work begins.  The ad-hoc/hallway conversations that so often spread domain knowledge far more effectively than formal meetings cease, as team members turn to email communications so that they have a "documentation trail" in case someone tries to throw them under the bus.

Bottom line, if the team has lost trust, your organization is critically ill, and drastic measures must be taken.  To build trust, you must create an environment where the following behaviors are present:

1. Absolute Honesty- As much as we feel pressure to please, we must be clear about what we are capable of and within what time frame.  It is your responsibility at the end of the day to get your tasks completed with sufficient quality, and hamstringing yourself before you begin is foolish.  This also means that your peers accept that you know your capabilities better than they do and support you in your assessments.  If the time estimated is unacceptable, it is up to the team to find a create solution as a collective, not tear down the person and pressure them into setting unreasonable expectations.
2. All communication is safe- Building on #1, the environment must be safe.  Everyone involved in the project should feel comfortable asking questions and voicing concerns without fear of retaliation.
3. Empowerment & Accountability- As a manager or executive, you need to trust your team.  If they need resources and can make the case for them, you should strive to provide them.  This also means that you hold them accountable for achieving the goals.  It is vital to any human resources decisions (hiring and firing) that you ensure that you have removed as many organizational and cultural challenges that you possibly can from the team.
4. Remove Single Function Groups- Having a Development Team, a QA team, a Business Analyst team, etc all separated off into their own worlds builds barriers to communication and trust.  If you are in the same geographic location there almost no excuse to not create cross functional teams and communication zones.  
5. Avoid the Silo- Jack does Jack's tasks and Jill does Jill's tasks.  Here you are creating silos of domain knowledge and making the blame game easier.  Don't see a software task as separate components, see it as a whole.  A developer and a QA analyst and a Business Analyst should be responsible for taking feature X from start to finish.  This helps you combat that situation where decisions by non-technical staff have little value into the actual development and the situation where decisions by developers aren't aligned with organizational interest.

You Assumed Process = Quality

Processes and methodology are tools, and tools can be misused.  Big mistake number three is assuming that just because you implement a process that your quality will improve.  "Well, we moved to short iterations, but we're still falling behind schedule", "We're making the developers write tests, but the code quality is still poor".

For a process to be successful it must be embraced by the team and practiced.  The testing quote above is something I see a lot in the wild, the organization says "the code quality isn't good enough, we need more tests!".  It's very appealing to blame a process for failure, since generally nobody will be held accountable for it.

The truth of the matter is that it is your people who create quality.  I'm sorry to say that most struggling organizations I come across have the wrong people organized the wrong way.  Going back to Agile there is something I must point out:

Implementing Agile on a weak team makes them WORSE

Software teams are comprised of knowledge workers and the ones who are skilled an passionate are going to succeed provided that the organizational culture supports them.  When a weak team is presented with Agile, it usually means that whatever sense of discipline and craftsmanship may have been there will quickly go out the window.

Your Organization Misunderstands That Software is a Craft

This is where you really need a champion in the organization to really communicate with all levels and stakeholders.  Software is a craft and as such it has far more in common with Artisan skills than Science (it's a mix of both, but the Artisan part is more important in day to day development).  

To become an Artisan requires a lifetime of learning and discipline

Some organizations don't believe in dedicating funds to training.  Other organization provide funds to training, but they assume that just because they sent a developer to a week of "Technology X boot camp" that they will return and be able to perform at a good level.  Have you ever heard of anyone getting piano lessons for a week and then becoming a concert pianist?  We learn from applying our skills in repetition.  Patterns emerge and confidence builds over time.

The Artisan system has been around for thousands of years, and it always involves shared learning and apprenticeship.  This is the biggest reason why it is absolutely crucial to foster communication and also why Paired Programming is increasingly adopted in organizations.  You will learn far more through continued interactions with your peers than you ever will from training or technical manuals.  Training and manuals are absolutely great for getting your feet wet, but excellence will be achieved far more effectively when working with peers.

Building an artisan team also requires that executives let go of the myth of 100% utilization.  You simply can not produce a good product if your utilization is 100% because it leaves no time for personal growth, mentoring, or improving existing code based upon what you know now versus what you knew then.  I often tell developers that I mentor that if you don't look back at code you wrote 6 months ago and hate it, you probably aren't learning anything.

If you look back at the points above, you can see why I left the craftsmanship portion for last on the list.  Building and retaining a team of Artisan software professionals requires the organizational culture to be in the right place before it has any chance of occurring.

The Moral of the Story

The vast majority of the time I am asked to assess and assist with a struggling organization when it comes to IT, the biggest problems start with the culture.  Once you've embraced a culture of communication, trust, and accountability then you can begin building that team of craftsmen who can take your shop to the next level.

MSDB tip, clean up your mail

Just a quick tip since I encountered this recently.  Came across a MSDB database in my journey that was exceptionally large, did some delving and found that they were using database mail to send attachments frequently and no one ever bothered to clean up the history.

By default, MSDB will store all those attachments you send, so you should clean them up or it will grow... well forever.  So Microsoft provides a system stored procedure that allows you to purge ones older than a specified date.  So for example if you wanted to purge everything older than 30 days, you could do something like this:

 declare @purgeDate datetime  
 SET @purgeDate = dateadd(dd, -30, getdate())  
 EXECUTE msdb.dbo.sysmail_delete_mailitems_sp   
   @sent_before = @purgeDate;