Getting an IT assessment: pitfalls to watch for

One key ongoing goal of mine is that I constantly strive to pay attention. In this case, specifically, through web logging reports, I can see the Google searches that drive people to this blog every day.  One of the most common of these, it turns out, is people searching on the phrase “how to improve IT department”. Another is “IT assessment”.  I somehow picture bleary-eyed CEOs and COOs, late at night, pondering how they can get more throughput or better results from IT, and turning to Google in their frustration.

Since it’s in essence a frequently requested topic, let’s talk about it.  I’ve been on both ends of such assessments, multiple times.  I’ve done them, and I’ve had them done for me.  Before entering into such an assessment, it’s worth considering some of the surrounding issues and common pitfalls.


I should first note that there are basically two kinds of assessment along these lines: tune-ups (which tend to be granular, tactical, and more specifically technical) and the “what’s fundamentally wrong with IT” kinds of assessments that are initiated when there’s long-standing dissatisfaction with the IT Department.  Most of my remarks concern the latter, which pops up distressingly often, usually at the behest of upper management.  The former, tune-ups, are what you, the CIO/CTO, should be initiating yourself at regular intervals, say every couple of years.  It’s a massive benefit to get some outside help (assuming it’s impartial) in looking carefully at your departments tools, practices, and staffing and seeing where the opportunities lie that might have otherwise gone unseen or unprioritized.

The other kind, addressing the question of what’s fundamentally wrong with the IT department, usually comes after business/technology alignment issues have persisted with upper management, whether that be your CEO, your board, or other senior executives in the company.  Such assessments are a little harder to swallow, but should equally be welcomed; if those perceptions exist, they need to be addressed, and outside help may aid you greatly by actually bolstering some of the same approaches that you’ve been trying to push within the company.  Sadly, outsiders sometimes are more readily believed than the people doing the work.

Let’s look at some of the issues and pitfalls more specifically.

1. Your staff, almost certainly, isn’t going to like the fact that the department is being “poked and prodded.” Make sure you lead by example in actively championing the benefits of getting this assessment.
2. It’s absolutely best to get an experienced, senior outsider to conduct this assessment.  For one thing, as I once heard Ted Nelson point out, “the fish cannot see the water”.  Even if you have the skills, experience, and tenure to conduct your own assessment, the weight carried by an outsider’s opinion, ironically, will be stronger than yours.
3. Be careful of hiring someone with an axe to grind (specifically, with services to sell).  For this reason, I recommend steering away from engaging the big consulting companies for such an assessment, or even smaller consulting companies, and suggest going with an independent.  Often, a larger firm will regard an IT assessment as simply a foot in the door, on the road to longer and more lucrative engagements.
4. Steer the effort, from its inception, in the direction of looking at IT/business interaction as a holistic issue. I’ve often seen such assessments unearth fundamental flaws in non-IT areas, most commonly in how the business played their role (or didn’t play a role) in project prioritization. Stakeholders across the business need to be prepared for such a finding.
   
5. No matter what level of involvement you have during the assessment, you need to realistically expect that more negatives will come out of the study than you think are accurate or justified.  After all, the company is not paying the assessor to discover that everything is fine, and most will tend to err on the side of exposing every possible blemish they perceive.  My advice, once again, is to cultivate a thick skin in yourself and your staff.
6. Equally, the findings of the assessment will often serve to confirm the points that you and your internal staff may have already been making to others at the company, but which were not being received well by stakeholders. It’s easy to have your staff, if not you yourself, feel some disgruntlement about that. To repeat: have a thick skin, and lead by example.
7. Insist that the assessment be useful, not simply voluminous and abstract.  It should sketch out some concrete plans of action, with possible timelines and a careful discussion of tradeoffs.  What’s not effective is for you or the company to simply put the resulting document into a drawer and do nothing about its findings.  I’ve actually seen this happen fairly frequently.
8. By the same token, don’t just merrily add the findings (as new initiatives) to the already teeming pile of things you have to get done.  As with all efforts, you need to weigh the tradeoffs, see how the internal pressures (from above) may have changed, and proceed accordingly.

Expanding on that last point: no assessment will substitute for what you need to do every day, as part of the basics of running an IT department. So, returning to those basics, as the CIO, how can you improve your IT department?  The major elements of your answer shouldn’t be much different from what any external assessment will tend to cover:

• measure the various areas (development, operations, QA, project management, etc.) over a reasonable period of time so that you’re dealing with actual data;
• prioritize the issues that are most injurious to the company and to the department’s perceived effectiveness; and
• plan specific measures for remediation;
• get buy-in to the plan
  

So the most important takeaway is that whether it’s internally or externally generated and pushed, prioritization is always king. You need to keep in mind what should be obvious: You can’t take a department from “zero to sixty” overnight.  Equally, you can’t solve all the deficiencies or problem spots at the same time.  Trying to do it all together, and all at the same time, is just plain nuts. Any assessment of quality will agree on that point.