Professional Blog

Over the past 5 years I have conducted Assessments of over 100 IT Programs and Projects, mainly for the Federal Government. A confirmed technophile and computer scientist, when I started this I was looking forward to finding juicy technical problems I could help solve. I did find a few and managed to save clients millions on impractical investments. However, it didn't take long to realize that most of the problems were related to missing, immature or out of control processes. While such risks tend to be less dramatic, they easily compound over time, and can stay under the manager's radar until they suddenly emerge to become major problems.

I based these assessments on structured interviews that involved all major areas of program/project management across the entire life-cycle; covered by some 70+ questions. After the interviews, the program was rated as having high, medium or low risk in each of the questions. Seven major areas had at least one component question with 1/3 or more of interviews scoring moderate to high risk. These areas were:

1. Resources. This question was unique because the PM was allowed to identify the risks and suggest a risk score for them. Not surprisingly, 74% of the PMs felt at risk. More surprisingly, the most common shortages reported involved in-house personnel, either through the loss of key personnel or shortages of personnel with the required skills. Most managers felt they had adequate funding to deliver their product. Some of the more common complaints about funding shortages for were for transition costs, unfunded mandates (including documentation), costs caused by delays waiting for deliverables from external programs, and budgets for technical refresh in sustainment.

2. System Engineering Documents/Processes. Non-technical processes and their documentation were found to be incomplete or outdated in two thirds of the assessments. Larger mature programs had well established processes, but saw little value in updating documentation to reflect current Agency standards and requirements. Smaller programs cited lack of resources to formalize and document their processes.

3. Transition Management. Nearly two thirds of the assessments found little or inadequate planning for transitioning to operations early in the program life-cycle Programs nearing the end of development frequently (39%) did not have transition processes, documentation, contracts and funding in place to smooth transition. The receiving organization often did not have the capability (both training and resources) to provide the required support. Funding for technology refresh during sustainment was also common risk area. System retirement was often poorly planned and underfunded. As soon as a program was identified as being at end-of-life it was frequently viewed as a source of resources and stripped of critical personnel and funding before the transition to the successor program was complete.

4. Net Centricity/Service Orientation. Thin client web based systems have become more of the norm, but many systems were not being developed to readily share information with other web based applications. There was generally a lack of clear standards, standard development tools, transition road maps, design guidelines and, most of all, a corporate culture encouraging sharing information. Better standards and tools have developed in recent years, but this is still a stretch for government programs.

5. Testing & Performance Measurement. Common causes of risks in testing and QA included pressure to field quickly and inadequate funding for developing performance metric capabilities and end-to-end traffic simulation. Smaller programs often did not have adequate corporate resources to help with developing test pans, testbeds and traffic simulation. Many programs had not adequately identified performance goals and metrics, and lacked quantitative data to estimate server and bandwidth requirements to ensure adequate performance under operational traffic loads.

6. Customer Communications. Nearly half of the Assessments were found to have significant risks in this area. Customer communication mechanisms ad hoc, without planned and documented processes. While most had good communications with their sponsors and immediate customers, they had difficulty collaborating with broader user groups and stakeholders. Many managers realized collaborative websites would be useful for communicating with large user communities and coordination within the team. However, Agency policies and lack of available collaborative services made it difficult to establish collaborative capabilities.

7. Development Process Maturity. Common sources of risk involved:

• Requirements. Problems included a lack of formalized requirements, traceability between requirements and system design and documented requirements that were not specific enough to guide development. Some of the most critical risk areas were difficulties managing customer expectations and "requirements creep." This was most likely to occur when there was no formal requirements review process to triage new requirements, where no senior individual or group had legitimate authority to approve and disapprove requirements.
• Risk Identification and Management. Newer and smaller programs, in particular, often did not routinely assess risks and develop mitigation strategies as an ongoing activity.
• Documented Development Processes and System Design. These documents were frequently not available from contractors performing the work, making the development effort a "black box" to the government. In particular, adequate system documentation and architectures were not often available for Quality Assurance (QA) and to help ensure continuity in the event of changes of contractors or key personnel.
• Configuration Management. Formal CM processes were often neglected in smaller projects. This risked developing a poorly documented patchwork of capabilities and modifications that were difficult to troubleshoot and maintain. In many cases these problems were found when there were not standard corporate CM processes or tools for tracking requirements and change requests.
• Processes of external organizations. Corporate processes external to the program or project frequently hindered rapid fielding and development. Some of these included: time to fill vacant billets, time to staff critical documents through the approval chain, processes for transitioning the developed system to operations, and the time required to transfer funds and get them on contract.

While these issues were unfortunately common, there is hope. We had the opportunity to conduct follow-up assessments on a number of these programs after a year or more. The improvements were dramatic. Over two thirds of the initial assessments indicated high risk (having more than 20% of the questions judged as moderate to high risk). Bringing potential problems to the attention of the teams, their leaders, and corporate management helped reduce that number to just one in ten. A little extra attention to filling in the gaps in project management can go a long way!