Identifying Risks to Software Development Projects

Threats to software development projects are often minimized or overlooked altogether because they are not as tangible as risks to projects in other industries. The risks are there though and just as capable of derailing the software development project as a project in any other industry.

Most project managers in the information technology field have had the experience of planning a software development project down to the last detail, planning the effort for each of the tasks in the plan down to the last hour and then having some unforeseen issue come along that derails the project and makes it impossible to deliver on time, or with the feature set originally envisioned.

Successful project managers in any industry must also be skillful risk managers. Indeed, the insurance industry has formalized the position of risk manager. To successfully manage the risks to your software development project, you first must identify those risks. This article was written to provide you with some tips and techniques to help you do that. There are a few terms that are not directly applicable to the activity of identifying risks that are helpful to understand before studying risk identification. These are some of those definitions:

• Risk event This is the event that will affect the project if it should happen.
• Threat A risk event that will have a negative impact on the scope, quality, schedule, or budget of the project should it happen.
• Opportunity Not all risks are threats, some are opportunities which will have a positive impact on scope, quality, schedule, or budget should they happen. Threats should be avoided, or their impacts diminished and opportunities encouraged, or their impacts enhanced.
• Probability The likelihood that a risk event will happen. This is what people in the gambling business call odds.
• Impact Usually refers to a comparative cardinal or ordinal rank assigned to a risk event. It may also refer to an absolute monetary value, period of time, feature set, or quality level.
• Risk Tolerance This refers to your organization’s approach to taking risks. Is it conservative? Does your organization welcome calculated risks?
• Risk Threshold Your organization’s risk tolerance will usually be expressed as a cardinal or ordinal comparator using the risk events probability and impact to produce the comparator. Risks whose Probability/Impact score exceed this threshold will be avoided or mitigated. Risks whose score is below the threshold are acceptable.
• Risk Contingency This is a sum allotted to the project for the purpose of managing risks. It should be split into two sums: one for managing identified risks and one for managing unidentified risks, or unknown unknowns. The sum can be either a monetary amount or an amount of time.

The project manager of a software development project can look to several sources for help in identifying risks: common risks (risks common to every software development project), risks identified with the performing organization, risks identified with the SDLC methodology chosen for the project, risks specific to a development activity, Subject Matter Experts, risk workshops, and surveys.

Common Risks

There are a number of risks that are common to every software development project regardless of size, complexity, technical components, tools, skill sets, and customers. The following list contains most of these:

• Missing requirements Requirements needed by the software system to be developed to meet the business goals and objectives of the project.
• Misstated requirements Requirements that have been captured but the original intent has been lost or misconstrued in the process of capturing them.
• Key or critical resources are lost to the projectThese resources are usually single contributors, or team members with skill sets in scarce supply for which there is a strong demand in the performing organization. The potential impact of losing the resource for any period of time will be increased if they are assigned tasks on the critical path.
• Bad estimation The estimations for effort required for developing the software are either significantly understated (bad) or overstated (also bad). Underestimation is the most common event. Work tends to be prolonged until it takes up all the time allotted by an overestimation.
• Missing or incomplete skill sets The results of this risk event will be the same as the results of bad estimation, but the risk will be mitigated differently. The result of a junior programmer being identified as an intermediate programmer may be a significant increase in the amount of effort required to produce their deliverables, or a complete inability to produce them.

These risk events should be captured by the project manager at the outset of any risk identification exercise, even though they will probably be identified by someone else on the team. Making them visible to the team in advance of any risk identification exercises will avoid time wasted in calling them out and may stimulate thinking about associated risks ("…..what if Jane were to be called away to a higher priority project, might that also cause Fred to be lost to the project?”).

Organizational Risks

These are risks that are unique to the organization performing the project. They may include some of the risks in the list of common risks, and other sources, but will also include risks that have no other sources.

The project manager should consult the archives of previous software development projects for the common risks, where project records have been archived. Gather the risk registers of all the previous projects (or at least enough to provide you with a representative selection of risk registers) and try to match risks in each register. It is highly unlikely that a risk will be common across all projects where there is a good selection of registers but you should closely examine risks that appear in two or more registers for applicability to your project.

Survey the project managers responsible for past software development projects in your organization where archives are not available. It is possible that these project managers may have archived project artifacts including their risk registers, in their personal space even if the organization does not have a structured approach to archival. Getting the benefit of seasoned project manager’s experience from past projects will also be beneficial for deciphering the risk captured in archived risk registers.

Risks will not be stated in duplicate language across different registers (or across different project managers for that matter). You will need to analyze the risk event statement to determine where two or more risk events are identical, despite different descriptions.

SDLC Specific Risks

Your software development project will be exposed to some risks and shielded from others depending on which SDLC (Software Development Life Cycle) methodology you choose to use for your project. Risk avoidance is a significant consideration when choosing an SDLC for the project and your project should choose the SDLC which avoids or reduces the impact of the risks most probable in your case. To that end the identification of risks and the choice of an SDLC are like the chicken and the egg: it is difficult to determine which comes first. Here’s a tip for sequencing the two. Choose your SDLC based on the type of software system being developed and the organization you are developing it in (How experienced is the organization with the tools and components involved? How experienced are they with each SDLC? What are the project priorities?, etc.). Once you’ve decided on an SDLC you can identify the risks associated with it and if the level of risk associated with it exceeds your organization’s risk tolerance, you can re-visit your choice.

There are risks inherent with each different type or category of SDLC. We will talk about a few of the most common risks for the most popular types or categories of SDLC.

Waterfall

Projects using the Waterfall methodology for development will be most prone to any risk event impacting the schedule and that is because there are no intermediate checkpoints in the method to catch problems early on in the build phase. Delays to any activity from requirements gathering to User Acceptance Testing will delay the final delivery for the project. Risk events which fall into the "delay” category will include: delays due to unfamiliarity with tools or components (e.g. programming languages, test tools), delays due to underestimation of effort, delays due to inexperience, and delays due to requirements contributors missing deadlines.

Delays are not the only risk events a waterfall project is susceptible to. Waterfall projects are not well designed to propagate learning across the project so a mistake made in one area of development could be repeated across other areas and would not come to light until the end of the project. These mistakes could mean that development could take longer than necessary or planned, that more re-work is necessary than was initially allowed for, that scope is reduced as a result of discarding bad code, or that product quality suffers.

The Waterfall method tends to be used on larger projects which have a greater duration than other development methodologies making them prone to change. It is the job of the Change Management process to handle all requested changes in an orderly fashion but as the duration of the project increases so too do the chances that the project will be overwhelmed with requests for change and buffers for analysis, etc. will be used up. This will lead to project delays and budget overruns.

Rapid Application Development (RAD)

The intent of Rapid Application Development is to shorten the time required to develop the software application. The primary benefit from this approach is the elimination of change requests – the theory being that if you provide a quick enough turn-around there will be no necessity for changes. This is a double edged sword though. The fact that the method relies on the absence of change requests will severely limit the project’s ability to accommodate them.

The risks that will be the most likely to occur on a project using this methodology will have to do with the software applications fitness for use. The market or business could change during the project and not be able to respond to a resulting change request within the original schedule. Either the schedule will be delayed while the change is made, or the change will not be made resulting in the build of a system that does not meet the client’s needs.

The RAD method requires a relatively small team and a relatively small feature set to support a quick turn-around. One possible result of having a small team is a failure to have a needed skill set on the team. Another will be the lack of redundancy in the skill sets which means that the illness of a team member cannot be absorbed without delaying the schedule or getting outside help.

Scrum

The distinguishing characteristic of this development method is the lack of a project manager. This role is replaced by a team lead. The team lead may be a project manager, but it is unlikely that the performing organization will seek out and engage an experienced project manager to fulfill this role. The method avoids management by a project manager to avoid some of the rigors of project management best practices in an effort to streamline development. The risk introduced by this approach is that there will be a lack of necessary discipline on the team: change management, requirements management, schedule management, quality management, cost management, human resources management, procurement management, and risk management.

The lack of project management discipline could leave the project open to an inability to accommodate change properly resulting in changes being ignored or changes being incorrectly implemented. Lack of experience in human resources management could result in an unresolved conflict, or inappropriate work assignments.

Iterative Methods

The main iterative methods are RUP (Rational Unified Process) and Agile. These methods take an iterative approach to design and development so are lumped together here. This method is intended to accommodate the changes to a project that a dynamic business requires. The cycle of requirements definition, design, build, and test is done iteratively with each cycle spanning a matter of weeks (how long the cycles are will depend on the methodology). Iterative development allows the project team to learn from past mistakes and incorporate changes efficiently.

Iterative methods all rely on dividing the system up into components that can be designed, built, tested, and deployed. One of the advantages of this method is its ability to deliver a working model early on in the project. One risk inherent in this method is the risk that the architecture does not support the separation of the system into components that can be demonstrated on their own. This introduces the risk of not learning from a mistake that won’t be found until the users test the system.

There is a trade off implied in iterative development: develop a core functionality that can be demonstrated first vs. develop the component that will yield the most learning. Choosing core functionality to develop may introduce the risk of failing to learn enough about the system being developed to help future iterations. Choosing the most complex or difficult component may introduce the risk of failing to produce the system the client needs.

Activity Specific Risks

Each activity in a development cycle has its own set of risks, regardless of the methodology chosen. The requirements gatheringactivity has the following risks: the requirements gathered may be incomplete, the requirements gathered may be misstated, or the requirements gathering exercise may take too much time.

The design portion of the cycle will have the following risks: the design may not interpret the requirements correctly so that the functionality built will not meet the customer’s needs. The design could be done in a way that calls for more complexity in the code than necessary. The design may be written in such a way that it is impossible for a programmer to develop code that will function properly. The design could be written in a way that is ambiguous or difficult to follow, requiring a lot of follow up questions or risking bad implementation. There may be several stages of design from a Commercial Specification all the way to a Detail Design Document. The interpretation of requirements through each stage exposes the stated requirements to misinterpretation.

Programmers may misinterpret the specifications, even when those are perfectly written, risking the development of an application that does not satisfy requirements. The unit, function, and system testing may be slipshod, releasing errors into the QA environment that consume extra time to resolve. Different programmers may interpret the same specification differently when developing modules or functions that must work together. For example, a section of functional specification may deal with both the input of one module and the output of another that are given to two different programmers to develop. The risk is that the discrepancy will not be found until the software is integrated and system tested.

Testing here refers to Quality Assurance testing and User Acceptance testing. While these two activities are different from a tester perspective, they are similar enough to lump together for our purposes. Actual testing effort may exceed the planned effort because of the number of errors found. An excessive number of errors found during testing will cause excessive rework and retesting. Test script writers may interpret the specifications they are working from differently than analysts, programmers, or the clients. User Acceptance Testers come from the business community so are susceptible to the risk of business demands reducing or eliminating their availability.

Subject Matter Experts (SMEs)

Subject Matter Experts are key to the success of the project because of their knowledge. Subject Matter Experts can contribute to all areas of the project but are especially important to requirements gathering, analysis of change requests, business analysis, risk identification, risk analysis, and testing. The key risk for SMEs is that the SMEs key to your project may not be available when they are promised. This will be especially harmful when the SME is responsible for a deliverable on the critical path.

Risk Workshops

Risk workshops are an excellent tool for identifying risks. The workshops have the advantage of gathering a group of Subject Matter Experts in a room so that their knowledge is shared. The result should be the identification of risks that would not have been discovered by polling the SMEs individually and the identification of mitigation strategies that can address multiple risk events.

Advice on how to conduct productive workshops is outside the scope of this article but there are a few tips I’ll give you that may help you get started:

1. Invite the right SMEs – you need to cover all phases and all activities of the project.
2. Communicate all the details of the project you are aware of. These include deliverables, milestones, priorities, etc.
3. Get the project sponsor’s active backing. This should include attendance at the workshop where feasible.
4. Invite at least one SME for each area or phase.
5. Split the group into sub-groups by area of expertise, or project phase where you have large numbers of SMEs.
6. Make certain the different groups or SMEs communicate their risks to each other to encourage new ways of looking at their areas.

The risk workshop does not end with the identification of risks. They must be analyzed, collated, assessed for probability and impact, and mitigation or avoidance strategies devised for them.

Surveys

Surveys or polls are an acceptable alternative to risk workshops where your Subject Matter Experts are not collocated. The lack of synergy that you get with a workshop must be made up by you, however. You’ll need to communicate all the information that could be helpful to the Subject Matter Experts you identify at the outset of the exercise. Once that is done, you can send out forms for the SMEs to complete which will capture the risk events, the source of the risk, the way the risk event might impact the project objectives, etc.

Collate the risks after you receive them, and look for connected risk events. Connected risk events can either be different approaches to describing the same risk which allow you to combine the two risk events into one, or different risks can be addressed by the same mitigation strategy.

Lack of participation is another disadvantage of the survey or poll method. You may be able to get by with a single SME in one project phase or area of expertise but will have to follow up on reluctant contributors. Don’t hesitate to ask for your project sponsor’s help in getting the level of participation you need. You may even get them to send the invitation and survey forms out initially.

Team Meetings

So far all the sources of identified risks we have discussed have been associated with the planning phase of the project. Executing properly during the planning phase will allow you to gather a comprehensive list of risks, but they will tend to more accurately reflect risks to the earlier project phases than to later phases. Once you’ve created your initial risk register you must keep that document current as you learn more about the project by doing the work and risks become obsolete because the work exposed to the risk has been completed.

Team meetings are the ideal place to update your risk register. The issues that will be brought forward as the team discusses its progress towards completing its deliverables are often related to the risks of meeting the deadlines for the deliverable. You may want to set apart a segment of your meeting for reviewing the impact and probability scores of existing risks to determine the impact the passage of one week has had on them. You should also monitor the team for any new risks they can identify. Risks that went unnoticed when the work was first planned may become visible as the start date for the work gets closer, or more is learned about the work. The project may identify new work as the planned work is done which was not contemplated when risks were initially identified.

You may want to conduct separate risk strategy meetings with your SMEs in cases where the team is insufficiently acquainted with project risks to make them active contributors to an up to date risk register. You should use this approach in addition to your team meetings when your software development project is large enough to require sub-projects. Review each active risk in the register and analyze it for the impact the passage of time has had on it. Typically as work approaches the likelihood of the risk event and/or the impact will increase. As more of the work is done, the likelihood and impact will tend to decrease.

You should monitor the project plan for work that has been completed. Risks to the work just completed will be obsolete and should no longer form part of the discussion of risk probability and impact.

This article is one of a series of articles dealing with managing project risks. Look for the other articles in the series on this web site:     Risk Analysis, Risk Strategies, Risk Maintenance, and Risk and CP (Critical Path).