Risk Management
The purpose of risk management is to reduce potential risks to an acceptable level before they occur throughout the life of the product or project. Risk management is a continuous, forward-looking process that is applied to anticipate and avert risks that may adversely impact the project. Risk management can be considered a project management or a systems engineering process. A balance must be achieved on each project in terms of overall risk management ownership, implementation, and day-to-day responsibility between by these two top-level processes.
Risk is a measure of the potential inability to achieve overall program objectives within defined cost, schedule, and technical constraints. It has two components: 1) the probability (or likelihood) of failing to achieve a particular outcome, and 2) the consequences (or impact) of failing to achieve that outcome. (DAU, 2003a) (In the domain of catastrophic risk analysis, risk has three components: 1) threat, 2) vulnerability, and 3) consequence.) (Willis et al. 2005)
Risk management involves defining a risk management strategy, identifying and analyzing risks, handling selected risks, and monitoring the progress in reducing risks to an acceptable level. (SEI 2010; DoD 2006; DAU 2003a; DAU 2003b; PMI 2008) (Opportunity and opportunity management is briefly discussed in Subsection 3 below.)
Risk Management Process Overview
The SE risk management process includes the following activities:
- Risk planning
- Risk identification
- Risk analysis
- Risk handling
- Risk monitoring
Risk Planning
Risk planning establishes and maintains a strategy for identifying, analyzing, handling, and monitoring risks within the project. The strategy, both the process and its implementation, is documented in a risk management plan (RMP).
The risk management process and its implementation should be tailored to each project, updated as appropriate throughout the life of the project, and the RMP transmitted in an appropriate means to the project team and key stakeholders.
The RMP should contain key risk management information, including: 1) a project summary; 2) project acquisition and contracting strategies; 3) key definitions; 4) a list of key documents; 5) process steps; 6) inputs, tools and techniques, and outputs per process step; 7) linkages between risk management and other project processes; 8) key ground rules and assumptions; 9) risk categories; 10) seller and buyer roles and responsibilities; and, 11) organizational and personnel roles and responsibilities. (Conrow 2003). The level of detail should be risk-driven: simple plans for low risk projects; detailed plans for high risk projects.
Risk Identification
Risk identification is the process of examining the project products, processes, and requirements to identify and document candidate risks. Risk identification should be performed continuously at the individual-level as well as through formerly structured events at both regular intervals and following major program changes (e.g., project initiation, re-baselining, change in acquisition phase).
It should use one or more top-level approaches (e.g., Work Breakdown Structure, key processes evaluation, key requirements evaluation) and one or more lower-level approaches (e.g., affinity, brainstorming, checklists and taxonomies, examining critical path activities, expert judgment, fishbone diagrams). (Conrow 2009) For example, lower-level checklists and taxonomies exist for software risk identification (Conrow and Shishido 1997, 83-89, p. 84; Boehm 1989, 115-125, Carr et al. 1993, p. A-2) and operational risk identification (Gallagher et al. 2005, p. 4), and have been used on a wide variety of programs. Both the top and lower-level approaches are essential but there is no single acceptable method—all approaches should be examined and used as appropriate.
Candidate risk documentation should include the following items where possible: 1) risk title; 2) a structured risk description; 3) applicable risk categories; 4) potential root causes; 5) relevant historical information (e.g., actions to date); and 6) responsible individual and manager. (Conrow 2003, p. 198)
It is important to use structured risk descriptions such as an If (an event occurs--trigger) Then (an outcome or affect occurs) Because (of the following reasons, or root cause) format. Another useful construct is a Condition (that exists) which leads to a potential Consequence (outcome). (Gluch 1994) These approaches help the analyst to better think through the potential nature of the risk.
Risk analysis and risk handling activities should only be performed on approved risks because of the scarcity of resources and opportunity costs in not efficiently focusing on the correct risks.
Risk Analysis
Risk analysis is the process of systematically evaluating each identified, approved risk to estimate the probability of occurrence (likelihood) and consequence of occurrence (impact), converting the results to a corresponding risk level or rating.
While there is no single “best” approach for a given risk category, risk scales and a corresponding matrix, simulations, and probabilistic risk assessments are often used for technical risks, while decision trees, simulations, and payoff matrices are used for cost risk, and simulations are used for schedule risk. Risk analysis approaches are sometimes grouped into qualitative and quantitative methods. A structured, repeatable methodology should be used in order to increase analysis accuracy and reduce uncertainty.
The most common qualitative method uses (typically) ordinal probability and consequence scales coupled with a risk matrix (also known as a risk cube or mapping matrix) to convert the resulting values to a risk level. Here, one or more probability of occurrence scales, coupled with three consequence of occurrence scales (cost, performance, schedule) are typically used. Mathematical operations should not be performed on ordinal scale values to prevent erroneous results. (Conrow 2003, pp. 187-364)
Once the risk level for each risk is determined, the risks need to be prioritized. Prioritization is typically performed by risk level (e.g., low, medium, high), risk score [the pair of max (probability), max (consequence) values], and other considerations such as time-frame, frequency of occurrence, and interrelationship with other risks. (Conrow 2003, pp. 187-364) An additional prioritization technique is to convert results into an estimated cost, performance, and schedule value (e.g., probability * dollar consequence). However, the result is only a point estimate and not a distribution of risk.
Widely used quantitative methods include decision trees and the associated expected monetary value analysis (Clemen and Reilly 2001), modeling and simulation (Law 2007; Mun 2010; Vose 2000), payoff matrices (Kerzner 2009, pp. 747-751), probabilistic risk assessments (Kumamoto and Henley 1996; NASA 2002), and other techniques. Risk prioritization can directly result from the quantitative methods employed.
For quantitative approaches, care is needed in developing the model structure, since the results will only be as good as the accuracy of the structure coupled with the characteristics of probability estimates or distributions (Law 2007; Evans, Hastings, and Peacock 2000) used to model risk that is present.
If multiple risk facets exist for a given item (e.g., cost risk, schedule risk, and technical risk), the different results should be integrated into a cohesive three-dimensional “picture” of risk. Sensitivity analyses can be applied to both qualitative and quantitative approaches in an attempt to understand how potential variability will affect results. Particular emphasis should be paid to compound risks (e.g., highly coupled technical risks with inadequate fixed budgets and schedules).
Risk Handling
Risk handling is the process that identifies and selects options and implements the desired option to reduce a risk to an acceptable level, given program constraints (budget, other resources) and objectives. (DAU 2003a, pp. 20-23, 70-78)
For a given system of interest, risk handling is primarily performed at two levels. At the system level, the overall ensemble of system risks is initially determined and prioritized, and second-level draft risk element plans (REPs) are prepared for handling the risks. For more complex systems, it is important that the REPs at the higher system-of-interest level are kept consistent with the system RMPs at the lower system-of-interest level, and that the top-level RMP preserves continuing risk traceability across the systems of interest.
The risk handling strategy selected is the combination of the most desirable risk handling option coupled with a suitable implementation approach for that option. (Conrow 2003) Risk handling options include assumption, avoidance, control (mitigation), and transfer. All four options should be evaluated and the best one chosen for each risk. An appropriate implementation approach is then chosen for that option. Hybrid strategies can be developed that include more than one risk handling option, but with a single implementation approach. Additional risk handling strategies can also be developed for a given risk and either implemented in parallel with the primary strategy or made a contingent and implemented if a particular trigger event occurs during the execution of the primary strategy. Often, option choice is difficult because of uncertainties in the risk probabilities and impacts. In such cases, buying information to reduce risk uncertainty via prototypes, benchmarking, surveying, modeling, etc. will clarify risk handling decisions (Boehm, 1981).
Risk Handling Plans
A risk handling plan (RHP, a REP at the system level), should be developed and implemented for all high and medium risks and selected low risks as warranted.
Each RHP should include : 1) a risk owner and management contacts, 2) selected option, 3) implementation approach, 4) estimated probability and consequence of occurrence levels at the start and conclusion of each activity, 5) specific measurable exit criteria for each activity, 6) appropriate metrics, and 7) resources needed to implement the RHP (e.g., personnel, funding, test equipment, etc.). (Conrow 2003, pp. 365-387)
Metrics included in each RHP should provide an objective means of determining whether the risk handling strategy is “on track,” and whether it needs to be updated. On larger projects these can include earned value, variation in schedule and technical performance measures (TPMs), and changes in risk level vs. time.
The activities present in each RHP should be integrated into the project’s integrated master schedule or equivalent; otherwise there will be ineffective risk monitoring and control.
Risk Monitoring
Risk monitoring is used to evaluate the effectiveness of risk handling activities against established metrics and provide feedback to the other risk management process steps. Risk monitoring results may also provide a basis to update RHPs, develop additional risk handling options and approaches, and re-analyze risks. In some cases, monitoring results may also be used to identify new risks, revise an existing risk with a new facet, or revise some aspects of risk planning. (DAU 2003a, p. 20) Some risk monitoring approaches that can be applied include: 1) earned value, 2) program metrics, 3) TPMs, 4) schedule analysis, and 5) variations in risk level. Risk monitoring approaches should be updated and evaluated at the same time and WBS level; otherwise, the results may be inconsistent.
Opportunity and Opportunity Management
In principle, opportunity management is the dual of risk management, with two components: probability of achieving an improved outcome, and impact of achieving the outcome. Thus, both should be addressed in risk management planning and execution. In practice, however, a positive opportunity exposure will not match a negative risk exposure in utility space, since the positive utility magnitude of improving an expected outcome is considerably less than the negative utility magnitude of failing to meet an expected outcome [Canada, 1971; Kahneman-Tversky, 1979]. Further, since many opportunity-management initiatives have failed to anticipate serious side effects, all candidate opportunities should be thoroughly evaluated for potential risks to prevent unintended consequences from occurring.
Linkages to Other Systems Engineering Management Topics
The measurement process provides indicators for risk analysis. Project planning involves the identification of risk and planning for stakeholder involvement. Project Assessment and Control monitors project risks. Decision management evaluates alternatives for selection and handling of identified and analyzed risks.
Practical Considerations
Key pitfalls and good practices related to systems engineering risk management are described in the next two sections.
Pitfalls
Some of the key pitfalls encountered in performing risk management are below.
Name | Description |
---|---|
Process Over-reliance |
|
Lack of Continuity |
|
Tool and Technique Over-reliance |
|
Lack of Vigilance |
|
Automatic Mitigation Selection |
|
Sea of Green |
|
Band-Aid Risk Handling |
|
Good Practices
Some good practices, gathered from the references are below.
Name | Description |
---|---|
Top Down and Bottom Up |
|
Early Planning |
|
Risk Analysis Limitations |
|
Robust Risk Handling Strategy |
|
Structured Risk Monitoring |
|
Update Risk Database |
|
References
Citations
Boehm, B. 1981. Software Engineering Economics. Upper Saddle River, NJ, USA:Prentice Hall.
Boehm, B. 1989. Software Risk Management. Los Alamitos, CA; Tokyo, Japan: IEEE Computer Society Press: 115-125.
Canada, J.R. 1971. Intermediate Economic Analysis for Management and Engineering. Upper Saddle River, NJ, USA:Prentice Hall.
Carr, M., S. Konda, I. Monarch, F. Ulrich, and C. Walker. 1993. Taxonomy-based risk identification. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie-Mellon University (CMU), CMU/SEI-93-TR-6.
Charette, R., L. Dwinnell, and J. McGarry. 2004. Understanding the roots of process performance failure. CROSSTALK: The Journal of Defense Software Engineering (August 2004): 18-24.
Clemen, R., and T. Reilly. 2001. Making hard decisions. Boston, MA, USA: Duxbury.
Conrow, E. 2003. Effective Risk Management: Some Keys to Success. 2nd ed. Reston, VA, USA: American Institute of Aeronautics and Astronautics (AIAA).
Conrow, E. 2008. Risk analysis for space systems. Paper presented at Space Systems Engineering and Risk Management Symposium, 27-29 February, 2008, Los Angeles, CA, USA.
Conrow, E., and P. Shishido. 1997. Implementing risk management on software intensive projects. IEEE Software 14 (3) (May/June 1997): 83-9.
DAU. 2003a. Risk Management Guide for DoD Acquisition: Fifth Edition, Version 2. Ft. Belvoir, VA, USA: Defense Acquisition University (DAU) Press.
DAU. 2003b. U.S. Department of Defense extension to: A guide to the project management body of knowledge (PMBOK(R) guide), first edition, version 1. 1st ed. Ft. Belvoir, VA, USA: Defense Acquisition University (DAU) Press.
DoD. 2006. Risk Management Guide for DoD Acquisition. Washington, D. C.: Office of the Under Secretary of Defense (Acquisition, Technology & Logistics)/Department of Defense, Sixth Edition, Version 1.
Evans, M., N. Hastings, and B. Peacock. 2000. Statistical distributions. 3rd ed. New York, NY, USA: Wiley-Interscience.
Gallagher, B., P. Case, R. Creel, S. Kushner, and R. Williams. 2005. A taxonomy of operational risk. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie-Mellon University (CMU), CMU/SEI-2005-TN-036.
Gluch, P. 1994. A Construct for Describing Software Development Risks. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie-Mellon University (CMU), CMU/SEI-94-TR-14.
Kerzner, H. 2009. Project management: A systems approach to planning, scheduling, and controlling. 10th ed. Hoboken, NJ: John Wiley & Sons.
Kahneman, D., and Tversky, A. 1979. Prospect theory: An analysis of decision under risk. Econometrica 47 (2) (Mar., 1979), pp. 263-292.
Kumamoto, H., and Henley, E. 1996. Probabilistic Risk Assessment and Management for Engineers and Scientists. 2nd ed. Piscataway, NJ, USA: Institute of Electrical and Electronics Engineers (IEEE) Press.
Law, A. 2007. Simulation modeling and analysis. 4th ed. New York, NY, USA: McGraw Hill.
Mun, J. 2010. Modeling risk. 2nd ed. Hoboken, NJ, USA: John Wiley & Sons.
NASA. 2002. Probabilistic risk assessment procedures guide for NASA managers and practitioners, version 1.1. Washington, D.C., USA: Office of Safety and Mission Assurance/National Aeronautics and Space Administration (NASA).
PMI. 2008. A guide to the project management body of knowledge (PMBOK guide). 4th ed. Newtown Square, PA, USA: Project Management Institute (PMI).
Scheinin, W. 2008. Start early and often: The need for persistent risk management in the early acquisition phases. Paper presented at Space Systems Engineering and Risk Management Symposium, 27-29 February 2008, Los Angeles, CA, USA.
SEI. 2010. Capability Maturity Model Integrated (CMMI) for Development, version 1.3. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie Mellon University (CMU).
Vose, D. 2000. Quantitative risk analysis. 2nd ed. New York, NY, USA: John Wiley & Sons.
Willis, H. H., A. R. Morral, T. K. Kelly, and J. J. Medby. 2005. Estimating terrorism risk. Santa Monica, CA, USA: The RAND Corporation, MG-388.
Primary References
Boehm, B. 1981. Software Engineering Economics. Upper Saddle River, NJ, USA:Prentice Hall.
Boehm, B. 1989. Software Risk Management. Los Alamitos, CA; Tokyo, Japan: IEEE Computer Society Press: 115-125.
Conrow, E. H. 2003. Effective Risk Management: Some Keys to Success. 2nd ed. Reston, VA, USA: American Institute of Aeronautics and Astronautics (AIAA).
DoD. 2006. Risk Management Guide for DoD Acquisition. Washington, D. C., USA: Office of the Under Secretary of Defense (Acquisition, Technology & Logistics)/Department of Defense, Sixth Edition, Version 1.
SEI. 2010. Capability Maturity Model Integrated (CMMI) for Development, version 1.3. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie Mellon University (CMU).
Additional References
Canada, J.R. 1971. Intermediate Economic Analysis for Management and Engineering. Upper Saddle River,NJ, USA: Prentice Hall.
Carr, M., S. Konda, I. Monarch, F. Ulrich, and C. Walker. 1993. Taxonomy-based risk identification. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie-Mellon University (CMU), CMU/SEI-93-TR-6.
Charette, R. 1990. Application strategies for risk management. New York, NY, USA: McGraw-Hill.
Charette, Robert. 1989. Software engineering risk analysis and management. New York, NY, USA: McGraw-Hill (MultiScience Press).
Charette, R., L. Dwinnell, and J. McGarry. 2004. Understanding the roots of process performance failure. CROSSTALK: The Journal of Defense Software Engineering (August 2004): 18-24.
Clemen, R. T., and T. Reilly. 2001. Making hard decisions. Boston, MA, USA: Duxbury.
Conrow, E. 2010. Space program schedule change probability distributions. Paper presented at American Institute of Aeronautics and Astronautics (AIAA) Space 2010, 1 September 2010, Anaheim, CA, USA.
Conrow, E. 2009. Tailoring risk management to increase effectiveness on your project. Presentation to the Project Management Institute, Los Angeles Chapter, 16 April, 2009, Los Angeles, CA.
Conrow, E. 2008. Risk analysis for space systems. Paper presented at Space Systems Engineering and Risk Management Symposium, 27-29 February, 2008, Los Angeles, CA, USA.
Conrow, E., and P. Shishido. 1997. Implementing risk management on software intensive projects. IEEE Software 14 (3) (May/June 1997): 83-9.
DAU. 2003a. Risk Management Guide for DoD Acquisition: Fifth Edition, Version 2. Ft. Belvoir, VA, USA: Defense Acquisition University (DAU) Press.
DAU. 2003b. U.S. Department of Defense extension to: A guide to the project management body of knowledge (PMBOK(R) guide), first edition, version 1. 1st ed. Ft. Belvoir, VA, USA: Defense Acquisition University (DAU) Press.
Dorofee, A., J. Walker, C. Alberts, R. Higuera, R. Murphy, and R. Williams, eds. 1996. Continuous risk management guidebook. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie-Mellon University (CMU).
Evans, M., N. Hastings, and B. Peacock. 2000. Statistical distributions. 3rd ed. New York, NY, USA: Wiley-Interscience.
Gallagher, B., P. Case, R. Creel, S. Kushner, and R. Williams. 2005. A taxonomy of operational risk. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie-Mellon University (CMU), CMU/SEI-2005-TN-036.
Gluch, P. 1994. A Construct for Describing Software Development Risks. Pittsburgh, PA, USA: Software Engineering Institute (SEI)/Carnegie-Mellon University (CMU), CMU/SEI-94-TR-14.
Haimes, Y. Y. 2009. Risk modeling, assessment, and management. Hoboken, NJ,USA: John Wiley & Sons, Inc.
Hall, E. 1998. Managing risk: Methods for software systems development. New York, NY, USA: Addison Wesley Professional.
INCOSE. 2011. INCOSE systems engineering handbook, version 3.2.1. San Diego, CA, USA: International Council on Systems Engineering (INCOSE), INCOSE-TP-2003-002-03.2.1.
ISO. 2009. Risk management—Principles and guidelines. Geneva, Switzerland: International Organization for Standardization (ISO), ISO 31000:2009.
ISO/IEC. 2009. Risk Management—Risk Assessment Techniques. Geneva, Switzerland: International Organization for Standardization (ISO)/International Electronical Commission (IEC), ISO/IEC 31010:2009.
ISO/IEC/IEEE. 2006. Systems and software engineering - Risk management. Geneva, Switzerland: International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC)/Institute of Electrical and Electronics Engineers (IEEE). ISO/IEC/IEEE 16085.
ISO. 2003. Space systems - Risk Management. Geneva, Switzerland: International Organization for Standardization (ISO), ISO 17666:2003.
Jones, C. 1994. Assessment and control of software risks. Upper Saddle River, NJ, USA: Prentice-Hall.
Kahneman, D., and Tversky, A. Prospect theory: An analysis of decision under risk. Econometrica 47(2) (Mar., 1979), pp. 263-292.
Kerzner, H. 2009. Project management: A systems approach to planning, scheduling, and controlling. 10th ed. Hoboken, NJ: John Wiley & Sons.
Kumamoto, Hi., and Henley, E.. 1996. Probabilistic Risk Assessment and Management for Engineers and Scientists. 2nd ed. Piscataway, NJ, USA: Institute of Electrical and Electronics Engineers (IEEE) Press.
Law, A. 2007. Simulation modeling and analysis. 4th ed. New York, NY, USA: McGraw Hill.
Mun, J. 2010. Modeling risk. 2nd ed. Hoboken, NJ, USA: John Wiley & Sons.
NASA. 2002. Probabilistic risk assessment procedures guide for NASA managers and practitioners. Version 1.1. Washington, D.C., USA: Office of Safety and Mission Assurance/National Aeronautics and Space Administration (NASA).
PMI. 2008. A guide to the project management body of knowledge (PMBOK guide). 4th ed. Newtown Square, PA, USA: Project Management Institute (PMI).
Scheinin, W. 2008. Start early and often: The need for persistent risk management in the early acquisition phases. Paper presented at Space Systems Engineering and Risk Management Symposium, 27-29 February 2008, Los Angeles, CA, USA.
USAF. 2005. SMC systems engineering primer & handbook: Concepts, processes, and techniques. 3rd ed. Los Angeles, CA, USA: Space & Missile Systems Center/U.S. Air Force (USAF).
Vose, D. 2000. Quantitative risk analysis. 2nd ed. New York, NY, USA: John Wiley & Sons.
Willis, H. H., A. R. Morral, T. K. Kelly, and J. J. Medby. 2005. Estimating terrorism risk. Santa Monica, CA, USA: The RAND Corporation, MG-388.
Article Discussion
Signatures
--Groedler 02:38, 31 August 2011 (UTC)
--Dholwell 16:59, 2 September 2011 (UTC)core edit.