Difference between revisions of "Economic Value of Systems Engineering"
| Line 2: | Line 2: | ||
With traditional projects, such as railroads, reservoirs, and refrigerators, a [[Systems Engineer (glossary)|systems engineer]] faced a self-contained system that typically had relatively stable requirements, a sound scientific base, and numerous previous precedents. As most modern systems become parts within one or more evolving [[System of Systems (SoS) (glossary)|systems of systems]] (SoS), the performance of effective SE now takes on an ever-higher economic value, as the systems feature a rapidly increasing scale, dynamism, interdependence, human-intensiveness, sources of vulnerability, and novelty. | With traditional projects, such as railroads, reservoirs, and refrigerators, a [[Systems Engineer (glossary)|systems engineer]] faced a self-contained system that typically had relatively stable requirements, a sound scientific base, and numerous previous precedents. As most modern systems become parts within one or more evolving [[System of Systems (SoS) (glossary)|systems of systems]] (SoS), the performance of effective SE now takes on an ever-higher economic value, as the systems feature a rapidly increasing scale, dynamism, interdependence, human-intensiveness, sources of vulnerability, and novelty. | ||
| − | This is corroborated by the [[Case Studies]] and [[Vignettes]] in Part 7. Shortfalls in SE lead to either cancellation of already expensive systems or even more expensive systems in terms of total cost of ownership or loss of human life. Part 7 presents the problems in the United States Federal Aviation Administration (FAA) Advanced Automation System (AAS), United States Federal Bureau of Investigation (FBI) Virtual Case File System, the [[Hubble Space Telescope Case Study]] | + | This is corroborated by the [[Case Studies]] and [[Vignettes]] in Part 7. Shortfalls in SE lead to either cancellation of already expensive systems or even more expensive systems in terms of total cost of ownership or loss of human life. Part 7 presents the problems in the [[FAA Advanced Automation System (AAS) Vignette|United States Federal Aviation Administration (FAA) Advanced Automation System (AAS)]], [[FBI Virtual Case File System Case Study |United States Federal Bureau of Investigation (FBI) Virtual Case File System]], the [[Hubble Space Telescope Case Study]], and [[Medical Radiation Case Study|the Therac-25 medical linear accelerator]]. |
| − | On the other hand, the Global Positioning System (GPS), Miniature Seeker Technology Integration Project (MSTI), and Next Generation Medical Infusion Pump Project all demonstrate that investment in thorough SE results in highly cost-effective systems. Figure 1 summarizes the analyses data by Werner Gruhl, which relates investment levels in SE to cost overruns of the United States National Aeronautics and Space Administration (NASA) projects (Stutzke 2005). The results indicate that there is a general correlation between the amount invested in SE within a program and cost overruns, demonstrating the critical role of properly allocating SE resources. | + | On the other hand, the [[Global Positioning System Case Study |Global Positioning System (GPS)]], [[MSTI Case Study|Miniature Seeker Technology Integration Project (MSTI)]], and [[Next Generation Medical Infusion Pump Case Study|Next Generation Medical Infusion Pump Project]] all demonstrate that investment in thorough SE results in highly cost-effective systems. Figure 1 summarizes the analyses data by Werner Gruhl, which relates investment levels in SE to cost overruns of the United States National Aeronautics and Space Administration (NASA) projects (Stutzke 2005). The results indicate that there is a general correlation between the amount invested in SE within a program and cost overruns, demonstrating the critical role of properly allocating SE resources. |
[[File:NASA Image Part 1.png|700px|thumb|center|'''Figure 1. Relation of SE Investments to NASA Program Cost Overruns (Stutzke 2005).''' Released by NASA HDQRT./Gruhl.]] | [[File:NASA Image Part 1.png|700px|thumb|center|'''Figure 1. Relation of SE Investments to NASA Program Cost Overruns (Stutzke 2005).''' Released by NASA HDQRT./Gruhl.]] | ||
Revision as of 18:36, 17 April 2013
Trends Toward Increasing the Value of Systems Engineering
With traditional projects, such as railroads, reservoirs, and refrigerators, a systems engineer faced a self-contained system that typically had relatively stable requirements, a sound scientific base, and numerous previous precedents. As most modern systems become parts within one or more evolving systems of systems (SoS), the performance of effective SE now takes on an ever-higher economic value, as the systems feature a rapidly increasing scale, dynamism, interdependence, human-intensiveness, sources of vulnerability, and novelty.
This is corroborated by the Case Studies and Vignettes in Part 7. Shortfalls in SE lead to either cancellation of already expensive systems or even more expensive systems in terms of total cost of ownership or loss of human life. Part 7 presents the problems in the United States Federal Aviation Administration (FAA) Advanced Automation System (AAS), United States Federal Bureau of Investigation (FBI) Virtual Case File System, the Hubble Space Telescope Case Study, and the Therac-25 medical linear accelerator.
On the other hand, the Global Positioning System (GPS), Miniature Seeker Technology Integration Project (MSTI), and Next Generation Medical Infusion Pump Project all demonstrate that investment in thorough SE results in highly cost-effective systems. Figure 1 summarizes the analyses data by Werner Gruhl, which relates investment levels in SE to cost overruns of the United States National Aeronautics and Space Administration (NASA) projects (Stutzke 2005). The results indicate that there is a general correlation between the amount invested in SE within a program and cost overruns, demonstrating the critical role of properly allocating SE resources.
Further Quantitative Evidence of the Value of Systems Engineering
Analysis of the effects of shortfalls in systems architecture and risk resolution (the results of insufficient SE) for software-intensive systems in the 161-project Constructive Cost Model II (COCOMO™ II) database, shows a statistically significant increase in rework costs as a function of project size measured in source lines of code (SLOC): averages of 18% rework for ten-thousand-SLOC projects and 91% rework for ten-million-SLOC projects. This data has influenced many major system projects to reconsider initial underinvestment in SE (e.g., Boehm et al. 2004), and well as to address “how much SE is enough” by balancing the risks of under-investing in SE against those of over-investing (often called “analysis paralysis”), as shown in Figure 2 (Boehm, Valerdi, & Honour 2008).
Typically, small projects can quickly compensate for neglected SE interface definition and risk resolution; however, as projects grow larger and have more independently-developed components, the cost of late rework negates any savings in reduced SE effort. Additionally, medium-sized projects have relatively flat operating regions, while very large projects pay extremely large penalties for neglecting thorough SE. Extensive surveys and case study analyses corroborate these results.
Survey data on software cost and schedule overruns in My Life Is Failure: 100 Things You Should Know to Be a Better Project Leader (Johnson 2006) indicates that the primary sources of the roughly 50% of the commercial projects with serious “software overruns” are the result of shortfalls in SE (lack of user input, incomplete requirements, unrealistic expectations, unclear objectives, and unrealistic schedules). The extensive survey of 46 government-contracted industry projects conducted by the Software Engineering Institute (SEI)/National Defense Industrial Association (NDIA) illustrated a strong correlation between higher project SE capability and higher project performance (Elm et al. 2007). Ongoing research that combined project data and survey data reported in “Toward an Understanding of The Value of SE” (Honour 2003) and “Effective Characterization Parameters for Measuring SE” (Honour 2010) has provided additional evidence as to the economic value of SE and further insights on critical factors the affect SE success.
A calibrated model for determining “how much SE is enough” has been developed and is discussed in (Valerdi 2008) called [COSYSMO][1] (Constructive Systems Engineering Cost Model). It estimates the number of person-months that a project needs for SE as a function of system size (i.e., requirements, interfaces, algorithms, and operational scenarios), modified by 14 factors (i.e., requirements understanding, technology risk, personnel experience, etc.), which dictates the amount of SE effort needed. Other economic considerations of SE include the costs and benefits of reuse (Wang, Valerdi & Fortune 2010), the management of SE assets across product lines (Fortune & Valerdi, 2013), the impact of SE on project risk (Madachy & Valerdi 2010), and the role of requirements volatility on SE effort (Pena & Valerdi 2010).
References
Works Cited
Boehm, B., Brown, A.W., Basili, V., and Turner, R. 2004. "Spiral Acquisition of Software-Intensive Systems of Systems." CrossTalk. May, pp. 4-9.
Boehm, B., R. Valerdi, and E.C. Honour. 2008. "The ROI of Systems Engineering: Some Quantitative Results for Software-Intensive Systems." Systems Engineering. 11(3): 221-234.
Elm, J. P., D.R. Goldenson, K. El Emam, N. Donatelli, and A. Neisa. 2008. A Survey of Systems Engineering Effectiveness-Initial Results (with Detailed Survey Response Data). Pittsburgh, PA, USA: Software Engineering Institute, CMU/SEI-2008-SR-034. December 2008.
Fortune, J., and R. Valerdi. 2013. "A Framework for Systems Engineering Reuse." Systems Engineering 16(2).
Honour, E.C. 2003. "Toward An Understanding of The Value of Systems Engineering." Proceedings of the First Annual Conference on Systems Integration, March 2003, Hoboken, NJ, USA.
Honour, E.C. 2010. "Effective Characterization Parameters for Measuring Systems Engineering." Proceedings of the 8th Annual Conference on Systems Engineering Research (CSER). March 17-19, 2010. Hoboken, NJ, USA.
Johnson, J. 2006. My Life Is Failure: 100 Things You Should Know to Be a Better Project Leader. Boston, MA, USA: Standish Group International.
Madachy, R., and R. Valerdi. 2010. Automating Systems Engineering Risk Assessment. 8th Conference on Systems Engineering Research, Hoboken, NJ.
Pena, M., and R. Valerdi. 2010. "Characterizing the Impact of Requirements Volatility on Systems Engineering Effort." 25th Forum on COCOMO and Systems/Software Cost Modeling, Los Angeles, CA.
Stutzke, R. 2005. Estimating Software-Intensive Systems. Boston, MA, USA: Addison Wesley.
Valerdi, R. 2008. The Constructive Systems Engineering Cost Model (COSYSMO): Quantifying the Costs of Systems Engineering Effort in Complex Systems. Saarbrücken, Germany: VDM Verlag.
Wang, G., R. Valerdi, and J. Fortune. 2010. “Reuse in Systems Engineering,” IEEE Systems Journal. 4(3): 376-384.
Primary References
Boehm, B., R. Valerdi, and E.C. Honour. 2008. "The ROI of Systems Engineering: Some Quantitative Results for Software-Intensive Systems." Systems Engineering, 11(3): 221-234.
Honour, E.C. 2010. "Effective Characterization Parameters for Measuring Systems Engineering." Proceedings of the 8th Annual Conference on Systems Engineering Research (CSER). March 17-19, 2010. Hoboken, NJ, USA.
Valerdi, R. 2008. The Constructive Systems Engineering Cost Model (COSYSMO): Quantifying the Costs of Systems Engineering Effort in Complex Systems. Saarbrücken, Germany: VDM Verlag.
Additional References
Elm, J.P., D.R. Goldenson, K. El Emam, N. Donatelli, and A. Neisa. 2008. A Survey of Systems Engineering Effectiveness-Initial Results (with Detailed Survey Response Data). Pittsburgh, PA, USA: Software Engineering Institute, CMU/SEI-2008-SR-034. December 2008.
Johnson, J. 2006. My Life Is Failure: 100 Things You Should Know to Be a Better Project Leader. Boston, MA, USA: Standish Group International.
Vanek, F., R. Grzybowski, P. Jackson, and M. Whiting. 2010. "Effectiveness of Systems Engineering Techniques on New Product Development: Results from Interview Research at Corning Incorporated." Proceedings of the 20th Annual INCOSE International Symposium. 12-15 July 2010. Chicago, IL.
SEBoK Discussion
Please provide your comments and feedback on the SEBoK below. You will need to log in to DISQUS using an existing account (e.g. Yahoo, Google, Facebook, Twitter, etc.) or create a DISQUS account. Simply type your comment in the text field below and DISQUS will guide you through the login or registration steps. Feedback will be archived and used for future updates to the SEBoK. If you provided a comment that is no longer listed, that comment has been adjudicated. You can view adjudication for comments submitted prior to SEBoK v. 1.0 at SEBoK Review and Adjudication. Later comments are addressed and changes are summarized in the Letter from the Editor and Acknowledgements and Release History.
If you would like to provide edits on this article, recommend new content, or make comments on the SEBoK as a whole, please see the SEBoK Sandbox.
blog comments powered by Disqus