Economic Value of Systems Engineering

This navigational tool is a prototype under development SEBoK 1.0. Comments are welcome for improvement.
Use [+] to expand and [-] to collapse. Clicking on a title will open that article.

{{#star:

}}

Trends Toward Increasing the Value of Systems Engineering

The systems engineering (SE) of traditional artifacts, such as railroads, reservoirs, and refrigerators, could count on having a relatively self-contained system to engineer, with relatively stable requirements , a sound scientific base, and numerous previous precedents. However, more and more systems need to be engineered to become parts of several evolving systems of systems (sos) with rapidly increasing scale, dynamism, interdependence, human-intensiveness, sources of vulnerability, and novelty, placing an ever higher economic value on the performance of effective SE. This is corroborated by the Case Studies and Vignettes in Part 7. The shortfalls in SE in the United States Federal Aviation Administration (FAA) Advanced Automation System (AAS), United States Federal Bureau of Investigation (FBI) Virtual Case File System, Hubble Space Telescope, and Therac-25 medical linear accelerator, led to either extremely expensive cancelled systems or much more expensive systems in terms of total cost of ownership or loss of human lives. On the other hand, the Global Positioning System (GPS), Miniature Seeker Technology Integration Project (MSTI), and Next Generation Medical Infusion Pump Project, all demonstrated that investment in thorough SE, led to highly cost-effective systems. Figure 1 summarizes the analyses by Werner Gruhl relating investment levels in SE to cost overruns of United States National Aviation Administration (NASA) projects (Stutzke 2005).

Figure 1. Relation of SE Investments to NASA Program Cost Overruns (Stutzke 2005) Released by NASA HDQRT./Gruhl)

Further Quantitative Evidence of the Value of Systems Engineering

Analysis of the effects of shortfalls in systems architecture and risk resolution (the results of inadequate SE) for software-intensive systems in the 161-project COnstructive COst MOdel II (COCOMO™ II) [1] database, showed a statistically-significant increase in rework costs as a function of project size: averages of 18% rework for ten thousand source lines of code (SLOC) projects and 91% rework for ten million SLOC projects. This result has been used to help enable major system projects to rectify their initial under investments in SE (e.g., Boehm et al. 2004), and to help determine “how much SE is enough” in general by balancing the risks of SE underinvestment and overinvestment (often called “analysis paralysis”), as shown in Figure 2 (Boehm-Valerdi-Honour 2008).

Figure 2. Risk-Balanced “How Much SE Is Enough” (Boehm, Valerdi, Honour 2008) Reprinted with permission of John Wiley & Sons Inc.

In general, small projects can quickly compensate for neglected SE interface definition and risk resolution, but as projects get larger, with more independently-developed components , the late-rework costs rise much more rapidly than the savings in reduced SE effort. Medium-sized projects have relatively flat operating regions, but very large projects pay extremely large penalties for neglecting thorough SE.

A calibrated model for further determining “how much SE is enough” has been developed in (Valerdi 2008). It estimates the number of person-months that a project will need for its SE as a function of system size (numbers and complexity of requirements , interfaces, operational scenarios, and key algorithms) modified by 14 factors that account for the effects on needed SE effort of requirements, architecture understanding, technology maturity, legacy-system migration, personnel capabilities , process maturity, tool support, etc.

The quantitative effort project-data determined results are corroborated by the results of extensive surveys and case study analyses. Survey data on software cost and schedule overruns in My Life Is Failure: 100 Things You Should Know to Be a Better Project Leader (Johnson 2006) indicate that the primary sources of the roughly 50% of the commercial projects with serious “software overruns” were actually due to shortfalls in SE (lack of user input, incomplete requirements, unrealistic expectations, unclear objectives, and unrealistic schedules). The extensive Software Engineering Institute (SEI) [2] - National Defense Industrial Association (NDIA) [3] survey of 46 government-contracted industry projects showed a strong correlation between higher project SE capability and higher project performance (Elm et al. 2007). Ongoing research combining project data and survey data reported in "Toward An Understanding of The Value of SE" (Honour 2003) and "Effective Characterization Parameters for Measuring SE" (Honour 2010) provide additional evidence of the economic value of SE, and further insights on SE critical success factors. Further information can be found in the “Value Proposition for SE” article in Part 5, Enabling Systems Engineering.

References

Works Cited

Boehm, B., Brown, A.W., Basili, V., andTurner, R. 2004. "Spiral Acquisition of Software-Intensive Systems of Systems," CrossTalk, May, pp. 4-9.

Boehm, B., R. Valerdi, and E.C. Honour. 2008. "The ROI of Systems Engineering: Some Quantitative Results for Software-Intensive Systems." Systems Engineering, 11(3): 221-234.

Elm, J. P., D.R. Goldenson, K. El Emam, N. Donatelli, and A. Neisa. 2008. A Survey of Systems Engineering Effectiveness-Initial Results (with Detailed Survey Response Data). Pittsburgh, PA, USA: Software Engineering Institute, CMU/SEI-2008-SR-034. December 2008.

Honour, E.C. 2003. "Toward An Understanding of The Value of Systems Engineering." First Annual Conference on Systems Integration, March 2003, Hoboken, NJ, USA.

Honour, E.C. 2010. "Effective Characterization Parameters for Measuring Systems Engineering." Proceedings of the 8th Annual Conference on Systems Engineering Research (CSER). March 17-19, 2010. Hoboken, NJ, USA.

Johnson, J. 2006. My Life Is Failure: 100 Things You Should Know to Be a Better Project Leader. Boston, MA, USA: Standish Group International.

Stutzke, R. 2005. Estimating Software-Intensive Systems. Boston, MA, USA: Addison Wesley.

Valerdi, R. 2008. The Constructive Systems Engineering Cost Model (COSYSMO): Quantifying the Costs of Systems Engineering Effort in Complex Systems. Saarbrücken, Germany: VDM Verlag.

Primary References

Boehm, B., R. Valerdi, and E.C. Honour. 2008. "The ROI of Systems Engineering: Some Quantitative Results for Software-Intensive Systems." Systems Engineering, 11(3): 221-234.

Honour, E.C. 2010. "Effective Characterization Parameters for Measuring Systems Engineering." Proceedings of the 8th Annual Conference on Systems Engineering Research (CSER). March 17-19, 2010. Hoboken, NJ, USA.

Valerdi, R. 2008. The Constructive Systems Engineering Cost Model (COSYSMO): Quantifying the Costs of Systems Engineering Effort in Complex Systems. Saarbrücken, Germany: VDM Verlag.

Additional References

Elm, J.P., D.R. Goldenson, K. El Emam, N. Donatelli, and A. Neisa. 2008. A Survey of Systems Engineering Effectiveness-Initial Results (with Detailed Survey Response Data). Pittsburgh, PA, USA: Software Engineering Institute, CMU/SEI-2008-SR-034. December 2008.

Johnson, J. 2006. My Life Is Failure: 100 Things You Should Know to Be a Better Project Leader. Boston, MA, USA: Standish Group International.

Vanek, F., R. Grzybowski, P. Jackson, and M. Whiting. 2010. "Effectiveness of Systems Engineering Techniques on New Product Development: Results from Interview Research at Corning Incorporated." Proceedings of the 20th Annual INCOSE International Symposium. 12-15 July 2010. Chicago, IL.

< Previous Article | Parent Article | Next Article >

Comments from 0.5 Wiki

This article is new to the SEBoK for version 0.75. As such, there are no associated 0.5 comments. Because of this, it is especially important for reviewers to provide feedback on this article. Please see the discussion prompts below.

SEBoK v. 1.9.1 released 30 September 2018

SEBoK Discussion

Please provide your comments and feedback on the SEBoK below. You will need to log in to DISQUS using an existing account (e.g. Yahoo, Google, Facebook, Twitter, etc.) or create a DISQUS account. Simply type your comment in the text field below and DISQUS will guide you through the login or registration steps. Feedback will be archived and used for future updates to the SEBoK. If you provided a comment that is no longer listed, that comment has been adjudicated. You can view adjudication for comments submitted prior to SEBoK v. 1.0 at SEBoK Review and Adjudication. Later comments are addressed and changes are summarized in the Letter from the Editor and Acknowledgements and Release History.

If you would like to provide edits on this article, recommend new content, or make comments on the SEBoK as a whole, please see the SEBoK Sandbox.

blog comments powered by Disqus