Systems Engineering Heuristics
Author: Peter Brook
Heuristics provide a way for an established profession to pass on its accumulated wisdom. This allows practitioners and others interested in how things are done to gain insights from what has been found to work well in the past and to apply the lessons learned. Heuristics will usually take the form of short expressions in natural language. These can be memorable phrases encapsulating rules of thumb, shortcuts or "words to the wise", giving general guidelines on professional conduct or rules, advice, or guidelines on how to act under specific circumstances. Common heuristics do not summarize all there is to know, yet they can act as useful entry points for learning more. This article overviews heuristics in general as well as some of those specifically supporting Systems Engineering practice.
Overview
Heuristics have always played an important part in the history of engineering and shaped its progress, especially before science developed to the point when it could also assist engineers. Systems Engineering is still at a stage at which there is no sufficiently reliable scientific basis for many of the systems being built, which has triggered a renewed interest in heuristics to fill the gap. This is especially true as the practice of Systems Engineering is extended to provide solutions to inherently complex, unbounded, ill-structured, or "wicked" problems (Churchman 1967).
Using heuristics does not guarantee success under all circumstances, but usefulness of a heuristic can be maximized if the known extent of its applicability is made clear. At their best, heuristics can act as aids to decision making, value judgements, and assessments.
Heuristics have the potential to be useful in a number of ways:
- reduce the amount of thinking (or computation) needed to make a good decision or a choice
- help in finding an acceptable solution to a problem
- identify the most important factors to focus on while addressing a complex problem
- improve the quality of decisions by drawing on best practices
- avoid repeating avoidable mistakes
- act as an entry point to wider knowledge of what has been found to work
Historical Background
Engineering first emerged as a series of skills acquired while transforming the ancient world, principally through buildings, cities, infrastructure, and machines of war. Since then, mankind has sought to codify the knowledge of "how to." Doing so allows each generation to learn from its predecessors, enabling more complex structures to be built with increasing confidence while avoiding repeated real-world failures. Setting out the aims for engineering in the 1st Century BCE, Vitruvius proposed a set of enduring principles: Strength, Utility and Beauty. He provided many examples of their applications to the fields of engineering of the time.
Vitruvius’ writings were rediscovered in the Middle Ages, forming the basis of the twin professions of architecture and engineering. Early cathedral builders encapsulated their knowledge in a small number of rules of thumb, such as: "maintain a low centre of gravity," "put 80% of the mass in the pillars," and "observe empirical ratios between cross-section and span for cross-members.". Designs were conservative, with large margins, the boundaries of which were largely unknown. Numerous excellent structures resulted, many of which have endured to this day. When the design margins were exceeded, for example out of a desire to build higher and more impressive structures, a high price could be paid, with the collapse of a roof, a tower, or even a whole building. From such failures, new empirical rules emerged. Much of this took place before the science behind the strength of materials or building secure foundations was understood. Only in recent times has computer simulation revealed the contribution towards certain failures played by dynamic effects, such as those of wind shear on tall structures.
Since then, engineering and applicable sciences have co-evolved: science providing the ability to predict and explain performance of engineered artifacts with greater assurance, and engineering developing new and more complex systems, requiring new scientific explanations and driving research agendas. In the modern era, complex and adaptive systems are being built which challenge conventional engineering sciences, with the builders turning to social and behavioral sciences, management sciences, and increasingly systems science to deal with some of the new forms of complexity involved and to guide the profession accordingly.
Koen (1985), went further, and argued that the whole of engineering was essentially heuristic in nature, since engineers could never know everything about whether what they built would meet their original intent, and how it would interact with an uncertain world. According to Koen, we must do what we can to tie these things down but make allowances for what we cannot know. He defined the engineering method as: the strategy for causing the best change in poorly understood or uncertain situation within available resources. He further claimed that heuristics are the only way of guiding our actions in these circumstances, and that the set of all heuristics agreed or used by a profession at any one time represents its ‘state of the art’. He also put forward a number of universal heuristic principles for the conduct of engineering, covering areas such as: rules of thumb, factors of safety, engineers’ attitude towards their work, managing risk and allocating resources. Although much of what Koen (a chemical engineer by profession) had to say remains relevant to systems engineers today, there is renewed optimism that scientific SE principles can be uncovered to underpin the discipline, just as they have done through the ages. (see Systems Engineering Principles)
Modern Interest
Renewed interest in the application of heuristics to the field of Systems Engineering stems from the seminal work of Rechtin and Maier (2009). Their book remains the best single repository of such knowledge, although efforts are now under way within INCOSE to update them for the 21st Century. Their motivation was to provide guidance for the emerging role of system architect as the person or team responsible for coordinating engineering effort towards devising solutions to complex problems and overseeing their implementation. Rechtin and Maier observed that it was in many cases better to apply "rules of thumb" than to attempt detailed analysis, especially when this was precluded by the number of variables involved, the complexity of the interactions between stakeholders, and the internal dynamics of system solutions and the organizations responsible for their realization.
An argument in favor of the wider use of heuristics was also made by Mervyn King (2016), who was Governor of the Bank of England during the 2008 global financial crash. Looking back, he said "it is better to be roughly right than precisely wrong": banks which observed the old bankers’ rule of maintaining capital assets equal to 70% of their loan book survived, while those who relied on complex (and flawed) mathematical models of derivatives failed. He has become a powerful advocate for the use of heuristics alongside formal economics to allow bankers and others to deal with the uncertainties of global financial affairs in the modern interconnected world.
Further backing for the contemporary use of heuristics comes from Simon (1957), who coined the term "satisficing" for a situation in which people seek solutions, or accept choices or judgments, that are "good enough" for their purposes, regardless of whether they can be further optimized by precise analysis. He made the point that some heuristics were scientifically derived from experiment or systematic collection and analysis of real-world data, while others were just rules of thumb based on real-world observation or experience.
This idea has been further developed over a number of years by Gigerenzer and co-workers (for example, Gigerenzer and Selten (2001)), who have conducted research on heuristics which assist in making rapid decisions in areas of limited predictability, when probability theory is no longer helpful. They have developed a series of what they call "fast and frugal" rules of general applicability which have outperformed more conventional analysis in such areas as medical diagnosis and performance science (Raab and Gigerenzer 2015). The application to Systems Engineering has so far been little explored.
Practical Use
Experience of using heuristics suggests they should be memorable and can be most effective when phrased informally. The best examples are more than just literal expressions; they should resonate with the readers and suggest additional meaning, encouraging them to find out more about why and when to use them.
There is an underlying issue here: if you haven’t experienced the situation for which the heuristic applies, it may mean little to you, and the inherent value may well be lost; but if you already have relevant experience, you may find it obvious. The use of heuristics is therefore linked to how we learn. In a world where experience is vital, but the number of opportunities to learn on the job is limited by the number of big projects one systems engineer works on in a lifetime, the learning process has to be accelerated. Heuristics can help here if linked to other knowledge sources and accessible at the right points in a career, as part of life-long learning.
A repository of heuristics can also act as a knowledge base in its own right, especially if other media, such as video clips or training materials, or even interactive media are added to encourage discussion and feedback. Such a repository might also link to other established knowledge sources or company websites. It can be organized to reflect accepted areas of practice or in a data base tagged with metadata to allow flexible retrieval. Maier and Rechtin (2009) suggested that a repository might also act as a "reading room," allowing users to move freely among associated subjects as if they were following their curiosity in a library or a bookshop. Such a repository could also allow users to assemble a set of heuristics most meaningful to them, relevant to their personal interest or professional sphere of activity.
A further possible use of heuristics was demonstrated by Beasley, et al (2014), who used a selection in a survey to uncover how key aspects of Systems Engineering were addressed within their organization. They asked their staff to mark the heuristics according to their importance to the business and whether they were observed in practice. Analysis of the answers allowed attention to be given to areas requiring improvement.
Broadly speaking, interest in heuristics now centers on their use in two main contexts: first, encapsulating engineering knowledge in an accessible form, where the practice is widely accepted and the underlying science understood; and second, overcoming the limitations of more analytical approaches, where the science is still of limited use. In either case, a good set of heuristics requires active maintenance to reflect the evolution of practice and our constantly developing understanding of what works best. A well-curated collection of heuristics allows practitioners to retain and represent the accumulated practical wisdom of the community of the Systems Engineering profession.
This article finishes with a few simple examples applicable to early stages, with some commentary to suggest their hidden meanings and where they might lead.
- Don’t assume that the original statement of the problem is necessarily the best, or even the right one. The same point is repeated in many heuristics, such as: "The hidden assumptions are likely to be the most damaging," and "The customer may know what he wants, but not what he needs." All this has to be handled with tact and respect for the user, but experience shows that failure to reach mutual understanding early on is a fundamental cause of failure, and strong relationships forged in the course of doing such work can pay off when solving more difficult issues which might arise later on.
- In the early stages of a project, unknowns are a bigger issue than known problems. Sometimes what is being asked for is obscure, and the whole context of the systems engineering remit difficult to know, especially in areas of high uncertainty. The starting question may be less ‘What are the Requirements for this?" and more "What’s going on here?". (Kay and King 2020) Unpeeling the layers of meaning behind the second question may require methods drawn from systems thinking – looking for root causes, for example – and can take a project into unexpected areas. Again, soft skills such as empathy and intuition may be more important than those of conventional engineering.
- Model before build, wherever possible. This heuristic draws one in to the deeper question of the general use and limitation of models as a way that systems engineering tries to predict desirable and undesirable emergent system properties before committing to build it. A related heuristic states "When you test a model of a system in the real world, you validate the model not the system," and a Theorem from System Science states "The only complete model of the system is the system itself." One heuristic leads to another, leading to reflection on how systems are built in the highly uncertain world of hyperconnected IT systems, where one might postulate that "The only complete model of the system in its environment is the system in its environment," which leads into using evolutionary lifecycles, rapid deployment of prototypes, agile life cycles, and so on. The original heuristic opens a door into 21st Century systems.
- Most of the serious mistakes are made early on. This heuristic summarizes much of what has just been said. Just to show that much what is believed now might have been known for some time, here is a quote from Plato: "The beginning is the most important part of the work." (Plato 375 BCE).
Finally a heuristic about heuristics – where they come from and what they are for – is taken from a fortune cookie and provides a summary: "The work will tell you how to do it." (Wilczek 2015) The serious point being made here – apart from showing that folk wisdom can have a part to play – is that the most important lessons about how to do Systems Engineering ultimately derive from the collective experience of the community of systems engineers as they undertake their profession. Even the science used to support heuristics must prove itself in practical situations.
References
Works Cited
Beasley, R., A. Nolan, and A.C. Pickard. 2014. "When ‘Yes’ is the Wrong Answer." INCOSE International Symposium, Las Vegas, NV, Jun 30-Jul 3, 2014.
Churchman, C.W. 1967. "Wicked Problems". Management Science. 14(4): B-141–B-146.
Gigerenzer, G. and R. Selten (eds.). 2001. Bounded Rationality. MIT Press.
Kay, J. and M. King. 2020. Radical Uncertainty: Decision-Making for an Unknowable Future. The Bridge St. Press.
King, M. 2016. The End of Alchemy: Money, Banking and the Future of the Global Economy. New York and London: W.W. Norton and Company.
Koen, B.V. 1985. Definition of the Engineering Method. Washington, DC: American Society for Engineering Education. ISBN-0-57823-101-3.
Maier, M. and E. Rechtin. 2009. The Art of Systems Architecting, 3rd edition. CRC Press.
Plato. 375 BCE. The Republic.
Raab, M. and G. Gigerenzer. 2015. "The Power of Simplicity: A Fast-and-Frugal Heuristics Approach to Performance Science", Frontiers in Psychology, October 29, 2015.
Simon, H.A. 1957. Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting. New York, NY: John Wiley and Sons.
Wilczek, F. 2015. A Beautiful Question: Finding Nature's Deep Design. New York, NY: Penguin Press.
Primary References
Gigerenzer, G. and R. Selten (eds.). 2001. Bounded Rationality. MIT Press.
Maier, M. and E. Rechtin. 2009. The Art of Systems Architecting, 3rd edition. CRC Press.
Rechtin, E. 1991. Systems Architecting: Creating and Building Complex Systems. Englewood Cliffs, NJ: Prentice-Hall.
Simon, H.A. 1957. Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting. New York, NY: John Wiley and Sons.
Additional References
Royal Academy of Engineering. 2010-2011. Philosophy of Engineering, Volumes 1 and 2. Proceedings of seminars held at Royal Academy of Engineering, June 2010 and October 2011. Accessed April 9, 2021. Available: https://www.raeng.org.uk/policy/supporting-the-profession/engineering-ethics-and-philosophy/philosophy