Analysis and Selection between Alternative Solutions

This article considers the activities of the systems approach related to the analysis and selection of a prefered solution from possible solution options in detail. Any of the activities described below may need to be considered concurrently with other activities in the Systems Approach. The final article in this knowledge area, Applying the Systems Approach, considers the dynamic aspects of how these activities are used as part of the Systems Approach and how this relates in detail to elements of Systems Engineering.

System Analysis

System Analysis is an activity within the Systems Approach to evaluate one or more system artefacts created during the Systems Approach activities:

• to define Assessment Criteria based on the required properties and behavior of an identified problem or opportunity system situation;
• to assess the Properties and Behavior of each candidate solution in comparison to these criteria;
• to compare the assessment of the candidate solutions and to identify if any of them could resolve the problem or exploit the opportunities, and if so to select which ones should be explored further.

As discussed in Synthesizing Possible Solutions the problem context for an engineered system will include a logical or ideal system solution description. It is assumed that the solution which “best” matches the ideal will be the most acceptable to the stakeholders. Note, as discussed below “best” should include an understanding of cost and risk as well as effectiveness. The problem context may include a soft system Conceptual Model describing the logical elements of a system to resolve the problem situation and how these are perceived by different stakeholders (Checkland 1999). This soft context view will provide additional criteria for the analysis process, which may become the critical issues in selecting between two equally effective solution alternatives.

Effectiveness Analysis

Effectiveness studies use the problem or opportunity system as a starting point.

The effectiveness of a synthesized system solution will included performance criteria associated with the system primary functions. These are derived from the systems purpose in enabling the realisation of stakeholder needs in one or more wider system contexts.

For a product system there are a set of generic non functional qualities which are associated with different types of solution pattern or technology, e.g. safety, security, reliability, maintainability, useability, etc. These criteria are often explicitly stated as part of the domain knowledge of related technical disciplines of technology domains.

For a service system or enterprise system the criteria will be more directly linked to the identified user need or enterprise goals. Typical qualities for such systems include agility, resilience, flexibility, upgradeability, etc.

In addition to assessments of the absolute effectiveness of a given solution system we must also be able to combine effectiveness with limitations of the cost and timescales included in the problem context. In general, the role of System Analysis is to identify those proposed solutions which can provide some effectiveness within the cost and time allocated to any given iteration of the Systems Approach, see Applying the Systems Approach for details. If none of the solutions can deliver effectiveness that justifies the proposed investment then it is necessary to return to the original framing of the problem. If at least one solution is assessed as sufficiently effective then a choice between solutions can be proposed.

Trade-off studies

In the context of the definition of a system, a trade-off study consists of comparing the characteristics of each candidate system element to determine the solution that best globally balances the assessment criteria. The various characteristics analyzed are gathered in cost analysis, technical risks analysis, and effectiveness analysis (NASA 2007). Each class of analysis is the subject of the following topics:

1. Assessment criteria are used to classify the various candidate solutions between themselves. They are absolute or relative. For example: maximum cost per unit produced is cc$, cost reduction shall be x%, effectiveness improvement is y%, and risk mitigation is z%.
2. Boundaries identify and limit the characteristics or criteria to be taken into account in the analysis. For example: kind of costs to be taken into account, acceptable technical risks, and type and level of effectiveness.
3. Scales are used to quantify the characteristics, properties, and/or criteria and to make comparisons. Their definition requires knowing the highest and lowest limits as well as the type of evolution of the characteristic (linear, logarithmic, etc.).
4. An assessment score is assigned to a characteristic or criterion for each candidate solution. The goal of the trade-off study is to succeed in quantifying the three variables (and their decomposition in sub-variables) of cost, risk, and effectiveness for each candidate solution. This operation is generally complex and requires the use of models.
5. The optimization of the characteristics or properties improves the scoring of interesting solutions.

A decision-making process is not an accurate science and trade-off studies have limits. The following concerns should be taken into account:

• Subjective criteria: for example, the component has to be beautiful. What is a beautiful component?
• Uncertain data: for example, inflation has to be taken into account to estimate the cost of maintenance during the complete life cycle. What will be inflation for the next five years?
• Sensitivity analysis: a global assessment score associated to every candidate solution is not absolute; it is recommended to get a robust selection by performing sensitivity analysis that considers small variations of assessment criteria values (weights). The selection is robust if the variations do not change the order of scores.

A thorough trade-off study specifies the assumptions, variables, and confidence intervals of the results.

Systems Principles of System Analysis

From the discussions above, the following general Principles (glossary) of Systems Analysis can be defined:

1. Systems Analysis is based on Assessment Criteria based upon a Problem or Opportunity System description.
   1. These criteria will be based around an Ideal System description, which assumes a hard system problem context can be defined.
   2. Criteria must consider required system behaviour and properties of the complete solution, in all possible wider system contexts and environments.
   3. These must consider non functional issues such as system safety, security, etc.
   4. This idea system description may be supported by soft system descriptions, from which additional “soft” criteria may be defined, e.g. a stakeholder preference for or against certain kinds of solution, relevant social, political or cultural conventions to be considered in the likely solution environment, etc.
2. The assessment criteria should include as a minimum the constraints on cost and time scales acceptable to stakeholders; but may also include preferences for or against certain kinds of solution, etc.
3. Trade studies provide a mechanism for conducting Analysis of alternative solutions.
   1. A trade Study should consider a “System of Assessment Criteria”, with appropriate awareness of the limitations and dependencies between individual criteria.
   2. Trade studies need to deal with both objective and subjective criteria. Care must be taken to assess the sensitivity of the overall assessment to particular criteria.

Linkages to other topics

Applying the Systems Approach

References

Primary References

ISO/IEC 2008. Systems and software engineering -- System life cycle processes. Geneva, Switzerland: International Organisation for Standardisation / International Electrotechnical Commissions. ISO/IEC/IEEE 15288:2008.

Jackson, S., D. Hitchins and H. Eisner. 2010. "What is the Systems Approach?" INCOSE Insight. 13(1) (April 2010): 41-43.

Additional References

---

<- Previous Article | Parent Article | Next Article ->