View source for System Analysis

[[System Analysis (glossary)|System analysis]] allows developers to objectively carry out quantitative assessments of [[System (glossary)|systems]] in order to select and/or update the most efficient [[Architecture (glossary)|system architecture]] and to generate derived engineering data. During engineering, assessments should be performed every time technical choices or decisions are made, to determine compliance with [[System Requirements|system requirements]]. System analysis provides a rigorous approach to technical decision-making. It is used to perform trade-off studies, and includes modeling and simulation, cost analysis, technical risks analysis, and effectiveness analysis. 

==Principles Governing System Analysis==
One of the major tasks of a systems engineer is to evaluate the engineering data and artifacts created during the [[Systems Engineering (glossary)|systems engineering]] (SE) process. The evaluations are at the center of system analysis, providing means and techniques
*to define [[Assessment Criterion (glossary)|assessment criteria]] based on system requirements;
*to assess [[Design Property (glossary)|design properties]] of each candidate solution in comparison to these criteria;
*to score globally the candidate solutions and to justify the scores; and
*to decide on the appropriate solution(s).

The [[Analysis and Selection between Alternative Solutions]] article in the [[Systems Approach Applied to Engineered Systems]] knowledge area (KA) of [[Systems|Part 2]] describes activities related to selecting between possible system solutions to an identified problem or opportunity.  The following general [[Principle (glossary)|principles]] of systems analysis are defined:

*Systems analysis is based on assessment criteria based upon a problem or opportunity system description.
**These criteria will be based around an ideal system description, which assumes a [[Hard System (glossary)|hard system]] problem context can be defined.
**Criteria must consider required system behavior and properties of the complete solution, in all possible wider system contexts and environments.
**These must consider non-functional issues such as system safety, security, etc. (Please see [[Systems Engineering and Specialty Engineering]] for additional discussion on incorporating non-functional elements.)
**This "ideal" system description may be supported by [[Soft System (glossary)|soft system]] descriptions, from which additional “soft” criteria may be defined. For example, a stakeholder preference for or against certain kinds of solutions, relevant social, political or cultural conventions to be considered, etc.
*The assessment criteria should include, at a minimum, the constraints on cost and time scales acceptable to stakeholders.
*Trade studies provide a mechanism for conducting analysis of alternative solutions.
**A trade study should consider a set of assessment criteria, with appropriate awareness of the limitations and dependencies between individual criteria.
**Trade studies need to deal with both objective and subjective criteria.  Care must be taken to assess the sensitivity of the overall assessment to particular criteria.
 
===Trade-off studies===
In the context of the definition of a system, a trade-off study consists of comparing the characteristics of each [[System Element (glossary)|system element]] and of each candidate [[Architecture (glossary)|system architecture]] to determine the solution that best globally balances the assessment criteria. The various characteristics analyzed are gathered in cost analysis, technical risks analysis, and effectiveness analysis (NASA 2007). 

Guidance on the conduct of trade studies for all types of system context are characterized in the above principles and described in more details in the [[Analysis and Selection between Alternative Solutions]] topic. Of particular interest to SE analysis are technical effectiveness, cost, and technical risk analysis.

===Effectiveness Analysis===
The effectiveness of an [[Engineered System (glossary)|engineered system]] solution includes several essential characteristics that are generally gathered in the following list of analyses, including (but not limited to): performance, usability, dependability, manufacturing, maintenance or support, environment, etc. These analyses highlight candidate solutions under various aspects. 

It is essential to establish a classification that limits the number of analyses to the really significant aspects, such as key performance parameters. The main difficulties of effectiveness analysis are to sort and select the right set of effectiveness aspects; for example, if the product is made for a single use, maintainability will not be a relevant criterion.

===Cost Analysis ===
A [[Cost (glossary)|cost]] analysis considers the full [[Life Cycle (glossary)|life cycle]] costs. A cost baseline can be adapted according to the project and the system. The global [[Life Cycle Cost (LCC) (glossary)|life cycle cost]] (LCC), or [[Total Ownership Cost (glossary)|total ownership cost]] (TOC), may include examplar labor and non-labor cost items such as those indicated in Table 1.

{|
|+'''Table 1. Types of Costs.''' (SEBoK Original)
!Type of Cost
!Description and Examples
|-
|'''Development'''
|Engineering, development tools (equipment and software), project management, test-benches, mock-ups and prototypes, training, etc.
|-
|'''Product manufacturing or service realization'''
|Raw materials and supplying, spare parts and stock assets, necessary resources to operation (water, electricity power, etc.), risks and nuances, evacuation, treatment and storage of waste or rejections produced, expenses of structure (taxes, management, purchase, documentation, quality, cleaning, regulation, controls, etc.), packing and storage, documentation required.
|-
|'''Sales and after-sales'''
|Expenses of structure (subsidiaries, stores, workshops, distribution, information acquisition, etc.), complaints and guarantees, etc.
|-
|'''Customer utilization'''
|Taxes, installation (customer), resources necessary to the operation of the product (water, fuel, lubricants, etc.), financial risks and nuisances, etc.
|-
|'''Supply chain'''
|Transportation and delivery
|-
|'''Maintenance'''
|Field services, preventive maintenance, regulation controls, spare parts and stocks, cost of guarantee, etc.
|-
|'''Disposal'''
|Collection, dismantling, transportation, treatment, waste recycling, etc.
|}

Methods for determining cost are described in the [[Planning]] topic.

===Technical Risks Analysis===
Every [[Risk (glossary)|risk]] analysis concerning every domain is based on three factors:
#Analysis of potential threats or undesired events and their probability of occurrence.
#Analysis of the consequences of these threats or undesired events and their classification on a scale of gravity.
#Mitigation to reduce the probabilities of threats and/or the levels of harmful effect to acceptable values.

The technical risks appear when the system cannot satisfy the system requirements any longer. The causes reside in the requirements and/or in the solution itself. They are expressed in the form of insufficient effectiveness and can have multiple causes: incorrect assessment of the technological capabilities; over-estimation of the technical maturity of a system element; failure of parts; breakdowns; breakage, obsolescence of equipment, parts, or software, weakness from the supplier (non-compliant parts, delay for supply, etc.), human factors (insufficient training, wrong tunings, error handling, unsuited procedures, malice), etc. 

Technical risks are not to be confused with project risks, even if the method to manage them is the same. Although technical risks may lead to project risks, technical risks address the system itself, not the process  for its development. (See [[Risk Management]] for more details.)

==Process Approach==
===Purpose and Principles of the Approach===
The system analysis process is used to: (1) provide a rigorous basis for technical decision making, resolution of requirement conflicts, and assessment of alternative physical solutions (system elements and [[Physical Architecture (glossary)|physical architectures]]); (2) determine progress in satisfying [[System Requirement (glossary)|system requirements]] and [[Derived Requirement (glossary)|derived requirements]]; (3) support risk management; and (4) ensure that decisions are made only after evaluating the cost, schedule, performance, and risk effects on the engineering or re-engineering of a system (ANSI/EIA 1998). This process is also called the decision analysis process by NASA (2007, 1-360) and is used to help evaluate technical issues, alternatives, and their uncertainties to support decision-making. (See [[Decision Management]] for more details.)

System analysis supports other system definition processes:
*[[Stakeholder Needs and Requirements|Stakeholder requirements definition]] and [[System Requirements|system requirements definition]] processes use system analysis to solve issues relating to conflicts among the set of requirements; in particular, those related to costs, technical risks, and effectiveness (performances, operational conditions, and constraints). [[System Requirement (glossary)|System requirements]] subject to high risks, or those which would require different architectures, are discussed.
*The [[Architectural Design: Logical]] and [[Architectural Design: Physical]] processes use it to assess characteristics or design properties of candidate [[Logical Architecture (glossary)|logical]] and physical architectures, providing arguments for selecting the most efficient one in terms of costs, technical risks, and effectiveness (e.g., performances, dependability, human factors, etc.).

Like any [[System Definition|system definition]] process, the system analysis process is iterative. Each operation is carried out several times; each step improves the precision of analysis.

===Activities of the Process ===
Major activities and tasks performed within this process include
*Planning the trade-off studies:
**Determine the number of candidate solutions to analyze, the methods and procedures to be used, the expected results (examples of objects to be selected: behavioral architecture/scenario, physical architecture, system element, etc.), and the justification items.
**Schedule the analyses according to the availability of models, engineering data (system requirements, design properties), skilled personnel, and procedures.
*Define the selection criteria model:
**Select the assessment criteria from non-functional requirements (performances, operational conditions, constraints, etc.), and/or from design properties.
**Sort and order the assessment criteria.
**Establish a scale of comparison for each assessment criterion, and weigh every assessment criterion according to its level of relative importance with the others.
*Identify candidate solutions, related models, and data.
*Assess candidate solutions using previously defined methods or procedures:
**Carry out costs analysis, technical risks analysis, and effectiveness analysis placing every candidate solution on every assessment criterion comparison scale.
**Score every candidate solution as an assessment score.
*Provide results to the calling process: assessment criteria, comparison scales, solutions’ scores, assessment selection, and possibly recommendations and related arguments.

===Artifacts and Ontology Elements===
This process may create several artifacts, such as
#A selection criteria model (list, scales, weighing)
#Costs, risks, and effectiveness analysis reports
#Justification reports

This process handles the ontology elements of Table 2 within system analysis.

{|
|+'''Table 2. Main Ontology Elements as Handled within System Analysis.''' (SEBoK Original)
|-
|'''Assessment Criterion'''
|In the context of system analysis, an assessment criterion is a characteristic used to assess or compare system elements, physical interfaces, physical architectures, functional architectures/scenarios, or any engineering elements that can be compared.
----
Identifier; name; description; relative weight; scalar weight
|-
|'''Assessment Selection'''
|In the context of system analysis, an assessment selection is a technical management element based on an assessment score that justifies the selection of a system element, a physical interface, a physical architecture, or a functional architecture/scenario.
|-
|'''Assessment Score'''
|In the context of system analysis, an assessment score is obtained assessing a system element, a physical interface, a physical architecture, a functional architecture/scenario using a set of assessment criteria.
----
Identifier; name; description; value
|-
|'''Cost'''
|In the context of systems engineering, a cost is an amount expressed in a given currency related to the value of a system element, a physical interface, and a physical architecture.
----
Identifier; name; description; amount; type (development, production, utilization, maintenance, disposal); confidence interval; period of reference; estimation technique
|-
|'''Risk'''
|An event having a probability of occurrence and consequences related to the system mission or on other characteristics. (Used for technical risk in engineering.). A risk is the combination of vulnerability a danger or threat.
----
Identifier; name description; status
|}

===Checking Correctness of System Analysis===
The main items to be checked within system analysis in order to get validated arguments are 
*Relevance of the models and data in the context of use of the system,
*Relevance of assessment criteria related to the context of use of the system,
*Reproducibility of simulation results and of calculations,
*Precision level of comparisons' scales,
*Confidence of estimates, and
*Sensitivity of solutions' scores related to assessment criteria weights.

See Ring, Eisner, and Maier (2010) for additional perspective.

===Methods and Modeling Techniques===
*'''General usage of models''': Various types of models can be used in the context of system analysis:
**'''Physical models''' are scale models allowing simulation of physical phenomena. They are specific to each discipline; associated tools include mock-ups, vibration tables, test benches, prototypes, decompression chamber, wind tunnels, etc.
**'''Representation models''' are mainly used to simulate the behavior of a system. For example, enhanced functional flow block diagrams (eFFBDs), statecharts, state machine diagrams (based in systems modeling language (SysML)), etc.
**'''Analytical models''' are mainly used to establish values of estimates. We can consider the deterministic models and probabilistic models (also known as stochastic models) to be analytical in nature. Analytical models use equations or diagrams to approach the real operation of the system. They can be very simple (addition) to incredibly complicated (probabilistic distribution with several variables).
*'''Use right models''' depending on the project progress
**At the beginning of the project, first studies use simple tools, allowing rough approximations which have the advantage of not requiring too much time and effort. These approximations are often sufficient to eliminate unrealistic or outgoing candidate solutions.
**Progressively with the progress of the project it is necessary to improve precision of data to compare the candidate solutions still competing. The work is more complicated if the level of innovation is high.
**A systems engineer alone cannot model a complex system; he has to be supported by skilled people from different disciplines involved.
*'''Specialist expertise''': When the values of assessment criteria cannot be given in an objective or precise way, or because the subjective aspect is dominating, we can ask specialists for expertise. The estimates proceed in four steps:
#Select interviewees to collect the opinion of qualified people for the considered field.
#Draft a questionnaire; a precise questionnaire allows an easy analysis, but a questionnaire that is too closed risks the neglection of significant points.
#Interview a limited number of specialists with the questionnaire, including an in-depth discussion to get precise opinions.
#Analyze the data with several different people and compare their impressions until an agreement on a classification of assessment criteria and/or candidate solutions is reached.

Often used analytical models in the context of system analysis are summarized in Table 3.

{|
|+'''Table 3. Often Used Analytical Models in the Context of System Analysis.''' (SEBoK Original)
!Type of Model
!Description
|-
|'''Deterministic models'''
|
*Models containing '''statistics''' are included in this category. The principle consists in establishing a model based on a significant amount of data and number of results from former projects; they can apply only to system elements/components whose technology already exists.
*Models by '''analogy''' also use former projects. The system element being studied is compared to an already existing system element with known characteristics (cost, reliability, etc.). Then these characteristics are adjusted based on the specialists' expertise.
*'''Learning curves''' allow foreseeing the evolution of a characteristic or a technology. One example of evolution: "Each time the number of produced units is multiplied by two, the cost of this unit is reduced with a certain percentage, generally constant."
|-
|'''Probabilistic models''' (also called stochastic models)
|The theory of probability allows classifying the possible candidate solutions compared to consequences from a set of events as criteria. These models are applicable if the number of criteria is limited and the combination of the possible events is simple. Take care that the sum of probabilities of all events is equal to one for each node.
|-
|'''Multi-criteria decisions models'''
|When the number of criteria is greater than ten, it is recommended that a multi-criteria decision model be established. This model is obtained through the following actions:
*Organize the criteria as a hierarchy (or a decomposition tree).
*Associate each criterion of each branch of the tree with a relative weight compare to each other of the same level.
*Calculate a scalar weight for each leaf criterion of each branch multiplying all the weights of the branch.
*Score every candidate solution on the leaf criteria; sum the scores to get a global score for each candidate solution; compare the scores.
*Using a computerized tool allows to perform sensitivity analysis to get a robust choice.
|}

==Practical Considerations==
Key pitfalls and good practices related to system analysis are described in the next two sections.

===Pitfalls===
Some of the key pitfalls encountered in planning and performing system analysis are provided in Table 4.

{|
|+'''Table 4. Pitfalls with System Analysis.''' (SEBoK Original)
!Pitfall
!Description
|-
|Analytical modeling is not a decision tool
|Analytical modeling gives analytical results from analytical data. It has to be considered as a help and not as a decision tool.
|-
|Models and system levels of decomposition
|A model can be well adapted to a level ''n'' of a system and to be incompatible with the model of the higher level which uses the data coming from the lower level. It is essential that the systems engineer ensures the coherence of the various models used.
|-
|Optimization is not a sum of optimized elements
|The general optimization of the system-of-interest is not the sum of its optimized systems and/or system elements.
|}

===Proven Practices===
Some proven practices gathered from the references are provided in Table 5.

{|
|+'''Table 5. Proven Practices with System Analysis.''' (SEBoK Original)
!Practice
!Description
|-
|Stay in the operational field
|Models can never simulate all the behavior/reactions of a system: they operate only in one limited field with a restricted number of variables. When a model is used, it is always necessary to make sure that the parameters and data inputs are part of the operation field. If not, there is a high risk of irregular outputs.
|-
|Evolve models
|Models shall evolve during the project: by modification of parameter settings, by entering new data when modified (modification of assessment criteria, functions to perform, requirements, etc.), by the use of new tools when those used reach their limits.
|-
|Use several types of models
|It is recommended to concurrently use several types of models in order to compare the results and/or to take into account another aspect of the system.
|-
|Keep context elements consistent
|Results of a simulation shall always be given in their modeling context: tool used, selected assumptions, parameters and data introduced, and variance of the outputs.
|}

==References== 

===Works Cited===
ANSI/EIA. 1998. ''Processes for Engineering a System''. Philadelphia, PA, USA: American National Standards Institute (ANSI)/Electronic Industries Association (EIA), ANSI/EIA-632-1998. 

NASA. 2007. ''Systems Engineering Handbook''. Washington, D.C.: National Aeronautics and Space Administration (NASA), NASA/SP-2007-6105.

Ring, J,  H. Eisner, and M. Maier.  2010.  "Key Issues of Systems Engineering, Part 3: Proving Your Design."  INCOSE ''Insight'' 13(2). 

===Primary References===
ANSI/EIA. 1998. ''[[ANSI/EIA 632|Processes for Engineering a System]]''. Philadelphia, PA, USA: American National Standards Institute (ANSI)/Electronic Industries Association (EIA), [[ANSI/EIA 632]]-1998. 

Blanchard, B.S., and W.J. Fabrycky. 2010. ''[[Systems Engineering and Analysis]].'' 5th ed. Prentice-Hall International Series in Industrial and Systems Engineering. Englewood Cliffs, NJ, USA: Prentice-Hall. 

NASA. 2007. ''[[NASA Systems Engineering Handbook|Systems Engineering Handbook]]''. Washington, D.C., USA: National Aeronautics and Space Administration (NASA), NASA/SP-2007-6105.

===Additional References===
Ring, J,  H. Eisner, and M. Maier.  2010.  "Key Issues of Systems Engineering, Part 3: Proving Your Design."  INCOSE ''Insight'' 13(2). 

----
<center>[[Architectural Design: Physical|< Previous Article]] | [[System Definition|Parent Article]] | [[System Realization|Next Article >]]</center>


{{DISQUS}}


[[Category: Part 3]][[Category:Topic]]
[[Category:System Definition]]