System Lifecycle Process Models: Incremental

System Life Cycle Process Models: Vee discussed the Vee Model. This article discusses incremental and evolutionary development models, beginning with variants of the Vee model.

Incremental and Evolutionary Development using the Vee Model

Overview of the Incremental Approach

Incremental and iterative development (IID) methods have been in use since the 1960s (and perhaps earlier). They allow a project to provide an initial capability followed by successive deliveries to reach the desired System-of-Interest.

The IID approach, shown in Figure 1, is used when:

• Rapid exploration and implementation of part of the system is desired,
• The requirements are unclear from the beginning,
• Funding is constrained, or
• The customer wishes to hold the System-of-Interest open to the possibilities of inserting new technology at a later time.
• Experimentation is required to develop successive prototype versions.

The features that distinguish IID from the single-pass plan-driven approach are velocity and adaptability.

Figure 1 - Incremental Development with multiple deliveries (Forsberg, Mooz, Cotterman. 2005, Pg 358)

Incremental development may also be “plan-driven” in nature when the requirements are known early in the life cycle but the development of the functionality is performed incrementally to allow for the latest technology insertion or potential changes in needs or requirements.

Incremental development also imposes constraints. The model shown in Figure 2 uses the increments to develop high-risk subsystems (or components) early, but the system cannot function until all increments are complete.

Figure 2 - Incremental Development with a Single Delivery (Forsberg, Mooz, Cotterman. 2005, Pg 358)

Overview of the Evolutionary Approach

A specific IID methodology called evolutionary development is common in research and development (R&D) environments. Figure 3 illustrates this approach. It was used in the evolution of the high temperature tiles for the NASA Space Shuttle (Forsberg 1995).

Figure 3 - Evolutionary Generic Model (Forsberg, Mooz, Cotterman. 2005, Pg. 357)

The real world development environment is complex to map because many different project cycles are underway simultaneously. Figure 4 shows the applied research era for the development of the space shuttle orbiter. Figure 4 illustrates multi-levels of simultaneous development, trade studies, and ultimately implementation.

Figure 4 - Evolution of components and orbiter subsystems (including space shuttle tiles) during creation of a large “single-pass” project (Forsberg 1995)

Iterative Software Development Process Models

The life cycle processes of software engineering follow the pattern of systems engineering: analysis, design, construction, verification and validation, acceptance, installation, sustainment, and retirement. However, software is a flexible and malleable medium, which facilitates iterative analysis, design, construction, verification, and validation to a greater degree than is usually possible for the physical components of a system. Each iteration of an iterative development model adds code to the growing software base; the expanded code base is tested, reworked as necessary, and demonstrated to satisfy the requirements for that baseline.

Process models for software development support iterative development on cycles of various lengths. Table 1 lists three iterative software development models presented in the Life Cycle Models topic area and the aspects of software development that are emphasized by those models.

Table 1 - Primary emphases of three iterative software development models

Iterative-development process models

Developing and modifying software involves creative processes that are subject to many external and changeable forces. Long experience has shown that it is impossible to “get it right” the first time, and that iterative development processes are preferable to linear, sequential development process models such as the well-known Waterfall model.

In iterative development, each cycle of the iteration subsumes the software of the previous iteration and adds new capabilities to the evolving product to produce a next, expanded version of the software. Iterative development processes provide the following advantages:

• continuous integration, verification, and validation of the evolving product,
• frequent demonstrations of progress,
• early detection of defects,
• early warning of process problems,
• systematic incorporation of the inevitable rework that occurs in software development, and
• early delivery of subset capabilities (if desired).

Iterative development takes many forms in software engineering, including these:

• An Incremental-build process is used to produce periodic (typically weekly) builds of increasing product capabilities;
• Agile development is used to closely involve a prototypical customer in an iterative process that may repeat on a daily basis;
• The Spiral model is used to confront and mitigate risk factors encountered in developing the successive versions of a product.

This section focuses on the Incremental-build model.

The Incremental-build model

The Incremental-build model is a build-test-demonstrate model of iterative cycles in which frequent demonstrations of progress and verification & validation of work to date are emphasized. The Incremental-build model is based on stable requirements and a software architectural specification. Prioritized requirements are allocated to various elements of the software architecture, which is the basis for specifying a prioritized sequence of builds. Each build adds new capabilities to the incrementally growing product. The development process ends when Version N (the final version) is verified, validated, demonstrated, and accepted by the customer.

Table 2 lists some partitioning criteria for incremental development. Partitions are decomposed into incremental build units of, typically, one calendar-week each. One-week increments and the number of developers available to work on the project determine the number of features that can be included in each incremental build. This, in turn, determines the overall schedule.

Table 2 - Some partitioning criteria for incremental builds

Figure 5 illustrates the details of the build-verify-validate-demonstrate cycles in the Incremental-build process. Each “build” includes detailed design, coding, integration, and review and testing by the developers. In cases where code is to be reused without modification, some or all of an incremental build may consist of review, integration, and test of the base code augmented with the reused code.

Figure 5 - Incremental build-verify-validate-demonstrate cycles (Fairley 2009, Figure 2.11)

Development of an increment may result in rework of previously developed components to better accommodate the new components being integrated, or to fix defects in previously developed components exposed by addition of the new components, or to fix defects in the new components. In any case, frequent demonstrations of progress (or lack thereof) are a major benefit of an Incremental-build development process.

Incremental verification, validation, and demonstration, as illustrated in Figure 17 overcomes two of the major problems of a Waterfall approach: 1) problems are exposed early and can be corrected as they occur; 2) minor in-scope changes to requirements that occur as a result of incremental demonstrations can be incorporated in subsequent incremental builds.

Figure 5 also illustrates that it may be possible to overlap successive builds of the product. It may be possible, for example, to start detailed design of the next version while the present version is being validated. Three factors determine the degree of overlap that can be achieved:

1. availability of sufficient personnel to concurrently pursue multiple activities,
2. adequate progress on the previous version to provide needed capabilities for the next version, and
3. the risk of significant rework that must be accomplished on the next overlapped build because of changes to the previous in-progress build.

The Incremental-build process works well when each team consists of 2 to 5 developers plus a team leader (who is also a technical contributor).

The Incremental-build model can be scaled up for large projects by partitioning the architecture into well-defined subsystems and allocating requirements and interfaces to each subsystem. The subsystems can be independently tested and demonstrated, perhaps using stubs and drivers for the subsystem interfaces, or perhaps using early, incremental versions of other evolving subsystems. System integration can proceed incrementally as intermediate versions of the various subsystems become operational.

A significant advantage of an Incremental-build process is that features built first are verified, validated, and demonstrated most frequently because subsequent builds incorporate the features of the earlier builds. In building the software to control a nuclear reactor, for example, the emergency shutdown software could be built first. Operation of emergency shutdown (scramming) would then be verified and validated in conjunction with the features of each successive build.

In summary, the Incremental-build model, like all iterative models, provides the advantages of continuous integration and validation of the evolving product, frequent demonstrations of progress, early warning of problems, early delivery of subset capabilities, and systematic incorporation of the inevitable rework that occurs in software development.

The role of prototyping in software development

In software engineering, a prototype is a mock-up of the desired functionality of some part of the system. This is in contrast to physical systems, where a prototype is usually a first full functionality version of a system. (Fairley 2009, p 74)

In the past, incorporating prototype software into production systems has created many problems. Prototyping is a useful technique that should be employed whenever appropriate; however, prototyping is not a process model for software development. When building a software prototype, we keep the knowledge we have gained but we do not use the code in the deliverable version of the system

In many cases it is more efficient and more effective to build the production code “from scratch” using the knowledge gained by prototyping than to re-engineer the prototype code.

Life cycle sustainment of software

Software, like all systems, requires sustainment efforts to enhance capabilities, to adapt to new environments, and to correct defects. The primary distinction for software is that sustainment efforts change the software; unlike physical entities, software components do not have to be replaced because of physical wear and tear. Changing the software requires re-verification and re-validation, which may involve extensive regression testing to determine that the change has the desired effect and that the change has not altered other aspects of functionality or behavior.

Retirement of software

Useful software is seldom retired. Software that is useful often experiences many upgrades during its lifetime so that a later upgrade may bear little similarity to the initial version. In some cases, software that ran in a former operational environment is executed on hardware emulators that provide a virtual machine on newer hardware. In other cases, a major enhancement may replace, and rename, an older version of the software, but the enhanced version provides all of the capabilities of the previous software in a compatible manner. Sometimes, however, a newer version of software may fail to provide compatibility with the older version, which necessitates other changes to a system.

Primarily Evolutionary and Concurrent Processes: The Incremental Commitment Spiral Model (ICSM)

Overview of the Incremental Commitment Spiral Model

A view of the Incremental Commitment Spiral Model is shown in Figure 6.

Figure 6 - The Incremental Commitment Spiral Model (Pew and Mavor 2007, Figure 2-5)

Each spiral will be concurrently rather than sequentially addressing requirements and solutions; products and processes; hardware, software and human factors aspects; and business case analysis of alternative product configurations or product line investments.

The stakeholders consider the risks and risk mitigation plans, and decide on a course of action. If the risks are acceptable and will covered by risk mitigation plans, the project would proceed into the next spiral.

The Development spirals after the first Development Commitment Review follow the three-team incremental development approach for achieving both agility and assurance shown in Figure 2 and discussed in System Life Cycle Process Drivers and Choices.

Other Views of the Incremental Commitment Spiral Model (ICSM)

Figure 7 presents an updated view of the ICSM life cycle process recommended in the National Research Council “Human-System Integration in the System Development Process” study (Pew and Mavor 2007). It was called the Incremental Commitment Model (ICM) in the study.

The ICSM builds on the strengths of current process models: early verification and validation concepts in the V-model, concurrency concepts in the Concurrent Engineering model, lighter-weight concepts in the Agile and Lean models, risk-driven concepts in the spiral model, the phases and anchor points in the Rational Unified Process (RUP) (Kruchten 1999; Boehm 1996), and recent extensions of the spiral model to address SoS capability acquisition (Boehm and Lane 2007).

Figure 7 - Phased View of the Generic Incremental Commitment Spiral Model Process (Pew and Mavor 2007 Fig 2-1)

The top row of Activities in Figure 7 indicates that a number of system aspects are being concurrently engineered at an increasing level of understanding, definition, and development. The most significant of these aspects are shown in Figure 20, an extension of a similar “hump diagram” view of concurrently engineered software activities developed as part of the RUP [Kruchten 1999].

Figure 8 - ICSM Activity Categories and Level of Effort (Pew and Mavor 2007 Fig 2-3)

As with the RUP version, the magnitude and shape of the levels of effort will be risk-driven and likely to vary from project to project.

Figure 8 indicates that a great deal of concurrent activity occurs within and across the various ICM phases, all of which needs to be synchronized and stabilized (a best-practice term taken from Microsoft Secrets (Cusumano and Selby 1996) to keep the project under control.

The review processes and use of independent experts are based on the highly successful AT&T Architecture Review Board procedures described in (Maranzano et al. 2005). Figure 9 shows the content of the Feasibility Evidence Description. Showing feasibility of the concurrently-developed elements helps synchronize and stabilize the concurrent activities.

Figure 9 - Feasibility Evidence Description Content (Pew and Mavor 2007 Box 2-2)

The Operations Commitment Review (OCR) is different, in that it addresses the often much higher operational risks of fielding an inadequate system. In general, stakeholders will experience a factor of two-to-ten increase in commitment level in going through the sequence of ECR to DCR milestones, but the increase in going from DCR to OCR can be much higher. These commitment levels are based on typical cost profiles across the various stages of the acquisition life cycle.

Underlying ICSM Principles

ICSM has four underlying principles. If a project just follows the diagrams without following the principles (as often happened with the original spiral model), the project will have a serious risk of failure. The four principles are:

1. Stakeholder value-based system definition and evolution. If a project fails to include success-critical stakeholders such as end-users, maintainers, or suppliers, these stakeholders will frequently feel little commitment to the project and will either underperform or refuse to use the results.
2. Incremental commitment and accountability. If success-critical stakeholders are not accountable for their commitments, they are likely to be drawn away to other pursuits when they are most needed.
3. Concurrent system and software definition and development. If definition and development of requirements and solutions; hardware, software, and human factors; or product and process definition are done sequentially, the project is likely both to go more slowly, and to make early, hard-to-undo commitments that cut off the best options for project success.
4. Evidence and risk-based decisionmaking. If key decisions are made based on assertions, vendor literature, or meeting an arbitrary schedule without access to evidence of feasibility, the project is building up uncertainties and risks. And in the words of Tom Gilb, “If you do not actively attack the risks, the risks will actively attack you.”

Model Experience to Date

During the National Research Council Human-Systems Integration study, it was found that the ICSM processes and principles corresponded well with best commercial practices. Examples are found in (Pew and Mavor 2007, Chapter 5)and(Royce 1998, Appendix D).

A further source of successful projects that have applied the ICSM principles is the annual series of Top-5 software-intensive systems projects published in CrossTalk (CrossTalk 2002-2005).

Primarily Interpersonal and Unconstrained Processes: Scrum and Agile

Figure 10 shows an example of Scrum as an agile process flow. As with most other agile methods, Scrum uses the Evolutionary Sequential process shown in Table 1 and described in Fixed-Requirements and Evolutionary Development Processes, in which systems capabilities are developed in short periods, usually around 30 days. The project then re-prioritizes its backlog of desired features and determines how many features the team (usually 10 people or less) can develop in the next 30 days.

As shown in Figure 10, once the features to be developed for the current Scrum have been expanded (usually in the form of informal stories) and allocated to the team members, the team establishes a daily rhythm of starting with a short meeting at which each team member does about a one-minute summary of progress since the last Scrum meeting, potential obstacles, and plans for the upcoming day.

Figure 10 - Example Agile process flow: Scrum (Boehm and Turner 2004; ControlChaos.com)

Architected Agile Methods

Over the last decade, several organizations have been able to scale up agile methods by using two layers of 10-person Scrum teams. This involves, among other things, having each Scrum team’s daily meeting followed up by a daily meeting of the Scrum team leaders, and by up-front investments in an evolving system architecture (Boehm et al. 2010). Figure 11 shows an example of the Architected Agile approach.

Figure 11 - Example of Architected Agile Process (Dingsoyr, et. al. 2010. Fig 9.6)

Agile Practices and Principles

Here are summaries of the general practices and principles shared by most agile methods. As seen with the Scrum and Architected Agile methods, generally-shared does not mean uniformly followed.

Iterative Development - any organization can improve its velocity to customer satisfaction.

1. The project team understands, respects, works, and behaves within a defined SE process.
2. The project is executed as fast as possible with minimum down time or staff diversion during the project. The critical path is managed.
3. All key players are physically or electronically collocated. Notebooks are team property and are available to all.
4. Baseline management and change control are achieved by formal, oral agreement based on “make a promise, keep a promise” discipline — participants hold each other accountable.
5. Opportunity exploration and risk reduction are accomplished by expert consultation and rapid model verification, coupled with close customer collaboration. Software development is done in a rapid development environment, while hardware is developed in a multi-disciplined model shop.
6. A culture of constructive confrontation pervades the project organization. The team takes ownership for success; it is never “someone else’s responsibility.”

Agile Development principles (adapted for SE) are as follows (see “Principles behind the Agile Manifesto” 2009):

1. First, satisfy the customer through early and continuous delivery of valuable software [and other system elements].
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
3. Deliver working software [and other system elements] frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
4. Business people and developers must work together daily throughout the project.
5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
6. The most efficient and effective method of conveying information is face-to-face conversation.
7. Working software [and other system elements] is the primary measure of progress.
8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence and good design enhances agility.
10. Simplicity — the art of maximizing the amount of work not done — is essential.
11. The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Concluding Remarks

While there are a number of different models describing the project environment, the Spiral model and the Vee model have become the dominant approaches to visualize the development process. Both the Vee and the Spiral are useful models that emphasize different aspects of the process.

The majority of projects will use Incremental and Iterative Development (IID). Evolutionary development is particularly appropriate to new products coming from research whether in the commercial or government environment.

---

Article Discussion

[Go to discussion page]

Signatures

<- Previous Article | Parent Article | Next Article ->