Product Systems Engineering Special Activities
Lead Author: Ricardo Pineda
Product systems engineering has activities that are unique to products. This article discusses many of them.
Readiness Level Assessments
As a new system is developed, it is essential to verify and validate that the developed system is mature enough to be released as an operational product or service. Technology readiness assessments (TRA) are established tools used to qualify technology development and help make investment decisions within complex development programs in order to deploy systems or elements of technology to an end user in a timely fashion.
This notion of maturity was formalized by the US National Aeronautics and Space Administration (NASA) (Mankins 1995) and later modified for use by the Department of Defense (DoD), the Air Force Research Laboratory (AFRL), and the US Department of Energy (DoE), as well as a growing number of non-governmental organizations. Technology readiness levels (TRL) are a metric developed to summarize the degree of maturity of a technology. The original NASA TRL scale has nine different levels from the basic principles observed and reported (TRL 1) to actual systems "flight proven" through successful mission operations (TRL 9). The TRL scale utilized by the DoD is portrayed in Table 1.
| Technology Readiness Level | Description |
|---|---|
| 1. Basic principles observed and reported. | Lowest level of technology readiness. Examples might include paper studies of a technology's basic properties. |
| 2. Technology concept and/or application formulated. | Invention begins. Examples are limited to analytic studies. |
| 3. An analytical and experimental critical function and/or characteristic proof of concept. | Includes analytical and lab studies to physically validate predictions of separate elements of the technology. Examples include components not yet integrated. |
| 4. Component validation in laboratory environment. | Basic technological components are integrated. This is relatively "low fidelity" compared to the eventual system. |
| 5. Component validation in relevant environment. | Basic technological components are integrated with reasonably realistic supporting elements so it can be tested in a simulated environment. |
| 6. Prototype demonstration in a relevant environment. | Representative prototype system tested in a relevant environment. Represents a major step up in a technology's demonstrated readiness. Examples include testing a prototype in a high-fidelity laboratory environment or in simulated operational environment. |
| 7. Prototype demonstration in an operational environment. | Prototype near, or at, planned operational system. Represents a major set up from TRL 6, requiring demonstration of an actual system prototype in an operational environment. |
| 8. System qualified through test and demonstration. | Technology proven to work in its final form and under expected conditions. Represents the end of true system development. Examples include developmental test and evaluation of the system. |
| 9. System proven through successful mission operations. | Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. |
The utilization of TRLs has an impact on the structure and operation of life cycles as described in Part 3; they allow better management and control of risks inherent with technology, as well as better control of costs and the schedule of program development. However, TRLs do not provide an assessment of the programmatic influence on a TRL, technology criticality and priority, software aging and readiness context, as pointed out by Smith (2005). While TRLs have proven to be useful in evaluating a technology’s performance, as demonstrated in the laboratory or in a test environment, they do not inform one whether or not the technology product can actually be produced in an affordable manner. The concept of manufacturing readiness levels (MRL) has been incorporated to expand the TRL idea so that it can incorporate producibility concerns. The MRL approach addresses questions such as the level of technology reproducibility, the cost of production, and technology manufacturing production environment early in the development phase (GAO 2003, DoD 2011).
Readiness levels are an active research area within academia and government agencies in regards to the integration of technology components into complex systems (integration readiness levels (IRLs)) to address interface maturity among existing and maturing technology developments. TRLs apply to the critical enabling technologies, which are usually embodied at the subsystem, assembly level, or system component level. Systems readiness levels (SRL) are used when going from individual technologies to the whole system. The SRL model is a function of the individual TRLs in a system and their subsequent integration points with other technologies, the IRL (Sauser 2006).
Another maturity aspect is related to the provisioning of products that are readily available and referred to as commercial off-the-shelf (COTS). Such products, be they hardware, software, or a mixture of both, have hopefully achieved the degree of maturity so that those acquiring them can rely upon their operational properties and that the documentation of the COTS products is sufficient to provide the proper guidance in their use.
The PSE should realize that the TRL assessment for COTS changes dramatically if the operational environment or other requirements are imposed that exceed the design limits of the COTS product (e.g., operations at very high or very cold temperatures, high shock, or vibration levels).
Product Certification
Product certifications are both domain and product specific, and typically relate to human safety and health, the need to meet a specific government regulation, or are required by underwriters for insurance purposes. Certifications are performed by a third party (independent of the developer) who provides a guarantee of the quality, safety, and reliability of the product to the customer or user.
The INCOSE SE Handbook defines product certification as "the process of certifying that a certain product has passed performance or quality assurance tests or qualification requirements stipulated in regulations such as a building code or nationally accredited test standards, or that it complies with a set of regulations governing quality or minimum performance requirements." (INCOSE 2012)
The INCOSE SE Handbook also defines four methods for verification: inspection, analysis, demonstration, and testing (INCOSE 2012). In addition, it defines certification as a fifth verification method, which is defined as verification against legal or industrial standards by an outside authority without direction to that authority as to how the requirements are to be verified. For example, electronic devices require a CE certification in Europe, and a UL certification in the US and Canada (INCOSE 2012).
The best known certification is the airworthiness certification, which relates to the safety of flight for aircraft. In the US, the test for this certification is performed by the Federal Aviation Administration (FAA). Government certifications are also common in the medical systems field where the Federal Drug Administration (FDA) is the primary certification agency. Some certifications are based on standards defined by technical societies, such as the American Society of Mechanical Engineers (ASME). The combination of the technical standards and a certification allows product developers to perform certifications that meet government standards without having the government directly involved in the process.
There are equivalent government organizations in other countries and for other regulated areas, such as communications, building safety, nuclear systems, transportation systems to include ships, trains and automobiles, environmental impact, and energy use. Systems engineers must be aware of the certifications that are required for the domain and product being developed. Certification agencies must be involved early in the development effort to ensure the necessary certifications are included in the system requirements, the system development plan, and the funding provided to accomplish the development. When system changes and upgrades are necessary, the systems engineers must determine if product re-certification is necessary and include it in the plans and funding for the system upgrade.
Enabling Product Certifications
There may be other certifications for enabling products that must be considered and appreciated by PSE, such as an operator certification of airplane pilots to ensure flight safety, and certification of nuclear plant operators to ensure prevention or mitigation of nuclear radiation effects. An example of this is shown in the certification program by the North American Electric Reliability Corporation (NERC):
In support of NERC’s mission, the System Operator Certification Program’s mission is to ensure that employers have a workforce of system operators that meet minimum qualifications. These minimum qualifications are set through internationally recognized processes and procedures for agencies that certify persons. The Certification Program promotes excellence in the area of system operator performance and encourages system operators to be inquisitive and informed. (NERC 2012)
Production qualification testing (PQT) is another type of certification which DAU (2005) describes as:
A technical test completed prior to the full-rate production (FRP) decision to ensure the effectiveness of the manufacturing process, equipment, and procedures. This testing also serves the purpose of providing data for the independent evaluation required for materiel release so that the evaluator can address the adequacy of the materiel with respect to the stated requirements. These tests are conducted on a number of samples taken at random from the first production lot, and are repeated if the process or design is changed significantly and when a second or alternative source is brought online.
Security certification and accreditation (C&A) is often required for the deployment of computing and networking equipment in a classified environment. Facility certification may be required to ensure that a building housing the equipment can provide the proper environment for safe and efficient operation of the equipment. High-altitude electromagnetic pulse (HEMP) certification may be required to ensure that a building and its equipment can withstand the effects of HEMP from nuclear weapons. A similar type of certification to HEMP is TEMPEST testing to ensure that sensitive electronic emissions are not allowed to leave high security facilities. TEMPEST is a code name referring to investigations and studies of compromising emission, and is not an acronym.
Technology Planning and Insertion
Technology planning can be an enterprise function or a program function. Technology planning as an enterprise function typically occurs on an annual basis to determine the funding necessary for independent research and development in the coming year. Technology planning as a program function occurs early in the program and often continues throughout the life of the system. The design of the product system is highly dependent on the availability of technologies that have acceptable risks and that meet the customer's cost, schedule, and performance requirements. These critical technologies will only be available when necessary if the systems engineers perform concept designs, technology assessments, and trade studies that define the critical technologies and the capabilities necessary before the system development activities that will use the critical technologies begin.
The MITRE Systems Engineering Guide (MITRE 2011) provides the following definition for technology planning:
Technology Planning is the process of planning the technical evolution of a program or system to achieve its future vision or end-state. Technology planning may include desired customer outcomes, technology forecasting and schedule projections, technology maturation requirements and planning, and technology insertion points. The goal is a defined technical end-state enabled by technology insertion over time.
Systems engineers who participate in technical planning must understand the future vision and system requirements, and relate these to the current and expected future technologies that can be applied to the system design during current development stages, as well as for potential future upgrades to the system. To do this, systems engineers must acquire and maintain knowledge of the existing and developing technology in their design domain. The systems engineer will also provide the essential connection between the system user and research communities to provide alignment between the technology developers and the system designers.
Technology planning and insertion usually requires that the systems engineer perform technology readiness assessments that rate the maturity levels and the risks associated with the planned technologies. Immature, risky technologies require risk reduction activities that include prototyping and product development and test activities that provide quantification of the capabilities and risks. The risk reduction activities provide the data necessary to assess and update the design to reduce its risk.
Product Road Mapping and Release Planning
Product road maps provide an outline that shows when products are scheduled for release and include an overview of the product's primary and secondary features. Both internal and external product road maps should be created. The form of the road map will depend on the development methodology being used. Waterfall, iterative, and spiral development models result in different road maps and release plans. The systems engineer must be an integral member of the team that creates road maps. Requirements should be mapped onto each of the planned releases. Test plans must be adapted to the development model and the release plans.
Product road maps should be aligned with the technology road maps that are applicable to the product. Technology maturity should be accomplished before the technologies are included in the product development plans and the road map for the product release that includes those technologies.
Product road maps are essential for software intensive systems that have many releases of software and capability upgrades. The identification of the requirements, the test plans, and the features provided for each release are an essential driver of the product development process. Clear definition of these items can make the difference between delivering the capabilities the customer is looking for and will support, or a product that fails to meet the needs of the customer and is abandoned.
Intellectual Property Management
Systems engineers must also manage intellectual property as part of their job. Existing systems engineering literature rarely covers this topic. However, there are many textbooks and management related literature that provide additional information, such as “Intellectual Property Rights for Engineers” (Irish 2005). Intellectual property may be considered as intangible output of the rational thought process that has some intellectual or informational value and is normally protected via using copyrights, patents, and/or trade secrets (Irish 2005). Listed below are some of the more important intellectual property types with brief explanations:
- Proprietary Information: Any information which gives a company (or enterprise) an advantage over its competitors is usually proprietary.
- Patents: A patent is the principle mechanism for protecting rights for an invention or discovery. In exchange for a full disclosure of how to practice it, the issuing government will grant the right to exclude others from practicing the invention for a limited amount of time, usually 15 to 20 years (in the US, a patent usually lasts for 17 years from the date of issue).
- Design Patents: In some countries, these are referred to by the more appropriate term design registrations or some other name. They protect rights in ornamental designs, provided the designs are new and inventive, i.e., non-obvious at the time they are made. In the US, the maximum length of a design patent is 14 years.
- Trademarks: A trademark identifies the source of origin for goods in commerce, and is not stronger than the actual use to which it has been put to and the diligence with which it has been protected from infringement, encroachment, or dilution. Under some circumstances, a trademark may be registered with governmental agencies. Among a company's most valuable assets is the corporate name, which also is the company's primary trademark.
- Copyrights: A claim of copyright protects such works as writings, musical compositions, and works of art from being copied by others, i.e., from plagiarism. A notice of claim of copyright must be made in the manner prescribed by law at the time of a protected work’s first publication.
Parts, Materials, and Process Management
The consequences of mission failure or an inability to deploy the system on time due to parts, materials, and process (PM&P) issues needs to be clearly understood by the systems engineer since these elements are fundamental to the overall mission reliability and program success. PM&P management is especially important in harsh environments (like outer space and underwater) and in situations where system failure can have catastrophic impacts on public safety (like nuclear power, bridges and tunnels, and chemical processing plants).
Generally, original equipment manufacturers (OEMs) engaged in the design and fabrication of electronic systems have a documented policy that deals with PM&P, sometimes in the form of a PM&P Management Manual. The elements of a PM&P control program include things such as
- PM&P requirements that apply to a system;
- the generation number of a program or project approved parts list (PAPL);
- the appointment of a PM&P control board (PMPCB);
- the development of a part stress derating policy and a part parameter derating policy for end of life use; and
- a definition of the minimum qualifications, quality controls, and screening requirements for parts.
PM&P management guidance is provided by MIL-HDBK-512 (DoD 2001) and ANSI/AIAA R-100 (2001), which identify the overall management process elements of a PM&P program. Additional issues to be addressed by PM&P include the following: hazardous materials, rare earth elements, conflict materials, and counterfeit materials.
References
Works Cited
ANSI/AIAA. 2001. Recommended Practice for Parts Management. Philadelphia, PA, USA: American National Standards Institute (ANSI)/American Institute of Aeronautics and Astronautics (AIAA), ANSI/AIAA R-100A-2001.
DAU. 2005. Glossary of Defense Acquisition Acronyms & Terms, 12th ed. Ft. Belvoir, VA, USA: Defense Acquisition University (DAU)/US Department of Defense (DoD). Available at: http://www.dau.mil/pubscats/PubsCats/13th_Edition_Glossary.pdf.
DoD. 2000. Department of Defense Directive (DoD-D) 5000.01: The Defense Acquisition System. Arlington, VA, USA: US Department of Defense, Acquisition, Technology, and Logistics (AT&L). Available at: http://www.dtic.mil/whs/directives/corres/pdf/500001p.pdf.
DoD. 2001. Department of Defense Handbook: Parts Management. Arlington, VA, USA: Department of Defense (DoD). MIL-HDBK-512A.
DoD. 2011 Department of Defense Technology Readiness Assessment (TRA) Guidance, Assistant Secretary of Defense for Research and Engineering (ASD(R&E)), May 2011.
FAA. 2011. Airworthiness Certificates Overview. Washington, DC, USA: Federal Aviation Administration (FAA). Available at: http://www.faa.gov/aircraft/air_cert/airworthiness_certification/aw_overview/.
GAO. 2003. Defense acquisitions: Assessments of Major Weapon Programs, GAO-03-476, US Government Accountability Office, May 2003.
INCOSE. 2012. Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities, version 3.2.2. San Diego, CA, USA: International Council on Systems Engineering (INCOSE), INCOSE-TP-2003-002-03.2.2.
Irish, V. 2005. Intellectual Property Rights for Engineers, 2nd ed. Herts, UK: Institution of Engineering and Technology (IET).
Mankins, J. 1995. Technology Readiness Levels—A White Paper. Washington, DC, USA: Advanced Concepts Office, Office of Space Access and Technology, National Aeronautics and Space Administration (NASA).
MITRE. 2011. "Systems Engineering Guide." Accessed September 11, 2012. Available at: http://www.mitre.org/work/systems_engineering/guide/.
Morgan, J 2007. Manufacturing Readiness Levels (MRLs) and Manufacturing Readiness Assessments (MRAs). ADA510027 Air Force Research Lab Wright-patterson Afb Oh Manufacturing Technology Directorate. September 2007. Accessed 06 November 2014 at Defense Technical Information Center http://www.dtic.mil/get-tr-doc/pdf?AD=ADA510027
NERC. 2012. "North American Electric Reliability Corporation (NERC)." Accessed September 11, 2012. Available at: http://www.nerc.com.
Sauser, B., D. Verma, J. Ramirez-Marquez, and R. Gove. 2006. From TRL to SRL: The Concept of System Readiness Levels. Proceedings of the Conference on Systems Engineering Research (CSER), April 7-8, 2006, Los Angeles, CA, USA.
Smith, J. 2005. An Alternative to Technology Readiness Levels for Non-Developmental Item (NDI) Software. Proceedings of the 38th Hawaii International Conference on Systems Sciences, January 3-6, 2005, Island of Hawaii, USA.
Primary References
Mankins, J. 1995. Technology Readiness Levels—A White Paper. Washington, DC, USA: Advanced Concepts Office, Office of Space Access and Technology, National Aeronautics and Space Administration (NASA).
MITRE. "Systems Engineering Guide." Available at http://www.mitre.org/work/systems_engineering/guide/
Sauser, B., D. Verma, J. Ramirez-Marquez, and R. Gove. 2006. From TRL to SRL: The Concept of System Readiness Levels. Proceedings of the Conference on Systems Engineering Research (CSER), Los Angeles, CA, April 7-8, 2006.
Additional References
ANSI/AIAA. 2001. Recommended Practice for Parts Management. Philadelphia, PA, USA: American National Standards Institute (ANSI)/American Institute of Aeronautics and Astronautics (AIAA), ANSI/AIAA R-100A-2001.
DoD. 2000. Department of Defense Directive (DoD-D) 5000.01: The Defense Acquisition System. Arlington, VA, USA: US Department of Defense, Acquisition, Technology, and Logistics (AT&L). Available at: http://www.dtic.mil/whs/directives/corres/pdf/500001p.pdf.
DoD. 2001. Department of Defense Handbook: Parts Management. Arlington, VA, USA: Department of Defense (DoD). MIL-HDBK-512A.
FAA. 2011. "Airworthiness Certificates Overview." Washington, DC, USA: Federal Aviation Administration (FAA). Available at: http://www.faa.gov/aircraft/air_cert/airworthiness_certification/aw_overview/.
Irish, V. 2005. Intellectual Property Rights for Engineers, 2nd ed. Herts, UK: Institution of Engineering and Technology (IET).
Smith, J. 2005. An Alternative to Technology Readiness Levels for Non-Developmental Item (NDI) Software. Proceedings of the 38th Hawaii International Conference on Systems Sciences, January 3-6, 2005, Island of Hawaii, USA.