Assessing Individuals

This article describes how to assess SE competency needs for individuals, how to assess the SE competency of individuals, and how to assess the SE performance of individuals.

Assessing Competency Needs

When developing a competency model for an organization, an initial decision is “make vs. buy.” If there is an existing SE competency model that fits the context and purpose, the organization might want to use the developed SE competency model directly. If existing models must be tailored or a new SE competency model developed, the organization might reference relevant literature on how to go about doing that.

Prior to understanding what SE competencies are needed, it is important to understand the situation in which the organization is embedded. As Figure 1 shows, MITRE has developed a framework characterizing different levels of systems complexity. (MITRE 2007, 1-12, pp. 1-12) This may help an organization frame which competencies are needed. An organization working primarily in the “traditional program domain” may need to emphasize a different set of competencies than an organization working primarily in the “messy frontier.” As an example, if the desired outcome improves existing capabilities, extensive technical knowledge in that specific area might be very important. If the desired outcome builds fundamentally new capability, technical knowledge in a broader set of areas might be useful. A higher level of competency in being able to balance stakeholder requirements might be needed if stakeholder involvement is characterized by multiple equities and distrust rather than when stakeholders concur.

Figure 1 MITRE Enterprise Systems Engineering Framework

The organization might consider both its current situation and its forward strategy. For example, if an organization has previously been working in a traditional systems engineering context (MITRE 2007) and the future strategy is to move into more enterprise systems engineering (ESE) work, the organization might want to develop a competency model both for what was important in the traditional SE context and for what is required for ESE work. An organization might also be moving to a different contracting environment where different competencies are more important, such as the ability to properly tailor the SE approach to “right size” the SE effort to balance cost and risk.

Once an organization has characterized the situation in which it is embedded, the next step is to understand which SE competencies are needed for that situation. The references from Table 5 explain various approaches to SE competency model development. Personality and social psychologists have also developed guidance on competency modeling.

Assessing Individual SE Competency

In order to fulfill the SE competency needs identified, one must be able to assess the existing level of competency for individuals. This assessment informs the interventions needed to further develop individual SE competency.

One approach to competency levels is based on “Levels of Cognition” in Bloom’s Taxonomy (Bloom 1984), presented below in rank order, from least complex to most complex.

• Remember – Recall or recognize terms, definitions, facts, ideas, materials, patterns, sequences, methods, principles, etc.

• Understand – Read and understand descriptions, communications, reports, tables, diagrams, directions, regulations, etc.

• Apply – Know when and how to use ideas, procedures, methods, formulas, principles, theories, etc.

• Analyze – Break down information into its constituent parts and recognize their relationship to one another and how they are organized; identify sublevel factors or salient data from a complex scenario.

• Evaluate – Make judgments about the value of proposed ideas, solutions, etc., by comparing the proposal to specific criteria or standards.

• Create – Put parts or elements together in such a way as to reveal a pattern or structure not clearly there before; identify which data or information from a complex set is appropriate to examine further or from which supported conclusions can be drawn.

Competency levels can also be situationally based. The levels for the SPRDE competency model are based on the complexity of the situation to which the person can appropriately apply the competency to (DAU 2010):

1. No exposure to or awareness of this competency.

2. Awareness: Applies the competency in the simplest situations.

3. Basic: Applies the competency in somewhat complex situations.

4. Intermediate: Applies the competency in complex situations.

5. Advanced: Applies the competency in considerably complex situations.

6. Expert: Applies the competency in exceptionally complex situations.

Other examples of proficiency levels include the INCOSE (INCOSE 2010) competency model proficiency levels of Awareness, Supervised Practitioner, Practitioner, and Expert, and the APPEL competency model levels of participate, apply, manage, and guide, respectively (Menrad and Lawson 29 September-3 October, 2008). NASA as part of the APPEL has also defined the proficiency levels of: I) Technical Engineer/Project Team Member, II) Subsystem Lead/Manager, III) Project Manager/Project Systems Engineer, and IV) Program Manager/Program Systems Engineer.

When using application as a measure of competency, it is important to have a measure of goodness. Just because someone is applying a competency in an exceptionally complex situation, it does not mean they are doing well in this application. Likewise, just because a person is managing and guiding, it does not mean they are doing this well. An individual might be fully competent in an area, but not be given an opportunity to use that competency. Decouple competency and application, unless the application has a measure of goodness for that application and the assumption is made that resources are utilized to their full potential.

Individual SE Competency vs. Performance

Even if an individual possesses exemplary SE competency, the specific context in which the individual is embedded may preclude exemplary performance of that competency. For example, an individual with exemplary risk management competency may be embedded in a team which does not utilize that talent or in an organization with flawed procedural policies which do not fully utilize this ability. Developing individual competencies is not enough to ensure exemplary SE performance. For more discussion on enabling the individual's context, see the sections on enabling teams and enabling enterprises (add links).

References

Please make sure all references are listed alphabetically and are formatted according to the Chicago Manual of Style (15th ed). See the BKCASE Reference Guidance for additional information.

Citations

Bloom, B. S. 1984. Taxonomy of educational objectives. New York, NY: Longman.

DAU. SPRDE-SE/PSE competency assessment: Employee's user's guide, 5/24/2010 version. in Defense Acquisition University (DAU)/U.S. Department of Defense [database online]. Ft. Belvoir, VA, USA, 2010. [1]

INCOSE. 2010. Systems engineering competencies framework 2010-0205. San Diego, CA, USA: International Council on Systems Engineering (INCOSE), INCOSE-TP-2010-003.

Menrad, R., and H. Lawson. 29 September-3 October, 2008. Development of a NASA integrated technical workforce career development model entitled: Requisite occupation competencies and knowledge--the ROCK. Paper presented at 59th International Astronautical Congress (IAC), Glasgow, Scotland.

MITRE. 2007. Enterprise architecting for enterprise systems engineering. SEPO Collaborations. June 2007, SAE International (accessed August 2010).

Primary References

All primary references should be listed in alphabetical order. Remember to identify primary references by creating an internal link using the ‘’’reference title only’’’ (title). Please do not include version numbers in the links.

Additional References

All additional references should be listed in alphabetical order.

---

Article Discussion

[Go to discussion page]