Difference between revisions of "SEBoK Review and Adjudication"

Latest revision as of 22:46, 2 May 2024

The participation of the worldwide systems engineering community in the open collaboration process played a significant role in shaping the development versions of SEBoK leading to release of v. 1.0 in September 2012 (see Acknowledgements and Release History). It is expected that community participation and review of the content will continue to keep the content of the SEBoK useful, current, and relevant.

This article outlines the process that was followed to solicit and adjudicate review comments that have been received on versions 0.25, 0.5, 0.75, 1.0 and 1.0.1 of the SEBoK . A comprehensive list of all comments received and the authors’ response to those comments is included in the adjudication matrices.

Since January 2013, versions 1.1 and beyond are being released under a new governance structure. The adjudication process described here is not applicable to these releases (versions 1.1. and beyond). Also, with every new release of the SEBoK since v. 1.1.2, the SEBoK Sandbox is also opened to receive comments and other contributions.

Reviewers and Review Comments

SEBoK v. 0.25

The first version of the SEBoK, v. 0.25, was released as a document for limited review in September 2010. A list of potential reviewers was compiled based on

direct requests to professional societies, corporations, and academic institutions; and
volunteers who responded to requests during SEBoK related presentations and talks at conferences and other events.

The SEBoK v. 0.25 document, along with a review form with over thirty questions and detailed instructions, were distributed to the potential reviewers.

Reviews were received from 114 reviewers who provided over 3000 comments by December 2010.

SEBoK v. 0.5

A number of changes in the structure and content of the SEBoK was made in response to the review comments received for SEBoK v. 0.25. The next version of the SEBoK, v. 0.5, was released in September 2011, as a wiki. In addition to providing easy access and navigation of the SEBoK, the wiki platform also enabled broader review by the worldwide systems engineering community. Release information and requests for review were sent out to an updated list of reviewers. Review comments on individual articles were predominantly received on the wiki. A review form was also distributed to receive overarching comments on the SEBoK as a whole and on the wiki implementation. Comments were also received in various other formats including emails, spreadsheets and marked-up documents until December 2011.

SEBoK v. 0.75

Though not originally planned, the third interim version of the SEBoK, v. 0.75, was released in March 2012 for a short limited period review until April 2012. Not all of the review comments were addressed in SEBoK v. 0.75. The wiki platform was well received by the worldwide SE community and so was retained for future releases of the SEBoK. More features for reviewing and navigating were added to the SEBoK wiki. The wiki remained open for review, though it was stated that the adjudication of those comments may not be addressed for SEBoK v. 1.0.

SEBoK v. 1.0 and v. 1.0.1

The first production version of the SEBoK, v. 1.0, was released in September 2012. In November 2012, v. 1.0.1, a micro update was released to correct a few errors but with no significant change in the content. Unlike previous developmental versions of the SEBoK, public review of SEBoK v. 1.0 and v. 1.0.1 was not intentionally solicited, but users were strongly encouraged to provide their comments and feedback.

SEBoK v. 1.1 and beyond

Beginning in January 2013, the SEBoK is operating under a new governance structure led by INCOSE, IEEE-CS, and the SERC acting jointly as stewards (see Acknowledgements and Release History). The SEBoK wiki continues to accept comments at the bottom of every page, but comments and various other types of contributions are also accepted through the SEBoK Sandbox. Adjudication matrices are no longer prepared or published.

Adjudication Process

A thorough process involving the authors and Part Team Leads (PTLs) was used to address every review comment received on the SEBoK. This process largely remained the same for all versions of the SEBoK that were released. Since January 2013, a new governance structure is in place where there are no PTLs, and contributions are being received in a variety of ways. The BKCASE associate and assistant editors actively adjudicate comments received, incorporating necessary changes in the next release of the SEBoK. The adjudication process described here is not applicable to releases v. 1.1 and beyond.

The following broad steps were followed for adjudication:

All comments received in various formats were all compiled in matrices as spreadsheets, segregated by SEBoK parts. When warranted, compound comments were broken down into individual comments to support effective response to the comments.
The comments were then distributed to the respective PTLs and authors.
The authors responded to every review comment by choosing one of the adjudication actions listed in Table 1 below. Where required, authors provided reasons and rationale for their decision on particular review comments.
Adjudication matrices were then re-compiled incorporating author responses as spreadsheets for further use and distribution.

**Table 1. Types of Adjudication Actions.** (SEBoK Original)
Action	Description
Accepted	The comment from the reviewer has been accepted in whole, and suggested action has been taken.
Modified	The comment from the reviewer has been accepted in part and suggested action has been taken with some modification(s). Details of the modification is provided.
Declined	Suggestion recommended by the reviewer has been declined. Rationale for this decision is provided
Noted	The comment made by the reviewer requires no further action.
Copy	This comment is similar to a comment already received and adjudicated (in this or earlier version of the SEBoK).
Deferred	Response to the comment is deferred to a future version of the SEBoK.

Adjudication Matrices

A detailed adjudication report was prepared based on the reviews received on SEBoK v. 0.25. For the review comments on the 0.5 and 0.75 versions of the SEBoK, the adjudication matrices have been prepared. Adjudication of comments received on the 1.0 and 1.0.1 versions are included in a single matrix. All matrices be may be downloaded (in PDF format) from the following links:

It may be noted that some review comments may be relevant only to the SEBoK version on which the comments were made; content may have changed since then. Names of individual reviewers have not been included in any of the above adjudication matrices. If you reviewed an earlier version of the SEBoK and would like to know your reviewer number and see the responses to your comments, please contact bkcase@stevens.edu.

SEBoK v. 2.10, released 06 May 2024

@@ Line 1: / Line 1: @@
-The participation of the worldwide systems engineering (SE) community in the open collaboration process has played a significant role in shaping the version 1.0 of the Guide to the Systems Engineering Body of Knowledge (SEBoK).  This article outlines the process followed to solicit and adjudicate review comments received on the developmental versions of the SEBoK (i.e. version 0.25, 0.5 and 0.75). A comprehensive list of all comments received and the authors’ response to those comments is also included as file attachments (in PDF / MS Excel formats).
+The participation of the worldwide systems engineering community in the open collaboration process played a significant role in shaping the development versions of SEBoK leading to release of v. 1.0 in September 2012 (see [[Acknowledgements and Release History]]). It is expected that community participation and review of the content will continue to keep the content of the SEBoK useful, current, and relevant.
+This article outlines the process that was followed to solicit and adjudicate review comments that have been received on versions 0.25, 0.5, 0.75, 1.0 and 1.0.1 of the SEBoK . A comprehensive list of all comments received and the authors’ response to those comments is included in the adjudication matrices.
+Since January 2013, versions 1.1 and beyond are being released under a [[BKCASE_Governance and Editorial Board|new governance structure]]. The adjudication process described here is not applicable to these releases (versions 1.1. and beyond). Also, with every new release of the SEBoK since v. 1.1.2, the  [http://www.sebokwiki.org/sandbox SEBoK Sandbox] is also opened to receive comments and other contributions.
 ==Reviewers and Review Comments==
 ===SEBoK v. 0.25===
-The first version of the SEBoK, version 0.25, was released as a document for limited review in September 2010. A list of potential reviewers was complied based on
+The first version of the SEBoK, v. 0.25, was released as a document for limited review in September 2010. A list of potential reviewers was compiled based on
 *direct requests to professional societies, corporations, and academic institutions; and
 *volunteers who responded to requests during SEBoK related presentations and talks at conferences and other events.
-The SEBoK v. 0.25 document, along with a review form with over thirty questions and detailed instructions were distributed to the potential reviewers.
+The SEBoK v. 0.25 document, along with a review form with over thirty questions and detailed instructions, were distributed to the potential reviewers.
 Reviews were received from 114 reviewers who provided over 3000 comments by December 2010.
 ===SEBoK v. 0.5===
-A number of changes in the structure and content of the SEBoK was made in response to the review comments received for SEBoK 0.25. The next version of the SEBoK, version 0.5, was released in September 2011, as a wiki. In addition to providing easy access and navigation of the SEBoK, the wiki platform also enabled broader review by the worldwide systems engineering community. Release information and requests for review were sent out to an updated list of reviewers. Review comments on individual articles were predominantly received on the wiki. A review form was also distributed to receive overarching comments on the SEBoK as a whole and on the wiki implementation. Comments were also received in various other formats including emails, spreadsheets and marked-up documents until December 2011.
+A number of changes in the structure and content of the SEBoK was made in response to the review comments received for SEBoK v. 0.25. The next version of the SEBoK, v. 0.5, was released in September 2011, as a wiki. In addition to providing easy access and navigation of the SEBoK, the wiki platform also enabled broader review by the worldwide systems engineering community. Release information and requests for review were sent out to an updated list of reviewers. Review comments on individual articles were predominantly received on the wiki. A review form was also distributed to receive overarching comments on the SEBoK as a whole and on the wiki implementation. Comments were also received in various other formats including emails, spreadsheets and marked-up documents until December 2011.
 ===SEBoK v. 0.75===
-Though not originally planned, the third interim version of the SEBoK, version 0.75, was released in March 2012 for a short limited period review until April 2012.  Not all of the review comments were addressed in SEBoK 0.75. The wiki platform was well received by the worldwide SE community and so was retained for future releases of the SEBoK.  More features for reviewing and navigating were added to the SEBoK wiki. The wiki remained open for review, though it was stated that the adjudication of those comments may not be addressed for SEBoK v 1.0.
+Though not originally planned, the third interim version of the SEBoK, v. 0.75, was released in March 2012 for a short limited period review until April 2012. Not all of the review comments were addressed in SEBoK v. 0.75. The wiki platform was well received by the worldwide SE community and so was retained for future releases of the SEBoK. More features for reviewing and navigating were added to the SEBoK wiki. The wiki remained open for review, though it was stated that the adjudication of those comments may not be addressed for SEBoK v. 1.0.
-===SEBoK v. 1.0===
-The wiki continues to remain open to review comments as SEBoK v. 1.0 is released and is now available worldwide. Comments received here, and those that were received after the review period of SEBoK v. 0.75 was closed would be adjudicated for future versions of the SEBoK.
+===SEBoK v. 1.0 and v. 1.0.1===
+The first production version of the SEBoK, v. 1.0, was released in September 2012. In November 2012, v. 1.0.1, a micro update was released to correct a few errors but with no significant change in the content. Unlike previous developmental versions of the SEBoK, public review of SEBoK v. 1.0 and v. 1.0.1 was not intentionally solicited, but users were strongly encouraged to provide their comments and feedback.
+===SEBoK v. 1.1 and beyond===
+Beginning in January 2013, the SEBoK is operating under a new governance structure led by INCOSE, IEEE-CS, and the SERC acting jointly as stewards (see [[Acknowledgements and Release History]]). The SEBoK wiki continues to accept comments at the bottom of every page, but comments and various other types of contributions are also accepted through the [http://www.sebokwiki.org/sandbox SEBoK Sandbox]. Adjudication matrices are no longer prepared or published.
 ==Adjudication Process==
-A thorough process involving the authors and Part Team Leads (PTLs) was used to address every review comment received on the SEBoK. This process largely remained the same for all versions of the SEBoK that were released.
+A thorough process involving the authors and Part Team Leads (PTLs) was used to address every review comment received on the SEBoK. This process largely remained the same for all versions of the SEBoK that were released. Since January 2013, a new governance structure is in place where there are no PTLs, and contributions are being received in a variety of ways. The [[BKCASE Governance and Editorial Board#BKCASE Editorial Board|BKCASE associate and assistant editors]] actively adjudicate comments received, incorporating necessary changes in the next release of the SEBoK. The adjudication process described here is not applicable to releases v. 1.1 and beyond.
+The following broad steps were followed for adjudication:
+#All comments received in various formats were all compiled in matrices as spreadsheets, segregated by SEBoK parts. When warranted, compound comments were broken down into individual comments to support effective response to the comments.
+#The comments were then distributed to the respective PTLs and authors.
+#The authors responded to every review comment by choosing one of the adjudication actions listed in Table 1 below. Where required, authors provided reasons and rationale for their decision on particular review comments.
+#Adjudication matrices were then re-compiled incorporating author responses as spreadsheets for further use and distribution.
-Once the review period for a SEBoK version was over, the following broad steps were followed:
-#All comments received in various formats were all compiled in spreadsheets, segregated by SEBoK parts. When warranted, compound comments were broken down into individual comments to support effective response to the comments.
-#The comments were then distributed to the respective PTLs and authors
-#The authors responded to every review comment by choosing one of the actions listed in Table 1 below. Where required, authors provided reasons and rationale for their decision on particular review comments.
-#Adjudication matrices with then re-compiled incorporating author responses as spreadsheets for further use and distribution.
-<center> ‘’’Table 1. Types of Adjudication Responses.’’’ (SEBoK Originial)
+<center>
+{|
+|+'''Table 1. Types of Adjudication Actions.''' (SEBoK Original)
+! Action
+! Description
+|-
+|'''Accepted'''
+|
+*The comment from the reviewer has been accepted in '''whole''', and suggested action has been taken.
+|-
+| '''Modified'''
+|
+*The comment from the reviewer has been accepted in '''part''' and suggested action has been taken with some modification(s).
+*Details of the modification is provided.
+|-
+|'''Declined'''
+|
+*Suggestion recommended by the reviewer has been declined.
+*Rationale for this decision is provided
+|-
+|''' Noted'''
+|
+*The comment made by the reviewer requires no further action.
+|-
+| '''Copy'''
+|
+*This comment is similar to a comment already received and adjudicated (in this or earlier version of the SEBoK).
+|-
+| '''Deferred'''
+|
+*Response to the comment is deferred to a future version of the SEBoK.
+|}
+</center>
 ==Adjudication Matrices==
+A detailed adjudication report was prepared based on the reviews received on SEBoK v. 0.25. For the review comments on the 0.5 and 0.75 versions of the SEBoK, the adjudication matrices have been prepared. Adjudication of comments received on the 1.0 and 1.0.1 versions are included in a single matrix. All matrices be may be downloaded (in PDF format) from the following links:
+* [http://sebokwiki.org/downloads/SEBOKVersion0.25AdjudicationReportFINAL1.pdf SEBoK v. 0.25 Adjudication Report]
+* [http://sebokwiki.org/downloads/SEBoK05_AdjudicationMatrix_FINAL.pdf SEBoK v. 0.5 Adjudication Matrix]
+* [http://sebokwiki.org/downloads/SEBoK075_AdjudicationMatrix_FINAL.pdf SEBoK v. 0.75 Adjudication Matrix]
+* [http://sebokwiki.org/downloads/SEBoK1.0_1.0.1_Adjudication%20Matrix.pdf SEBoK v. 1.0 & 1.0.1 Adjudication Matrix]
+It may be noted that some review comments may be relevant only to the SEBoK version on which the comments were made; content may have changed since then. Names of individual reviewers have not been included in any of the above adjudication matrices. If you reviewed an earlier version of the SEBoK and would like to know your reviewer number and see the responses to your comments, please contact [mailto:bkcase@stevens.edu bkcase@stevens.edu].
-{{DISQUS}}
+<center>'''SEBoK v. 2.10, released 06 May 2024'''</center>