Validating the ISO/IEC 15504 measure of software requirements analysis process capability
IEEE Transactions on Software Engineering
ISO/IEC 15504 is an emerging international standard on software process assessment. It defines a number of software engineering processes, and a scale for measuring their capability. One of the defined processes is software requirements analysis (SRA). A basic premise of the measurement scale is that higher process capability is associated with better project performance (i.e., predictive validity). This paper describes an empirical study that evaluates the predictive validity of SRA process
... y of SRA process capability. Assessments using ISO/IEC 15504 were conducted on 56 projects world wide over a period of two years. Performance measures on each project were also collected using questionnaires, such as the ability to meet budget commitments and staff productivity. The results provide strong evidence of predictive validity for the SRA process capability measure used in ISO/IEC 15504, but only for organisations with more than 50 IT Staff. Specifically, a strong relationship was found between the implementation of requirements analysis practices as defined in ISO/IEC 15504 and the productivity of software projects. For smaller organisations evidence of predictive validity was rather weak. This can be interpreted in a number of different ways: that the measure of capability is not suitable for small organisations, or that the SRA process capability has less affect on project performance for small organisations. Briefly, our results indicate that for large organisations, SRA process capability as measured in ISO/IEC 15504 is related to project productivity. This means that improvements in SRA process capability are associated with a reduction in the cost of software projects. This is interpreted to be due to a reduction in rework during the project. However, no relationship was found with other measures of performance, nor 3 The Capability Maturity Model for Software. 4 The logic of this sequencing is that this is the natural evolutionary order in which, historically, software organisations improve  , and that practices early in the sequence are prerequisite foundations to ensure the stability and optimality of practices implemented later in the sequence . 5 In this paper we only refer to the PDTR version of the ISO/IEC 15504 document set since this was the one used during our empirical study. The PDTR version reflects one of the stages that a document has to go through on the path to international standardisation. The PDTR version is described in detail in  . 6 Here we use the term "SPA" in the general sense, not in the sense of the SEI specific assessment method (which was also called a SPA). 7 A recent survey of sponsors of assessments indicated that baselining process capability and tracking process improvement progress are two important reasons for conducting a SPA  .