Maintenance of Certification for Family Physicians (MC-FP) examination is designed to measure a single construct: clinical decision-making abilities within the scope of practice of family medicine. Implied in the construct of clinical decision-making abilities is the ability to recall relevant elements from a large fund of pertinent medical knowledge. While clinical decision-making abilities could be perceived as comprising several separate constructs (based on, for example, clinical categories

more »

... or organ systems), that approach would require the development of multiple assessment scales with a passing criteria specific to each. Instead, the overarching construct of clinical decision-making ability, which encompasses those more specific areas, has been selected by the ABFM because it more closely mirrors the pass/fail decision process used to discern which candidates receive certification. In any instance, the construct that the ABFM attempts to measure needs to be sufficiently unidimensional to produce precise, error-free estimates of a candidate's performance. This brief article will discuss the dimensionality of the MC-FP examination and its implications for construct validity, namely the validation that the examination in fact accurately measures the ability of family physicians to make appropriate clinical decisions. Dimensionality Why is dimensionality important? Simply put, it is desirable to measure only one thing at a time. Just as physical measurements attempt to measure one thing at a time (eg, a patient's blood pressure reading should not be biased by his or her height, weight, or sex), psychometricians, the measurement experts that help design the ABFM's examinations, also aspire to mea-sure one latent trait at a time. It is only when dimensions are clearly isolated that one can understand the meaning of the measure and make a valid inference about an examination score. Dimensionality of the MC-FP Examination As we have mentioned previously, the psychometric model that the ABFM employs to score its examinations is the Rasch model, a one-parameter item response theory measurement model. The Rasch model converts raw scores to linear measures and controls for the difficulty of the version of a test a candidate received. 1 In addition to using typical fit indicators, the most effective way to detect multidimensionality in the analysis of data based on Rasch measurements is to use a principal components analysis (PCA) of standardized residual correlations. 2 In short, the Rasch model uses ordinal data to construct a one-dimensional measurement system. Of course, real data are never perfectly unidimensional, so the presence of more than one latent dimension in the data always exists to some extent. When the data perfectly fit the Rasch model (this includes all items and persons examined), all systematic variation is explained by a single dimension. Data that are not in perfect accord with the model leave behind residuals that have a random normal structure and predictable variance. 2 To evaluate the dimensionality of the MC-FP examination, we perform the aforementioned industry standard tests of fit and PCA of standardized residual correlations. An investigation of how the data fit to the model, both overall and by individual item analysis, can help us discern whether multiple dimensions are present and exactly where in the dataset these dimensions might be. To demonstrate this, let us share an analysis we performed using the core portion of the 2010 examination. The dataset included 3697 examinees and the 423 Conflict of interest: The authors are from the ABFM.