Practice points  It is important to always evaluate the quality of a high stakes assessment, such as an OSCE, through the use of a range of appropriate metrics.  When judging the quality of an OSCE, it is very important to employ more than one metric to gain an all round view of the assessment quality. Abstract With an increasing use of criterion based assessment techniques in both undergraduate and postgraduate healthcare programmes, there is a consequent need to ensure the quality and

more »

... of these assessments. The obvious question for those responsible for delivering assessment is how is this 'quality' measured, and what mechanisms might there be that allow improvements in assessment quality over time to be demonstrated? Whilst a small base of literature exists, few papers give more than one or two metrics as measures of quality in OSCEs. In this guide, aimed at assessment practitioners, the authors aim to review the metrics that are available for measuring quality and indicate how a rounded picture of OSCE assessment quality may be constructed by using a variety of such measures, and also to consider which characteristics of the OSCE are appropriately judged by which measure(s). The authors will discuss the quality issues both at the individual station level and across the complete clinical assessment as a whole, using a series of 'worked examples' drawn from OSCE data sets from the authors' institution.