The evaluation of quality assurance: developing and testing practical methods for managers

T. TZAVARAS CATSAMBAS
2002 International Journal for Quality in Health Care  
Objective of the study. To develop an approach for evaluating quality assurance (QA) activities and programs in health care settings and to test different evaluation methods. Design. This was not a formal scientific study, but rather a research and development (R&D) study, which followed the following steps: (1) reviewing the literature; (2) clarifying critical issues for all key aspects of QA activities; (3) drafting a guide to provide a flexible vehicle for different approaches; (4) testing
more » ... d adapting the guide as it evolved in three countries; and (5) testing two evaluation tools (self-assessment and appreciative evaluation) in Chile. Setting and study participants. The evaluation guide was tested by evaluating QA structures, activities, and programs at the country, regional, and facility levels in Zambia, Niger, and Chile. Results. The study resulted in an evaluation guide, which includes an implementation outline, an evaluation matrix, and an appendix of evaluation tools and methods. The guide helps evaluators: agree on a proposed evaluation's scope and design; develop an evaluation methods plan; and address QA history, advocacy, culture, and structure, as well as QA activities and accomplishments. Specific results of the country evaluations in Zambia, Niger, and Chile are presented in separate articles in this supplement. Conclusion. The QA programs in which the evaluation guide was tested differed in many ways, such as health system structure, decision to focus on particular services, political level implementing QA, policy environment, leadership, and program evolution. The implementation guide presents an outline of the key implementation steps for an evaluation, and includes checklists and model forms (e.g. sample agenda for a team planning meeting, sample list of questions to focus the evaluation). The evaluation matrix presents indicators by QA component and key question, and it enables evaluators to build an approach and select methods. The appendix describes the various tools and methods presented in the first two sections of the evaluation guide. The United States Agency for International Development various methods. QAP evaluators teamed up with outside experts to conduct three evaluations, in Zambia, Niger and (USAID)-funded Quality Assurance Project II (QAP) required an approach and testing methods to evaluate QA Chile (each is presented in separate articles in this supplement). QAP technical staff engaged in probing discussions about activities and programs. As the work of designing evaluation methods was underway, USAID missions asked QAP to the 'core QA approach' versus 'field adaptations', and the implications of this for how to evaluate activities and pro-evaluate selected QA programs. This provided the opportunity to review different approaches interactively with grams. Rather than develop one approach for evaluating QA, the team developed an evaluation guide that would allow field visits, to build an evaluation guide, and to field-test Address reprint requests to Tessie Tzavaras Catsambas, EnCompass LLC, 7803 Fox Gate Court,
doi:10.1093/intqhc/14.suppl_1.75 pmid:12572790 fatcat:2ot7qprtprf2nezdn72zzg6y2e