Aim and objectives This study aimed to analyse the reliability of the CLEO scale. Material and methods From the 171 modelled clinical situations in Pharmaclass, the 50 most frequent were chosen. For each situation a PI was retrospectively and randomly selected between November 2019 and November 2020 in the AVIC-ENNE database. It contained 1263 PI transmitted after of Pharmaclass alerts' analysis in two 1700 beds health facilities. A multiprofessional panel of 11 clinicians have rated

more »

... ly the PIs using the CLEO scale. CLEO evaluates the clinical, economical and organisational impact of PI. The panel re-rated the PIs after a 1-month washout period. Intra-class correlation coefficients in absolute agreement on single unit (ICC A,1 ) are calculated using the 'Psych' package on Rstudio to measure inter-and intra-rater reliabilities of the panel. Results The PIs were rated as having a minor, medium, major or vital clinical impact in, respectively, 10%, 70%, 16% and 4% of situations. Direct drug management costs were reduced by the PI in 24%, unchanged in 62% and increased in 14% of the situations. The care process did not change in 78% of the situations, 20% of PIs improved it and 2% of PIs altered it. On average less than 3 min are needed per evaluation. Inter-rater reliability (ICC A,1 ) was poor for clinical (ICC A,1 = 0.297) and organiaational (ICC A,1 = 0.338) dimensions and moderate for economic dimensions (ICC A,1 = 0.665). Intrarater reliability was moderate for clinical (ICC A,1 = 0.611) and organisational (ICC A,1 = 0.726) dimensions and excellent for economic dimensions (ICC A,1 = 0.914). Conclusion and relevance Almost all of AVICENNE PIs prevent a temporary or permanent damage or the need of care to reduce their gravity. The CLEO tool offers a limited validity when used by untrained clinicians. Symbolic artificial intelligence reinforces the therapeutic safety of patients and the relevance of care.