To compare reviewers recommendations and editorial decisions in The Lancet, a high-impact journal, and Croatian Medical Journal (CMJ), a small general medical journal. Case study of research manuscripts submitted to the CMJ (n=140 manuscripts; 308 review forms) and a sample of similar manuscripts submitted to The Lancet (n=141; 348 review forms) in 1999 and the first half of 2000. Reviewers recommendations and their influence on editorial decisions on manuscripts were analyzed by logistic

more »

... sion. Agreement between reviewers was assessed by the kappa statistic. Although reviewers scores were identical in the two journals (median=3 for both journals, range 0 to 5), Lancet reviewers more often recommended rejection than CMJ reviewers (44% vs 17%; chi-square=52.1, p=0.029), and agreed best on rejection (kappa=0.29 [95% CI=0.04 to 0.53] vs kappa=-0.04 [-0.45 to 0.36] for CMJ). Lancet editors were even stricter than their reviewers and accepted for publication only 53% [95% CI=37 to 68] of manuscripts graded acceptable by the reviewers, compared with 85% [73 to 91] for CMJ editors (chi-square=10.0, p=0.001). For nine questions about manuscript quality in the review form, multiple regression analysis showed significant association between editorial decision and reviewers scores for the suitability of research design (B=0.70, OR=2.01, 95% CI=1.40 to 2.89, p<0.001) and discussion of systematic/random error (B=0.32, OR=1.38 [1.03 to 1.85], p=0.031) for The Lancet, and scores for novelty of information (B=0.56, OR=1.75 [1.35 to 2.27], p<0.001) for the CMJ. Reviewers of big journals, such as The Lancet, are stricter in their recommendations reviewers of a smaller journal, such as the CMJ. The Lancet editors rely on reviewers to identify methodologically superior studies, whereas CMJ editors look for the novelty of information in a manuscript.