For a congruence level H ⊂ GL 2 ( Z), the formation of the modular curve X H , that is, of the coarse moduli space of the level H modular stack X H , is known to commute with arbitrary base change in a wide range of cases. We exhibit infinitely many H, for instance H = Γ 1 (4), for which this coarse base change property fails. In our examples, failure is witnessed for base change to F 2 and for any Z (2) -fiberwise dense open substack of X H . These examples fill in several open entries in a table in the book of Katz and Mazur.