It is expected that the interplay between non-trivial band topology and strong electron correlation will lead to very rich physics. Thus a controlled study of the competition between topology and correlation is of great interest. Here, employing large-scale quantum Monte Carlo (QMC) simulations, we provide a concrete example of the Kane-Mele-Hubbard (KMH) model on an AA stacking bilayer honeycomb lattice with inter-layer antiferromagnetic interaction. Our simulation identified several different

more »

... phases: a quantum spin-Hall insulator (QSH), a xy-plane antiferromagnetic Mott insulator (xy-AFM) and an inter-layer dimer-singlet insulator (dimer-singlet). Most importantly, a bona fide topological phase transition between the QSH and the dimer-singlet insulators, purely driven by the inter-layer antiferromagnetic interaction is found. At the transition, the spin and charge gap of the system close while the single-particle excitations remain gapped, which means that this transition has no mean field analogue and it can be viewed as a transition between bosonic SPT states. At one special point, this transition is described by a (2+1)d O(4) nonlinear sigma model (NLSM) with exact SO(4) symmetry, and a topological term at exactly Θ = π. Relevance of this work towards more general interacting SPT states is discussed.