AbstractWe consider Feshbach resonance in an optical cavity where photons interact with atoms and molecules dispersively. From mean-field theory we obtain multiple fixed-point solutions, which is strongly related to the phenomenon of bistability. Adiabatic evolutions demonstrate hysteretic behaviors by varying pump-cavity detuning from opposite directions. We also use the quantum model to check mean-field results which match perfectly. The analysis here may enrich the study of particle-photon interaction systems.