We argue, based on general principles, that topological order is essential to realize fractionalization in gapped insulating phases in dimensions d ≥ 2. In d=2 with genus g, we derive the existence of the minimum topological degeneracy q^g if the charge is fractionalized in unit of 1/q, irrespective of microscopic model or of effective theory. Furthermore, if the quasiparticle is either boson or fermion, it must be at least q^2g.