We consider the model theoretic notion of convex orderability, which fits strictly between the notions of VC-minimality and dp-minimality. In some classes of algebraic theories, however, we show that convex orderability and VC-minimality are equivalent, and use this to give a complete classification of VC-minimal theories of ordered groups and abelian groups. Consequences for fields are also considered, including a necessary condition for a theory of valued fields to be quasi-VC-minimal. For example, the p-adics are not quasi-VC-minimal.