We consider the estimation problem of an unknown drift parameter within classes of non-degenerate diffusion processes. The Maximum Likelihood Estimator (MLE) is analyzed with regard to its pathwise stability properties and robustness towards misspecification in volatility and even the very nature of noise. We construct a version of the estimator based on rough integrals (in the sense of T. Lyons) and present strong evidence that this construction resolves a number of stability issues inherent to the standard MLEs.