System modeling is important for detecting abnormalities in the control system. Both controller and plant models are required for modeling. The controller model can be obtained from the program, but it is difficult to obtain the plant model. In this paper, we created a stochastic model of the plant by the Markov decision process and examined the method of anomaly detection. We implemented the proposed method using a plant simulator and conducted an anomaly detection experiment of simulated cyber-attacks.