This paper combines Markov Random Field and subspaces to perform object tracking. We first sample some particles using particle filter, and then divide each particle to patches. For each particle, we optimize each patch's position and use Markov Random Field to represent the structure of the patches, including each patch's own position and the relations between neighbor patches. We also evaluate each patch and the whole sub image according to their subspaces respectively. Experimental results demonstrated the efficiency of our method.