On-line Learning of Perceptron from Noisy Data by One and Two Teachers

Tatsuya Uezu, Yoshiko Maeda, Sachi Yamaguchi
2006 Journal of the Physical Society of Japan  
We analyze the on-line learning of a Perceptron from signals produced by a single Perceptron suffering from external noise or by two independent Perceptrons without noise. We adopt typical three learning rules in both single-teacher and two-teacher cases. For the single-teacher case, we treat the input and output noises and for the two-teacher case, we assume that signals are given by two teachers with a definite probability. In the single-teacher case, in order to improve the learning when it
more » ... oes not succeed in the sense that the student vector does not converge to the teacher vector, we use two methods: a method based on the optimal learning rate and an averaging method. Furthermore, we obtain an asymptotic form of the generalization error using an optimal learning rate for the three learning rules, and we estimate noise parameters using the simulation data by the averaging method. In the two-teacher case, for the Hebbian rule, we give analytical solutions of order parameters. Furthermore, we estimate noise parameters using the Perceptron rule by the averaging method. The theoretical results agree quite well with the numerical simulations.
doi:10.1143/jpsj.75.114007 fatcat:owuo4bnlf5aofkcotmoxu2xsvm