Initialization of Weights in Deep Belief Neural Network Based on Standard Deviation of Feature Values in Training Data Vectors

Nader Rezazadeh
2017 IJCSN-International Journal of Computer Science and Network   unpublished
Nowadays, the feature engineering approach has become very popular in deep neural networks. The purpose of this approach is to extract higher-level and more efficient features compared to those of learning data and to improve the learning of machines. One of the common ways in feature engineering is the use of deep belief networks. In addition, one of the problems in deep neural networks' training is the training process. The problems of the training process will be further enhanced in the
more » ... of an increase in the dimensions of the features and the complexity of the relationship between the initial features and the higher-level features. In the present paper, we attempt to set the initial weights based on the standard deviation of the feature vector values. Hence, a part of the training process is initially conducted and a better starting point can be provided for the weight training process. However, the impact of this method, to a large extent, depends on the relationship between the training data itself and the degree of independence of the training data's feature values. Experiments conducted in this field have achieved acceptable results.