New Keypoint Matching Method Using Local Convolutional Features for Power Transmission Line Icing Monitoring

Qiangliang Guo, Jin Xiao, Xiaoguang Hu
2018 Sensors  
Power transmission line icing (PTLI) problems, which cause tremendous damage to the power grids, has drawn much attention. Existing three-dimensional measurement methods based on binocular stereo vision was recently introduced to measure the ice thickness in PTLI, but failed to meet requirements of practical applications due to inefficient keypoint matching in the complex PTLI scene. In this paper, a new keypoint matching method is proposed based on the local multi-layer convolutional neural
more » ... work (CNN) features, termed Local Convolutional Features (LCFs). LCFs are deployed to extract more discriminative features than the conventional CNNs. Particularly in LCFs, a multi-layer features fusion scheme is exploited to boost the matching performance. Together with a location constraint method, the correspondence of neighboring keypoints is further refined. Our approach achieves 1.5%, 5.3%, 13.1%, 27.3% improvement in the average matching precision compared with SIFT, SURF, ORB and MatchNet on the public Middlebury dataset, and the measurement accuracy of ice thickness can reach 90.9% compared with manual measurement on the collected PTLI dataset. Sensors 2018, 18, 698 2 of 15 implementation steps of these methods can be summarized as: camera calibration, keypoints matching, and ice thickness calculation. The accuracy of keypoint matching has a crucial impact on measurement results. Nevertheless, instead of proposing a new algorithm, the aforementioned literature employs the improved classic feature description and matching methods to verify the feasibility of 3D measurement. A typical keypoint matching method mainly includes feature description, feature matching and outlier removal. Although the advanced feature-matching methods and outlier removal approaches can effectively enhance the final performance of keypoint matching [11] [12] [13] [14] , discriminative feature description is the foundation of the aforementioned processes, especially in the complex PTLI scene. Thus, the focus of this work is on extracting discriminative features and applying it to keypoint matching in the PTLI. The image noise, similarity of foreground and background, high texture repetition, and low distinction of icing types are the main factors affecting the accuracy of keypoint matching in a PTLI scene. Under such conditions, it is difficult to achieve discriminative features using traditional hand-crafted features, such as SIFT (Scale-Invariant Feature Transform) [15] , SURF (Speeded Up Robust Features) [16] and ORB (Oriented FAST and Rotated BRIEF) [17] . As a result, false matching may be caused. In contrast, the features of CNN have certain invariance on translation, distortion, and scaling, together with strong robustness and fault tolerance. Additionally, the learning features have better performance in description of internal information of data and expressiveness [18] . Based on the aforementioned advantages, convolutional features are widely used in matching tasks [19] [20] [21] [22] [23] [24] . In Fischer et al. [19] , CNN deep features were compared with standard SIFT descriptors in terms of region matching and turned out to be superior to SIFT under several typical challenges. In Zagoruyko et al. [20], several CNN-based models were built for comparing image patches, which contain two-channel-based ones, two-stream multi-resolution models, and SPP-based (Spatial-Pyramid-Pooling) Siamese networks. The models can significantly outperform the state-of-art on several benchmark datasets. In Han et al. [21], three fully-connected layers with ReLU (Rectified Linear Units) nonlinearity were used to compute the similarity between the extracted features. In , patch-level correspondence was realized by training deep convolutional models for the extraction of image descriptors. Zbontar et al. [23] addressed the matching cost problem by learning a similarity measure on small image patches using CNN. Combining with the post-processing steps, dense stereo matching can be achieved. Furthermore, Luo et al. [24] replaced the concatenation layer and subsequent processing layers by a single product layer, which shows better performance on efficiency than the works in [23] . Different from the models in [19] [20] [21] [22] [23] [24] , a new keypoint matching method is presented in this paper based on the local multi-layer CNN features, termed Local Convolutional Features (LCFs), which can extract more discriminative features better than the conventional CNNs. In summary, our main contributions include:
doi:10.3390/s18030698 pmid:29495416 pmcid:PMC5876525 fatcat:slaba2kntfh7ld6luwjnukv3zq