Please wait a minute...


Remote Sensing Technology and Application  2021, Vol. 36 Issue (2): 247-255    DOI: 10.11873/j.issn.1004-0323.2021.2.0247
Research on Gap-repairing Algorithm based on Convolutional Neural Network Prediction Result
Zhao Liu(),Tong Zhao(),Feifan Liao
Institute of Traffic Engineering and Geospatial Information,Department of Civil Engineering,Tsinghua University,Beijing 100084,China
Download:  HTML  PDF (6143KB) 
Export:  BibTeX | EndNote (RIS)      

With the development of convolutional neural network technology, recent research has paid more attention to the improvement of accuracy and the improvement of semantic information. Mask R-CNN network is a further improved segmentation network of Faster R-CNN. It has a good segmentation effect in high-resolution remote sensing image feature recognition. However, since the convolutional neural network can only be trained and predicted with small tile images, there is a large semantic information error in the prediction results. Faced with this problem, this paper proposed a gap-repairing algorithm based on the defect of prediction result of convolutional neural network. The approach use overlapsize algorithm to improve the matching degree between the prediction result and the ground-truth result at first. Then fill the gap through the correlation function in the PostGIS database to repair the small tile, which can make it be spliced ??into a complete picture. The research and experiment results showed that the algorithm could improve the image semantic information well and has practicability.

Key words:  Convolutional neural network      Instance segmentation      Mask R-CNN      Gap-repairing algorithm     
Received:  20 June 2020      Published:  24 May 2021
ZTFLH:  P237  
Corresponding Authors:  Tong Zhao     E-mail:;
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
Articles by authors
Zhao Liu
Tong Zhao
Feifan Liao

Cite this article: 

Zhao Liu,Tong Zhao,Feifan Liao. Research on Gap-repairing Algorithm based on Convolutional Neural Network Prediction Result. Remote Sensing Technology and Application, 2021, 36(2): 247-255.

URL:     OR

Fig.1  Mask R-CNN network framework
Fig.2  Mask R-CNN network prediction process
Fig.3  Principle of Overlapsize algorithm
Fig.4  Gap-repairing algorithmic flow
Fig.5  Farmland instance segmentation dataset examples
Fig.6  Varying curve of total loss value
Fig.7  Varying curve of classified loss value
Fig.8  Varying curve of bounding box loss value
Fig.9  Varying curve of mask loss value
Mask R-CNN/%33.655.235.3
Table 1  Network training accuracy
Fig.10  Experimental comparison of overlapsize algorithm
Fig.11  Comparisons of gap-repairing algorithm
1 Krizhevsky A,Sutskever I,Hinton G E.ImageNet Classification with Deep Convolutional Neural Networks[C]∥ Advances in Neural Information Processing Systems,2012:1097-1105.
2 Girshick R, Donahue J, Darrell T, et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 580-587.
3 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C]∥ International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, Cham,2015:234-241.
4 Badrinarayanan V,Kendall A,Cipolla R.Segnet: A Deep Convolutional Encoder-decoder Architecture for Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
5 He K, Zhang X, Ren S, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
6 Girshick R. Fast r-cnn[C]∥ Proceedings of the IEEE International Conference on Computer Vision,2015:1440-1448.
7 Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks[C]∥ Advances in Neural Information Processing Systems, 2015: 91-99.
8 He K, Gkioxari G, Dollár P, et al. Mask R-CNN[C]∥ Proceedings of the IEEE International Conference on Computer Vision, 2017: 2961-2969.
9 Lin T Y, Dollár P, Girshick R, et al. Feature Pyramid Networks for Object Detection[C]∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017: 2117-2125.
10 Nie S, Jiang Z, Zhang H, et al. Inshore Ship Detection based on Mask R-CNN[C]∥ IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 2018: 693-696.
11 Zhang Y, You Y, Wang R, et al. Nearshore Vessel Detection based on Scene-mask R-CNN in Remote Sensing Image[C]∥ 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC). IEEE, 2018: 76-80.
12 Zhao K, Kang J, Jung J, et al. Building Extraction from Satellite Images Using Mask R-CNN with Building Boundary Regularization[C]∥ CVPR Workshops. 2018: 247-251.
13 Wen Q, Jiang K, Wang W, et al. Automatic Building Extraction from Google Earth Images Under Complex Backgrounds based on Deep Instance Segmentation Network[J]. Sensors, 2019, 19(2): 333. doi:.
doi: 10.3390/s19020333
14 Zhang C, Li Z, Liu Q, et al. Toward Efficient Simultaneous Detection and Segmentation[C]∥ 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), IEEE, 2018: 1-5.
15 Tian Jin, Zhang Chi, Wang Yongsen, et al. Ground Mark Detection based on Mask R-CNN[C]∥ The 22nd Annual Meeting of Network New Technologies and Applications of China Computer Users Association Network Application Branch2018.
15 田锦, 张弛, 王永森, 等. 基于 Mask R-CNN 的地面标识检测[C]∥ 中国计算机用户协会网络应用分会 2018 年第二十二届网络新技术与应用年会论文集, 2018.
16 Bai Baolin. Vehicle Identification and Detection based on Improved Mask R-CNN[D]. Anhui: Anhui University, 2018.
16 白宝林. 基于改进的 Mask R-CNN 的车辆识别及检测[D]. 安徽:安徽大学, 2018.
17 Deng Liuyuan, Yang Ming, Wang Chunxiang, et al. Surround View Cameras based Instance Segmentation Method for Autonomous Vehicles[J]. Huazhong University of Science and Technology(Natural Science Edition),2018,46(12):24-29.邓琉元, 杨明, 王春香, 等. 基于环视相机的无人驾驶汽车实例分割方法[J]. 华中科技大学学报 (自然科学版), 2018 .46(12):24-29.
18 Chen Mingyang, Zhao Zhigang, Pan Zhikuan, et al. Image Instance Segmentation based on Deep Learning[J]. Journal of Qingdao University (Natural Science Edition), 2019,32(1): 46-50, 54.
18 陈茗杨, 赵志刚, 潘振宽, 等. 基于深度学习的图像实例分割[J]. 青岛大学学报:自然科学版, 2019,32(1): 46-50, 54.
19 Zhang Xudong, Wang Yuting, Fan Zhiguo, et al. RGB-D Multi-class Instance Segmentation based on Double Pyramid Feature Fusion Model[J]. Control and Decision,2020,35(7):1561-1568.
19 张旭东, 王玉婷, 范之国,等. 基于双金字塔特征融合网络的RGB-D多类实例分割[J].控制与决策,2020,35(7):1561-1568.
20 Lin T Y, Dollár P, Girshick R, et al. Feature Pyramid Networks for Object Detection[C]∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 2117-2125.
21 Liu Y, Yao J, Lu X, et al. DeepCrack: A Deep Hierarchical Feature Learning Architecture for Crack Segmentation[J]. Neurocomputing, 2019, 338: 139-153.
No Suggested Reading articles found!