Research on Gap-repairing Algorithm based on Convolutional Neural Network Prediction Result

doi:10.11873/j.issn.1004-0323.2021.2.0247

Remote Sensing Technology and Application

2021, Vol. 36

Issue (2): 247-255 DOI: 10.11873/j.issn.1004-0323.2021.2.0247

Research on Gap-repairing Algorithm based on Convolutional Neural Network Prediction Result

Zhao Liu(

),Tong Zhao(

),Feifan Liao

Institute of Traffic Engineering and Geospatial Information，Department of Civil Engineering，Tsinghua University，Beijing 100084，China

Download:

HTML

PDF (6143KB)
Export: BibTeX | EndNote (RIS)

Abstract

With the development of convolutional neural network technology， recent research has paid more attention to the improvement of accuracy and the improvement of semantic information. Mask R-CNN network is a further improved segmentation network of Faster R-CNN. It has a good segmentation effect in high-resolution remote sensing image feature recognition. However， since the convolutional neural network can only be trained and predicted with small tile images， there is a large semantic information error in the prediction results. Faced with this problem， this paper proposed a gap-repairing algorithm based on the defect of prediction result of convolutional neural network. The approach use overlapsize algorithm to improve the matching degree between the prediction result and the ground-truth result at first. Then fill the gap through the correlation function in the PostGIS database to repair the small tile， which can make it be spliced ??into a complete picture. The research and experiment results showed that the algorithm could improve the image semantic information well and has practicability.

Key words: Convolutional neural network Instance segmentation Mask R-CNN Gap-repairing algorithm

Received: 20 June 2020 Published: 24 May 2021

ZTFLH:

P237

Corresponding Authors: Tong Zhao E-mail: liuz@tsinghua.edu.cn;zhaot18@mails.tsinghua.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Zhao Liu
	Tong Zhao
	Feifan Liao

Cite this article:

Zhao Liu,Tong Zhao,Feifan Liao. Research on Gap-repairing Algorithm based on Convolutional Neural Network Prediction Result. Remote Sensing Technology and Application, 2021, 36(2): 247-255.

URL:

http://www.rsta.ac.cn/EN/10.11873/j.issn.1004-0323.2021.2.0247 OR http://www.rsta.ac.cn/EN/Y2021/V36/I2/247

Fig.1 Mask R-CNN network framework

Fig.2 Mask R-CNN network prediction process

Fig.3 Principle of Overlapsize algorithm

Fig.4 Gap-repairing algorithmic flow

Fig.5 Farmland instance segmentation dataset examples

Fig.6 Varying curve of total loss value

Fig.7 Varying curve of classified loss value

Fig.8 Varying curve of bounding box loss value

Fig.9 Varying curve of mask loss value

Table 1 Network training accuracy

Fig.10 Experimental comparison of overlapsize algorithm

Fig.11 Comparisons of gap-repairing algorithm

1	Krizhevsky A，Sutskever I，Hinton G E.ImageNet Classification with Deep Convolutional Neural Networks［C］∥ Advances in Neural Information Processing Systems，2012：1097-1105.
2	Girshick R， Donahue J， Darrell T， et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition， 2014： 580-587.
3	Ronneberger O， Fischer P， Brox T. U-Net： Convolutional Networks for Biomedical Image Segmentation［C］∥ International Conference on Medical Image Computing and Computer-assisted Intervention. Springer， Cham，2015：234-241.
4	Badrinarayanan V，Kendall A，Cipolla R.Segnet： A Deep Convolutional Encoder-decoder Architecture for Image Segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495.
5	He K， Zhang X， Ren S， et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
6	Girshick R. Fast r-cnn［C］∥ Proceedings of the IEEE International Conference on Computer Vision，2015：1440-1448.
7	Ren S， He K， Girshick R， et al. Faster R-CNN： Towards Real-time Object Detection with Region Proposal Networks［C］∥ Advances in Neural Information Processing Systems， 2015： 91-99.
8	He K， Gkioxari G， Dollár P， et al. Mask R-CNN［C］∥ Proceedings of the IEEE International Conference on Computer Vision， 2017： 2961-2969.
9	Lin T Y， Dollár P， Girshick R， et al. Feature Pyramid Networks for Object Detection［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017： 2117-2125.
10	Nie S， Jiang Z， Zhang H， et al. Inshore Ship Detection based on Mask R-CNN［C］∥ IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium， IEEE， 2018： 693-696.
11	Zhang Y， You Y， Wang R， et al. Nearshore Vessel Detection based on Scene-mask R-CNN in Remote Sensing Image［C］∥ 2018 International Conference on Network Infrastructure and Digital Content （IC-NIDC）. IEEE， 2018： 76-80.
12	Zhao K， Kang J， Jung J， et al. Building Extraction from Satellite Images Using Mask R-CNN with Building Boundary Regularization［C］∥ CVPR Workshops. 2018： 247-251.
13	Wen Q， Jiang K， Wang W， et al. Automatic Building Extraction from Google Earth Images Under Complex Backgrounds based on Deep Instance Segmentation Network［J］. Sensors， 2019， 19（2）： 333. doi：. doi: 10.3390/s19020333
14	Zhang C， Li Z， Liu Q， et al. Toward Efficient Simultaneous Detection and Segmentation［C］∥ 2018 IEEE Fourth International Conference on Multimedia Big Data （BigMM）， IEEE， 2018： 1-5.
15	Tian Jin， Zhang Chi， Wang Yongsen， et al. Ground Mark Detection based on Mask R-CNN［C］∥ The 22nd Annual Meeting of Network New Technologies and Applications of China Computer Users Association Network Application Branch2018.
15	田锦，张弛，王永森，等. 基于 Mask R-CNN 的地面标识检测［C］∥ 中国计算机用户协会网络应用分会 2018 年第二十二届网络新技术与应用年会论文集， 2018.
16	Bai Baolin. Vehicle Identification and Detection based on Improved Mask R-CNN［D］. Anhui： Anhui University， 2018.
16	白宝林. 基于改进的 Mask R-CNN 的车辆识别及检测［D］. 安徽：安徽大学， 2018.
17	Deng Liuyuan， Yang Ming， Wang Chunxiang， et al. Surround View Cameras based Instance Segmentation Method for Autonomous Vehicles［J］. Huazhong University of Science and Technology（Natural Science Edition），2018，46（12）：24-29.邓琉元，杨明，王春香，等. 基于环视相机的无人驾驶汽车实例分割方法［J］. 华中科技大学学报（自然科学版）， 2018 .46（12）：24-29.
18	Chen Mingyang， Zhao Zhigang， Pan Zhikuan， et al. Image Instance Segmentation based on Deep Learning［J］. Journal of Qingdao University （Natural Science Edition）， 2019，32（1）： 46-50， 54.
18	陈茗杨，赵志刚，潘振宽，等. 基于深度学习的图像实例分割［J］. 青岛大学学报：自然科学版， 2019，32（1）： 46-50， 54.
19	Zhang Xudong， Wang Yuting， Fan Zhiguo， et al. RGB-D Multi-class Instance Segmentation based on Double Pyramid Feature Fusion Model［J］. Control and Decision，2020，35（7）：1561-1568.
19	张旭东，王玉婷，范之国，等. 基于双金字塔特征融合网络的RGB-D多类实例分割［J］.控制与决策，2020，35（7）：1561-1568.
20	Lin T Y， Dollár P， Girshick R， et al. Feature Pyramid Networks for Object Detection［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017： 2117-2125.
21	Liu Y， Yao J， Lu X， et al. DeepCrack： A Deep Hierarchical Feature Learning Architecture for Crack Segmentation［J］. Neurocomputing， 2019， 338： 139-153.

No Suggested Reading articles found!

Viewed

Full text

Abstract

Cited

Shared

Discussed