Please wait a minute...
img

官方微信

遥感技术与应用  2021, Vol. 36 Issue (2): 247-255    DOI: 10.11873/j.issn.1004-0323.2021.2.0247
CNN 专栏     
基于卷积神经网络预测结果的缝隙修复算法研究
刘钊(),赵桐(),廖斐凡
清华大学 土木工程系交通与地球空间信息研究所,北京 100084
Research on Gap-repairing Algorithm based on Convolutional Neural Network Prediction Result
Zhao Liu(),Tong Zhao(),Feifan Liao
Institute of Traffic Engineering and Geospatial Information,Department of Civil Engineering,Tsinghua University,Beijing 100084,China
 全文: PDF(6143 KB)   HTML
摘要:

随着卷积神经网络技术的发展,近来的研究越来越注重于准确率的提升以及语义信息的完善。其中Mask R-CNN网络是对Faster R-CNN进一步改进后的实例分割网络,在高分遥感图像地物识别具有良好的分割效果。但由于卷积神经网络只能用小瓦片图像进行训练和预测,而导致预测结果存在较大的语义信息误差。面对这种问题,提出了针对卷积神经网络预测结果缺陷的缝隙修复算法,即先使用Overlapsize算法改善预测结果与真实结果的匹配程度,再通过PostGIS数据库中的相关函数填补缝隙,使小瓦片能真正拼接成完整大图。研究及实验结果表明:该算法能够很好地改善图像语义信息,具有实用性。

关键词: 卷积神经网络实例分割Mask R-CNN缝隙修复算法    
Abstract:

With the development of convolutional neural network technology, recent research has paid more attention to the improvement of accuracy and the improvement of semantic information. Mask R-CNN network is a further improved segmentation network of Faster R-CNN. It has a good segmentation effect in high-resolution remote sensing image feature recognition. However, since the convolutional neural network can only be trained and predicted with small tile images, there is a large semantic information error in the prediction results. Faced with this problem, this paper proposed a gap-repairing algorithm based on the defect of prediction result of convolutional neural network. The approach use overlapsize algorithm to improve the matching degree between the prediction result and the ground-truth result at first. Then fill the gap through the correlation function in the PostGIS database to repair the small tile, which can make it be spliced ??into a complete picture. The research and experiment results showed that the algorithm could improve the image semantic information well and has practicability.

Key words: Convolutional neural network    Instance segmentation    Mask R-CNN    Gap-repairing algorithm
收稿日期: 2020-06-20 出版日期: 2021-05-24
ZTFLH:  P237  
通讯作者: 赵桐     E-mail: liuz@tsinghua.edu.cn;zhaot18@mails.tsinghua.edu.cn
作者简介: 刘钊(1967-),男,江苏镇江人,副教授,主要从事GIS及其应用、云GIS、时空大数据及遥感图像处理等方面的研究。E?mail:liuz@tsinghua.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
刘钊
赵桐
廖斐凡

引用本文:

刘钊,赵桐,廖斐凡. 基于卷积神经网络预测结果的缝隙修复算法研究[J]. 遥感技术与应用, 2021, 36(2): 247-255.

Zhao Liu,Tong Zhao,Feifan Liao. Research on Gap-repairing Algorithm based on Convolutional Neural Network Prediction Result. Remote Sensing Technology and Application, 2021, 36(2): 247-255.

链接本文:

http://www.rsta.ac.cn/CN/10.11873/j.issn.1004-0323.2021.2.0247        http://www.rsta.ac.cn/CN/Y2021/V36/I2/247

图1  Mask R-CNN网络框架图
图2  Mask R-CNN网络预测流程图
图3  Overlapsize算法原理图
图4  缝隙修复算法流程图
图5  农田实例分割数据集示例
图6  总损失值变化曲线
图7  分类损失值变化曲线
图8  边界框损失值变化曲线
图9  掩膜损失值变化曲线
精度APAP50AP75
Mask R-CNN/%33.655.235.3
本实验/%34.055.535.4
表1  网络训练精度
图10  Overlapsize算法实验对比图
图11  缝隙修复算法对比图
1 Krizhevsky A,Sutskever I,Hinton G E.ImageNet Classification with Deep Convolutional Neural Networks[C]∥ Advances in Neural Information Processing Systems,2012:1097-1105.
2 Girshick R, Donahue J, Darrell T, et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 580-587.
3 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C]∥ International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, Cham,2015:234-241.
4 Badrinarayanan V,Kendall A,Cipolla R.Segnet: A Deep Convolutional Encoder-decoder Architecture for Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
5 He K, Zhang X, Ren S, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
6 Girshick R. Fast r-cnn[C]∥ Proceedings of the IEEE International Conference on Computer Vision,2015:1440-1448.
7 Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks[C]∥ Advances in Neural Information Processing Systems, 2015: 91-99.
8 He K, Gkioxari G, Dollár P, et al. Mask R-CNN[C]∥ Proceedings of the IEEE International Conference on Computer Vision, 2017: 2961-2969.
9 Lin T Y, Dollár P, Girshick R, et al. Feature Pyramid Networks for Object Detection[C]∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017: 2117-2125.
10 Nie S, Jiang Z, Zhang H, et al. Inshore Ship Detection based on Mask R-CNN[C]∥ IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 2018: 693-696.
11 Zhang Y, You Y, Wang R, et al. Nearshore Vessel Detection based on Scene-mask R-CNN in Remote Sensing Image[C]∥ 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC). IEEE, 2018: 76-80.
12 Zhao K, Kang J, Jung J, et al. Building Extraction from Satellite Images Using Mask R-CNN with Building Boundary Regularization[C]∥ CVPR Workshops. 2018: 247-251.
13 Wen Q, Jiang K, Wang W, et al. Automatic Building Extraction from Google Earth Images Under Complex Backgrounds based on Deep Instance Segmentation Network[J]. Sensors, 2019, 19(2): 333. doi:10.3390/s19020333.
doi: 10.3390/s19020333
14 Zhang C, Li Z, Liu Q, et al. Toward Efficient Simultaneous Detection and Segmentation[C]∥ 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), IEEE, 2018: 1-5.
15 Tian Jin, Zhang Chi, Wang Yongsen, et al. Ground Mark Detection based on Mask R-CNN[C]∥ The 22nd Annual Meeting of Network New Technologies and Applications of China Computer Users Association Network Application Branch2018.
15 田锦, 张弛, 王永森, 等. 基于 Mask R-CNN 的地面标识检测[C]∥ 中国计算机用户协会网络应用分会 2018 年第二十二届网络新技术与应用年会论文集, 2018.
16 Bai Baolin. Vehicle Identification and Detection based on Improved Mask R-CNN[D]. Anhui: Anhui University, 2018.
16 白宝林. 基于改进的 Mask R-CNN 的车辆识别及检测[D]. 安徽:安徽大学, 2018.
17 Deng Liuyuan, Yang Ming, Wang Chunxiang, et al. Surround View Cameras based Instance Segmentation Method for Autonomous Vehicles[J]. Huazhong University of Science and Technology(Natural Science Edition),2018,46(12):24-29.邓琉元, 杨明, 王春香, 等. 基于环视相机的无人驾驶汽车实例分割方法[J]. 华中科技大学学报 (自然科学版), 2018 .46(12):24-29.
18 Chen Mingyang, Zhao Zhigang, Pan Zhikuan, et al. Image Instance Segmentation based on Deep Learning[J]. Journal of Qingdao University (Natural Science Edition), 2019,32(1): 46-50, 54.
18 陈茗杨, 赵志刚, 潘振宽, 等. 基于深度学习的图像实例分割[J]. 青岛大学学报:自然科学版, 2019,32(1): 46-50, 54.
19 Zhang Xudong, Wang Yuting, Fan Zhiguo, et al. RGB-D Multi-class Instance Segmentation based on Double Pyramid Feature Fusion Model[J]. Control and Decision,2020,35(7):1561-1568.
19 张旭东, 王玉婷, 范之国,等. 基于双金字塔特征融合网络的RGB-D多类实例分割[J].控制与决策,2020,35(7):1561-1568.
20 Lin T Y, Dollár P, Girshick R, et al. Feature Pyramid Networks for Object Detection[C]∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 2117-2125.
21 Liu Y, Yao J, Lu X, et al. DeepCrack: A Deep Hierarchical Feature Learning Architecture for Crack Segmentation[J]. Neurocomputing, 2019, 338: 139-153.
[1] 张峰极,吴艳兰,姚雪东,梁泽毓. 基于改进DenseNet网络的多源遥感影像露天开采区智能提取方法[J]. 遥感技术与应用, 2020, 35(3): 673-684.
[2] 郑宗生,胡晨雨,黄冬梅,邹国良,刘兆荣,宋巍. 基于迁移学习及气象卫星云图的台风等级分类研究[J]. 遥感技术与应用, 2020, 35(1): 202-210.
[3] 崔刚,吴金胜,于镇,周玲. 深度学习对不同分辨率影像冬小麦识别的适用性研究[J]. 遥感技术与应用, 2019, 34(4): 748-755.
[4] 王颖洁,张荞,张艳梅,蒙印,郭文. 基于深度卷积神经网络的油罐目标检测研究[J]. 遥感技术与应用, 2019, 34(4): 727-735.
[5] 李淑敏, 冯权泷, 梁其椿, 张学庆. 基于深度学习的国产高分遥感影像飞机目标自动检测[J]. 遥感技术与应用, 2018, 33(6): 1095-1102.
[6] 田德宇,张耀南,赵国辉,韩立钦. 基于卷积神经网络的遥感沙漠绿地提取方法[J]. 遥感技术与应用, 2018, 33(1): 151-157.
[7] 何海清,庞燕,陈晓勇. 面向遥感影像场景的深度卷积神经网络递归识别模型[J]. 遥感技术与应用, 2017, 32(6): 1078-1082.