Please wait a minute...
img

官方微信

遥感技术与应用  2022, Vol. 37 Issue (6): 1328-1338    DOI: 10.11873/j.issn.1004-0323.2022.6.1328
冰雪遥感专栏     
基于分形维数和角二阶矩辅助的卷积神经网络云雪识别研究
朱博(),钟方洁,赵军锁
中国科学院软件研究所 天基综合信息系统重点实验室,北京 100190
Fractal Dimension and Angular Second Moment-assisted Convolutional Neural Network for Cloud and Snow Recognition
Bo Zhu(),Fangjie Zhong,Junsuo Zhao
Science & Technology on Integrated Information System Laboratory,Institute of Software,Chinese Academy of Sciences,Beijing 100190,China
 全文: PDF(9604 KB)   HTML
摘要:

对于天基对地观测成像任务,遥感影像中的云量往往决定了数据是否可用。然而云雪光谱特征近似,使得二者较难区分。云雪识别研究的目的是提高数据有效性判断的能力,具有实际应用价值,是遥感数据应用中的重要环节。研究提出一种提高局部特征信息提取能力的小块结构并构建了轻量化卷积神经网络模型作为骨干网络用于区分类云(云、雪、亮地物)与其他地物,通过分形维数与角二阶矩分析云、雪、地物的纹理及灰度特征形成二叉树辅助网络对类云进行精细化识别,网络权重层只有6层(4个卷积层,两个全连接层)。通过对天智1号、SPOT4/5/6、Pleiades等不同几何分辨率的数据进行训练与分析,并与随机森林、SVM、传统方法等进行对比,在云、雪、云雪共存等场景下,该方法能够较好地识别云、雪、(亮)地物,识别准确率达89%。方法适用于全色、多光谱、高光谱等遥感数据云雪识别,同时结构简洁、参数量少。

关键词: 光学遥感云雪识别分形维数角二阶矩卷积神经网络二叉树辅助网络    
Abstract:

For space-based earth observation imaging missions, the cloud cover contained in an remote sensing image often determines whether the data is available. However, the spectral characteristics of cloud and snow are similar, which makes it difficult to distinguish them. The purpose of the research on cloud and snow recognition is to improve the ability to judge the validity data. So an novel method is designed to solve it. Firstly, an improved lightweight convolutional neural network, which is built based on the proposed DCP (Double Convolution Parallel) structure, is used as the backbone to classify the quasi-cloud (cloud, snow and highly reflective ground objects) and other ground objects. Secondly, the textures and gray features of cloud and snow and ground objects are analyzed by a binary tree network formed by fractal dimension and angular second moment for fine recognition in further. The network weight layers are only six (four convolutional layers and two full connection layers). The proposed method is trained on the data sets containing cloud, snow and cloud-snow with different ground sample resolutions from Tianzhi-1 and SPOT4/5/6 and Pleiades. When compared on the accuracy with reference methods, such as random forest, SVM, traditional methods and binary tree methods, our method provide an increasing accuracy to 89.08%. The experimental results shows: (1) The comparative experiment between network structures shows that the DCP could effectively improve the model ability of feature information extraction and promote faster convergence; (2) Texture features analysis of remote sensing images makes the recognition process not completely dependent on convolutional neural network, so as to reduce the network depth and weight parameters; (3) The combination of traditional remote sensing analysis method and neural network is better than one of them alone, which can improve the accuracy of cloud and snow recognition. The proposed method is suitable for cloud and snow recognition on panchromatic, multispectral and hyperspectral remote sensing imagery.

Key words: Optical remote sensing    Cloud and snow recognition    Fractal dimension    Angular second moment    Convolutional neural network    Binary tree
收稿日期: 2021-11-05 出版日期: 2023-02-15
ZTFLH:  TP79  
基金资助: 国家自然科学基金项目(62027801)
作者简介: 朱 博(1983-),男,江苏徐州人,硕士,工程师,主要从事遥感图像智能感知、遥感图像质量评价研究。E?mail:zhubo@iscas.ac.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
朱博
钟方洁
赵军锁

引用本文:

朱博,钟方洁,赵军锁. 基于分形维数和角二阶矩辅助的卷积神经网络云雪识别研究[J]. 遥感技术与应用, 2022, 37(6): 1328-1338.

Bo Zhu,Fangjie Zhong,Junsuo Zhao. Fractal Dimension and Angular Second Moment-assisted Convolutional Neural Network for Cloud and Snow Recognition. Remote Sensing Technology and Application, 2022, 37(6): 1328-1338.

链接本文:

http://www.rsta.ac.cn/CN/10.11873/j.issn.1004-0323.2022.6.1328        http://www.rsta.ac.cn/CN/Y2022/V37/I6/1328

图1  数据集中部分云、雪、其他下垫面图

数据集

名称

载荷

大小

/像素

数据量

(幅/景)

分辨率

/m

天智-1全色2 560×2 048606
SPOT4全色/多光谱2 000×2 0483010/20
SPOT5全色/多光谱2 000×2 000202.5/10
SPOT6全色/多光谱2 500×2 500301.5/6
Pleiades多光谱3 500×2 500202
表1  实验数据信息
图2  基于分形维数和角二阶矩的卷积神经网络模型
函数大小通道数量
Conv13,3116
Conv23,3132
Conv33,3132
Conv43,3164
Max Pooling 12,2/p:1/s:1
Max Pooling 22,2/p:1/s:1
FC 164,16,1650
FC 2502
Learning rate0.01
Activation FunctionReLU
Initialized weightHe
Parameters updatingSGD[36]
Dropout1 ratio0.5
Dropout2 ratio0.5
表2  骨干网络参数
图3  基于分形维数和角二阶矩的卷积神经网络模型原始网络与改进网络训练与测试对比
图4  改进前后的神经网络对验证数据识别结果比较
图5  云、雪、其他下垫面分形维数
图6  云、雪、地物分形维数比较
图7  云、雪、地物角二阶矩图
图8  云、雪、地物角二阶矩比较
图9  二叉树辅助网络判读结果示意图
图10  5种算法的云、雪、地物识别效果
图11  5种算法的云、雪、地物识别结果比较
DingBTRFSVMOurs
天智1号80.679.476.681.790.6
SPOT486.384.585.983.689.7
SPOT582.578.675.479.889.2
SPOT677.275.573.278.487.3
Pleiades82.481.380.38088.6
平均准确率80.1484.379.376.8689
表3  识别结果统计
图12  云、雪、地物识别结果
1 Hou Shuwei, Sun Wenfang, Zheng Xiaosong. Overview of cloud detection methods in remote sensing images[J]. Space Electronic Technology, 2014, 11(3): 68-77.
1 侯舒维, 孙文方,郑小松.遥感图像云检测方法综述[J].空间电子技术, 2014, 11(3): 68-77.
2 Liu P. Review on the technology of threshold selection in image segmentation[D]. Beijing:Institute of Computing Technology, Chinese Academy of Science, 2004.
3 Rossow W B, Schiffer R A. ISCCP cloud data products[J]. Bulletin of the American Meteorological Society,1991,27(1): 855-863.
4 Saunders R W, Kriebel K T. An improved method of detecting clear sky and cloudy radiances from AVHRR data[J]. International Journal of Remote Sensing, 1988,9(1):123-150.DOI:10.1080101431168808954841 .
doi: 10.1080101431168808954841
5 Stowe L L, Davis P A, McClain P E. Scientific basis and initial evaluation of the CLAVR21 global clear/cloud classification algorithm for advanced very high resolution radiometer[J]. Atmos and Oceanic Technology, 1999,16(6):656-681.DOI:10.1175/15200426(2001)018<0169:APLCSC>2.0.CO;2 .
doi: 10.1175/15200426(2001)018<0169:APLCSC>2.0.CO;2
6 Ding Baohui, Li Wenchao, Wang Fuming. Analysis of fractal image and design of fractal dimension calculation program[J]. Journal of University of Science and Technology Beijing, 1999, 21(3): 304-307.
6 丁宝华,李文超,王福明.分形图像分析与分形维数计算程序的设计[J],北京科技大学学报,1999,21(3):304-307.
7 Wang Quan, Sun Lin, Wei Jing, et al. Improvement of universal dynamic threshold cloud detection algorithm and its application in high resolution satellite[J]. Acta Optica Sinica, 2018, 38(10): 1028002-1-1028002-10.王权,孙林,韦晶,等.动态阈值云检测算法改进及在高分辨率卫星上的应用[J].光学学报,2018,38(10):1028002-1
doi: 10.3788/AOS201838.1028002
7 -1028002-10.DOI:10.3788/AOS201838.1028002 .
doi: 10.3788/AOS201838.1028002
8 Xiang Peisu. Cloud detection algorithm for MODIS images based on K-means clustering and Otsu method[J]. Geospatial Information,2020,18(4):31-34.
8 向培素.Kmeans与Otsu相结合的MODIS云检测算法[J].地理空间信息,2020, 18(4):31-34.
9 Wang Shuhua, Zhao Yuming, Zhou Xiaosi, et al. Automatic classification of cloud and mist based on weighted fractal dimension[J].Infrared and Laser Engineering,2020,31(1):18-22.
9 王淑华,赵宇明,周小四,等.基于灰度连通域加权分数维的云雾自动分离算法[J].红外与激光工程,2020,31(1):18-22.
10 Liu Q, Hu H, Sha D X, et al. Hyperspectral infrared sounder cloud detection using deep neural network model[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 5500705. DOI: 10.1109/LGRS.2020.3023683 .
doi: 10.1109/LGRS.2020.3023683
11 Wang Yongji, Ming Yanfang, Liang Tianchen, et al. Research on GF-6 WFV data cloud detection based on improved LCCD algorithm[J]. Acta Optica Sinica, 2020, 40(16): 1628001.王永吉,明艳芳,梁天辰,等.基于改进的LCCD算法的GF-6 WFV数据云检测研究[J].光学学报, 2020, 40(16): 1628001.DOI:10.3788/ AOS202040. 1628 001 .
doi: 10.3788/ AOS202040. 1628 001
12 Hu Xueqian, Sun Lin, Sui Songman, et al .Landsat-8 OLI cloud and snow recognition based on improved CDAG cloud detection algorithm[J].Radio Engineering, 2021, 51(12):1461-1470.
12 胡学谦,孙林,隋淞蔓,等.基于改进 CDAG云检测算法的Landsat-8 OLI云雪识别[J].无线电工程,2021,51(12):1461-1470.
13 Liu Jun, Zheng Qunfeng, Xing Xiuwei, et al. Automatic and accurate cloud detection for Fengyun meteorological satellite[J]. Bulletin of Surveying and Mapping,2019(12): 45-49.
doi: 10.13474/j.CNki.11-2246.2019.0384
13 刘军,郑群峰,刑秀为,等.风云气象卫星影像自动精细云检测[J].测绘通报,2019(12):45-49. DOI:10.13474/j.CNki.11-2246.2019.0384 .
doi: 10.13474/j.CNki.11-2246.2019.0384
14 Zhong Jing, Kang Yifei, Han Yu, et al. Automatic cloud detection of GF-1 satellite imagery[J]. Geospatial Information, 2018, 16(4): 52-56.
14 钟静,康一飞,韩昱,等.高分一号卫星影像自动云检测[J]. 地理空间信息, 2018, 16 (4): 52-56.
15 Kittler J, Pairman D. Contextual pattern recognition applied to cloud detection and identification[J]. IEEE Transaction on Geoscience and Remote Sensing, 1985, 23(6): 855-863. DOI:10.1109/TGRS.1985.289471 .
doi: 10.1109/TGRS.1985.289471
16 Mandelbrotb B. The fractal geometry of nature[M]. New York: W H Freeman and Co, 1982.
17 Christodoulou C I, Michaelides S C, Pattichis C S. Multifeature texture analysis for the classification of clouds in satellite imagery[J]. IEEE Transaction on Geoscience and Remote Sensing, 2003, 41(11): 2662-2668. DOI:10.1109/TGRS.2003.815404 .
doi: 10.1109/TGRS.2003.815404
18 Cao Qiong, Zheng Hong, Li Xingshan. A method for detecting cloud in satellite remote sensing image based on texture[J]. Acta Aeronautica et Astronautica Sinica, 2007, 28(3): 661-666.
18 曹琼,郑红,李行善.一种基于纹理特征的卫星遥感图像云探测方法[J].航空学报,2007,28(3): 661-666.
19 Li Bingchun. Texture feature retrieval and applications using gray Co-occurrence matrices[J]. Journal of Kashgar Teachers College,2006,27(6):35-37.
19 李丙春.基于共生矩阵的图像纹理特征提取及应用[J].喀什师范学院学报,2006,27(6):35-37.
20 Zhou Quan. The Classification of remote sensing cloud images based on multiscale features[D]. Hefei: University of Science and Technology of China, 2010.
20 周全. 基于多尺度特征的遥感云图分类[D].合肥:中国科技大学, 2010
21 Ding Haiyan. A study on automatic cloud detection for intelligent satellite[D]. Beijing: Graduate University of the Chinese Academy of Sciences, 2011.
21 丁海燕.智能小卫星自动云判方法研究[D].北京:中国科学院研究生院,2011.
22 Weng Liguo, Liu Wanan, Shi Bicheng, et al. Cloud/Snow classification based on multi-dimensional multi-grained cascade forest in plateau region[J]. Journal of Computer Applications, 2018, 38(8): 2218-2223.
22 翁理国,刘万安,施必成,等.基于多维多粒度级联森林的高原地区云雪分类[J].计算机应用, 2018, 38(8): 2218-2223.
23 Stowe L L. Global distribution of cloud cover derived from NOAA/AVHRR operational satellite data[J]. Advances in Space Research, 1991, 11(3): 51-54.
24 Simpson J J, Gobat J J. Improved cloud detection in GOES scenes over the oceans[J]. Remote Sensing of Environment, 1995, 52(2): 79-94.
25 Gao Jun, Jing Yiguo. A fully convoluted neural network-based cloud detection method for satellite remote sensing images[J]. Infrared Technology, 2019, 41(7): 607-615.
25 高军,荆益国.基于全卷积神经网络的卫星遥感图像云检测方法[J].红外技术,2019,41(7): 607-615.
26 Qu Jianhua, Yan Junjie, Xue Juan, et al. Research on the cloud detection model of FY3D/MESI and EOS/MODIS based on deep learning[J]. Journal of Meteorology and Environment, 2019, 35(3): 87-93.
26 瞿建华,鄢俊洁,薛娟,等. 基于深度学习的FY3D/MERSI和EOS/MODIS云检测模型研究[J].气象与环境学报,2019,35(3):87-93.
27 Cao Hui, Weng Liguo, Zhang Dezheng. Cloud and snow image detection in Qinghai-Tibet Area based on dense dilated convolution neural network[J]. Computer Measurement & Control, 2019, 27(9): 169-173.
27 曹辉,翁理国,张德正.基于密集连接空洞卷积神经网络的青藏地区云雪图像分类[J].计算机测量与控制, 2019, 29(9): 169-173.
28 Qiao H W, Zhang P, Li Z, et al. A new geostationary satellite-based snow cover recognition method for FY-4Y AGRI[J]. IEEE J-STARS,2021,14:11372-11385. DOI:10.1109/JSTARTS.2021.3125015 .
doi: 10.1109/JSTARTS.2021.3125015
29 Xu M, Deng F R, Jia S, et al. Attention mechanism-based generative adversarial networks for cloud removal in Landsat images[J].Remote Sensing of Environment,2022,271:112902. DOI:10.1016/j.rse.2022.112902 .
doi: 10.1016/j.rse.2022.112902
30 Xiang M X, Li Y X, Fei G Q, et al. Development of cloud recognition system for ground-based cloud images based on machine vision[C]∥ Proceedings of the 5th Advanced Information Technology, Electronic and Automation Control Conference(IAEAC),2021:98-101. DOI:10.1109/IAEAC 50856.2021.9391069 .
doi: 10.1109/IAEAC 50856.2021.9391069
31 Sergey I, Christian S. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]∥ Proceedings of the 32nd International Conference on Machine Learning (ICML), 2015: 448-456.
32 Nair V, Hinton G E. Rectified linear units improve restricted Boltzmann machine[C]∥ Proceedings of the 27th International Conference on Machine Learning(ICML),2010,47:16-29.
33 Huang G, Liu Z, Weinberger K Q, et al. Densely connected convolutional networks[C]∥ Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2017: 2261-2269.
34 Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research,2014,15(1):1929-1958. DOI:10.5555/2627435.2670313 .
doi: 10.5555/2627435.2670313
35 Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space[C]∥ Proceedings of the 1st International Conference on Learning Representations (ICLR), 2013: 1-12.
36 Hensman J, Fusi N, Lawrence N D. Gaussian processes for big data[C]∥ Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence (UAI), 2013:282-290.
37 Zhang Zhi, Dong Fuan, Wu Youli. Estimation of fractal dimension for 2-D gray image[J]. Computer Applications, 2005, 25(12): 2853-2855.
37 张志,董福安,伍友利.二维灰度图像的分形维数计算[J].计算机应用,2005,25(12):2853-2855.
38 Shan Na, Zheng Tianyao, Wang Zhensong. High-speed and high-accuracy algorithm for cloud detection and its application[J]. Journal of Remote Sensing, 2009, 13(6): 1138-1155.
38 单娜,郑天垚,王贞松.快速高准确度云检测算法及其应用[J].遥感学报,2009,13(6):1138-1155.
39 Chen Qiang, Tian Jie, Huang Haining. Study on SAS image segmentation using SVM based on statistical and texture features[J]. Chinese Journal of Scientific Instrument, 2013, 34(6): 1413-1420.
39 陈强,田杰,黄海宁,等.基于统计和纹理特征的SAS图像SVM分割研究[J].仪器仪表学报, 2013, 34 (6): 1413-1420.
40 Han Dongchen, Shang Yaceng, Lai Yueshen. Illumination-Invariant image for shadow removal method[J]. Journal of Xi’an Technological University,2021,41(6):691-695.
40 韩东辰,尚雅层,来跃深.一种光照无关图阴影去除方法[J].西安工业大学学报, 2021, 41 (6): 691-695.
41 Hong D F, Gao L R, Yokoya N,et al. More diverse means better: Multimodal deep learning meets remote-sensing imagery classification[J].IEEE Transactions on Geoscience and Remote,2021,59(5):4340-4354.DOI:10.1109/TGRS.2020.3016820 .
doi: 10.1109/TGRS.2020.3016820
[1] 付涵,范湘涛,严珍珍,杜小平. 基于深度学习的遥感图像目标检测技术研究进展[J]. 遥感技术与应用, 2022, 37(2): 290-305.
[2] 吕冬梅,马玥,李华朋. 基于CNN的吉林一号卫星城市土地覆被制图潜力评估[J]. 遥感技术与应用, 2022, 37(2): 368-378.
[3] 郑磊,何直蒙,丁海勇. 基于ENVINet5的高分辨率遥感影像稀疏塑料大棚提取研究[J]. 遥感技术与应用, 2021, 36(4): 908-915.
[4] 刘钊,赵桐,廖斐凡. 基于卷积神经网络预测结果的缝隙修复算法研究[J]. 遥感技术与应用, 2021, 36(2): 247-255.
[5] 陈敏,潘佳威,李江杰,徐璐,刘加敏,韩健,陈奕云. 结合VGGNet与Mask R-CNN的高分辨率遥感影像建设用地检测[J]. 遥感技术与应用, 2021, 36(2): 256-264.
[6] 胡乃勋,陈涛,甄娜,牛瑞卿. 基于卷积神经网络的面向对象露天采场提取[J]. 遥感技术与应用, 2021, 36(2): 265-274.
[7] 陈妮,应丰,王静,李健. 基于U-Net的高分辨率遥感图像土地利用信息提取[J]. 遥感技术与应用, 2021, 36(2): 285-292.
[8] 屈炀,袁占良,赵文智,陈学泓,陈家阁. 基于多时序特征和卷积神经网络的农作物分类[J]. 遥感技术与应用, 2021, 36(2): 304-313.
[9] 张峰极,吴艳兰,姚雪东,梁泽毓. 基于改进DenseNet网络的多源遥感影像露天开采区智能提取方法[J]. 遥感技术与应用, 2020, 35(3): 673-684.
[10] 唐廷元,付波霖,何素云,娄佩卿,闭璐. 基于GF-1和Sentinel-1A的漓江流域典型地物信息提取[J]. 遥感技术与应用, 2020, 35(2): 448-457.
[11] 郑宗生,胡晨雨,黄冬梅,邹国良,刘兆荣,宋巍. 基于迁移学习及气象卫星云图的台风等级分类研究[J]. 遥感技术与应用, 2020, 35(1): 202-210.
[12] 王颖洁,张荞,张艳梅,蒙印,郭文. 基于深度卷积神经网络的油罐目标检测研究[J]. 遥感技术与应用, 2019, 34(4): 727-735.
[13] 崔刚,吴金胜,于镇,周玲. 深度学习对不同分辨率影像冬小麦识别的适用性研究[J]. 遥感技术与应用, 2019, 34(4): 748-755.
[14] 李淑敏, 冯权泷, 梁其椿, 张学庆. 基于深度学习的国产高分遥感影像飞机目标自动检测[J]. 遥感技术与应用, 2018, 33(6): 1095-1102.
[15] 廖凯涛,齐述华,王成,王点. 结合GLAS和TM卫星数据的江西省森林高度和生物量制图[J]. 遥感技术与应用, 2018, 33(4): 713-720.