Fractal Dimension and Angular Second Moment-assisted Convolutional Neural Network for Cloud and Snow Recognition

doi:10.11873/j.issn.1004-0323.2022.6.1328

Remote Sensing Technology and Application

2022, Vol. 37

Issue (6): 1328-1338 DOI: 10.11873/j.issn.1004-0323.2022.6.1328

Fractal Dimension and Angular Second Moment-assisted Convolutional Neural Network for Cloud and Snow Recognition

Bo Zhu(

),Fangjie Zhong,Junsuo Zhao

Science & Technology on Integrated Information System Laboratory，Institute of Software，Chinese Academy of Sciences，Beijing 100190，China

Download:

HTML

PDF (9604KB)
Export: BibTeX | EndNote (RIS)

Abstract

For space-based earth observation imaging missions， the cloud cover contained in an remote sensing image often determines whether the data is available. However， the spectral characteristics of cloud and snow are similar， which makes it difficult to distinguish them. The purpose of the research on cloud and snow recognition is to improve the ability to judge the validity data. So an novel method is designed to solve it. Firstly， an improved lightweight convolutional neural network， which is built based on the proposed DCP （Double Convolution Parallel） structure， is used as the backbone to classify the quasi-cloud （cloud， snow and highly reflective ground objects） and other ground objects. Secondly， the textures and gray features of cloud and snow and ground objects are analyzed by a binary tree network formed by fractal dimension and angular second moment for fine recognition in further. The network weight layers are only six （four convolutional layers and two full connection layers）. The proposed method is trained on the data sets containing cloud， snow and cloud-snow with different ground sample resolutions from Tianzhi-1 and SPOT4/5/6 and Pleiades. When compared on the accuracy with reference methods， such as random forest， SVM， traditional methods and binary tree methods， our method provide an increasing accuracy to 89.08%. The experimental results shows：（1） The comparative experiment between network structures shows that the DCP could effectively improve the model ability of feature information extraction and promote faster convergence；（2） Texture features analysis of remote sensing images makes the recognition process not completely dependent on convolutional neural network， so as to reduce the network depth and weight parameters；（3） The combination of traditional remote sensing analysis method and neural network is better than one of them alone， which can improve the accuracy of cloud and snow recognition. The proposed method is suitable for cloud and snow recognition on panchromatic， multispectral and hyperspectral remote sensing imagery.

Key words: Optical remote sensing Cloud and snow recognition Fractal dimension Angular second moment Convolutional neural network Binary tree

Received: 05 November 2021 Published: 15 February 2023

ZTFLH:

TP79

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Bo Zhu
	Fangjie Zhong
	Junsuo Zhao

Cite this article:

Bo Zhu,Fangjie Zhong,Junsuo Zhao. Fractal Dimension and Angular Second Moment-assisted Convolutional Neural Network for Cloud and Snow Recognition. Remote Sensing Technology and Application, 2022, 37(6): 1328-1338.

URL:

http://www.rsta.ac.cn/EN/10.11873/j.issn.1004-0323.2022.6.1328 OR http://www.rsta.ac.cn/EN/Y2022/V37/I6/1328

Fig.1 Parts of samples of cloud， snow and underlying surfaces in the different datasets

Table 1 Information of data sets

Fig.2 Fractal dimension and angular second moment-assisted convolutional neural network

Table 2 Backbone parameters

Fig.3 Comparison of training and testing accuracy between the plain and improved convolutional neural network

Fig.4 Comparison between the plain and improved CNN on validation dataset

Fig.5 Fractal dimension values of cloud， snow and underlying surfaces

Fig.6 Comparison of fractal dimension between cloud， snow and underlying surfaces

Fig.7 ASMs of cloud， snow and underlying surfaces

Fig.8 Comparison of ASMs between cloud， snow and underlying surfaces

Fig.9 Recognition of some cloud， snow and underlying surfaces by binary tree network

Fig.10 Effect of the five methods on different data

Fig.11 Comparison of the five methods

Table 3 Statistics of recognition results

Fig.12 Recognition on cloud， snow and underlying surfaces

1	Hou Shuwei， Sun Wenfang， Zheng Xiaosong. Overview of cloud detection methods in remote sensing images［J］. Space Electronic Technology， 2014， 11（3）： 68-77.
1	侯舒维，孙文方，郑小松.遥感图像云检测方法综述［J］.空间电子技术， 2014， 11（3）： 68-77.
2	Liu P. Review on the technology of threshold selection in image segmentation［D］. Beijing：Institute of Computing Technology， Chinese Academy of Science， 2004.
3	Rossow W B， Schiffer R A. ISCCP cloud data products［J］. Bulletin of the American Meteorological Society，1991，27（1）： 855-863.
4	Saunders R W， Kriebel K T. An improved method of detecting clear sky and cloudy radiances from AVHRR data［J］. International Journal of Remote Sensing， 1988，9（1）：123-150.DOI： . doi: 10.1080101431168808954841
5	Stowe L L， Davis P A， McClain P E. Scientific basis and initial evaluation of the CLAVR21 global clear/cloud classification algorithm for advanced very high resolution radiometer［J］. Atmos and Oceanic Technology， 1999，16（6）：656-681.DOI： . doi: 10.1175/15200426（2001）018<0169：APLCSC>2.0.CO；2
6	Ding Baohui， Li Wenchao， Wang Fuming. Analysis of fractal image and design of fractal dimension calculation program［J］. Journal of University of Science and Technology Beijing， 1999， 21（3）： 304-307.
6	丁宝华，李文超，王福明.分形图像分析与分形维数计算程序的设计［J］，北京科技大学学报，1999，21（3）：304-307.
7	Wang Quan， Sun Lin， Wei Jing， et al. Improvement of universal dynamic threshold cloud detection algorithm and its application in high resolution satellite［J］. Acta Optica Sinica， 2018， 38（10）： 1028002-1-1028002-10.王权，孙林，韦晶，等.动态阈值云检测算法改进及在高分辨率卫星上的应用［J］.光学学报，2018，38（10）：1028002-1 doi: 10.3788/AOS201838.1028002
7	-1028002-10.DOI： . doi: 10.3788/AOS201838.1028002
8	Xiang Peisu. Cloud detection algorithm for MODIS images based on K-means clustering and Otsu method［J］. Geospatial Information，2020，18（4）：31-34.
8	向培素.Kmeans与Otsu相结合的MODIS云检测算法［J］.地理空间信息，2020， 18（4）：31-34.
9	Wang Shuhua， Zhao Yuming， Zhou Xiaosi， et al. Automatic classification of cloud and mist based on weighted fractal dimension［J］.Infrared and Laser Engineering，2020，31（1）：18-22.
9	王淑华，赵宇明，周小四，等.基于灰度连通域加权分数维的云雾自动分离算法［J］.红外与激光工程，2020，31（1）：18-22.
10	Liu Q， Hu H， Sha D X， et al. Hyperspectral infrared sounder cloud detection using deep neural network model［J］. IEEE Geoscience and Remote Sensing Letters， 2022， 19： 5500705. DOI： . doi: 10.1109/LGRS.2020.3023683
11	Wang Yongji， Ming Yanfang， Liang Tianchen， et al. Research on GF-6 WFV data cloud detection based on improved LCCD algorithm［J］. Acta Optica Sinica， 2020， 40（16）： 1628001.王永吉，明艳芳，梁天辰，等.基于改进的LCCD算法的GF-6 WFV数据云检测研究［J］.光学学报， 2020， 40（16）： 1628001.DOI： . doi: 10.3788/ AOS202040. 1628 001
12	Hu Xueqian， Sun Lin， Sui Songman， et al ．Landsat-8 OLI cloud and snow recognition based on improved CDAG cloud detection algorithm［J］．Radio Engineering， 2021， 51（12）：1461-1470．
12	胡学谦，孙林，隋淞蔓，等.基于改进 CDAG云检测算法的Landsat-8 OLI云雪识别［J］.无线电工程，2021，51（12）：1461-1470.
13	Liu Jun， Zheng Qunfeng， Xing Xiuwei， et al. Automatic and accurate cloud detection for Fengyun meteorological satellite［J］. Bulletin of Surveying and Mapping，2019（12）： 45-49. doi: 10.13474/j.CNki.11-2246.2019.0384
13	刘军，郑群峰，刑秀为，等.风云气象卫星影像自动精细云检测［J］.测绘通报，2019（12）：45-49. DOI： . doi: 10.13474/j.CNki.11-2246.2019.0384
14	Zhong Jing， Kang Yifei， Han Yu， et al. Automatic cloud detection of GF-1 satellite imagery［J］. Geospatial Information， 2018， 16（4）： 52-56.
14	钟静，康一飞，韩昱，等.高分一号卫星影像自动云检测［J］. 地理空间信息， 2018， 16 （4）： 52-56.
15	Kittler J， Pairman D. Contextual pattern recognition applied to cloud detection and identification［J］. IEEE Transaction on Geoscience and Remote Sensing， 1985， 23（6）： 855-863. DOI： . doi: 10.1109/TGRS.1985.289471
16	Mandelbrotb B. The fractal geometry of nature［M］. New York： W H Freeman and Co， 1982.
17	Christodoulou C I， Michaelides S C， Pattichis C S. Multifeature texture analysis for the classification of clouds in satellite imagery［J］. IEEE Transaction on Geoscience and Remote Sensing， 2003， 41（11）： 2662-2668. DOI： . doi: 10.1109/TGRS.2003.815404
18	Cao Qiong， Zheng Hong， Li Xingshan. A method for detecting cloud in satellite remote sensing image based on texture［J］. Acta Aeronautica et Astronautica Sinica， 2007， 28（3）： 661-666.
18	曹琼，郑红，李行善.一种基于纹理特征的卫星遥感图像云探测方法［J］.航空学报，2007，28（3）： 661-666.
19	Li Bingchun. Texture feature retrieval and applications using gray Co-occurrence matrices［J］. Journal of Kashgar Teachers College，2006，27（6）：35-37.
19	李丙春.基于共生矩阵的图像纹理特征提取及应用［J］.喀什师范学院学报，2006，27（6）：35-37.
20	Zhou Quan. The Classification of remote sensing cloud images based on multiscale features［D］. Hefei： University of Science and Technology of China， 2010.
20	周全. 基于多尺度特征的遥感云图分类［D］.合肥：中国科技大学， 2010
21	Ding Haiyan. A study on automatic cloud detection for intelligent satellite［D］. Beijing： Graduate University of the Chinese Academy of Sciences， 2011.
21	丁海燕.智能小卫星自动云判方法研究［D］.北京：中国科学院研究生院，2011.
22	Weng Liguo， Liu Wanan， Shi Bicheng， et al. Cloud/Snow classification based on multi-dimensional multi-grained cascade forest in plateau region［J］. Journal of Computer Applications， 2018， 38（8）： 2218-2223.
22	翁理国，刘万安，施必成，等.基于多维多粒度级联森林的高原地区云雪分类［J］.计算机应用， 2018， 38（8）： 2218-2223.
23	Stowe L L. Global distribution of cloud cover derived from NOAA/AVHRR operational satellite data［J］. Advances in Space Research， 1991， 11（3）： 51-54.
24	Simpson J J， Gobat J J. Improved cloud detection in GOES scenes over the oceans［J］. Remote Sensing of Environment， 1995， 52（2）： 79-94.
25	Gao Jun， Jing Yiguo. A fully convoluted neural network-based cloud detection method for satellite remote sensing images［J］. Infrared Technology， 2019， 41（7）： 607-615.
25	高军，荆益国.基于全卷积神经网络的卫星遥感图像云检测方法［J］.红外技术，2019，41（7）： 607-615.
26	Qu Jianhua， Yan Junjie， Xue Juan， et al. Research on the cloud detection model of FY3D/MESI and EOS/MODIS based on deep learning［J］. Journal of Meteorology and Environment， 2019， 35（3）： 87-93.
26	瞿建华，鄢俊洁，薛娟，等. 基于深度学习的FY3D/MERSI和EOS/MODIS云检测模型研究［J］.气象与环境学报，2019，35（3）：87-93.
27	Cao Hui， Weng Liguo， Zhang Dezheng. Cloud and snow image detection in Qinghai-Tibet Area based on dense dilated convolution neural network［J］. Computer Measurement & Control， 2019， 27（9）： 169-173.
27	曹辉，翁理国，张德正.基于密集连接空洞卷积神经网络的青藏地区云雪图像分类［J］.计算机测量与控制， 2019， 29（9）： 169-173.
28	Qiao H W， Zhang P， Li Z， et al. A new geostationary satellite-based snow cover recognition method for FY-4Y AGRI［J］. IEEE J-STARS，2021，14：11372-11385. DOI： . doi: 10.1109/JSTARTS.2021.3125015
29	Xu M， Deng F R， Jia S， et al. Attention mechanism-based generative adversarial networks for cloud removal in Landsat images［J］.Remote Sensing of Environment，2022，271：112902. DOI： . doi: 10.1016/j.rse.2022.112902
30	Xiang M X， Li Y X， Fei G Q， et al. Development of cloud recognition system for ground-based cloud images based on machine vision［C］∥ Proceedings of the 5th Advanced Information Technology， Electronic and Automation Control Conference（IAEAC），2021：98-101. DOI： . doi: 10.1109/IAEAC 50856.2021.9391069
31	Sergey I， Christian S. Batch normalization： accelerating deep network training by reducing internal covariate shift［C］∥ Proceedings of the 32nd International Conference on Machine Learning （ICML）， 2015： 448-456.
32	Nair V， Hinton G E. Rectified linear units improve restricted Boltzmann machine［C］∥ Proceedings of the 27th International Conference on Machine Learning（ICML），2010，47：16-29.
33	Huang G， Liu Z， Weinberger K Q， et al. Densely connected convolutional networks［C］∥ Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition （CVPR），2017： 2261-2269.
34	Srivastava N， Hinton G， Krizhevsky A， et al. Dropout： A simple way to prevent neural networks from overfitting［J］. Journal of Machine Learning Research，2014，15（1）：1929-1958. DOI： . doi: 10.5555/2627435.2670313
35	Mikolov T， Chen K， Corrado G， et al. Efficient estimation of word representations in vector space［C］∥ Proceedings of the 1st International Conference on Learning Representations （ICLR）， 2013： 1-12.
36	Hensman J， Fusi N， Lawrence N D. Gaussian processes for big data［C］∥ Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence （UAI）， 2013：282-290.
37	Zhang Zhi， Dong Fuan， Wu Youli. Estimation of fractal dimension for 2-D gray image［J］. Computer Applications， 2005， 25（12）： 2853-2855.
37	张志，董福安，伍友利.二维灰度图像的分形维数计算［J］.计算机应用，2005，25（12）：2853-2855.
38	Shan Na， Zheng Tianyao， Wang Zhensong. High-speed and high-accuracy algorithm for cloud detection and its application［J］. Journal of Remote Sensing， 2009， 13（6）： 1138-1155.
38	单娜，郑天垚，王贞松.快速高准确度云检测算法及其应用［J］.遥感学报，2009，13（6）：1138-1155.
39	Chen Qiang， Tian Jie， Huang Haining. Study on SAS image segmentation using SVM based on statistical and texture features［J］. Chinese Journal of Scientific Instrument， 2013， 34（6）： 1413-1420.
39	陈强，田杰，黄海宁，等.基于统计和纹理特征的SAS图像SVM分割研究［J］.仪器仪表学报， 2013， 34 （6）： 1413-1420.
40	Han Dongchen， Shang Yaceng， Lai Yueshen. Illumination-Invariant image for shadow removal method［J］. Journal of Xi’an Technological University，2021，41（6）：691-695.
40	韩东辰，尚雅层，来跃深.一种光照无关图阴影去除方法［J］.西安工业大学学报， 2021， 41 （6）： 691-695.
41	Hong D F， Gao L R， Yokoya N，et al. More diverse means better： Multimodal deep learning meets remote-sensing imagery classification［J］.IEEE Transactions on Geoscience and Remote，2021，59（5）：4340-4354.DOI： . doi: 10.1109/TGRS.2020.3016820

No Suggested Reading articles found!

Viewed

Full text

Abstract

Cited

Shared

Discussed