Please wait a minute...
img

官方微信

遥感技术与应用  2021, Vol. 36 Issue (4): 810-819    DOI: 10.11873/j.issn.1004-0323.2021.4.0810
数据与图像处理     
基于改进NSST-PCNN的光学与SAR图像融合去云方法
陈子涵1,2,3(),王峰1,2,许宁1,2,尤红建1,2,3()
1.中国科学院空间信息处理与应用系统技术重点实验室,北京 100090
2.中国科学院空天信息创新研究院,北京 100090
3.中国科学院大学,北京 100049
Cloud Removal by Fusing Optical and SAR Images based on Improved PCNN in NSST Domain
Zihan Chen1,2,3(),Feng Wang1,2,Ning Xu1,2,Hongjian You1,2,3()
1.Key Laboratory of Technology in Geo-spatial Information Processing and Application System,Chinese Academy of Sciences,Beijing 100090,China
2.Aerospace Information Research Institute,Chinese Academy of Sciences,Beijing 100090,China
3.University of Chinese Academy of Sciences,Beijing 100049,China
 全文: PDF(2856 KB)   HTML
摘要:

由于光学遥感穿透性差,光学图像常受到云层等天气因素干扰而影响其遥感应用。现有基于多时相或单幅图像修复的方法受地物变化及缺乏先验信息的影响,难以恢复云下真实地物信息。利用SAR图像不受云层、光照等因素干扰的特点,提出一种与SAR图像融合的光学图像去云方法。首先利用分形网络演化算法(FNEA)结合形状及光谱特性对云区进行检测,接着采用非下采样剪切波变换(NSST)对光学与SAR图像进行分解,最后对分解后系数结合云区检测结果进行融合,其中低频信息基于改进加权能量和进行融合,高频则结合方向信息熵及脉冲耦合神经网络(PCNN)模型进行融合。以高分一号、二号光学和高分三号SAR图像数据进行实验。结果表明,该方法相较其他5种算法在云区与参考图像有更高的相似性,可以更好地保持纹理及细节特征,在有效解决云层遮挡问题的同时实现图像增强,有利于后续图像分类、目标识别以及图像判别等遥感应用。

关键词: 图像去云图像融合SAR非下采样剪切波变换脉冲耦合神经网络    
Abstract:

Due to the poor penetrability of optical remote sensing, optical images are often disturbed by weather factors such as clouds, which affect the applications of remote sensing. The existing methods based on multi- temporal or single image are difficult to recover the real features under the cloud because of the change of features and the lack of prior information. Based on the fact that the SAR images are not interfered by the factors such as cloud and illumination, an cloud removal method by fusing SAR and optical images is proposed. Firstly, the cloud area is detected by the Fractal Net Evolution Approach (FNEA) combined with the shape and spectral characteristics. Secondly, the optical and SAR images are decomposed by the Non-Subsampled Shearlet Transform (NSST). Finally, the decomposed coefficients are dealed with the cloud area detection results, in which the low-frequency information is fused by the improved weighted energy sum, and the high-frequency information is fused by the direction information entropy and Pulse Coupled Neural Network (PCNN). Take the GF-1/GF-2 optical and GF-3 SAR images as the experimental data source. The results show that compared with the other five algorithms, our method has higher similarity with the reference image in the cloud area and can better maintain the texture and detail features, which effectively solve the problem of cloud occlusion while realizing image enhancement and is beneficial to the following remote sensing applications such as image classification, target recognition and image discrimination.

Key words: Cloud removal    Image fusion    SAR    Non-Subsampled Shearlet Transform(NSST)    Pulse Coupled Neural Network(PCNN)
收稿日期: 2020-03-29 出版日期: 2021-09-26
ZTFLH:  TP751  
基金资助: 国家重点研发计划项目(2017YFB0502901);国防科工局十三五民用航天技术预先研究项目(D040402)
通讯作者: 尤红建     E-mail: chenzihan2014@mails.ucas.ac.cn;hjyou@mail.ie.ac.cn
作者简介: 陈子涵(1996-),女,北京人,硕士研究生,主要从事遥感图像处理与应用方面研究。E?mail:chenzihan2014@mails.ucas.ac.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
陈子涵
王峰
许宁
尤红建

引用本文:

陈子涵,王峰,许宁,尤红建. 基于改进NSST-PCNN的光学与SAR图像融合去云方法[J]. 遥感技术与应用, 2021, 36(4): 810-819.

Zihan Chen,Feng Wang,Ning Xu,Hongjian You. Cloud Removal by Fusing Optical and SAR Images based on Improved PCNN in NSST Domain. Remote Sensing Technology and Application, 2021, 36(4): 810-819.

链接本文:

http://www.rsta.ac.cn/CN/10.11873/j.issn.1004-0323.2021.4.0810        http://www.rsta.ac.cn/CN/Y2021/V36/I4/810

图1  NSST分解示意图
图2  PCNN简化模型[15]
图3  融合去云流程图
图4  云区检测流程图
图5  云区检测结果
全色图像SAR图像地区像素大小
采集时间分辨率/m采集时间分辨率/m
117.04.300.817.03.180.5广州2 000×1 500
21 400×1 000
318.03.272.018.05.151.0苏州1 350×1 120
41 500×1 100
表1  数据信息
图6  第一组融合去云结果
图7  第二组融合去云结果
图8  第三组融合去云结果
图9  第四组融合去云结果
AHFS-EBIIPoissonNSCT-PAPCNNNSST-PCNNProposed
SSIM0.360.360.600.620.620.71
PSNR8.6014.9820.2520.2620.2620.87
RMSE94.7045.4324.7724.7524.7623.07
EN7.117.077.067.277.287.26
AG2.692.382.354.034.173.32
SSIM-A---0.480.470.61
表2  第一组图像各方法客观评价结果
AHFS-EBIIPoissonNSCT-PAPCNNNSST-PCNNProposed
SSIM0.320.430.350.350.350.52
PSNR8.7616.1014.6916.3816.3917.17
RMSE92.9940.0047.0538.7038.6535.34
EN7.297.387.387.557.567.51
AG5.495.595.0911.2811.757.93
SSIM-A---0.310.370.61
表3  第三组图像各方法客观评价结果
时间/s图6图7图8图9
AHF16.115.896.466.22
S-EBII2 926.671 643.24665.571 251.8
Poisson121.60274.4318.6564.93
NSCT-PAPCNN4 393.312 128.532 951.362 450.87
NSST-PCNN298.37103.39137.28110.21
Proposed239.5398.95114.41101.32
表4  各算法运行时间对比
1 Sheng H, Li X, Cheng Q, et al. Missing information reconstruction of remote sensing data: a technical review[J]. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3): 61-85.DOI:10.1109/MGRS.2015. 2441912.
doi: 10.1109/MGRS.2015. 2441912
2 Criminisi A, Perezé P, Toyama K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212.
3 Zhou Xiaojun,Guo Jia,Zhou Chengxian, et al. An algorithm of cloud removal for remote sensing image based on improved homomorphic filtering[J].Radio Engineering, 2015, 45(3): 14-18.
3 周小军,郭佳,周承仙,等. 基于改进同态滤波的遥感图像去云算法[J]. 无线电工程, 2015,45(3):14-18.
4 Liu H, Bi X, Lu G, et al. Exemplar-based image inpainting with multi-resolution information and the graph cut technique[J]. IEEE Access, 2019, 7:101641-101657. DOI:10.1109/ACCESS.2019. 2931064.
doi: 10.1109/ACCESS.2019. 2931064
5 Li Z, Shen H, Cheng Q, et al. Thick cloud removal in high- resolution satellite images using stepwise radiometric adjustment and residual correction[J]. Remote Sensing, 2019, 11(16): 1925. DOI:10.3390/rs11161925.
doi: 10.3390/rs11161925
6 Hu C, Huo L Z, Zhang Z, et al. Multi-temporal Landsat data automatic cloud removal using poisson blending[J]. IEEE Access,2020,8:46151-46161. DOI:10.1109/ACCESS.2020. 2979291.
doi: 10.1109/ACCESS.2020. 2979291
7 Zhou Binxing. Study of Thin cloud removal method fot muti-spectral remote sensing images[D]. Chengdu: University of Electronic Science and Technology of China, 2020.
7 周斌兴. 多光谱遥感影像薄云去除方法研究[D].成都:电子科技大学,2020.
8 Huang B, Li Y, Han X, et al. Cloud removal from optical satellite imagery with SAR imagery using sparse representation[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(5): 1046-1050. DOI:10.1109/LGRS.2014.2377476.
doi: 10.1109/LGRS.2014.2377476
9 Zhu C, Zhao Z, Zhu X, et al. Cloud removal for optical images using SAR structure data[C]∥ IEEE International Conference on Signal Processing, 2016.
10 Li W, Li Y, Chan C J. Thick cloud removal with optical and SAR imagery via convolutional-mapping-deconvolutional network[J]. IEEE Transactions on Geoscience and Remote Sensing,2020,58(4):2865-2879. DOI:10.1109/TGRS.2019. 2956959.
doi: 10.1109/TGRS.2019. 2956959
11 Liu Jian, Lei Yingjie, Xing Yaqiong, et al. Fusion technique for SAR and gray visible image based on hidden Markov model in non-subsample shearlet transform domain[J]. Control and Decision, 2016, 31(3): 453-457.
11 刘健,雷英杰,邢雅琼,等. 基于NSST域隐马尔可夫树模型的SAR和灰度可见光图像融合[J]. 控制与决策, 2016, 31(3): 453-457.
12 Wu Yiquan, Wang Zhilai. Multispectral and panchromatic image fusion using chaotic bee colony optimization in NSST domain[J]. Journal of Remote Sensing, 2017, 21(4):549-557.
12 吴一全,王志来. 混沌蜂群优化的NSST域多光谱与全色图像融合[J]. 遥感学报, 2017, 21(4): 549-557.
13 Chu T, Tan Y, Liu Q, et al. Novel fusion method for SAR and optical images based on Non-subsampled shearlet transform[J]. International Journal of Remote Sensing, 2020, 41(12): 4588-4602. DOI:10.1080/01431161.2020.1723175.
doi: 10.1080/01431161.2020.1723175
14 Easley G, Labate D, Lim W Q. Sparse directional image representations using the discrete shearlet transform[J]. Applied Computational Harmonic Analysis, 2008, 25(1): 25-46. DOI:10.1016/j.acha.2007.09. 003.
doi: 10.1016/j.acha.2007.09. 003
15 Blasch E P. Biological information fusion using a PCNN and belief filtering[C]∥ Proceedings of the International Joint Conference on Neural Networks, 1999.
16 Jiao Jiao, Wu Lingda. Fusion of multispectral and panchromatic images via morphological filter and improved PCNN in NSST domain[J]. Journal of Image and Graphics, 2019, 24(3): 115-126.
16 焦姣,吴玲达. 形态学滤波和改进PCNN的NSST域多光谱与全色图像融合[J]. 中国图象图形学报, 2019, 24(3): 115-126.
17 Baate A, Schape A. Multiresolution segmentation: An optimization approach for highquality multiscale image segmentation[J]. Journal of Photogrammetry and Remote Sensing, 2007, 58: 3-4.
18 Zhu Qiaoyun, Da Xing. Dodging method for multi-source remote sensing images based on wallis filter[J]. Geomatics and Spatial Information Technology, 2012, 35(10): 130-132.
18 朱巧云,答星. 基于Wallis滤波器的异源遥感影像匀光方法[J]. 测绘与空间地理信息, 2012,35(10):130-132.
19 Liu Y, Liu S, Wang Z. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164. DOI:10.1016/j.inffus.2014.09.004.
doi: 10.1016/j.inffus.2014.09.004
20 Yin M, Liu X, Liu Y, et al. Medical image fusion with parameter- adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1): 49-64.
21 Shen Yu, Chen Xiaopeng, Yang Qian. Image fusion of multidirectional sum modified Laplacian and Tetrolet transform[J]. Journal of Image and Graphics,2020, 25(4): 721-731.
21 沈瑜,陈小朋,杨倩.多方向Laplacian能量和与Tetrolet变换的图像融合[J].中国图象图形学报, 2020, 25(4): 721-731.
22 Wang Zhishe,Yang Fengbao,Chen Lei,et al. SAR and visible image enhanced fusion based on texture segmentation and Top-Hat transformation[J].Acta Optica Sinica,2014,34(10): 85-92.
22 王志社,杨风暴,陈磊, 等. 基于纹理分割和Top-Hat变换的合成孔径雷达与可见光图像增强融合[J]. 光学学报, 2014, 34(10): 85-92.
23 Hou B. SAR image despeckling based on nonsubsampled shearlet transform[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2012, 5(3): 809-823. DOI:10.1109/ JSTARS.2012.2196680.
doi: 10.1109/ JSTARS.2012.2196680
24 Shi Peili, Guo Lei, Li Huihui. SAR and optical images fusion method based on NSCT[J]. Application Research of Computers, 2013, 30(5): 1568-1570.
24 时丕丽,郭雷,李晖晖. 基于NSCT的SAR与可见光图像融合方法[J]. 计算机应用研究, 2013, 30(5): 1568-1570.
25 Pérez P, Gangnet M, Blake A. Poisson image editing[J]. Acm Transactions on Graphics,2003,22(3):313. DOI:10.1145/1201775.882269
doi: 10.1145/1201775.882269
26 Kulkarni S C, Rege P P. Pixel level fusion techniques for SAR and optical images: A review [J]. Information Fusion 2020, 59:13-29. DOI: 10.1016/j.inffus.2020.01.003.
doi: 10.1016/j.inffus.2020.01.003
[1] 王晨丞,王永前,王利花. 基于SAR纹理信息的农作物识别研究——以农安县为例[J]. 遥感技术与应用, 2021, 36(2): 372-380.
[2] 金亚秋. 空间微波遥感研究与应用—丛书述评[J]. 遥感技术与应用, 2021, 36(1): 1-10.
[3] 周金霖,陈锟山,古琼昇,曾江源,许镇. 基于多次散射目标模型的雷达成像研究[J]. 遥感技术与应用, 2021, 36(1): 132-140.
[4] 任慧敏,宋冬梅,王斌,甄宗晋,刘斌,张婷. 基于新极化特征参数的SAR海洋溢油检测[J]. 遥感技术与应用, 2020, 35(4): 934-942.
[5] 王树果, 马春锋, 赵泽斌, 魏龙. 基于Sentinel-1及Landsat 8数据的黑河中游农田土壤水分估算[J]. 遥感技术与应用, 2020, 35(1): 13-22.
[6] 杨燕,李震,黄磊,田帮森. 高分辨率SAR影像提取冰川面积与冰面河[J]. 遥感技术与应用, 2019, 34(6): 1155-1161.
[7] 崔刚,吴金胜,于镇,周玲. 深度学习对不同分辨率影像冬小麦识别的适用性研究[J]. 遥感技术与应用, 2019, 34(4): 748-755.
[8] 李春江,沈国状,张继超. 基于灰色系统理论的植被物理参数与极化分解参数的关联分析[J]. 遥感技术与应用, 2019, 34(4): 839-846.
[9] 苏华,张明慧,李静,陈修治,汪小钦. 基于光学与SAR因子的森林生物量多元回归估算[J]. 遥感技术与应用, 2019, 34(4): 847-856.
[10] 徐梦竹, 徐佳, 邓鸿儒, 袁春琦. 基于全极化SAR影像的海岛地物分类[J]. 遥感技术与应用, 2019, 34(3): 647-654.
[11] 李春江, 沈国状, 张继超. 基于灰色系统理论的植被物理参数与极化分解参数的关联分析—以鄱阳湖湿地为例[J]. 遥感技术与应用, 2019, 34(2): 284-292.
[12] 宋小霞, 王静, 储小青. 基于多普勒频移的SAR海表流场反演[J]. 遥感技术与应用, 2019, 34(2): 293-302.
[13] 李丹, 杨斌, 陈财. 基于Sentinel-1A数据反演九寨沟地震地表形变场[J]. 遥感技术与应用, 2018, 33(6): 1141-1148.
[14] 李姣姣, 刘玉, 陈锟山. 基于香农熵的极化SAR相干矩阵信息量评价[J]. 遥感技术与应用, 2018, 33(5): 842-849.
[15] 王仕哲, 柯长青. 基于ALOSPALSAR的喜马拉雅山冰川流速分布及变化[J]. 遥感技术与应用, 2018, 33(5): 942-955.