Please wait a minute...
img

官方微信

遥感技术与应用  2022, Vol. 37 Issue (4): 829-838    DOI: 10.11873/j.issn.1004-0323.2022.4.0829
深度学习专栏     
一种基于ASR和PAPCNN的NSCT域遥感影像融合方法
吕开云1,2,3(),侯昭阳1,龚循强1,3(),杨硕1
1.东华理工大学 测绘工程学院,江西 南昌 330013
2.自然资源部 海洋环境探测技术与应用重点实验室,广东 广州 510300
3.自然资源部 环鄱阳湖区域矿山环境监测与治理重点实验室,江西 南昌 330013
A Remote Sensing Image Fusion Method based on ASR and PAPCNN in NSCT Domain
Lü Kaiyun1,2,3(),Zhaoyang Hou1,Xunqiang Gong1,3(),Shuo Yang1
1.Faculty of Geomatics,East China University of Technology,Nanchang 330013,China
2.Key Laboratory of Marine Environmental Survey Technology and Application,Ministry of Natural Resources,Guangzhou 510300,China
3.Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake,Ministry of Natural Resources,Nanchang 330013,China
 全文: PDF(5308 KB)   HTML
摘要:

针对稀疏字典的高冗余性和脉冲耦合神经网络(PCNN)参数设置的主观性问题,提出一种结合自适应稀疏表示(ASR)和参数自适应脉冲耦合神经网络(PAPCNN)的非下采样轮廓波变换(NSCT)域遥感影像融合方法。该方法将多光谱影像通过YUV空间变换得到的亮度分量Y与全色影像进行NSCT分解为高低频子带。对低频子带采用基于ASR的融合规则,根据影像块的梯度信息实现自适应稀疏表示。对高频子带采用PAPCNN模型,以选择PCNN的最优参数,再经过相应逆变换得到融合结果。实验结果表明:该方法对不同卫星影像在定性和定量评价上的总体效果均优于其他8种方法。

关键词: 遥感影像融合非下采样轮廓波变换自适应稀疏表示参数自适应脉冲耦合神经网络    
Abstract:

In order to solve the problems of the high redundancy of the sparse dictionary and the subjectivity of Pulse-Coupled Neural Network (PCNN) parameter setting, a remote sensing image using fusion method based on Adaptive Sparse Representation (ASR) and Parameter Adaptive Pulse Coupled Neural Network (PAPCNN) in Non-Subsampled Contourlet Transform (NSCT) domain is proposed in this paper. Luminance components and panchromatic images are decomposed by NSCT to obtain high and low frequency sub-bands, and the luminance component Y is obtained from the multi-spectral image through YUV spatial transformation. ASR-based fusion rules are used for sparse representation of low frequency sub-band and adaptive sparse representation is realized according to the gradient information of the image block. The PAPCNN model is adopted to select the optimal parameters of PCNN in the high frequency sub-band. Finally, the fusion result is obtained through the corresponding inverse transformation. The experimental results of different satellite images show that the overall effect of the proposed method is better than the other six methods by using qualitative evaluation and quantitative evaluation.

Key words: Remote sensing image fusion    Non-Subsampled Contourlet Transform    Adaptive Sparse Representation    Parameter Adaptive Pulse Coupled Neural Network
收稿日期: 2021-12-02 出版日期: 2022-09-28
:  TP751  
基金资助: 自然资源部海洋环境探测技术与应用重点实验室开放基金项目(MESTA-2021-B001);国家自然科学基金项目(42101457);江西省自然科学基金项目(20202BABL202030);江西省教育厅科学技术科技项目(GJJ150591);东华理工大学放射性地质与勘探技术国防重点学科实验室开放基金项目(REGT1219)
通讯作者: 龚循强     E-mail: kylv@ecut.edu.cn;xqgong1988@ ecut.edu.cn
作者简介: 吕开云(1974-),男,湖南武冈人,博士,副教授,主要从事遥感影像处理研究。E?mail: kylv@ecut.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
吕开云
侯昭阳
龚循强
杨硕

引用本文:

吕开云,侯昭阳,龚循强,杨硕. 一种基于ASR和PAPCNN的NSCT域遥感影像融合方法[J]. 遥感技术与应用, 2022, 37(4): 829-838.

Lü Kaiyun,Zhaoyang Hou,Xunqiang Gong,Shuo Yang. A Remote Sensing Image Fusion Method based on ASR and PAPCNN in NSCT Domain. Remote Sensing Technology and Application, 2022, 37(4): 829-838.

链接本文:

http://www.rsta.ac.cn/CN/10.11873/j.issn.1004-0323.2022.4.0829        http://www.rsta.ac.cn/CN/Y2022/V37/I4/829

图1  二级NSCT分解示意图
图2  PAPCNN模型的体系结构
图3  方法流程图
图4  实验数据(a) GF-2 MS (b)GF-2 PAN (c)SPOT-6 MS (d)SPOT-6 PAN
图5  PAPCNN不同迭代次数的结果
图6  GF-2影像融合效果
图7  SPOT 6影像融合效果
方法SD↑IE↑AG↑SF↑CC↑DD↓ERGAS↓
GS9.0625.0972.0344.0830.7087.02110.067
Wavelet9.0545.0512.0454.2690.7704.8599.895
CNN9.2815.1422.1674.3300.8214.6868.954
SR8.8195.0501.3653.0760.7805.20410.287
ASR9.6455.1111.3132.9720.9383.4225.680
PAPCNN8.8055.0871.9863.9770.7795.2669.369
NSCT-ASR9.5215.1402.1674.3340.8604.1738.178
NSCT-PAPCNN9.4825.1382.1724.3460.7705.4949.696
本文方法9.5565.1432.1744.3430.8754.1448.155
表1  GF-2源影像融合结果的性能评价
方法SD↑IE↑AG↑SF↑CC↑DD↓ERGAS↓
GS3.4293.1620.6981.5920.9243.04214.648
Wavelet3.6413.5400.6961.5910.9633.06513.705
CNN3.2873.2730.7761.6310.9183.52616.812
SR3.1462.9650.3280.9060.9393.42216.585
ASR4.9233.6000.3500.9250.9891.92010.650
PAPCNN3.2523.1830.8181.9050.8983.45728.292
NSCT-ASR4.9043.6050.7741.6520.9712.06011.784
NSCT-PAPCNN3.2803.2130.7871.6580.9043.54629.482
本文方法4.9763.6310.7961.6790.9742.0459.066
表2  SPOT-6源影像融合结果的性能评价
1 Li Shutao, Li Congyu, Kang Xudong. Development status and future prospects of multi-source remote sensing image fusion[J]. Journal of Remote Sensing, 2021, 25(1): 148-166.
1 李树涛, 李聪妤, 康旭东. 多源遥感图像融合发展现状与未来展望[J]. 遥感学报, 2021, 25(1): 148-166.
2 Ding Haiyong, Guo Ruirui, Luo Haibin. Denoising of remote sensing images using adaptive threshold in NSCT domain by concerning texture information[J]. Reomte Sensing Techonogy and Application, 2017, 32(3):435-442.
2 丁海勇, 郭瑞瑞, 罗海滨. 顾及纹理信息的遥感图像NSCT域自适应阈值去噪[J]. 遥感技术与应用, 2017, 32(3):435-442.
3 Chen Yngxia, Chen Yan, Liu Cong. Joint AIHS and particle swarm otimization for pan-sharpening[J]. Acta Geodaetica er Cartographica, 2019, 48(10): 1296-1304.
3 陈应霞, 陈艳, 刘丛. 遥感影像融合AIHS转换与粒子群优化算法[J]. 测绘学报, 2019, 48(10): 1296-1304.
4 Wang Hanhun, Lu Yansheng, Chen Minjiang. Remote sensing image fusion by using discrete multiwavelet transform[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2006, 34(8): 7-9.
4 王海晖, 卢炎生, 陈闽江. 基于离散多小波变换的遥感图像融合方法[J]. 华中科技大学学报(自然科学版), 2006, 34(8): 7-9.
5 Liu H, Xiao G F, Tan Y L, et al. Multi-source remote sensing image registration based on contourlet transform and multiple feature fusion[J]. International Journal of Automation and Computing, 2019, 16(5): 575-588. DOI: 10.1007/s11633-018-1163-6 .
doi: 10.1007/s11633-018-1163-6
6 Bhatnagar G, Wu Q J, Liu Z. Directive contrast based multimodal medical image fusion in NSCT domain[J]. IEEE transactions on multimedia,2013,15(5):1014-1024. DOI: 10.1109/ TMM.2013.2244870 .
doi: 10.1109/ TMM.2013.2244870
7 Xiong Z, Guo Q, Liu M, et al. Pan-sharpening based on convolutional neural network by using the loss function with no-reference[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14: 897-906. DOI:10.1109/JSTARS.2021.3086877 .
doi: 10.1109/JSTARS.2021.3086877
8 Vivone G, Dalla Mura M, Garzelli A, et al. A benchmarking protocol for pansharpening: Dataset,preprocessing, and quality assessment[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 6102-6118. DOI:10.1109/JSTARS.2020.3038057 .
doi: 10.1109/JSTARS.2020.3038057
9 Zhang Minghua, Luo Hongling, Song Wei, et al. Feature extraction of hyperspectral image based on sparse representation and learning graph regularity[J]. Acta Photonica Sinica,2021, 50(4): 0410002.张明华, 罗红玲, 宋巍, 等. 基于稀疏表示和学习图正则的高光谱图像特征提取[J]. 光子学报, 2021, 50(4): 0410002. DOI:10.3788/gzxb20215004.0410002
doi: 10.3788/gzxb20215004.0410002
10 Fangzhou Nan, Xu Ya, Liu Wei, et al. Denoising methods of OBS data based on sparse representation[J]. Chinese Journal of Geophysics,2018,61(4):1519-1528.
10 南方舟, 徐亚, 刘伟, 等. 基于稀疏表达的OBS去噪方法[J]. 地球物理学报, 2018, 61(4): 1519-1528.
11 Xu Ning, Xiao Xinyao, You Hongjian, et al. A pansharpening method based on HCT and joint sparse model[J]. Acta Geodaetica er Cartographica, 2016, 45(4): 434-441.
11 许宁, 肖新耀, 尤红建, 等. HCT变换与联合稀疏模型相结合的遥感影像融合[J]. 测绘学报, 2016, 45(4): 434-441.
12 Liu Fan, Pei Xiaopeng, Zhang Jing,et al.Remote sensing ima-ge fusion based on optimized dictionary learning[J]. Journal of Electronics & Information Technology, 2018, 40(12): 2804-2811.
12 刘帆,裴晓鹏,张静,等.基于优化字典学习的遥感图像融合方法[J]. 电子与信息学报,2018,40(12):2804-2811.
13 Wu Yiquan, Tao Feixiang. Multispectral and panchromatic omage fusion based on improved projected gradient NMF in NSST domain[J]. Acta Optica Sinica,2015,35(4):0410005-1-0410005-10.吴一全, 陶飞翔. 改进投影梯度NMF的NSST域多光谱与全色图像融合[J]. 光学学报, 2015, 35(4): 0410005-1-0410005-10.
14 Jiao Jiao, Wu Lingda, Yu Shaobo, et al. Image fusion method using multi-scale analysis and improved PCNN[J]. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(6): 988-996.
14 焦姣, 吴玲达, 于少波. 混合多尺度分析和改进PCNN相结合的图像融合方法[J]. 计算机辅助设计与图形学学报, 2019, 31(6): 988-996.
15 Yin M, Liu X N, Liu Y. Medical image fusion with parameter-adaptive pulse coupled-neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1): 49-64. DOI: 10.1109/TIM.2018.2838778 .
doi: 10.1109/TIM.2018.2838778
16 Liu Y, Chen X, Peng H, et al. Multi-focus image fusion with a deep convolutional neural network[J]. Information Fusion, 2017, 36: 191-207. DOI: 10.1016/j.inffus.2016.12.001 .
doi: 10.1016/j.inffus.2016.12.001
17 Yang B, Li S T. Multi-focus image fusion and restoration with sparse representation[J]. IEEE transactions on Instrumentation and Measurement, 2010, 59(4), 884-892. DOI: 10.1109/TIM.2009.2026612 .
doi: 10.1109/TIM.2009.2026612
18 Liu Y, Wang Z F. Simultaneous image fusion and denoising with adaptive sparse representation[J]. IET Image Processing, 2015, 9(5): 347-357. DOI: 10.1049/iet-ipr.2014.0311 .
doi: 10.1049/iet-ipr.2014.0311
19 Cheng Feifei, Fu Zhitao, Huang Liang, et al. Non-subsampled shearlet transform remote sensing image fusion combined with parameter-adaptive PCNN[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10): 1380-1389.
19 成飞飞, 付志涛, 黄亮, 等. 结合自适应PCNN的非下采样剪切波遥感影像融合[J]. 测绘学报, 2021, 50(10): 1380-1389.
20 Chen Y L, Park S K, Ma Y D,et al. A new automatic parameter setting method of a Simplified PCNN for image segmentation[J]. IEEE Transactions on Neural Networks, 2011, 22(6): 880-892. DOI: 10.1109/TNN.2011.2128880 .
doi: 10.1109/TNN.2011.2128880
[1] 杨昊翔,张丽,闫敏,林光辉. 基于高时空分辨率融合影像的红树林总初级生产力遥感估算[J]. 遥感技术与应用, 2021, 36(2): 453-462.
[2] 曹 闻,李弼程,彭天强. 一种基于小波包变换的遥感影像融合方法[J]. 遥感技术与应用, 2003, 18(4): 248-253.