A Remote Sensing Image Fusion Method based on ASR and PAPCNN in NSCT Domain

doi:10.11873/j.issn.1004-0323.2022.4.0829

Remote Sensing Technology and Application

2022, Vol. 37

Issue (4): 829-838 DOI: 10.11873/j.issn.1004-0323.2022.4.0829

A Remote Sensing Image Fusion Method based on ASR and PAPCNN in NSCT Domain

Lü Kaiyun^1,^2,³(

),Zhaoyang Hou¹,Xunqiang Gong^1,³(

),Shuo Yang¹

^1.Faculty of Geomatics，East China University of Technology，Nanchang 330013，China
^2.Key Laboratory of Marine Environmental Survey Technology and Application，Ministry of Natural Resources，Guangzhou 510300，China
^3.Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake，Ministry of Natural Resources，Nanchang 330013，China

Download:

HTML

PDF (5308KB)
Export: BibTeX | EndNote (RIS)

Abstract

In order to solve the problems of the high redundancy of the sparse dictionary and the subjectivity of Pulse-Coupled Neural Network （PCNN） parameter setting， a remote sensing image using fusion method based on Adaptive Sparse Representation （ASR） and Parameter Adaptive Pulse Coupled Neural Network （PAPCNN） in Non-Subsampled Contourlet Transform （NSCT） domain is proposed in this paper. Luminance components and panchromatic images are decomposed by NSCT to obtain high and low frequency sub-bands， and the luminance component Y is obtained from the multi-spectral image through YUV spatial transformation. ASR-based fusion rules are used for sparse representation of low frequency sub-band and adaptive sparse representation is realized according to the gradient information of the image block. The PAPCNN model is adopted to select the optimal parameters of PCNN in the high frequency sub-band. Finally， the fusion result is obtained through the corresponding inverse transformation. The experimental results of different satellite images show that the overall effect of the proposed method is better than the other six methods by using qualitative evaluation and quantitative evaluation.

Key words: Remote sensing image fusion Non-Subsampled Contourlet Transform Adaptive Sparse Representation Parameter Adaptive Pulse Coupled Neural Network

Received: 02 December 2021 Published: 28 September 2022

TP751

Corresponding Authors: Xunqiang Gong E-mail: kylv@ecut.edu.cn;xqgong1988@ ecut.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Lü Kaiyun
	Zhaoyang Hou
	Xunqiang Gong
	Shuo Yang

Cite this article:

Lü Kaiyun,Zhaoyang Hou,Xunqiang Gong,Shuo Yang. A Remote Sensing Image Fusion Method based on ASR and PAPCNN in NSCT Domain. Remote Sensing Technology and Application, 2022, 37(4): 829-838.

URL:

http://www.rsta.ac.cn/EN/10.11873/j.issn.1004-0323.2022.4.0829 OR http://www.rsta.ac.cn/EN/Y2022/V37/I4/829

Fig 1 Schematic of a two-level NSCT decomposition

Fig. 2 Adaptive PCNN model architecture

Fig 3 Algorithm flow chart

Fig4 Experimental data

Fig.5 Results of PAPCNN with different iteration number

Fig.6 GF-2 image fusion effect

Fig7 SPOT 6 image fusion effect

Table 1 Performance evaluation of GF-2 source image fusion results

Table 2 Performance evaluation of SPOT-6 source image fusion results

1	Li Shutao， Li Congyu， Kang Xudong. Development status and future prospects of multi-source remote sensing image fusion［J］. Journal of Remote Sensing， 2021， 25（1）： 148-166.
1	李树涛，李聪妤，康旭东. 多源遥感图像融合发展现状与未来展望［J］. 遥感学报， 2021， 25（1）： 148-166.
2	Ding Haiyong， Guo Ruirui， Luo Haibin. Denoising of remote sensing images using adaptive threshold in NSCT domain by concerning texture information［J］. Reomte Sensing Techonogy and Application， 2017， 32（3）：435-442.
2	丁海勇，郭瑞瑞，罗海滨. 顾及纹理信息的遥感图像NSCT域自适应阈值去噪［J］. 遥感技术与应用， 2017， 32（3）：435-442.
3	Chen Yngxia， Chen Yan， Liu Cong. Joint AIHS and particle swarm otimization for pan-sharpening［J］. Acta Geodaetica er Cartographica， 2019， 48（10）： 1296-1304.
3	陈应霞，陈艳，刘丛. 遥感影像融合AIHS转换与粒子群优化算法［J］. 测绘学报， 2019， 48（10）： 1296-1304.
4	Wang Hanhun， Lu Yansheng， Chen Minjiang. Remote sensing image fusion by using discrete multiwavelet transform［J］. Journal of Huazhong University of Science and Technology （Natural Science Edition）， 2006， 34（8）： 7-9.
4	王海晖，卢炎生，陈闽江. 基于离散多小波变换的遥感图像融合方法［J］. 华中科技大学学报（自然科学版）， 2006， 34（8）： 7-9.
5	Liu H， Xiao G F， Tan Y L， et al. Multi-source remote sensing image registration based on contourlet transform and multiple feature fusion［J］. International Journal of Automation and Computing， 2019， 16（5）： 575-588. DOI： . doi: 10.1007/s11633-018-1163-6
6	Bhatnagar G， Wu Q J， Liu Z. Directive contrast based multimodal medical image fusion in NSCT domain［J］. IEEE transactions on multimedia，2013，15（5）：1014-1024. DOI： . doi: 10.1109/ TMM.2013.2244870
7	Xiong Z， Guo Q， Liu M， et al. Pan-sharpening based on convolutional neural network by using the loss function with no-reference［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2020， 14： 897-906. DOI： . doi: 10.1109/JSTARS.2021.3086877
8	Vivone G， Dalla Mura M， Garzelli A， et al. A benchmarking protocol for pansharpening： Dataset，preprocessing， and quality assessment［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2021， 14： 6102-6118. DOI： . doi: 10.1109/JSTARS.2020.3038057
9	Zhang Minghua， Luo Hongling， Song Wei， et al. Feature extraction of hyperspectral image based on sparse representation and learning graph regularity［J］. Acta Photonica Sinica，2021， 50（4）： 0410002.张明华，罗红玲，宋巍，等. 基于稀疏表示和学习图正则的高光谱图像特征提取［J］. 光子学报， 2021， 50（4）： 0410002. DOI： doi: 10.3788/gzxb20215004.0410002
10	Fangzhou Nan， Xu Ya， Liu Wei， et al. Denoising methods of OBS data based on sparse representation［J］. Chinese Journal of Geophysics，2018，61（4）：1519-1528.
10	南方舟，徐亚，刘伟，等. 基于稀疏表达的OBS去噪方法［J］. 地球物理学报， 2018， 61（4）： 1519-1528.
11	Xu Ning， Xiao Xinyao， You Hongjian， et al. A pansharpening method based on HCT and joint sparse model［J］. Acta Geodaetica er Cartographica， 2016， 45（4）： 434-441.
11	许宁，肖新耀，尤红建，等. HCT变换与联合稀疏模型相结合的遥感影像融合［J］. 测绘学报， 2016， 45（4）： 434-441.
12	Liu Fan， Pei Xiaopeng， Zhang Jing，et al.Remote sensing ima-ge fusion based on optimized dictionary learning［J］. Journal of Electronics & Information Technology， 2018， 40（12）： 2804-2811.
12	刘帆，裴晓鹏，张静，等.基于优化字典学习的遥感图像融合方法［J］. 电子与信息学报，2018，40（12）：2804-2811.
13	Wu Yiquan， Tao Feixiang. Multispectral and panchromatic omage fusion based on improved projected gradient NMF in NSST domain［J］. Acta Optica Sinica，2015，35（4）：0410005-1-0410005-10.吴一全，陶飞翔. 改进投影梯度NMF的NSST域多光谱与全色图像融合［J］. 光学学报， 2015， 35（4）： 0410005-1-0410005-10.
14	Jiao Jiao， Wu Lingda， Yu Shaobo， et al. Image fusion method using multi-scale analysis and improved PCNN［J］. Journal of Computer-Aided Design & Computer Graphics， 2019， 31（6）： 988-996.
14	焦姣，吴玲达，于少波. 混合多尺度分析和改进PCNN相结合的图像融合方法［J］. 计算机辅助设计与图形学学报， 2019， 31（6）： 988-996.
15	Yin M， Liu X N， Liu Y. Medical image fusion with parameter-adaptive pulse coupled-neural network in nonsubsampled shearlet transform domain［J］. IEEE Transactions on Instrumentation and Measurement， 2019， 68（1）： 49-64. DOI： . doi: 10.1109/TIM.2018.2838778
16	Liu Y， Chen X， Peng H， et al. Multi-focus image fusion with a deep convolutional neural network［J］. Information Fusion， 2017， 36： 191-207. DOI： . doi: 10.1016/j.inffus.2016.12.001
17	Yang B， Li S T. Multi-focus image fusion and restoration with sparse representation［J］. IEEE transactions on Instrumentation and Measurement， 2010， 59（4）， 884-892. DOI： . doi: 10.1109/TIM.2009.2026612
18	Liu Y， Wang Z F. Simultaneous image fusion and denoising with adaptive sparse representation［J］. IET Image Processing， 2015， 9（5）： 347-357. DOI： . doi: 10.1049/iet-ipr.2014.0311
19	Cheng Feifei， Fu Zhitao， Huang Liang， et al. Non-subsampled shearlet transform remote sensing image fusion combined with parameter-adaptive PCNN［J］. Acta Geodaetica et Cartographica Sinica， 2021， 50（10）： 1380-1389.
19	成飞飞，付志涛，黄亮，等. 结合自适应PCNN的非下采样剪切波遥感影像融合［J］. 测绘学报， 2021， 50（10）： 1380-1389.
20	Chen Y L， Park S K， Ma Y D，et al. A new automatic parameter setting method of a Simplified PCNN for image segmentation［J］. IEEE Transactions on Neural Networks， 2011， 22（6）： 880-892. DOI： . doi: 10.1109/TNN.2011.2128880

No Suggested Reading articles found!

Viewed

Full text

Abstract

Cited

Shared

Discussed