Hyperspectral image super-resolution using spatial-spectral attention and spectral transformer
,
and
*
Article Sidebar
Full Text: 
PDF
Main Article Content
Abstract
Hyperspectral image super-resolution (HSI-SR) aims to reconstruct high-resolution hyperspectral images (HR-HSI) from low-resolution observations (LR-HSI). Existing convolutional neural network (CNN)-based methods excel at extracting local spatial features but struggle to capture long-range spectral dependencies, while full spatial Transformer architectures incur high computational cost. We propose ESSA-SSAM-SpecTrans, a hybrid architecture that extends the Efficient Spectral-Spatial Attention (ESSA) baseline with two novel modules: (1) a Separated Spatial-Spectral Attention Module (SSAM) that independently models spatial and spectral features in 2D image space via dual-branch attention, and (2) a Spectral Transformer that applies multi-head self-attention along the spectral dimension, treating each spectral band as a token with complexity O(C2·HW) instead of O((HW)2·C). Experiments on two benchmark datasets with a scale factor of ×4 demonstrate the effectiveness of the proposed approach. On the CAVE dataset, the model achieves a PSNR of 37.78 dB, an SSIM of 0.960, and an ERGAS of 3.18, the best results among the compared methods on these three metrics. On the Pavia Centre dataset, it achieves the lowest SAM (4.80°) and ERGAS (5.51), indicating strong spectral fidelity on real-world remote-sensing data.
Article Details
Conflict of Interest
© 2026 The authors. This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License.
References
Bioucas-Dias, J.M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P., & Chanussot, J. (2012). Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(2), 354–379.
Computational Intelligence Group (GIC). University of the Basque Country (UPV/EHU). (2011). Pavia centre hyperspectral scene (ROSIS sensor). Hyperspectral Remote Sensing Scenes dataset repository, acquired by the ROSIS-03 sensor over Pavia, Italy; courtesy of Prof. P. Gamba, University of Pavia.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., et al. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
Fu, Y., Liang, Z., & You, S. (2021). Bidirectional 3d quasi-recurrent neural network for hyperspectral image super-resolution. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 2674–2688.
Hardie, R.C., Barnard, K.J., & Armstrong, E.E. (1998). High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering, 37(1), 247–260.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR) (pp. 770–778).
Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132–7141).
Hu, J.F., Huang, T.Z., Deng, L.J., Dou, H.X., Hong, D., & Vivone, G. (2022). Fusformer: A transformer-based fusion approach for hyperspectral image super-resolution. IEEE Geoscience and Remote Sensing Letters, 19, 1–5.
Hu, Q., Ma, J., Gao, Y., Jiang, J., & Yuan, Y. (2024). MAUN: Memory-augmented deep unfolding network for hyperspectral image reconstruction. IEEE/CAA Journal of Automatica Sinica, 11(5), 1139–1150.
Jiang, J., Sun, H., Liu, X., & Ma, J. (2020). Learning spatial-spectral prior for superresolution of hyperspectral imagery. IEEE Transactions on Computational Imaging, 6, 1082–1096.
Khan, S., Naseer, M., Hayat, M., Zamir, S.W., Khan, F.S., & Shah, M. (2022). Transformers in vision: A survey. ACM Computing Surveys, 54(10s), 1–41.
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
Ledig, C., Theis, L., Husza´r, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., & Shi, W. (2017). Photo-realistic single image superresolution using a generative adversarial network. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR) (pp. 105–114).
Li, Q., Wang, Q., & Li, X. (2020). Mixed 2d/3d convolutional network for hyperspectral image super-resolution. Remote Sensing, 12(10), 1660.
Li, Y., Zhang, H., & Shen, Q. (2017). Spectral-spatial classification of hyperspectral imagery with 3d convolutional neural network. Remote Sensing, 9(1), 67.
Li, Y., Zhang, L., Ding, C., Wei, W., & Zhang, Y. (2018). Single hyperspectral image superresolution with grouped deep recursive residual network. In: Proc. IEEE Int. Conf. Multimedia Big Data (BigMM) (pp. 1–4).
Liu, J., Wu, Z., Xiao, L., & Wu, X.J. (2022). Model inspired autoencoder for unsupervised hyperspectral image super-resolution. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–12.
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In: Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV) (pp. 9992–10002).
Qu, Y., Qi, H., & Kwan, C. (2018). Unsupervised sparse dirichlet-net for hyperspectral image super-resolution. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR) (pp. 2511–2520).
Simoes, M., Bioucas-Dias, J., Almeida, L.B., & Chanussot, J. (2014). A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Transactions on Geoscience and Remote Sensing, 53(6), 3373–3388.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L ., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
Wang, Q., Li, Q., & Li, X. (2020). Spatial-spectral residual network for hyperspectral image super-resolution. arXiv preprint arXiv:2001.04609.
Wang, X., Hu, Q., Cheng, Y., & Ma, J. (2023). Hyperspectral image super-resolution meets deep learning: A survey and perspective. IEEE/CAA Journal of Automatica Sinica, 10(8), 1668–1691.
Woo, S., Park, J., Lee, J.Y., & Kweon, I.S. (2018). CBAM: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV) (pp. 3–19).
Xie, Q., Zhou, M., Zhao, Q., Xu, Z., & Meng, D. (2020). MHF-Net: An interpretable deep network for multispectral and hyperspectral image fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3), 1457–1473.
Yasuma, F., Mitsunaga, T., Iso, D., & Nayar, S.K. (2010). Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing, 19(9), 2241–2253.
Yokoya, N., Yairi, T., & Iwasaki, A. (2011). Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing, 50(2), 528–537.
Zhang, C., Zhang, M., Li, Y., Gao, X., & Qiu, S. (2021). Difference curvature multidimensional network for hyperspectral image super-resolution. Remote Sensing, 13(17), 3455.
Zhang, M., Zhang, C., Zhang, Q., Guo, J., Gao, X., & Zhang, J. (2023). ESSAformer: Efficient transformer for hyperspectral image super-resolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 23016– 23027).
Zhu, Z., Hou, J., Chen, J., Zeng, H., & Zhou, J. (2020). Hyperspectral image super-resolution via deep progressive zero-centric residual learning. IEEE Transactions on Image Processing, 30, 1423–1438.