Attention-based Multi-Reference Learning for Image Super-Resolution



CSNLN

RSRGAN

SRNTT

AMRSR (ours)

References

CSNLN

RSRGAN

SRNTT

AMRSR (ours)

References
Abstract

This paper proposes a novel Attention-based Multi-Reference Super-resolution network (AMRSR) that, given a low-resolution image, learns to adaptively transfer the most similar texture from multiple reference images to the super-resolution output whilst maintaining spatial coherence. The use of multiple reference images together with attention-based sampling is demonstrated to achieve significantly improved performance over state-of-the-art reference super-resolution approaches on multiple benchmark datasets. Reference super-resolution approaches have recently been proposed to overcome the ill-posed problem of image super-resolution by providing additional information from a high-resolution reference image. Multi-reference super-resolution extends this approach by providing a more diverse pool of image features to overcome the inherent information deficit whilst maintaining memory efficiency. A novel hierarchical attention-based sampling approach is introduced to learn the similarity between low-resolution image features and multiple reference images based on a perceptual loss. Ablation demonstrates the contribution of both multi-reference and hierarchical attention-based sampling to overall performance. Perceptual and quantitative ground-truth evaluation demonstrates significant improvement in performance even when the reference images deviate significantly from the target image.
AMRSR Approach

 
Overview of AMRSR
 

 
Hierarchical Attention-based Similarity
 
Paper

Citation

Marco Pesavento, Marco Volino, and Adrian Hilton, "Attention-based Multi-Reference Learning for Image Super-Resolution", IEEE International Conference on Computer Vision (ICCV), 2021.

Bibtex
@misc{pesavento2021attentionbased,
      title={Attention-based Multi-Reference Learning for Image Super-Resolution}, 
      author={Marco Pesavento and Marco Volino and Adrian Hilton},
      year={2021},
      eprint={2108.13697},
      archivePrefix={arXiv},
      primaryClass={cs.CV}

}
        
Acknowledgement

This research was supported by UKRI EPSRC Platform Grant EP/P022529/1.