BAM: a balanced attention mechanism to optimize single image super-resolution
Abstract
Recovering texture information from the aliasing regions has always been a major challenge for single image super-resolution (SISR) task. These regions are often submerged in noise so that we have to restore texture details while suppressing noise. To address this issue, we propose an efficient Balanced Attention Mechanism (BAM), which consists of Avgpool Channel Attention Module (ACAM) and Maxpool Spatial Attention Module (MSAM) in parallel. ACAM is designed to suppress extreme noise in the large-scale feature maps, while MSAM preserves high-frequency texture details. Thanks to the parallel structure, these two modules not only conduct self-optimization, but also mutual optimization to obtain the balance of noise reduction and high-frequency texture restoration during the back propagation process, and the parallel structure makes the inference faster. To verify the effectiveness and robustness of BAM, we applied it to 10 state-of-the-art SISR networks. The results demonstrate that BAM can efficiently improve the networks' performance, and for those originally with attention mechanism, the substitution with BAM further reduces the amount of parameters and increases the inference speed. Information multi-distillation network (IMDN), a representative lightweight SISR network with attention, when the input image size is 200 × 200, the FPS of proposed IMDN-BAM precedes IMDN {8.1%, 8.7%, 8.8%} under the three SR magnifications of × 2, × 3, × 4, respectively. Densely residual Laplacian network (DRLN), a representative heavyweight SISR network with attention, when the scale is 60 × 60, the proposed DRLN-BAM is {11.0%, 8.8%, 10.1%} faster than DRLN under × 2, × 3, × 4. Moreover, we present a dataset with rich texture aliasing regions in real scenes, named realSR7. Experiments prove that BAM achieves better super-resolution results on the aliasing area.
Additional Information
© 2022 Springer Nature. Received 25 April 2022. Accepted 22 June 2022. Published 26 July 2022. The authors would like to thank the Associate Editor and the Reviewers for their constructive comments. This work is supported by OPPO Research Institute. And this work is pre-print at https://arxiv.org/abs/2104.07566 Availability of code and data. The source code is released at https://github.com/dandingbudanding/BAM.Attached Files
Submitted - 2104.07566.pdf
Files
Name | Size | Download all |
---|---|---|
md5:29dd5d995bbd27c5905e4a6a271e6847
|
15.7 MB | Preview Download |
Additional details
- Alternative title
- BAM: A Lightweight and Efficient Balanced Attention Mechanism for Single Image Super Resolution
- Eprint ID
- 115851
- Resolver ID
- CaltechAUTHORS:20220726-997394000
- OPPO Research Institute
- Created
-
2022-07-27Created from EPrint's datestamp field
- Updated
-
2022-07-27Created from EPrint's last_modified field