Illumination guided domain adaptation object detection in thermal imagery

Large-scale RGB datasets have played vital role in the huge success of training deep learning models. As a result, in many image applications, such as object detection, they have helped deliver superior performance in processing images from traditional cameras. However, under adverse illumination conditions, the visible-light cameras tend to capture poor and low-quality images. This may severely deteriorate the final performance in object detection. In contrast, thermal infrared cameras are very robust in various lighting conditions and thus can well serve as an efficient replacement or supplement to the regular cameras. However, training a robust thermal object detector faces its own limitations caused by the lack of large-scale labeled thermal data. To rectify this problem, we propose to supply the small-scale thermal dataset by leveraging the large-scale visible dataset, so as to enhance the performance of object detection models in the thermal domain. This leads to a novel domain adaptation method for thermal object detection, which integrates the illumination-guided attention mechanism module and the Prototype-based Adaptation module. The first module takes advantage of illumination information to selectively focus on domain-invariant features, which are important for robust adaptation across different domains. The second module, on the other hand, performs both category-agnostic and category-specific adaptation to further enhance the model's adaptation ability. By leveraging the output of the attention mechanism module, the adaptation module can more effectively align the distribution between the visible and thermal domains, leading to better performance of detector in the thermal domain. Through extensive experiments, we have demonstrated that our approach achieves superior performance compared to state-of-the-art methods on both the KAIST and FLIR datasets. Our code is available at https://github.com/ZLab540/IATDA.