TY - GEN
T1 - Multi-view Spectral Polarization Propagation for Video Glass Segmentation
AU - Qiao, Yu
AU - Dong, Bo
AU - Jin, Ao
AU - Fu, Yu
AU - Baek, Seung Hwan
AU - Heide, Felix
AU - Peers, Pieter
AU - Wei, Xiaopeng
AU - Yang, Xin
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - In this paper, we present the first polarization-guided video glass segmentation propagation solution (PGVS-Net) that can robustly and coherently propagate glass segmentation in RGB-P video sequences. By leveraging spatiotemporal polarization and color information, our method combines multi-view polarization cues and thus can alleviate the view dependence of single-input intensity variations on glass objects. We demonstrate that our model can outperform glass segmentation on RGB-only video sequences as well as produce more robust segmentation than per-frame RGB-P single-image segmentation methods. To train and validate PGVS-Net, we introduce a novel RGB-P Glass Video dataset (PGV-117) containing 117 video sequences of scenes captured with different types of camera paths, lighting conditions, dynamics, and glass types.
AB - In this paper, we present the first polarization-guided video glass segmentation propagation solution (PGVS-Net) that can robustly and coherently propagate glass segmentation in RGB-P video sequences. By leveraging spatiotemporal polarization and color information, our method combines multi-view polarization cues and thus can alleviate the view dependence of single-input intensity variations on glass objects. We demonstrate that our model can outperform glass segmentation on RGB-only video sequences as well as produce more robust segmentation than per-frame RGB-P single-image segmentation methods. To train and validate PGVS-Net, we introduce a novel RGB-P Glass Video dataset (PGV-117) containing 117 video sequences of scenes captured with different types of camera paths, lighting conditions, dynamics, and glass types.
UR - http://www.scopus.com/inward/record.url?scp=85188247878&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85188247878&partnerID=8YFLogxK
U2 - 10.1109/ICCV51070.2023.02122
DO - 10.1109/ICCV51070.2023.02122
M3 - Conference contribution
AN - SCOPUS:85188247878
T3 - Proceedings of the IEEE International Conference on Computer Vision
SP - 23161
EP - 23171
BT - Proceedings - 2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023
Y2 - 2 October 2023 through 6 October 2023
ER -