TY - GEN
T1 - Deep multispectral painting reproduction via multi-layer, custom-ink printing
AU - Shi, Liang
AU - Babaei, Vahid
AU - Kim, Changil
AU - Foshey, Michael
AU - Hu, Yuanming
AU - Sitthi-Amorn, Pitchaya
AU - Rusinkiewicz, Szymon
AU - Matusik, Wojciech
N1 - Publisher Copyright:
© 2018 Copyright held by the owner/author(s). Publication rights licensed to Association for Computing Machinery.
PY - 2018/12/4
Y1 - 2018/12/4
N2 - We propose a workflow for spectral reproduction of paintings, which captures a painting's spectral color, invariant to illumination, and reproduces it using multi-material 3D printing. We take advantage of the current 3D printers' capabilities of combining highly concentrated inks with a large number of layers, to expand the spectral gamut of a set of inks. We use a data-driven method to both predict the spectrum of a printed ink stack and optimize for the stack layout that best matches a target spectrum. This bidirectional mapping is modeled using a pair of neural networks, which are optimized through a problem-specific multi-objective loss function. Our loss function helps find the best possible ink layout resulting in the balance between spectral reproduction and colorimetric accuracy under a multitude of illuminants. In addition, we introduce a novel spectral vector error diffusion algorithm based on combining color contoning and halftoning, which simultaneously solves the layout discretization and color quantization problems, accurately and efficiently. Our workflow outperforms the state-of-the-art models for spectral prediction and layout optimization. We demonstrate reproduction of a number of real paintings and historically important pigments using our prototype implementation that uses 10 custom inks with varying spectra and a resin-based 3D printer.
AB - We propose a workflow for spectral reproduction of paintings, which captures a painting's spectral color, invariant to illumination, and reproduces it using multi-material 3D printing. We take advantage of the current 3D printers' capabilities of combining highly concentrated inks with a large number of layers, to expand the spectral gamut of a set of inks. We use a data-driven method to both predict the spectrum of a printed ink stack and optimize for the stack layout that best matches a target spectrum. This bidirectional mapping is modeled using a pair of neural networks, which are optimized through a problem-specific multi-objective loss function. Our loss function helps find the best possible ink layout resulting in the balance between spectral reproduction and colorimetric accuracy under a multitude of illuminants. In addition, we introduce a novel spectral vector error diffusion algorithm based on combining color contoning and halftoning, which simultaneously solves the layout discretization and color quantization problems, accurately and efficiently. Our workflow outperforms the state-of-the-art models for spectral prediction and layout optimization. We demonstrate reproduction of a number of real paintings and historically important pigments using our prototype implementation that uses 10 custom inks with varying spectra and a resin-based 3D printer.
KW - 3d printing
KW - Multi-spectral imaging
KW - Spectral reproduction
UR - http://www.scopus.com/inward/record.url?scp=85066105607&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85066105607&partnerID=8YFLogxK
U2 - 10.1145/3272127.3275057
DO - 10.1145/3272127.3275057
M3 - Conference contribution
AN - SCOPUS:85066105607
T3 - SIGGRAPH Asia 2018 Technical Papers, SIGGRAPH Asia 2018
BT - SIGGRAPH Asia 2018 Technical Papers, SIGGRAPH Asia 2018
PB - Association for Computing Machinery, Inc
T2 - SIGGRAPH Asia 2018 Technical Papers - International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH Asia 2018
Y2 - 4 December 2018 through 7 December 2018
ER -