Prime Focus Spectrograph (PFS) for Subaru Telescope: Recovery of the VPHG efficiency

The predicted efficiency of the Prime Focus Spectrograph (PFS) for the Subaru telescope at Mauna Kea took a serious hit at the discovery of an error in the mounting of its volume phase holographic gratings (VPHG). Alerted by unexpected jumps in spectral flux between the blue and red channels in the first two spectrograph modules as they became available on sky, inspection of the gratings as mounted into the fourth and last module, still present in the lab, confirmed that all its gratings - three low-resolution (blue, red, NIR) gratings and a medium resolution grating and prism assembly (grism) - were indeed all mounted upside down. In this paper, after reporting on the observations leading to this discovery, we describe the corrective actions taken, illustrating by on-sky spectra full recovery of performance. Finally, we discuss the causes for this potentially catastrophic error, with an emphasis on the difficulty of ensuring verification at all levels (preliminary and final design, procurement, manufacturing, and AIT phases) for the giant instruments we are currently building and designing. We provide guidelines for future instrument designers in order to minimize the risk of such flaws happening again.