Critical landscape topology for optimization on the symplectic group

Optimization problems over compact Lie groups have been studied extensively due to their broad applications in linear programming and optimal control. This paper analyzes an optimization problem over a noncompact symplectic Lie group Sp(2N,ℝ), i.e., minimizing the Frobenius distance from a target symplectic transformation, which can be used to assess the fidelity function over dynamical transformations in classical mechanics and quantum optics. The topology of the set of critical points is proven to have a unique local minimum and a number of saddlepoint submanifolds, exhibiting the absence of local suboptima that may hinder the search for ultimate optimal solutions. Compared with those of previously studied problems on compact Lie groups, such as the orthogonal and unitary groups, the topology is more complicated due to the significant nonlinearity brought by the incompatibility of the Frobenius norm with the pseudo-Riemannian structure on the symplectic group.