Characterization of the Complexity of Computing the Minimum Mean Square Error of Causal Prediction

This paper investigates the complexity of computing the minimum mean square prediction error for wide-sense stationary stochastic processes. It is shown that if the spectral density of the stationary process is a strictly positive, computable continuous function then the minimum mean square error (MMSE) is always a computable number. Nevertheless, we also show that the computation of the MMSE is a #P1 complete problem on the set of strictly positive, polynomial-time computable, continuous spectral densities. This means that if, as widely assumed, FP1 ≠ #P1, then there exist strictly positive, polynomial-time computable continuous spectral densities for which the computation of the MMSE is not polynomial-time computable. These results show in particular that under the widely accepted assumptions of complexity theory, the computation of the MMSE is generally much harder than an NP1 complete problem.