Fundamental Limits for Iterated Function Optimization on Turing Machines

This paper studies the effective convergence of iterative methods for solving convex minimization problems using block Gauss-Seidel algorithms. It investigates whether it is always possible to algorithmically terminate the iteration in such a way that the outcome of the iterative algorithm satisfies any predefined error bound. It is shown that the answer is generally negative. Specifically, it is shown that even if a computable continuous function which is convex in each variable possesses computable minimizers, a block Gauss-Seidel iterative method might not be able to effectively compute any of these minimizers. This means that it is impossible to algorithmically terminate the iteration such that a given performance guarantee is satisfied. The paper discusses two reasons for this behavior and gives simple and concrete examples.