Computability theory for multidimensional signal processing^☆ Boche et al. (2025).

This chapter studies the effective convergence of iterative methods for solving convex minimization problems using block Gauss–Seidel algorithms. It investigates whether it is always possible to algorithmically terminate the iteration in such a way that the outcome of the iterative algorithm satisfies any predefined error bound. It is shown that the answer is generally negative. Specifically, it is shown that even if a computable continuous function which is convex in each variable possesses computable minimizers, a block Gauss–Seidel iterative method might not be able to effectively compute any of these minimizers. This means that it is impossible to algorithmically terminate the iteration such that a given performance guarantee is satisfied. We discuss two reasons for this behavior. First, it might happen that certain steps in the Gauss–Seidel iteration cannot be effectively implemented on a digital computer. Second, all computable minimizers of the problem may not be reachable by the Gauss–Seidel method. Simple and concrete examples for both behaviors are provided. We also discuss some consequences of these results for statistical learning theory, the theory of large language models, and nested learning.