Relative frequencies of non-homogeneous Markov chains in simulated annealing and related algorithms

We consider a class of non-homogeneous Markov chains arising in simulated annealing and related stochastic search algorithms. Using only elementary first principles, we analyze the convergence and rate of convergence of the relative frequencies of visits to states in the Markov chain. We describe in detail three examples, including the standard simulated annealing algorithm, to show how our framework applies to specific stochastic search algorithms - these examples have not previously been recognized to be sufficiently similar to share common analytical grounds. Our analysis, though elementary, provides the strongest sample-path convergence results to date for simulated annealing type Markov chains. Our results serve to illustrate that by taking a purely sample-path view, surprisingly strong statements can be made using only relatively elementary tools.