The regression problem of fitting a "smooth", discrete curve to data points on a Riemannian manifold is formulated here as an unconstrained, finite-dimensional optimization problem. Smoothness of a discrete curve, seen as a sequence of close points on the manifold, is assessed and encouraged by a regularity term in the objective function. This term is built upon a generalization of finite differences to manifolds introduced in this work. Tuning of the balance between fitting and regularity (or energy-efficiency) is achieved by adjusting two parameters. The proposed framework is described in detail and is then applied to the special orthogonal group SO(n), i.e., the set of rotations in ℝn. A Riemannian version of the nonlinear conjugate gradient method is used to minimize the resulting objective. To this end, an explicit formula for the derivative of the matrix logarithm is derived, yielding explicit formulas for the gradient of the objective. Numerical results are presented and show that smooth curves in SO(n) can be obtained in a few hundred iterations with the proposed algorithm.