The optimal sampling frequency in a Sigma-Delta analog-to-digital converter with a fixed bitrate at the output is studied. We consider the mean squared error performance metric where the input signal statistics are known. Fixing the output bitrate introduces a trade-off between the sampling rate and the number of bits used to quantize each sample. That is, while increasing the sampling rate reduces the in-band quantization noise, it also reduces the number of bits available to quantize each sample and therefore increases the magnitude of the quantization noise. The optimal sampling rate is the result of the interplay between these two phenomena. In this work we analyze the sampling rate of a Sigma-Delta modulator of arbitrary order under the approximation that the quantization error behaves like additive white noise that is uncorrelated with the signal. We show that for a signal with a spectrum that is constant over its bandwidth, the optimal sampling rate is either the Nyquist rate or the maximal sampling rate corresponding to the output bitrate. The choice between the two is approximately a function of the Sigma-Delta system order and the bitrate per unit bandwidth.