On the uncertainty in the regularized solution

Regularization is a fundamental technique in the processing of measurement data. These data are typically noisy and sparse, and in some cases, only a projection of the data on a low-dimensional subspace is available. Very often, direct analysis of measured data yields unphysical results, due to sensitivity to noise and neglect of the underlying distribution of the measured physical phenomena. Therefore, regularization is extensively applied in physics. One of the most common regularization methods was developed by a few researchers (most notably Tikhonov and Phillips) during the 1940s-1960s. In the early 1980s, Geman and Geman reformulated regularization in a Bayesian framework, in which every candidate function f (x) is assigned a probability, with the expectation of the random variable f (x₀) equal to the value of the regularized interpolating function at x₀. Taking this probabilistic approach 'to second order', the variance of this random variable, which reflects the amount of uncertainty inherent in the solution, is computed. When estimating physical phenomena, not only the estimated value is important, but also the uncertainty associated with it (for example, more measurements should yield a more certain solution). This review provides a formal definition of this uncertainty, explores some numerical issues with its computation, and extends it to compute the uncertainties of the integral and derivative, with the intriguing result that in some cases the uncertainty of the function and the derivative are highly uncorrelated-for example, there are cases in which the point at which the function is least reliable is the one at which the derivative is most reliable. Lastly, a related definition of optimal sampling is discussed. The relevance of optimal sampling to physics is that it offers a method to determine the location of the measuring devices which yields the most reliable restoration of the measured physical phenomenon.