Estimation of roughness measurement bias originating from background subtraction

When measuring the roughness of rough surfaces, the limited sizes of scanned areas lead to its systematic underestimation. Levelling by polynomials and other filtering used in real-world processing of atomic force microscopy data increases this bias considerably. Here a framework is developed providing explicit expressions for the bias of squared mean square roughness in the case of levelling by fitting a model background function using linear least squares. The framework is then applied to polynomial levelling, for both one-dimensional and two-dimensional data processing, and basic models of surface autocorrelation function, Gaussian and exponential. Several other common scenarios are covered as well, including median levelling, intermediate Gaussian--exponential autocorrelation model and frequency space filtering. Application of the results to other quantities, such as Rq, Sq, Ra and~Sa is discussed. The results are summarized in overview plots covering a range of autocorrelation functions and polynomial degrees, which allow graphical estimation of the bias.