Figures of Merit for General (Non-Uniform) Distribution Generators

A general distribution generator depends on the quality of the underlying BRNG. You can use several basic approaches to test general distribution generators.

Random number distributions are characterized by various measures: probability moments, central and absolute moments, quantiles, mode, scattering, skewness, and excess (kurtosis) coefficients, and so on. All the ordinary sample characteristics converge in probability to the corresponding measures of distribution when the sample size tends to infinity [Cram46]. Commonly, the characteristics based on the distribution moments are asymptotically normal with large sample sizes. Some classes of sample characteristics that are not based on sampling moments are also asymptotically normal, while others have quite different asymptotic behavior. When the limit probability distribution is known, you can build a statistical test to check whether a particular sample characteristic agrees with the corresponding measure of the distribution.

Of greatest practical value for simulation purposes are sample mean and variance that are the main properties of the distribution bias and scattering. All the VSL random number generators undergo testing for agreement between distribution sampling moments (mean and variance) and theoretical values calculated for various sample sizes and distribution parameters.

Another class of valuable tests aims to check how well the sample distribution function agrees with the theoretical one. The most important tests are:

  1. chi-square Pearson goodness-of-fit test (for discrete and continuous distributions)

  2. Kolmogorov-Smirnov goodness-of-fit test (for continuous distributions)

Every VSL distribution is tested with chi-square Pearson test over various sample sizes and distribution parameters.

It may be useful to transform the sequence that is being tested into one of the distributions, for example, into a uniform, normal, or multidimensional normal distribution. Then the transformed sequence is tested using a set of statistical tests that are specific for the distribution to which the sequence was transformed.

Tests that are based on simulation are Monte Carlo applications. Their choice is optional and should be made in accordance with the application field of the generator, the only requirement being an opportunity to verify the results obtained against the theoretical value. A good example of such a test application, which is used in checking the VSL generators for quality, is the self-avoiding random walk [Ziff98].

For more complete information about compiler optimizations, see our Optimization Notice.