What You See and What You Don’t See: The Hidden Moments of a Probability Distribution

What You See and What You Don’t See: The Hidden Moments of a Probability Distribution

Empirical distributions have their in-sample maxima as natural censoring. We look at the "hidden tail", that is, the part of the distribution in excess of the maximum for a sample size of n. Using extreme value theory, we examine the properties of the hidden tail and calculate its moments of order p. The method is useful in showing how large a bias one can expect, for a given n, between the visible in-sample mean and the true statistical mean (or higher moments), which is considerable for α close to 1. Among other properties, we note that the "hidden" moment of order 0, that is, the exceedance probability for power law distributions, follows an exponential distribution and has for expectation 1 n regardless of the parametrization of the scale and tail index.

See also: https://www.youtube.com/watch?v=64WDwef5jKI&feature=youtu.be