A statistical model is said to be calibrated if the expected value of the responses for a given mean estimate matches this mean estimate. Testing for calibration has only been considered recently in the literature and we propose a new approach based on calibration bands. Calibration bands denote a set of lower and upper bounds such that the probability that the true means lie simultaneously inside those bounds exceeds some given confidence level. Such bands were constructed by Yang-Barber (2019) for sub-Gaussian distributions. Dimitriadis et al. (2023) then introduced narrower bands for the Bernoulli distribution. We use the same idea in order to extend the construction to the entire exponential dispersion family that contains for example the binomial, Poisson, negative binomial, gamma and normal distributions. Moreover, we show that the obtained calibration bands allow us to construct various tests for calibration. As the construction of the bands does not rely on asymptotic results, we emphasize that our tests can be used for any sample size.

Based on a joint work with Mario Wuthrich and Selim Gatti (ETH Zurich)