Mostly Calibrated

Prequential testing of a forecaster is known to be manipulable if the test must pass an informed forecaster for all possible true distributions. Stewart (2011) provides a non-manipulable prequential likelihood test that only fails an informed forecaster on a small, category I, set of distributions. We present a prequential test based on calibration that also fails the informed forecaster on at most a category I set of true distributions and is non-manipulable. Our construction sheds light on the relationship between likelihood and calibration with respect to the distributions they reject.