Abstract :

Averaging is everywhere in modern AI (e.g., attention layers, ensembling, cross-validation, gradient aggregation) and every average implicitly answers a question: how should observations be weighted? The sample mean is efficient and minimax but fragile under heavy tails; robust estimators resist heavy tails but typically sacrifice efficiency. We show a soft-shrinkage calibration improves on both fronts: observations are smoothly downweighted by their distance from a pilot estimate, with the shrinkage scale calibrated so that the total rejected weight equals a prescribed budget. Any reasonable pilot is upgraded to sub-Gaussian concentration with the minimax constant $\sqrt{2}$ under finite variance alone, and the budget doubles as an exact breakdown point. The mechanism trades variance for bias, and the bias can be quantified exactly. This yields a second-order theory establishing an MSE expansion that identifies when shrinkage beats the sample mean, Berry–Esseen and moderate deviation approximations, and asymptotically exact confidence intervals. Beyond the theory, the results give concrete guidance for designing mean estimators in statistics and AI pipelines, and bridge modern sub-Gaussian estimation with classical robust statistics. This is joint work with Antônio Catão.