For simplicity, we will assume we are estimating θ, that is, g ( θ ) = θ.

To diminish the shrinkage at the extremes, the loss function (centered around the overall group location) must penalize errors far from the overall location more heavily than the square loss. We also need an origin M to anchor the weighting function w ( θ i ). This origin M might be chosen subjectively, using prior knowledge and experience, or empirically, and could be interpreted as the mean of the θ i ’s.

Calculation yields the optimal estimator in (1.1) as δ i e x p ( X ) = a [ b ( μ 1 i − r v ) − c ] + a ′ [ c ′ + ( 1 − b ′ ) ( μ 1 i + r v ) ] a b + a ′ ( 1 − b ′ ) , where a = exp ( r ( M − μ 1 i ) + v r 2 / 2 ) ), b = Φ ( ( M − ( μ 1 i − r v ) v ), c = v ϕ ( M − ( μ 1 i − r v ) v ), a ′ = exp ( r ( μ 1 i − θ ˜ ) + v r 2 / 2 ) ), b ′ = Φ ( θ ˜ − ( μ 1 i + v r ) v ), c ′ = v ϕ ( θ ˜ − ( μ 1 i + r v ) v ), and μ 1 i , v are the posterior mean and variance, respectively, of a Normal likelihood with a Normal prior.

This and other loss functions can lead to tractable results, but it is more convenient for the purposes of the present paper to work with losses that also have a direct interpretation in terms of heavy tailed (robust) priors, which penalize discrepancies more heavily and are naturally scaled.

Figure 1 shows the Cauchy over Gaussian loss, being quite close to a square loss around zero but growing fast without bound for “exceptional” values.

In words, the optimal estimator under the Robust Cauchy over Gaussian penalty (and Gaussian prior) is the posterior expectation under a Cauchy Prior, since with square loss, the optimal estimator is the posterior expectation.

So, a bridge has been established between Robust Losses and Robust Priors through equation (1.1). We call it comprehensive Robustness, the use of Robust Loss Functions, or the use of Robust (heavy tailed) Priors. For convenience, now we go to the “Robust Prior Route”.

In what follows we try different models, in two methods: Empirical Bayes and Fully Robust Bayes, trying to solve, or at least alleviate, the Clemente problem, that is, the lack of robustness of the exponential family models with conjugate (light-tailed) priors and squared loss function.

In this section, we motivate briefly the use of heavy tailed priors as a tool for robustness. Back in the situation of subsection 1.2, let us make the usual assumption of a Normal Likelihood and Prior: (3.1) X i ∼ Normal ( μ i , 1 ) , i = 1 , … , 18 (3.2) μ i ∼ Normal ( M , σ 0 2 ) , where ( M , σ 0 2 ) has been assigned, for instance via an Empirical Bayesian method as in [7] or using previous year batting averages (for 1969, the global batting average was 0.248, see for example http://www.baseball-reference.com/leagues/MLB/1969.shtml). These assumptions, coupled with square error loss, lead to the posterior conditional expectation as the optimal estimator, which can be written as: (3.3) E ( μ i | x i ) = x i + 1 1 + σ 0 2 ( M − x i ) .

It is convenient here to define shrinkage as | E ( μ i | x i ) − x i |. There are several ways to analyze the lack of robustness of (3.3), but the one most relevant here is: all batters, whether exceptional or average, are shifted to the mean of the means M by a fixed proportion c = 1 / ( 1 + σ 0 2 ), so the shrinkage is c · | M − x i |. That is the “Clemente Problem”. Notice that increasing the prior variance σ 0 2 is not a fix to the problem: it would certainly reduce the shrinkage, but to all batters in equal proportion, even to those who are not exceptional. We are not proposing an indiscriminate reduction of the shrinkage, but rather a differential shrinkage. The posterior mean (3.3) is myopic to the exception. Now (3.3) is the logical consequence of the assumptions. Thus, the only logical way to change it is to change the assumptions. We could change the loss, but equivalently, we change the tail behavior of the priors: using flatter tails gives Bayes Theorem the input that the exceptional is more likely, and probabilistic (not ad-hoc impositions) coherence then acts as a “robustifier” of the estimation.

Our first alternative is a Double Exponential prior: (3.4) μ ∼ DE ( M , ν 0 ) = 1 ν 0 2 exp ( − ( 2 ) ν 0 | μ − M | ) , where ν 0 = ( 2 ) σ 0 log ( 2 ) Φ − 1 ( 0.75 ), to match the quartiles of the Normal prior. Finally for even heavier tails we explore with a Cauchy prior. (3.5) μ ∼ C ( M , γ 0 ) = 1 π γ 0 1 1 + ( μ − M ) 2 / γ 0 2 , where γ 0 = σ 0 Φ − 1 ( 0.75 ), again to match Normal quartiles.

For exact and approximate results with these, and other priors, see for example [17], but Figures 2 and 3 tell the story. In Figure 2 the observation is kept fixed at zero and the prior location is moved to create a conflict between one data point and a prior. With a Normal prior the shrinkage grows linearly without bound. The other two priors yield robust estimators. For a Double Exponential prior, the posterior expectation becomes essentially a ‘limited translation estimator” in Efron and Morris’ terminology. The influence of the overall mean M is bounded and monotonic. Finally, for the Cauchy prior, the posterior expectation is not monotonic in the conflict between the MLE and the General Mean. Furthermore, the prior is progressively discarded in favor of the MLE, as the general mean and MLE diverge. It is also quite interesting that the shrinkage of the two robust priors is almost the same close to the center, that is, around the general average M, and actually they give more shrinkage near the center than the Normal.

In Figure 3 we are showing the actual values of the transformed data X i , i = 1 , … , 18 on the x axis. The prior location M is fixed at the sample overall average. Around the middle-ground the three estimators are very close together. But at some point, the robust estimators separate from the Normal, in the direction of the MLE, being the adjustment towards the MLE of the Cauchy somewhat stronger than that of the Double-Exponential.

We will consider two types of strategies, Empirical Bayes and Full Bayesian strategies.

All proposed models were fitted using Stan (version 2.32) [19] through package rstan (version 2.37.2) [20]. Table 2 shows the batting averages predicted for each of the models, and in Figure 5 we display the milder shrinkage of the extremes using robust priors, particularly Cauchy priors, when compared to that of Model 1 and Model 4.

Robustifying the priors pays dividends twice: the relative shrinkage of the extremes is lower and, at the same time, the error of prediction is diminished up to 7 % for Models 6 and 7, which use the Cauchy-Scaled Beta2 and the Cauchy-Scale2 Beta 2 priors respectively. Model 3, which incorporates the Cauchy prior has, for example, a square prediction error 5% lower than Model 1, and predicts for Clemente a more respectful 0.314 average, much higher than the 0.290 from Model 1. Something similar may be said for Model 6 and Model 7. On the other hand, the price paid seems less than modest: computational tools as approximations and MCMC algorithms that make the computations routine are now available.

Thus by a very modest cost in computation, the “robustified” model has achieved both goals, decreasing the MSE and solving or at least alleviating the Clemente problem.

In terms of alternative approaches of Statistics that merge direct and indirect evidence, the difference between (sensible and objective versions) of Empirical and Fully Bayes Hierarchical Modeling is relatively small as compared with the difference between heavy and light tail priors.

In general, the Clemente problem is closely related to the implicit dogmatism inherent, not in Bayes in general, but in “Conjugate Bayes with Square Loss”. The way out seems to be: either an Empirical or Fully Bayesian Hierarchical Model, but making emphasis on Robustness.

In many contexts, Bayesian models are compared using methods focused on the predictive performance in cross-validation using tools as Watanabe-Akaike Information Criterion [24] and Leave One Out cross validation (LOO-CV) criterion [22]. WAIC and LOO were calculated using R-package loo [23]. Table 3 shows the results

According to Table 3, these two popular methods have little discriminatory power between models but slightly prefer conjugate ones. This is in contrast to a prediction perspective. The 80 % intervals of the Robust Models 3, 6 and 7, cover the Clemente Data but not the others (see Figure 6). It is striking that the best models in terms of WAIC and LOO are the poorest in prediction coverage. Coverage is a main objective of this article, as a consequence of structured Bayesian modeling. In summary, the best predictive models (in terms of coverage and in terms of Mean Squared Errors) are robust models, based on the Cauchy and Scaled-Beta 2 distributions. This is the case whether from an Empirical Bayes or Full Bayesian methodology. On the other hand, methods for model selection like WAIC and LOO performed badly in prediction and coverage.

It can be argued about the great relevance of Objective Robust Bayesian Analysis (ORBA). To put this article in perspective, we should mention some contributions. The first, although its approach is subjective, is [1], which uses the theory of Regularly Varying (RV) functions to assess robustness for general location and scale parameters. The second is [11], where the Generalized Polynomial Tails Comparison (GPTC) Theorem is proved, and the properties of specific robust priors are analyzed. In [15], the relationship between RV and GPTC is established. On the other hand, [5] introduced the “Horseshoe Estimator,” which obeys certain desiderata convenient for robust analysis in hierarchical models.

In the present article, we add the following insights: 1) We established a bridge between robust losses and robust priors, by weight ratios times the square loss, using the expression (1.1) to compute the optimal. Example 2 provides a specific example of how to assess a robust loss through the ratio of a flat-tailed prior over a Gaussian. The theoretical duality between losses and priors has been mentioned before, for example, by [2, p. 161], but we also search for the consequences of looking at robust procedures through the glass of robust penalties. Furthermore, note that the connection between priors and losses is done through an empirical Bayes component, since the loss is centered and scaled through an EB reasoning. 2) We employ systematically here a robust prior for scales, the Scaled Beta2, which is a convenient alternative to the overused Inverted-Gamma prior. Furthermore, we show the Cauchy-Scaled Beta2 prior, which is an explicit objective prior that obeys the desiderata of a “Horseshoe” prior, as well as the Cauchy-Scaled Beta2 prior with general closed form marginal distributions for location parameters. 3) We illustrate, using a classical data set, that it is possible to alleviate the “Clemente problem” and at the same time reduce the mean square error of prediction as compared with non-robust conjugate approaches and with the James-Stein estimator. 4) We illustrate that the differences between sensible versions of Empirical Bayes and Objective Bayes are relatively small in practice. Much more important is the difference between robust Bayes and conjugate Bayes. This, coupled with the fact that Robust Bayes is, in general, less dogmatic, in the sense that in conflict prior information is discarded or at least with limited influence, makes Objective Robust Bayes a strong candidate for statistical synthesis.

The scope of applications of Robust Objective Procedures is wide-ranging and includes any problem calling for the use of hierarchical models. Just to mention a few, in the analysis of population dynamics for the orchid genus Caladenia presented in [21], hierarchical models based on vague conjugate priors were used, but although the model selection procedures clearly pointed towards a hierarchical model as the selected one, they gave results that were not biologically sound, probably due to excessive shrinkage toward the species with more data. Another example is the unfair “pulling down” of hitherto perfect-scoring hospitals in hospital profiling, simply because a couple of hospitals have poor performance [see 14]. “Shrinkage is a good thing, but non-robust-conjugate methods yield too much of a good thing”.