Any method of description requires relatively arbitrary choices and conventions. Here we begin by choosing variables that have values for each individual in our study population: a variable Y (the target variable) and variables X 1 , … , X K (the forecasting variables). Inferential statistical theory has accustomed us to think of these choices as causal modeling, but when description is our goal, Y and the X k are simply what we want to describe. For whatever reason, we want to know how they are related in the study population.

Next we choose a family of algorithms ( P θ ) θ ∈ Θ , where each algorithm P θ uses the X k to give probability distributions for Y. The P θ are our forecasters, and the probability distributions they give are our forecasts.

We will want to limit the complexity of the P θ . In inferential statistics, simplicity is said to be a virtue because complex forecasters are likely to overfit — i.e., fail to generalize beyond the study populations. When are goal is description rather than inference, either because we are uninterested in other populations or because we cannot make assumptions that would justify inference to other populations, a more immediate virtue of simplicity is salient. Description is description only when it is simple enough to be understood.

Being familiar and conventional is another virtue of a descriptive forecasting family. Description requires convention, and it can only be communicated to those who know the convention.

Once the variables and the forecasting family are chosen, we can evaluate the θ according to their relative forecasting success within the study population. Let Θ D be the subset of Θ consisting of θ whose forecasts perform reasonably well in our judgment; we may call it the description range. Its elements are the descriptive forecasters of Y. Often we will be most interested in some particular aspect of the descriptive forecasters. When θ = ( μ , σ 2 ), for example, we might be interested in μ. In other cases, we might be interested in the difference Y A − Y B when values of the X k for two individuals A and B are given. In general, the range of h ( θ ) when θ ranges over Θ D is our description range for h ( θ ).

For each ordered pair ( θ 1 , θ 2 ) of elements of Θ, we pit θ 2 bet against θ 1 . Forecaster predicts Y with the distribution P θ 1 ; Skeptic predicts Y with the distribution P θ 2 . Skeptic has unit capital. Forecaster offers to sell Skeptic any non-negative payoff S ( Y ) for E θ 1 ( S ), his expected value for S. In deciding what nonnegative payoff S to buy, Skeptic can use his distribution P θ 2 in different ways, but when he uses Kelly betting he buys the payoff S given by (3.1) S ( Y ) : = P θ 2 ( Y ) / P θ 1 ( Y ) at the price Forecaster requires: E θ 1 P θ 2 ( Y ) P θ 1 ( Y ) , which is equal to 1 . This bet turns Skeptic’s initial unit capital into S ( y ) = P θ 2 ( y ) / P θ 1 ( y ). This ratio, the betting score, provides a measure of θ 2 ’s forecasting success relative to θ 1 .

For our descriptive purposes, Kelly betting can be considered one more convention. It does, however, have well known optimization properties for Skeptic when Skeptic is confident in Q as a forecast.9⁹

See [8]. For more on Kelly betting, see [18, Chapter 10] and [60]. For other roles Kelly betting can play in statistical theory, see [50] and [58].

The ratio P θ 2 ( y ) / P θ 1 ( y ) is familiar to statisticians under the name likelihood ratio. We will usually chose our forecasting family so that a unique maximum of P θ ( y ) always exists. The value of θ that achieves the maximum, say θ ˆ, is the maximum-likelihood estimate when y is observed, and the likelihood ratio L ( θ ) = P θ ( y ) P θ ˆ ( y ) is a number between zero and one that measures θ’s forecasting performance.

Using cutoffs suggested by R. A. Fisher in 1956,10¹⁰

See [22, p. 71] or page 75 of the posthumous third edition (1973).

For Fisher, the intervals were inferences. A number of other authors, beginning with A. W. F. Edwards [17] and Richard Royall [45], have adopted Fisher’s proposal to use L ( θ ) for inference. These authors have developed methods for computing, tabulating, and displaying L ( θ ), intervals of its values, and other summaries for a large variety of models. I will not attempt to review this very extensive work, but it is obviously an asset for the descriptive proposal I am making here.

Fisher called L ( θ ) the likelihood of θ. This name has endured for a century, and no mathematical statistician will be able to put it out of mind while reading the rest of this paper. Yet I will avoid it as much as possible, because its inferential connotation cannot be circumvented. On the other hand, I am using the familiar notation: θ ˆ, L ( θ ), and also l ( θ ) for ln ( L ( θ ) ).

Suppose we observe successive trials of an event, and each algorithm in our forecasting family has a fixed probability that it uses each time as its forecast. Formally, Θ = [ 0 , 1 ], and Forecaster θ always gives θ as its forecast.

If we observe 100 trials, and the event happens 70 times, then θ ˆ = 0.7, and L ( θ ) : = θ 0.7 70 1 − θ 0.3 30 . Our scheme for rating the forecasters yields these approximate description ranges:

Very good	0.64 < θ < 0.76
Good	0.61 < θ < 0.78
Satisfactory	0.59 < θ < 0.80

The forecaster θ = 1 / 2 may have been of particular interest, and we may want to emphasize that its performance was unsatisfactory.

Not surprisingly, Fisher’s categories are roughly consistent with inferential practice. The standard error of the maximum-likelihood estimate 0.7 is 0.046, suggesting a 95 % confidence interval of ( 0.61 , 0.79 ). Like this confidence interval, our description ranges do not contain 1 / 2. But unlike the confidence interval, the description ranges merely describe; they merely tell us which constant forecasts perform relatively well in the data. This does not involve attribution of independence in any sense to the trials themselves.

The calculation of error probabilities from statistical data was first made practical by Laplace’s central limit theorem, and the calculation was explained to statisticians by Joseph Fourier (1768–1830). Fourier had been an impassioned participant in the French revolution and an administrator under Napoleon. After the royalists regained power, a prominent royalist who had been Fourier’s student, Chabrol de Volvic, rescued him from impoverishment with an appointment to the Paris statistics bureau. This assignment left him time to perfect the theory of heat diffusion for which he is best known, but as part of his work at the statistics bureau, he published two marvelously clear essays on the use of probability in statistics, in 1826 and 1829. According to Bernard Bru, Marie-France Bru, and Olivier Bienaymé, these were the only works on mathematical probability read by statisticians in the early 19th century.11¹¹

See [9, p. 198]. The annual reports issued by the bureau during Fourier’s tenure list no editor on their cover pages. Fourier was no doubt primarily responsible for editing them, and I identify him as the editor in the references [23, 24].

To illustrate Laplace’s asymptotic theory, Fourier studied data on births and marriages gleaned from 18th-century parish and governmental records in Paris. He was particularly interested in the length of a masculine generation — the average time, for fathers of sons, from the father’s birth to the birth of his first son. On the basis of 505 cases, he estimated this average time to be 33.31 years. In his bureau’s report for 1829 [24, Table 64, p. 143ff], he gave the bounds on the estimate’s error shown in Table 1.

Laplace’s theory is applicable, of course, only if the 505 cases are a random sample, and they are not. They are a convenience sample, consisting of 18th-century fathers of sons in Paris for which the needed parish records could be found. This convenience sample is of interest, but Fourier’s inferential analysis of it is unjustified. A descriptive analysis is needed.

For description, we do not need Fourier’s implicit assumption that the 505 cases were a random sample. We need a conventional forecasting family. Let us use the most conventional family, the normal family with mean μ and variance  σ 2 . Here θ = ( μ , σ 2 ) and θ ˆ = ( y ‾ , s ) = ( 33.31 , 7.642 ), where y ‾ is the average of the 505 ages, and s is their standard deviation. Fourier did not report the data, but he reported the average y ‾ = 33.31 years, and we can calculate the standard deviation s = 7.642 years from the error probabilities he reported.

Writing l ( θ ) for ln ( L ( θ ) ), we have (3.3) l ( μ , σ 2 ) = n ln ( s ) − ln ( σ ) − s 2 + ( y ‾ − μ ) 2 2 σ 2 + 1 2 , where n = 505. The values of ( μ , σ 2 ) for which (3.3) exceeds ln ( 1 / 2 ) constitute the very good description range for ( μ , σ 2 ), those for which it exceeds ln ( 1 / 5 ) the good range, those for which it exceeds ln ( 1 / 15 ) the satisfactory range.

Following Fourier, we are interested only in description ranges for μ. So our question is what values of μ are included in these three description ranges. So we maximize (3.3) for each μ, by setting σ 2 equal to s 2 + ( y ‾ − μ ) 2 . This gives (3.4) l ( μ , s 2 + ( y ‾ − μ ) 2 ) = n ln ( s ) − 1 2 ln ( s 2 + ( y ‾ − μ ) 2 ) . This is greater than ln ( 1 / C ) when μ is in the interval (3.5) y ‾ ± s C 2 / n − 1 . Here we have obtained the log-likelihood (3.4) for the parameter of interest μ by maximizing over the unwanted parameter σ 2 . In inferential likelihood theory, the result of such a maximization is sometimes called a profile likelihood. The inferential use of profile likelihoods is hard to justify and sometimes misleading [45, p. 159], but their game-theoretic descriptive use is unobjectionable. We want to know how well each value of μ can perform in the betting competition. Its performance depends on the value of σ 2 it is paired with. We pair it with the value of σ 2 that enables it to do its best to see how well it can perform.

Table 2 uses (3.5) to calculate description ranges. Comparing it with Table 1, we see that the width of Fourier’s 95% confidence interval is between that of our good description range and our satisfactory description range. The relation as n increases between confidence intervals and the description ranges given by (3.5) has been studied by [10] and [42].

Some organizations in the United States have recently surveyed their employees about perceptions of discrimination. To avoid the complexities involved in real examples, consider the following fictional example.

An organization wants to know whether its employees of different genders and racial identities differ systematically in their perception of discrimination. Most of the employees respond to a survey asking whether they have suffered discrimination because of their gender or race. The employees saying yes are distributed as shown in Table 3.

According to the usual test for the difference between two proportions, the difference between the rows (male vs female) and the difference between the columns (BIPOC vs White) are both statistically significant at the 5% level. But the 20 percentage-point difference between BIPOC males and BIPOC females is not, as its standard error is 1 3 2 3 1 20 + 1 10 ≈ 0.18 = 18 percentage points . These simple significance tests seem informative. The differences declared statistically significant seem general enough to be regarded as features of the organization, but we hesitate to say this about the difference declared not statistically significant.

Yet the theory of significance testing does not fit the occasion. Have the individuals in the study (or their responses to the survey) been chosen at random from some larger population? Certainly not. For anyone who has been inside an organization long enough to see its employees come and go, seeing or guessing the reasons, the idea that they constitute a random sample is phantasmagoria. Nor can we agree that their responses are independent with respect to some “data-generating mechanism”. Many of them see the same media and talk with each other.

If we took the theory of significance testing seriously for this example, we would also worry about multiple testing. The 5% error rate we claim for our tests is valid under the theory’s assumptions only when we make just a single comparison. We have made three comparisons and might make more.

The theory’s assumptions are not met, and we have abused the theory. But there is a larger issue. The theory is irrelevant from the outset, because its goals are irrelevant. The organization did not undertake the survey in order to make inferences about a larger or a different population or about some data-generating mechanism. The organization wanted only to know about itself. It wanted to know how its employees’ perceptions vary with gender and race. This calls for a descriptive analysis.

For a descriptive analysis, we can use the obvious forecasting family, in which each cell in the 2 × 2 table has its own forecast: θ = ( θ bf , θ bm , θ wf , θ wm ) , where θ bf is the forecast that a BIPOC female will say yes to the survey, etc. According to the data in Table 3, θ ˆ = 8 10 , 12 20 , 20 50 , 20 120 , and L ( θ ) = θ bf 4 / 5 8 ( 1 − θ bf ) 1 / 5 2 θ bm 3 / 5 12 ( 1 − θ bm ) 2 / 5 8 θ wf 2 / 5 20 ( 1 − θ wf ) 3 / 5 30 θ wm 1 / 6 20 ( 1 − θ wm ) 5 / 6 100 .

We found earlier that the 20 percentage-point difference between BIPOC males and BIPOC females is not statistically significant. In this descriptive analysis, the question can be reframed this way: what differences between BIPOC males and BIPOC females within the study population are forecast by very good forecasters? We can answer the question by looking at all the θ = ( θ bf , θ bm , θ wf , θ wm ) that rank as very good forecasters by having a value of L ( θ ) greater than 1 / 2 and finding the range of their values for θ bf − θ bm . The range is from a little more than 0 to about 0.4. We can say that there are very good forecasters who give nearly the same forecasts for the two groups.

When the individuals responding to a yes-no survey are categorized in more than one way, or when other data is collected about them, we may prefer to use a more sophisticated forecasting family, such as logistic regression. The logic will remain the same. For particular interesting values of the forecasting variables, we can calculate the range of forecasts given by good forecasters. We can similarly calculate ranges for odds ratios. The computations involved are not trivial, but software environments adequate to the task would not be need to be more complex for the user than those that now use logistic regression for nominally inferential analyses.

The descriptive approach can be compared with the inferential approach used in 2016 by the University of Michigan’s Diversity, Equity & Inclusion Initiative. Michigan sought inferential legitimacy by using random samples. As they explained in their report on the faculty survey [41, p. 6],

Given the large faculty population at the University of Michigan, this study used a sample survey approach rather than a census of all faculty. A carefully selected sample, with randomization, allows researchers to make scientifically based inferences to the population as a whole.

In any case, the authors chose 1 , 500 out of 6 , 700 faculty members at random to complete the survey. The survey results were then analyzed using logistic regression, and a number of differences were observed to be statistically significant. It was found, for example, that female faculty were 130% more likely to feel discriminated against than male faculty (i.e., the odds ratio for a positive response to the question was equal to 2.3 and significantly different from 1). The results of the survey were clearly meaningful, but the inferential logic is problematic. As David A. Freedman [27] has shown, randomization probabilities do not justify logistic regression. Our descriptive theory is not affected by this problem and is just as applicable to a complete census as to a random or non-random sample.

One obvious alternative is to fit models — i.e., calculate descriptive statistics such as means, standard deviations, and regression coefficients — while de-emphasizing the question of their precision and refraining from calculating significance tests or confidence intervals. This has often been the practice in fields, such as geodesy, that have been more influenced by the Gaussian tradition than by the Laplacean tradition.13¹³

Marie-Françoise Jozeau has documented the competition of the two traditions in geodesy [36, 47].

In 2004, in his Regression Analysis: A Constructive Critique [5], the sociologist Richard A. Berk gave three cheers for a purely descriptive use of multiple regression: estimate the regression coefficients, but do not calculate p-values or confidence intervals. The book has been widely cited, but so far as I know few researchers have followed this advice. What use is a regression coefficient if you have no sense of its precision?

John Tukey’s advocacy of data analysis was another effort to shift attention from inference to description [15]. But Tukey’s distinction between exploratory data analysis and confirmatory data analysis undermines the effort to promote description to a goal in itself. The terminology suggests that we explore in order to discover what we will then try to confirm.

In 1995, Richard A. Berk and his colleagues Bruce Western and Robert E. Weiss proposed that the neo-Bayesian philosophy could justify statistical modeling for entire populations [7]. I have not been able to understand their argument.

Another notable effort to make inferential tools descriptive was mounted earlier by [25]. In the simplest cases, their proposal involves permuting residuals and interpreting a p-value as a measure of how unusual the actual data is in the population of alternative data thus generated. Along with many other mathematical statisticians, I was very intrigued by this proposal when it appeared, but I always found its intuition elusive.

The most common justification of using inferential tools with non-random samples is, of course, the argument that assumptions are never exactly satisfied. George Box’s slogan, “all models are wrong but some are useful” is evoked to justify calculating p-values and confidence intervals. This leaves unanswered, however, the question of what we are being asked to have confidence in. Any claim that the calculations are mere description is contradicted by the language being used.

I have proposed scoring relative success by the likelihood ratio P 1 ( y ) / P 2 ( y ). This is hardly a new idea; it goes back at least to Laplace. What this paper adds is a betting interpretation. The ratio is the factor by which algorithm 1 multiplies its capital betting against algorithm 2.

The ratio P 1 ( y ) / P 2 ( y ) can also be understood in terms of information theory. John L. Kelly Jr. entitled his 1956 paper “A new interpretation of information rate”; for him Kelly betting was merely an interesting sidelight to information theory. The connection has been elaborated using the idea of minimum description length [44, 30]. But these ideas too have failed to gain traction in the natural and social sciences, and we cannot expect that they will. As beautiful and simple as they are for mathematicians, the ideas are not part of our wider culture.

If we set the betting interpretation aside and change the scale by taking logarithms, P 1 ( y ) / P 2 ( y ) becomes ln ( P 1 ( y ) ) − ln ( P 2 ( y ) ). As this makes clear, we are comparing the two forecasts using log loss. There are many other loss functions. Why shouldn’t we give others equal attention?

The answer, of course, is that we are not setting the betting interpretation aside. A vast amount of work has been done, in decision theory and more recently in machine learning, on evaluating predictions using loss functions. But in most fields of natural and social science, this work has gained little to no traction. I think this is because the scales of measurement have no meaning to the scientists and their larger public. Even though the cutoffs and other conventions we must use will be relatively arbitrary, the idea of measuring relative success in prediction by success in betting and resistance against betting does have meaning for scientists and the publics with whom they want to communicate.