Statistical Inference

Statistical inference is the process of drawing conclusions from a piece of statistical analysis with a view to making recommendations for an organization, process, or policy. It is a process that requires both objective and subjective judgments, since statistics rarely indicate causation or provide definitive answers rather than indications. While the level of nuance such inference requires is understandable in an academic context, it does not always fit well in policy making or discourse for the public.

Statistical analysis is generally based on the concept of the confidence interval, or level of confidence in the results. A distribution of observed results is examined and compared to what would be expected. Calculations are then used to determine the level of significance of the difference between what was found and what was expected.

For general market research or academic research in business or management, a confidence interval of 95 percent is normally expected. This means that when a given result is declared to be significant, it can be assumed to be correct 95 percent of the time and incorrect for the remaining 5 percent. Researchers will most commonly conduct several different but related tests to try to reduce the likelihood of making a declaration that turns out to be false; for example, a questionnaire will have a number of questions about different aspects of a potential new product rather than just one. The level of confidence can be increased by adding more observations or respondents, but this must be balanced against the extra cost and time required, and uncertainty can never be eliminated completely. A confidence interval of 95 percent is a two-sigma approach, which means that it defines as nonsignificant any result that is within two standard deviations, or sigmas, from the mean. Even very rigorous testing, such as the search for the Higgs boson at the European Organization for Nuclear Research (CERN), which uses a five-sigma approach, will declare a false result once in every 3.5 million tests in which a significant result is observed.

Most forms of statistical analysis do not deal with causation. Cross-tabulations or correlation tests are used to indicate when an observed distribution of results is sufficiently far from what was expected to be declared significant, but such results do not actually explain what is happening. It is the responsibility of the researcher to interpret and provide all reasonable explanations for the results. In some cases, the results will be overwhelming and noncontroversial; for example, the relationship between certain demographic variations, such as level of income, ethnicity, and gender, and the propensity to vote for a particular political party is well established. In other cases, explaining the situation will involve much less concrete relationships and so will rely more on the conceptual framework that gave rise to the test in the first place and the degree of trust placed in the researcher.

On occasion, statistical analysis does address causation to some degree. Tests based on linear regression aim to explain a portion of overall change in a measurement through particular variables; changes in labor productivity explain a proportion of the overall change in the gross domestic product of a country, for example. A similar approach is used in the case of factor analysis. Another technique, structural equation modelling (SEM), is intended to provide an explanatory causal path leading from variables to final results, such as showing how emotional antecedent states in combination with environmental variables can determine consumer behavior. In these cases, inference focuses on the degree of confidence that can be placed in a particular explanation of a complex phenomenon.

—John Walsh, PhD