Uncertainty

"'It lacks now three minutes of the half-hour,' shouted the captain, as
he gave me the longitude and the time. I admired the businesslike air
of the /Olympia/; but I have the feeling still that the captain was just
a little too precise in his reckoning. That may be all well enough,
however, where there is plenty of sea-room. But over-confidence, I
believe, was the cause of the disaster to the liner /Atlantic/, and many
more like her. The captain knew too well where he was."

Captain Joshua Slocum, in /Sailing Alone Around the World/, notes a
problem that we also address in the social sciences--dealing with
uncertainty. False precision and false accuracy may not sink any ships
in the Social Sciences, but one has only to look to the "shock
treatment" approach of Sachs and others (now renounced by Sachs) to see
that, when Economists and other Social Scientists are too sure of their
models, bad things can happen--bad things that can lead to a great deal
of suffering. That is, of course, when governments decide to listen to us!

In quantitative research, there are some fairly well developed protocols
for addressing issues of uncertainty (one example is the practice of
reporting confidence intervals and p-values). There are some
potentially very dangerous problems, however, in statistics when
analysts can produce a large number of regression results at very low
cost. We can simply choose the results we like, when probability favors
a particular model. It is easy, in many cases, to run 100 regression
models and report the one for which the results fit our theory. We
conveniently ignore, however the other 99 models, which would make our
95% confidence intervals look a bit questionable!

Of course, as many quantitative researchers have pointed out, these
issues are also a problem in qualitative research, where protocols for
reporting uncertainty have not developed in the same way as in
quantitative analysis. While respected scholars such as King, Keohane,
and Verba suggest estimating uncertainty in qualitative research, it is
unclear to me how I can accurately depict uncertainty without providing
a great deal of information about the process of doing research and how
I drew a particular set of conclusions.

This problem may explain the reason that I seem to observe a correlation
between the variable "historical institutionalism" (a dummy variable
coded 1 for Steinmo, Thelen, etc...) and the variable "length of
published manuscript."