Logomacy
Between logomachy and logomancy . . .

Transterrestrial Musings

Thoughts On Objectivity

In both science, and journalism.

The notion that journalists are, or should be, or can be “objective” is perhaps the profession’s most fatal conceit. As Virginia Postrel says, what’s important is to be fair, something that they often don’t even attempt, as demonstrated by CNN and its performance in the debates.

I’m dubious. Even granting that “objective” is something of a term of art among journalists that doesn’t quite correspond to what a philosopher or scientist might mean in terms of attempting to avoid prejudice, bias, or wishful thinking, I don’t see how you can aim at fairness without first being able to assess what parts of your reporting might be unfair. And to do that, it seems to me you have to try to be objective…unless you’re just going to reduce everything to a procedure, as in “he said; she said” journalism.

From my point of view, the problem isn’t that journalists try to be objective where they should be trying to be fair–it’s that they’re so damn bad at objectivity. And it doesn’t reassure me that fairness over objectivity would be an improvement when the biggest critics of objectivity as a journalistic goal (e.g. Chomsky) want to downplay it precisely so they can hide their biases and better achieve their agendas. “Fake but accurate” is exactly what that approach is trying to legitimize. It is unfair that the journalist can’t present what he knows to be true based on his expert judgment, just because there’s no actual “objective” evidence. But because journalistic standards still require objectivity, he supplies fake evidence (and maybe even believes it to be true because of his biases), and with luck gets caught out. I say that if you believe that the journalist is obligated to provide the actual documents for other people to examine, and not just assert that they exist, you believe in objectivity not fairness; you believe that there is a truth of the matter that can be gotten at through examination of the evidence*, and not just a requirement to announce your biases.

Rand Simberg’s post above is in reference to a Virginia Postrel post on Objectivity. I haven’t read the book, but to infer anything about the appropriateness of objectivity as an epistemic virtue from a discussion of its history is to commit the genetic fallacy. I’m not at all sure whether the Daston and Gallison, the authors of Objectivity, would agree with Postrel’s take-away that “Real objectivity would turn the journalist into a C-Span camera, simply recording data without any sort of selection or pattern-making,” but I am sure that it is a core epistemic virtue for journalists to start by simply recording the data without any sort of selection or pattern-making. As the folks at Language Log have demonstrated over and over and over again, if you want the truth you have first accurately record what really was said. That doesn’t mean that you end there, and the journalist’s job is just to faithfully transcribe and then print it–but it has to start there.

* if it can be gotten at at all…

Marginal Revolution: Why Most Published Research Findings are False

There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims. However, this should not be surprising. It can be proven that most claimed research findings are false. - John Ioannidis

The argument is from a paper by John Ionnidis, but Alex Tabarrok gives a much easier to read analysis of the fairly simply Bayesian reasoning behind it. Essentially, this is the classic problem of false positives vs. true positives when the condition being tested for is rare in the population (e.g. presence of AIDs in non-high-risk groups, or in this case the truth of a hypothesis).

It might be tempting to argue that the case of a hypothesis under test being true isn’t typically as bad as the general assumptions being made to drive the argument, since the researchers presumably have some thought or intuition that drives them to pick a particular hypothesis to test (they’re not just throwing darts at a board), but consider that works both ways. Despite the common complaint that this or that study is “just another case of science proving what everybody already knows (and so a waste of money)”, I suspect very few researchers deliberately pick hypotheses that are widely believed to be true, particularly if there’s a lot of evidence and research backing up that belief. That’s not, generally speaking, believed to be the way to advance the frontiers of scientific knowledge. But in that case, the sample is biased in the other direction–a random hypothesis to test would include already-known-to-be-true hypotheses in the same proportion that they occur in the population of all hypotheses, so the hypotheses actually attracting attention are less likely to be true than random chance would dictate. Whether the scientist’s intuition towards selecting true hypothesis is a bigger bias than the elimination of all the ones believed to be true is something you can’t really be sure of, so I’d be really cautious about asserting that P(hypothesis is true) must be a lot better than Ionaddis’ calculations allow for.