In conversation over the week-end at the Los Angeles Times Festival of Books, American virologist David Baltimore, who won the Nobel Prize for Medicine in 1975 for his work on the genetic mechanisms of viruses, commented that over the years (and especially while he was president of CalTech) he had received many manuscripts claiming to have solved some great scientific problem or to have overthrown the existing scientific paradigm to provide some grand theory of everything. Most prominent scientists have drawers full of similar submissions, almost always from people who work alone and outside of the scientific community.
Unfortunately, none of these offerings has done anything remotely close to what was claimed, and Dr. Baltimore offered some fascinating insight into why he thinks that’s so. At its best, he noted, good science is a collaborative, community effort. On the other hand, crackpots work alone.
I suspect that this idea is as true in our business as it is in science generally. Working collaboratively and in community lessens the likelihood that our work will be fraught with the bias to which we are so prone and increases the likelihood that all analysis and any conclusions drawn therefrom will be questioned, checked and tested.
Of course, in this context “collaborative” must mean that there is a free and uninhibited exchange of ideas, viewpoints and conclusions. It most decidedly is not a bunch of people agreeing vehemently with the boss or those with a seal of approval from the boss. In both the scientific and the investment process, principled, data-driven questioning and even disagreement should not only be tolerated, it needs to be encouraged. Only then can some approximation of truth (or at least reality) be an expected outcome.
As Dr. Baltimore emphasized, “Science is about changing our understanding of something by investigating its behavior.” In this sense, investigating includes analysis, testing, reaching tentative conclusions and then repeating – perhaps again and again. If what we think cannot be substantiated by data, evidence and experience, even after the most rigorous attempts to falsify it, we have no business claiming it to be true or investing as if it were true, no matter how intuitive, elegant or sales-worthy it may be.
Addendum: This post came about out of a wonderful conversation with Tom Brakke, of the outstanding blog, The Research Puzzle, over lunch today. Thanks, Tom.
Above the Market 
Pingback: Live from the LA Times Festival of Books | Above the Market
The idea that isolation can magnify the bias of the individual has merit, but also explains why some scientists are driven to prove their theories.
Thanks for reading.
I would suggest that scientists are driven to prove their theories because that’s the way the scientific process works. Unevidenced assertions have no place in science — nor should they. Thus it can never be enough for a scientist to claim that something is true without support. Once a theory is proposed (and it can only be done upon sufficient evidence), others then try to confirm or falsify the findings. If the results are confirmed, the strength of the theory is advanced (the theory of special relativity is well-established in physics, for example). If the results are falsified and established as such, the theory must be discarded or adjusted and subjected to more testing. Unfortunately, we aren’t nearly so rigorous in many other areas of life (say, economics and public policy).
The challenge of researcher bias is so much more complex than you seem to grasp. A collaborative effort with repeated trial and error is as prone to a data mining error as the “crackpots” you are so willing to disparage. “Crackpots” are distributed randomly throughout society and are as likely to show up on a college faculty as anywhere else.
Let me assure you that the research process that you describe is extraordinarily susceptible to what is referred to as the “multiple comparison problem” in statistics. You address one of the most complex problems we find in society (the most critical issue for anyone doing research of any kind) with a couple of paragraphs. You made it clear that you want to keep the world of ideas limited to only the well-credentialed, but you have actually shed zero light on the subject of scientific thought or a productive research methodology.
Thanks for reading.
Two quick responses:
1. Over the week-end, Dr. Baltimore and Sean Carroll (the CalTech physicist) replied to a response similar to yours relating to how frequently a new scientific paradigm arose from a non-credentialed source since the advent of the institutionalization of science by asserting that it hasn’t happened. That made me wonder if perhaps the institution’s self-preservation “instinct” was impeding good science from sources outside the mainstream. My guess (without knowing and without researching) is “yes,” but I’d like to see the evidence either way (to the extent it is available). My focus was on collaboration rather than disputes between “ins” and “outs.” I see no reason to doubt that good collaboration would help the non-credentialed do better science.
2. I agree that collaborative work is prone to error too. But as I noted in the post, good collaboration within the research group and the work of the scientific community — if done well — can limit the risk and enhance the process. As the post notes, “in this context ‘collaborative’ must mean that there is a free and uninhibited exchange of ideas, viewpoints and conclusions. It most decidedly is not a bunch of people agreeing vehemently with the boss or those with a seal of approval from the boss. In both the scientific and the investment process, principled, data-driven questioning and even disagreement should not only be tolerated, it needs to be encouraged. Only then can some approximation of truth (or at least reality) be an expected outcome.”
I do appreciate your basic premise regarding the scientific process, and I would actually be surprised to find a truly important scientific discovery that came from a non-credentialed source. I just find it too easy to dismiss anyone because of their station in life, and a little mean-spirited as well. I am sure there are plenty of PhD’s that prefer to work alone.
Certainly anything worthwhile in the scientific realm must be subjected to peer review, and needs to pass independent falsifiability tests. I do believe, however, that anything truly worthwhile in the world of trading and investing does not make it into the public realm, and once it does it loses its value by definition. Ed Thorp (a credentialed guy who basically works alone) claims to have used his own version of Black-Scholes for enormous profit for years before Black and Scholes published it. Once it became common knowledge we got a more efficient option market, but the information edge for Thorp disappeared.
I doubt Rennaisance Capital or Bridgewater will be submitting their work for peer review anytime soon. But I guess that supports your basic premise for collaboration since they are both large groups of well-credentialed people. I just think that the subject of researcher bias is much too subtle and complex to be explained in anything shorter than an extensive research paper, and probably requires a full book.
I understand the attraction to working alone. But I don’t think it’s a coincidence that virtually every major scientist has a drawer full of “new paradigm” manuscripts that come from people they label as something like crackpots and that they virtually all work alone. Those who prefer to work alone had better develop a good series of mechanisms effectively to check their work. They should also allow for routine and rigorous informal peer review if they want to try to limit their errors and ultimately succeed.
The investing world is different in that maintaining one’s competitive advantage is so important. That’s why a series of internal mechanisms to foster challenge and debate is (in my view) so important. A solo investor way well succeed, but if I’m hiring someone to manage my money, a want a good, collaborative team. The probabilities will be better.
Thanks again for reading and participating.
Pingback: “I’m Joining a Cult!” (said nobody, ever) | Above the Market
Pingback: Using Buffers to Become a Better Investor