When to stop searching?


Frequently, particularly when searching for work related to possibly novel research ideas I or others at FXPAL have had, it is not easy to determine when to stop searching. This dilemma comes up any time anyone is searching for something we are not sure exists.  After doing N searches, and finding nothing, how certain can we be that it isn’t there?

An unusual example of an existence search came up as I was doing background research for my review of N. David Mermin’s book Quantum Computer Science that was recently published in ACM SIGACT News. As part of the review, I wanted to give a sense for the extent that Mermin’s thoughts and writings have influenced scholarly and popular thought on quantum mechanics. I thought I remembered that he was the originator of the “Shut up and calculate” interpretation of quantum mechanics, but I wanted to fact check before putting it in my review. Would this search be a hard or easy one?

Almost the first thing I found was an article in which the author describes having done exactly this search. The tentative conclusion was that Mermin had been the originator. Particularly surprising was that the search was conducted by Mermin himself. If he was the originator, why did he need to do such a search? For witticisms, as with research, you want to claim to be the originator only if you were the first.  Mermin needed to find out if he was the victim of the Matthew Effect in which credit is assigned to the most eminent of plausible candidates or if he, by “unconscious plagiarism,” had unwittingly stolen from Feynman who was widely credited with the “Shut up and calculate” quote.

Mermin’s account is worth reading. I decided to review his book because I had so thoroughly enjoyed reading a handful of his papers and was glad for an excuse to read more of his entertaining and exceptionally lucid writing. I will only quote the very end of his account:

“So, dear reader, if you have evidence that Feynman really did say “shut up and calculate,” please send it to me. I will not be happy to receive it. I’d rather be a Matthew victim than a plagiarist. But I’d like to know the truth.”

“N. David Mermin (ndm4@cornell.edu) is a semi−retired professor of physics at Cornell University, where he nervously awaits your evidence that Feynman said it.”

Mermin had to decide when to stop searching. But his coda indicates that he was not entirely satisfied when he stopped. At this point he threw it open to the public to see what a world full of searchers could discover. His article “Could Feynman Have Said This?” was published in 2004. From my follow up search, it appears that while there are still plenty of loose attributions of this quote to Feynman, the consensus is that the quote is original to Mermin. I am satisfied that I can end my search here. Should I be?

Furthermore, with respect to another search I performed recently, should I conclude that I perpetrated a Matthew Effect on myself since no one has found a quote I thought I remembered even though my question asking about its origin has received over 1000 views?


  1. Great story! This reminds me of the information availability problem I propose for evaluating the effectiveness an interactive information retrieval system, e.g., http://thenoisychannel.com/2009/03/17/precision-and-recall/ It would be great to be able judge such systems in terms of the correctness of the outcome (does the user correctly conclude whether the information of interest is available?); user confidence in the outcome (is the user satisfied?), and efficiency (the user’s time or labor expenditure).

  2. […] I’ve had occasion to perform genealogical searches for my family as well as for others. Genealogical searches can be rewarding, but more often than not you wind up with nothing. So when starting on such searches one expects that little can be found; only one’s optimism determines whether to continue searching. […]

Comments are closed.