This paper presents the tale of a replication experiment. The main characters are operating systems, Hessians, scaling, double-peaked likelihoods, and the limits of PC computing. Some of these characters, especially the Hessians, looked scary at first, but turned out to be sheep in wolves' clothing. In other words, the story has a happy ending. To appreciate the twists and turns, we go back and start at the beginning. Once upon a time, indeed, in the June 2003 issue of the AER, B. D. McCullough

more »

... and H. D. Vinod (2003; "MV" hereafter) set out to test the AER replication policy. While many AER authors were invited to participate in this replication event, few answered the call. We did. MV singled out our cooperation and honoring of the AER replication policy. MV replicated the results in our 1999 AER paper (Shachar and Nalebuff, 1999). You might have expected that we would be happy. But we were not. MV were concerned not only with replication but also with reliability of nonlinear estimation procedures. Specifically, they were concerned that nonlinear solvers can produce inaccurate answers. They believe that this is a systemic problem with empirical research in economics. Thus, they proposed a four-step method to verify the solution from a nonlinear solver. Using data from our paper to illustrate their point they conclude (referring to our 1999 article as "SN"): [T]he problem posed by SN has completely exhausted the limits of PC computing, and more powerful computational methods will be needed to analyze this problem properly ... the SN problem, as posed, cannot be reliably solved on a PC.