Oral argument in Lott v. Levitt
SteveLevittThe oral argument heard before the 7th Circuit Appeals Court can be found here.
Levitt's original correction letter.
Levitt's original correction letter.
Showing posts with label SteveLevitt. Show all posts
Showing posts with label SteveLevitt. Show all posts
More on "Dispute Over the Economics of File Sharing Intensifies"
Research SteveLevittDavid Glenn has a long discussion about the paper published in the Journal of Political Economy by Oberholzer-Gee and Koleman S. Strumpf on file sharing. Glenn has a useful and well-done discussion of the debate. For example, he notes:
As too often seems to be the way of academic debate from people in part of the academy, this has a personal attack on Stan Liebowitz:
I agree with Craig Newmark that Stan Liebowitz's concern that he couldn't redo their results and that they wouldn't share their data is enough to motivate many academics, including Liebowitz.
Other comments by CaveatBettor, Peter Klein, Josh Wright, and sivacracy.
Mr. Strumpf replies that most of Mr. Liebowitz's criticisms are trivial, even if correct. "Almost every point raised in Liebowitz's various pieces," he says, "involves incidental points which are not central to our conclusions."
Mr. Liebowitz, however, says his objections cast doubts over the entire study. Because he has not been able to scrutinize the OpenNap data at the heart of the study, his criticisms are largely aimed at a few non-OpenNap-based auxiliary tests that are presented at the end of the paper. But those tests are so weakly constructed, he says, that they call into question the validity of all the work.
One of the tests has to do with seasonal variations in record sales. "The number of file-sharing users in the United States drops 12 percent over the summer ... because college students are away from their high-speed Internet connections," Mr. Oberholzer-Gee and Mr. Strumpf write. Because of that seasonal dip, we might expect to see higher CD sales during the summer. But summer CD sales, as a proportion of the full year's sales, have not risen during the file-sharing era. Ergo, there doesn't seem to be much relationship between file sharing and CD sales.
That all sounds logical enough. But Mr. Liebowitz points out that one of the argument's premises—"the number of file-sharing users in the United States drops 12 percent over the summer"—is profoundly misleading.
The authors' citation is to a report on monthly file-sharing usage prepared by BigChampagne, a company that measures traffic on peer-to-peer networks. Mr. Liebowitz has a copy of the same report, which covers the period from August 2002 through May 2006. It's true that summer file sharing drops by an average of 12 percent (11.7 percent, to be precise) during the three summers—2003, 2004, 2005—covered in that report. But all of that effect comes from a severe drop in the summer of 2003, during a much-publicized wave of industry lawsuits against file sharers. During the summer of 2004, file sharing was flat, and during the summer of 2005, it actually rose slightly. So the test Mr. Oberholzer-Gee and Mr. Strumpf have set up—which is based on the ratio of summer-to-full-year CD sales—tells us nothing, Mr. Liebowitz says.
"If one my undergraduates did that, I would fail him," says Bruce D. McCullough, a professor of decision sciences at Drexel University who became interested in the dispute because he is a proponent of data transparency in economics publishing. "To take one decline, one flat, and one advance, and to suggest that it always goes down in the summer is just wrong." . . .
As too often seems to be the way of academic debate from people in part of the academy, this has a personal attack on Stan Liebowitz:
In an e-mail message to The Chronicle, Mr. Strumpf defends his work and suggests that Mr. Liebowitz's zeal stems from the fact that an academic center he directs, the Center for the Analysis of Property Rights and Innovation, receives grants from the Recording Industry Association of America and other commercial interests. "One might ask why Professor Liebowitz has remained so engrossed with our study," he writes.
I agree with Craig Newmark that Stan Liebowitz's concern that he couldn't redo their results and that they wouldn't share their data is enough to motivate many academics, including Liebowitz.
Other comments by CaveatBettor, Peter Klein, Josh Wright, and sivacracy.
The Controversy over how the Journal of Political Economy treated papers on Downloading Music
Research SteveLevittI have covered the controversy here. Stan Liebowitz has a follow up piece on the who debate here. His abstract notes:
Through a stroke of luck, a referee report in the review process at the JPE has been positively identified as the Oberholzer-Gee/Strumpf (O/S) response to my earlier comment. Regardless of the response's provenance, what counts is whether it solidly refuted my comment. This 'sequel' analyzes the O/S response. The O/S response only deals with four of the nine points discussed in my comment, leaving the five remaining critiques unchallenged. The conclusion of my review is that the O/S response fails as a defense of these four points and contains many of the same types of errors that marred their original paper. This sequel also discusses the history of this dispute including O/S' various reasons for not making their data available. Finally, this sequel provides full documentation on the JPE's decision not to publish the comment.
Did the Journal of the Political Economy make a serious mistake in accepting a paper?
Research SteveLevittJohn Palmer has an extremely lengthy excerpt from an investigative report on a paper on file sharing that was published by the JPE (for a previous post see here). This discussion involves a case where the editor was apparently told about severe data errors in the piece before publication, told that the authors were unwilling to share their data before publication and have still refused to do so (coming out with multiple conflicting reasons for not doing this), and questions about whether the rejection of a comment pointing out the data errors was done properly. When told that there was improper behavior with the data the editor's response was interpreted by Stan Liebowitz to mean this: "There appears to be no way within the profession to adjudicate such claims. In fact, the profession tends to operate as if all economists are truthful all the time."
Craig Newmark has his comments here. Peter Klein has comments here, who classifies this under the general category of "Academic Journal Fakery." Other posts here (Paul Walker) and here ("provocateurjim") also draw concerns over what happened in this case.
Craig Newmark has his comments here. Peter Klein has comments here, who classifies this under the general category of "Academic Journal Fakery." Other posts here (Paul Walker) and here ("provocateurjim") also draw concerns over what happened in this case.
Lott v. Levitt: The Appeal with all the briefs filed
SteveLevittA couple of weeks ago I posted the original brief and Levitt's response in my Appeal in the Levitt case. Here is my final reply brief.
The Levitt correction letter is available here.
The Levitt correction letter is available here.
Lott v. Levitt: The Appeal
SteveLevittThe settlement in the original lower court case resulted in Levitt's "correction letter" and allowed for me to appeal the decision on the first count. Here is a copy of my filing in the appeal and Levitt's response. This article on the Chronicle of Higher Education blog reviews the discussion, though I wish that it had mentioned the "e-mails," not just the "e-mail," and that the libel per quod had been brought up in the lower court, not just now. It would have been nice if the Chronicle reporter, David Glenn, had noted that the settlement with Levitt in the lower court preserved the right to appeal the first count.
The Problematic "famous file-sharing paper"
Research SteveLevittCraig Newmark has previously covered the problems with Felix Oberholzer-Gee and Koleman Strumpf's widely cited and discussed paper in the Journal of Political Economy entitled "The Effect of File Sharing on Record Sales: An Empirical Analysis." Stan Liebowitz had found numerous errors in the paper, but was not given access to their data. Unfortunately, under Steven Levitt's editorship at the JPE, this general type of claim has arisen previously. Anyway, Craig has a follow up posting on all this. Florenz Plassmann has translated an article from Handelsblatt, a leading German financial publication. This is what Craig posts from the article:
Given the huge coverage of this by the American press, it would be useful if someone in the American Press would write about this problem.
Thanks very much to Florenz Plassmann for translating this.
Recently, this paper has sparked a heated discussion. The relevance of the debate extends far beyond the paper in question. It questions the reliability of empirical studies in economics, and may ultimately challenge the way in which the crème de la crème of scientific journals deals with scientific evidence.
The key question is: how can a study that is based on secret data that nobody has double-checked be printed without close examination by one of the most prestigious economics journals? This is especially puzzling because the supplier of the data has a special interest in a certain result. The study of the two economists from Harvard and Kansas is based on proprietary data on music downloads, which the authors received from the file sharing services "MixmasterFlame" and "FlameNap."
. . . .
Liebowitz knew of the filesharing study before it was published because it had been circulated as a working paper. In his letter he told Levitt that, despite repeated requests, the authors did not provide him with an opportunity to check their results. Could he please use his influence as editor of the "JPE" to make such checks possible? Levitt declined to tell Handelsblatt whether he followed up on this request.
It appears that he did not. Even one year after publication, the authors still keep their data to themselves. Oberholzer-Gee told Handelsblatt that they had to sign an agreement not to share the data to get them from the file sharing service. The authors argued that they had to "protect their sources" and declined to provide Handelsblatt with either a copy of the agreement or the name of a reference at the file sharing service who could confirm their version.
Liebowitz pressed Levitt, the editor of the "JPE," to at least correct several mistakes and ambiguities before publishing the paper.
For example, the authors write that about half the reductions in music CD sales are the result of the increase in market share of music discount stores with smaller inventories. Liebowitz argues that this cannot possibly be correct. He calculates that, even under extreme assumptions, the reduction in inventories can at most account for one-sixth of the decrease in sales. "It is unbelievable that a top-journal like the "JPE" would publish such claims without any evidence," Liebowitz complains in his letter, and he points Levitt to an entire series of additional errors or ambiguities.
Levitt forwarded Liebowitz’ letter to the authors, who ignored it—their study was published with only minor changes. Since then, file sharing services can refer to an academic paper in one of the top economics journals to defend themselves against the music industry.
In principle, like many other journals, the "JPE" requires that authors publish not only their results but also disclose the data and the methods that they use to derive them. However, this requirement does not apply to Oberholzer-Gee and Strumpf—their paper was accepted before the requirement became binding. "This has nothing to do with science," criticizes Bruce McCullough, professor of decision sciences at Drexel University in Philadelphia. "Without scrutiny, there can be no science," says the expert on the replicability of empirical results in economics. . . .
Given the huge coverage of this by the American press, it would be useful if someone in the American Press would write about this problem.
Thanks very much to Florenz Plassmann for translating this.
Steve Levitt's Correction Letter
SteveLevitt
Many comments have been posted on this letter based upon the news story written on it in the Chronicle of Higher Education, but few have seen Levitt's actual correction letter. Levitt at least acknowledged that he sent multiple problematic "emails" over more than one day. Among those commenting on the original news stories please see Ted Frank, Ben Zycher, John Palmer, Craig Newmark, Robert Wallach, Clayton Cramer, Larry White, Steve Sailer, Xlrq, Jonathan Adler, Michael Munger, Steve Sailer again, Glenn Reynolds, Say Uncle, Jode Shoo, Singular Values, and
The Corner. Unfortunately, at this point, I am not allowed to really comment on this. One part of Levitt's letter that has not gotten any attention is the one that I think was his most important, his statement that:
The Chronicle of Higher Education has had a couple of articles on all this:
See this from the Chronicle.
See also this:
One point that wasn't directly mentioned by anyone is that Levitt's response when asked to backup is claim that others hadn't replicated my research was that the research papers were not refereed. Not only is Levitt acknowledging that the papers backed up and replicated my research, but he is admitting that the papers that did so were refereed.
The date on this letter does not match when I received it.
Update: See also this follow up filing.
Here is the exchange that Levitt had with economist John McCall on May 24th and 25th, 2005:
Email from John McCall to Steven LevittYou also state that others have tried to replicate [Lott’s] research and have failed. Please supply me with appropriate citations so that I might check for myself.
Email from Steven Levitt to John McCallThere was a NRC/ natl acad of sciences panel I was part of about research on guns that came out in 2004. That will point you in the right direction. . . .
Email from John McCall to Steven LevittHi Steve,
I went to the website you recommended -- have not gone after the round table proceedings yet -- I also found the following citations -- have not read any of them yet, but it appears they all replicate Lott's research. The Journal of Law and Economics is not chopped liver. . . .
Cordially,
John McCall PhD
Email from Steven Levitt to John McCallJohn,
It was not a peer refereed edition of the Journal. For $15,000 he was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.
Steve
Email from John McCall to Steven LevittReturning to the $15,000 bribe issue of JLE and -- although I have not yet gone (I will find it tomorrow) to the NRC/natl. acad. of sciences panel you recommended -- I am wondering whether those deliberations were published somewhere, and, if so, who paid for that. Could such not have been essentially the same thing? I noticed that this issue of JLE was the results of a conference on crime safety and guns that was sponsored by AEI and the Yale Center for Studies in Law. I understand how your best friend the editor could have been outraged, and I hope he had the principle to resign his position in protest. However, we all eventually realize that an editor is but a small cog in a big wheel.
Email from Steven Levitt to John McCallJohn, if you read the paper by Duggan in JPE, and Ayres and Donohue in Stanford Law Review, and the NAS/NRC report (which was not paid for by anyone, it is done by the National Academy of Sciences), you will see the other side of the debate. Steve
Judge Castillo issues decision on Lott v. Levitt
SteveLevittA copy of the judge's decision can be downloaded here. A Copy of Levitt's Correction Letter can be obtained here.
1) The Judge has found that one of the two counts of defamation involving Levitt can proceed.
p. 15: ". . . Levitt's email sounds as if he was "in possession of objectively verifiable facts." In his email Levitt states: 'It was not a peer refereed edition of the Journal. For $15,000 [Lott] was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.' First, it would be unreasonable to interpret Levitt's unqualified statement that the journal edition was not 'peer refereed' as Levitt merely giving his opinion on the 'peers' chosen to review,or referee, the Special issue. Indeed, the editor of the Journal might be able to verify the truth or falsity of whether the Special Issue was reviewd by peers. Furthermore, while Levitt argues that one person's "'peer' in the academic realm may be another person's 'hack'," this distinction is not reasonable when discussing the review process at a top university's academic journal. Second, a reasonable reader would not interpret Levitt's assertion that "For $15,000 [Lott] was able to buy an issue and put in only work that supported him" as simply a statement of Levitt's opinion. Levitt's email appears to state objectively verifiable facts: that Lott paid $15,000 to control the content of the Special Issue. . . . Levitt's motion to dismiss Count II of Lott's Complaint is denied."
2) p. 7: "The applicable standard, however, is not that of the 'world of academic research and scholarship' that Lott describes. Rather, the critical question is how a 'reasonable reader' would interpret the phrase. The reasonable reader in this case is the general population . . . . In everyday language replicating results does not necessarily mean analyzing identical data in identical ways . . . ."
Response: I think that the market for the book was also aimed at academics. The book is apparently marketed to a large number of economics classes and is read by academics.
Levitt's correction letter
Many comments have been posted on this letter based upon the news story written on it in the Chronicle of Higher Education, but few have seen Levitt's actual correction letter. Among those commenting on the original news stories please see Ted Frank, Ben Zycher, John Palmer, Craig Newmark, Robert Wallach, Clayton Cramer, Larry White, Steve Sailer, Xlrq, Jonathan Adler, Michael Munger, Steve Sailer again, Glenn Reynolds, Say Uncle, Jode Shoo, Singular Values, and
The Corner. Unfortunately, at this point, I am not allowed to really comment on this. One part of Levitt's letter that has not gotten any attention is the one that I think was his most important, his statement that:"I also was aware at the time of the May 2005 emails to you that in connection with the preparation of conference issues for the JLE that the organizer of each conference issue needs to provide funding to the JLE to cover publication and mailing expenses. I did not mean to suggest that Dr. Lott did anything unlawful or improper in arranging for the payment of the publication expenses for the Conference Issue."
The Chronicle of Higher Education has had a couple of articles on all this:
See this from the Chronicle.
See also this:
According to the motion, new facts have come to Mr. Lott’s attention since last year that significantly alter the character of his complaint. For one thing, he says, new information has come out about what he calls Mr. Levitt’s malice toward him. The motion alleges that Mr. Levitt has publicly referred to Mr. Lott as “the anti-Christ” and that Mr. Levitt “offered publicly to pay colleagues if they would humiliate” Mr. Lott. (Mr. Levitt did not immediately reply to a request for comment today.)
One point that wasn't directly mentioned by anyone is that Levitt's response when asked to backup is claim that others hadn't replicated my research was that the research papers were not refereed. Not only is Levitt acknowledging that the papers backed up and replicated my research, but he is admitting that the papers that did so were refereed.
The date on this letter does not match when I received it.
Update: See also this follow up filing
1) The Judge has found that one of the two counts of defamation involving Levitt can proceed.
p. 15: ". . . Levitt's email sounds as if he was "in possession of objectively verifiable facts." In his email Levitt states: 'It was not a peer refereed edition of the Journal. For $15,000 [Lott] was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.' First, it would be unreasonable to interpret Levitt's unqualified statement that the journal edition was not 'peer refereed' as Levitt merely giving his opinion on the 'peers' chosen to review,or referee, the Special issue. Indeed, the editor of the Journal might be able to verify the truth or falsity of whether the Special Issue was reviewd by peers. Furthermore, while Levitt argues that one person's "'peer' in the academic realm may be another person's 'hack'," this distinction is not reasonable when discussing the review process at a top university's academic journal. Second, a reasonable reader would not interpret Levitt's assertion that "For $15,000 [Lott] was able to buy an issue and put in only work that supported him" as simply a statement of Levitt's opinion. Levitt's email appears to state objectively verifiable facts: that Lott paid $15,000 to control the content of the Special Issue. . . . Levitt's motion to dismiss Count II of Lott's Complaint is denied."
2) p. 7: "The applicable standard, however, is not that of the 'world of academic research and scholarship' that Lott describes. Rather, the critical question is how a 'reasonable reader' would interpret the phrase. The reasonable reader in this case is the general population . . . . In everyday language replicating results does not necessarily mean analyzing identical data in identical ways . . . ."
Response: I think that the market for the book was also aimed at academics. The book is apparently marketed to a large number of economics classes and is read by academics.
Levitt's correction letter
Many comments have been posted on this letter based upon the news story written on it in the Chronicle of Higher Education, but few have seen Levitt's actual correction letter. Among those commenting on the original news stories please see Ted Frank, Ben Zycher, John Palmer, Craig Newmark, Robert Wallach, Clayton Cramer, Larry White, Steve Sailer, Xlrq, Jonathan Adler, Michael Munger, Steve Sailer again, Glenn Reynolds, Say Uncle, Jode Shoo, Singular Values, and
The Corner. Unfortunately, at this point, I am not allowed to really comment on this. One part of Levitt's letter that has not gotten any attention is the one that I think was his most important, his statement that:
The Chronicle of Higher Education has had a couple of articles on all this:
See this from the Chronicle.
See also this:
One point that wasn't directly mentioned by anyone is that Levitt's response when asked to backup is claim that others hadn't replicated my research was that the research papers were not refereed. Not only is Levitt acknowledging that the papers backed up and replicated my research, but he is admitting that the papers that did so were refereed.
The date on this letter does not match when I received it.
Update: See also this follow up filing
New Research that Abortion Increases Violent Crime Gets Attention
Abortion Crime Economics JohnDonohue SteveLevittFrom today's Chicago Sun Times:
A high-profile economist is challenging the conclusion in the best-selling book Freakonomics by University of Chicago professor Steven D. Levitt that the legalization of abortion in the early 1970s led to a major drop in murder and other violent crimes a generation later.
John R. Lott Jr., a former U. of C. economist now teaching in New York, says the Supreme Court's 1973 Roe v. Wade decision actually caused violent crime to rise.
Lott and fellow researcher John Whitley plan to publish a paper in October in Economic Inquiry that questions Levitt's research on abortion and crime.
Lott and Levitt already were feuding over Lott's charge that Levitt had defamed him in Freakonomics.
In their new paper, Lott and Whitley say that legalization of abortion prompted a cultural change that increased the number of children born out of wedlock. Those children of unwed mothers caused murders to rise by more than 700 cases in 1998 alone, saddling the public with more than $3.3 billion in "victimization costs," the paper says.
On the other hand, Levitt's research found that Roe v. Wade resulted in a savings of $30 billion a year that crime would have cost the public.
More unwed mothers
His Freakonomics, co-authored by Stephen Dubner and published last year, says legalized abortion led to a large drop in murder and other violent crime in the late 1980s and early '90s, and continues to reduce crime.
The book suggests that if the aborted fetuses had instead been born, they would have become adults more likely to commit crimes because they were unwanted by their mothers.
To illustrate the point, the book says the five states that allowed abortion three years before Roe vs. Wade saw major declines in violent crime between 1988 and 1994 -- earlier than the other states.
But Lott says the Levitt study did not fully consider the increase of children born out of wedlock. His theory is that with the option of abortion, women became more likely to have premarital sex, but then had their babies and raised them as single parents.
Children born out of wedlock have had smaller investments in "human capital" by their parents and are more likely to get into trouble when they grow older, Lott says.
On average, his paper says, about 5 percent of whites were born out of wedlock from 1965 to 1969, rising two decades later to 16 percent. For blacks, the figure rose from about 35 percent to about 62 percent, the paper says.
Before legalized abortion, more than 70 percent of children born out of wedlock ended up in families with a father, but the fraction fell to 44 percent in 1984, according to the paper. . . . .
If you want to read the research, you can find it here. THe newspaper article says that I concede that abortion through its effect on "unwanted" births slightly reduces violent crime, but what I believe that I said is that it is possible. The net effect however is abortion increases violent crime.
John R. Lott Jr., a former U. of C. economist now teaching in New York, says the Supreme Court's 1973 Roe v. Wade decision actually caused violent crime to rise.
Lott and fellow researcher John Whitley plan to publish a paper in October in Economic Inquiry that questions Levitt's research on abortion and crime.
Lott and Levitt already were feuding over Lott's charge that Levitt had defamed him in Freakonomics.
In their new paper, Lott and Whitley say that legalization of abortion prompted a cultural change that increased the number of children born out of wedlock. Those children of unwed mothers caused murders to rise by more than 700 cases in 1998 alone, saddling the public with more than $3.3 billion in "victimization costs," the paper says.
On the other hand, Levitt's research found that Roe v. Wade resulted in a savings of $30 billion a year that crime would have cost the public.
More unwed mothers
His Freakonomics, co-authored by Stephen Dubner and published last year, says legalized abortion led to a large drop in murder and other violent crime in the late 1980s and early '90s, and continues to reduce crime.
The book suggests that if the aborted fetuses had instead been born, they would have become adults more likely to commit crimes because they were unwanted by their mothers.
To illustrate the point, the book says the five states that allowed abortion three years before Roe vs. Wade saw major declines in violent crime between 1988 and 1994 -- earlier than the other states.
But Lott says the Levitt study did not fully consider the increase of children born out of wedlock. His theory is that with the option of abortion, women became more likely to have premarital sex, but then had their babies and raised them as single parents.
Children born out of wedlock have had smaller investments in "human capital" by their parents and are more likely to get into trouble when they grow older, Lott says.
On average, his paper says, about 5 percent of whites were born out of wedlock from 1965 to 1969, rising two decades later to 16 percent. For blacks, the figure rose from about 35 percent to about 62 percent, the paper says.
Before legalized abortion, more than 70 percent of children born out of wedlock ended up in families with a father, but the fraction fell to 44 percent in 1984, according to the paper. . . . .
If you want to read the research, you can find it here. THe newspaper article says that I concede that abortion through its effect on "unwanted" births slightly reduces violent crime, but what I believe that I said is that it is possible. The net effect however is abortion increases violent crime.
LA Times Letter to Editor: Re "Shooting holes in a lawsuit," Opinion, May 31
SteveLevittUnfortunately, this letter was substantially shortened and somewhat altered the meaning due to space limits on letters. It is also unfortunate that the LA Times did not publish any letters from other academics who had written in.
Accusations on gun law research are unfounded
June 12, 2006
Re "Shooting holes in a lawsuit," Opinion, May 31
Jon Wiener claims that "nobody" tried to replicate my research, which demonstrated that right-to-carry laws reduce crime rates.
This is false. Not only has everyone who tried managed to replicate my findings, but many academics have gone beyond that and shown that right-to-carry laws reduce violent crime under a variety of approaches (see johnrlott.tripod.com/postsbyday/RTCResearch.html).
Wiener fails to note that since my research was published, not a single peer-reviewed publication has found that right-to-carry laws increase crime, and there is research that finds even larger drops than I found.
Wiener's Google search implies that many other scholars agree with his claims of research fraud. Yet Wiener fails to note that only about a sixth of the postings are actually scholars of any type.
Wiener claims that my defamation suit wants to "silence" certain claims of fraud. I and other academics have tried to engage Stephen D. Levitt in discussions on the accuracy of his claims, but he won't respond to us.
Instead, [Levitt] falsely charges that the research with which he disagrees was published only because the University of Chicago Press overrode the journal's editor, and that I bought the press' decision.
JOHN R. LOTT JR.
Here is one of the other letters that I know of from two academics:
In his op-ed piece, “Gun-research 'Freak'-out” (LA Times, May 31), Jon Wiener says, “Blocking the sale of a book based on a literal interpretation of a single word would be outrageous.” But this is not the issue. In Freakonomics, Steven Levitt and Stephen Dubner say, “When other scholars have tried to replicate [Lott’s] results, they found that right-to-carry laws simply don't bring down crime.” This statement is false under any reasonable interpretation. We replicated Lott’s results, as have many other scholars. If Levitt did not know this, then he was lax in his scholarship.
Levitt should know that among scholars “unable to replicate” is interpreted as “something is seriously wrong here.” Moreover, the phrase carries the suggestion of, at best, incompetence and, more likely, dishonesty.
Levitt should express his disagreement with Lott in a way that does not imply incompetence or dishonesty. It is thoroughly reasonable to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.
Nicolaus Tideman
Blacksburg, VA
Florenz Plassmann
Ithaca, NY
See also:
Dear Mr. Goldberg:
At the end of his op-ed piece, “Gun-research 'Freak'-out” (LA Times, May 31), Jon Wiener says, “Blocking the sale of a book based on a literal interpretation of a single word would be outrageous.” But the relevant issue is not “a literal interpretation of a single word.” In Freakonomics, Steven Levitt and Stephen Dubner say, “When other scholars have tried to replicate his results, they found that right-to-carry laws simply don't bring down crime.” This statement is false under any reasonable interpretation of the words.
In our understanding, and in the understanding of colleagues we consulted, what is generally meant in economics by replicating someone’s work is gathering the same data and analyzing them in the way that the original researcher had analyzed them. We took an interest in the issue of guns and crime when Lott’s work was first published. We gathered the data that he had used, analyzed it in the way that he described and got essentially the same results. Thus we replicated Lott’s work.
In some circumstances, what is meant by replicating someone’s work is to gather similar data and analyze it in the same way. But that interpretation would not be relevant here, because (as often happen in economics) the data that Lott used were all of the data that were available.
Should “replicate” be stretched to mean “undertake similar analyses and reach similar results”? Well, we analyzed the guns and crime data in additional ways that we regarded as interesting and reached results that were broadly in agreement with those that Lott had reached.
Is there any reason why our work (“Does the Right to Carry Concealed Handguns deter Countable Crimes? Only a Count Analysis Can Say,” Journal of Law and Economics, October 2001, Volume 44, Number 2, Part 2, pp.771-798), should not count as a replication of Lott’s work? Wiener quotes in apparent endorsement Levitt’s assertion that, “for $15,000 [Lott] was able to buy an issue [of the Journal of Law and Economics] and put in only work that supported him.” This is not what happened. We presented our work at a conference that Lott organized. At Lott’s invitation, we submitted our work for a special issue of the Journal of Law and Economics that Lott arranged. A letter from an editor (Sam Peltzman, Professor of Economics at the University of Chicago) informed us that we would need to address the concerns of a referee who had reviewed our work before a decision would be made as to whether it would be published. A later letter for Peltzman informed us that our revision in response to the referee’s comments was accepted. Thus our work was not “put in” the Journal of Law and Economics by Lott but rather accepted for publication after scholarly review.
The basic issue between Lott and Levitt is not whether the results of either scholar can be replicated, but rather what statistical analyses are appropriate to make inferences about the effect on crime of allowing more citizens to carry guns. It is not unreasonable to require Levitt to find a way to express his analytical disagreement with Lott without implying that others who analyze the data in the way that Lott did do not get the same results. While lawyers are better suited to opine on what legal measures are appropriate to achieve this end, it seems to us not unreasonable, prima facie, to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.
Sincerely,
Florenz Plassmann
Associate Professor of Economics, Binghamton University
Nicolaus Tideman
Professor of Economics, Virginia Tech
June 12, 2006
Re "Shooting holes in a lawsuit," Opinion, May 31
Jon Wiener claims that "nobody" tried to replicate my research, which demonstrated that right-to-carry laws reduce crime rates.
This is false. Not only has everyone who tried managed to replicate my findings, but many academics have gone beyond that and shown that right-to-carry laws reduce violent crime under a variety of approaches (see johnrlott.tripod.com/postsbyday/RTCResearch.html).
Wiener fails to note that since my research was published, not a single peer-reviewed publication has found that right-to-carry laws increase crime, and there is research that finds even larger drops than I found.
Wiener's Google search implies that many other scholars agree with his claims of research fraud. Yet Wiener fails to note that only about a sixth of the postings are actually scholars of any type.
Wiener claims that my defamation suit wants to "silence" certain claims of fraud. I and other academics have tried to engage Stephen D. Levitt in discussions on the accuracy of his claims, but he won't respond to us.
Instead, [Levitt] falsely charges that the research with which he disagrees was published only because the University of Chicago Press overrode the journal's editor, and that I bought the press' decision.
JOHN R. LOTT JR.
Here is one of the other letters that I know of from two academics:
Levitt should know that among scholars “unable to replicate” is interpreted as “something is seriously wrong here.” Moreover, the phrase carries the suggestion of, at best, incompetence and, more likely, dishonesty.
Levitt should express his disagreement with Lott in a way that does not imply incompetence or dishonesty. It is thoroughly reasonable to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.
Nicolaus Tideman
Blacksburg, VA
Florenz Plassmann
Ithaca, NY
See also:
At the end of his op-ed piece, “Gun-research 'Freak'-out” (LA Times, May 31), Jon Wiener says, “Blocking the sale of a book based on a literal interpretation of a single word would be outrageous.” But the relevant issue is not “a literal interpretation of a single word.” In Freakonomics, Steven Levitt and Stephen Dubner say, “When other scholars have tried to replicate his results, they found that right-to-carry laws simply don't bring down crime.” This statement is false under any reasonable interpretation of the words.
In our understanding, and in the understanding of colleagues we consulted, what is generally meant in economics by replicating someone’s work is gathering the same data and analyzing them in the way that the original researcher had analyzed them. We took an interest in the issue of guns and crime when Lott’s work was first published. We gathered the data that he had used, analyzed it in the way that he described and got essentially the same results. Thus we replicated Lott’s work.
In some circumstances, what is meant by replicating someone’s work is to gather similar data and analyze it in the same way. But that interpretation would not be relevant here, because (as often happen in economics) the data that Lott used were all of the data that were available.
Should “replicate” be stretched to mean “undertake similar analyses and reach similar results”? Well, we analyzed the guns and crime data in additional ways that we regarded as interesting and reached results that were broadly in agreement with those that Lott had reached.
Is there any reason why our work (“Does the Right to Carry Concealed Handguns deter Countable Crimes? Only a Count Analysis Can Say,” Journal of Law and Economics, October 2001, Volume 44, Number 2, Part 2, pp.771-798), should not count as a replication of Lott’s work? Wiener quotes in apparent endorsement Levitt’s assertion that, “for $15,000 [Lott] was able to buy an issue [of the Journal of Law and Economics] and put in only work that supported him.” This is not what happened. We presented our work at a conference that Lott organized. At Lott’s invitation, we submitted our work for a special issue of the Journal of Law and Economics that Lott arranged. A letter from an editor (Sam Peltzman, Professor of Economics at the University of Chicago) informed us that we would need to address the concerns of a referee who had reviewed our work before a decision would be made as to whether it would be published. A later letter for Peltzman informed us that our revision in response to the referee’s comments was accepted. Thus our work was not “put in” the Journal of Law and Economics by Lott but rather accepted for publication after scholarly review.
The basic issue between Lott and Levitt is not whether the results of either scholar can be replicated, but rather what statistical analyses are appropriate to make inferences about the effect on crime of allowing more citizens to carry guns. It is not unreasonable to require Levitt to find a way to express his analytical disagreement with Lott without implying that others who analyze the data in the way that Lott did do not get the same results. While lawyers are better suited to opine on what legal measures are appropriate to achieve this end, it seems to us not unreasonable, prima facie, to require a publisher to revise a sentence for future sales if the existing sentence is false and defamatory.
Sincerely,
Florenz Plassmann
Associate Professor of Economics, Binghamton University
Nicolaus Tideman
Professor of Economics, Virginia Tech
Will Donohue and Levitt predict that a massive new crime wave is coming?
Abortion Crime Economics JohnDonohue SteveLevittGuttmacher Institute, associated with Planned Parenthood, claims that: "The rate of unintended births — unintended pregnancies carried to term — rose by 44 percent among poor women from 1994 to 2001, but declined by 8 percent for wealthier women." If true, I assume that those few who still believe that abortion massively reduces violent crime rates after the errors were discovered in the original research will be predicting that we will soon see a big increase in violent crime rates. If up to 80 percent of the changes in murder rates can be explained by abortion as has been argued, those who believe this relationship must be worried we could be in for big increases in murder. (Of course, the abortion is more likely to slightly increase crime.) People born in 1994 will become teenagers starting next year. Presumably, proponents of this theory will start warning people very soon.
News story on defamation suit: Lott v. Levitt
SteveLevitt(Update) From the University of Chicago student newspaper the Maroon:
Freakonomics claim sparks defamation lawsuit
By Kim Velsey
April 21, 2006 in News
John Lott, Jr., a former visiting professor at the University filed a defamation lawsuit on April 10 against economic professor Steven Levitt, co-author of the New York Time bestseller Freakonomics
Lott said the book misrepresents his work on guns and crime, according to court documents. The lawsuit does not name journalist Stephen Dubner, though he co-wrote the book with Levitt.
Freakonomics, which melds Levitt’s economic essays with Dubner’s flowing prose, remains high on the bestseller list. The book’s success, however, may have prompted the legal action, as the lawsuit references the popularity of Freakonomics as a factor contributing to Lott’s damaged reputation.
The lawsuit states that the book “damages Lott’s reputation in the eyes of the academic community in which he works, and in the minds of hundreds of thousands of academics, college students, graduate students, and members of the general public who read Freakonomics.”
The contested material is on pages 133–134 of Freakonomics, in which Levitt writes that researchers have been unable to confirm Lott’s conclusion that right-to-carry gun laws actually reduce crime.
Freakonomics states, “Then there was the troubling allegation that Lott actually invented some of the survey data that supports his more-guns/less-crime theory. Regardless of whether or not the data was faked, Lott’s admittedly intriguing hypothesis doesn’t seem to be true. When other scholars have tried to replicate results, they found that right-to-carry laws simply don’t bring down crime.”
According to the lawsuit, Lott acknowledges that his findings have come under scrutiny in the academic community, but he maintains that he used “different data or methods to analyze the relationship between gun control laws and crime.”
The lawsuit states that scholars who have replicated Lott’s work have achieved the same results. “Every time that an economist or researcher have tried to replicate [Lott’s] results, he or she has confirmed Lott’s conclusion.”
Carl Moody, a professor of economics at the College of William and Mary, said he successfully replicated Lott’s findings and published the results in 2001. Moody said Levitt’s accusation is wrong.
The lawsuit, which also names Levitt’s publisher HarperCollins, states that the publisher acted with malice by failing to verify the statements. It seeks a court order to halt sales of Freakonomics until the statements are retracted or amended and also demands that Levitt and HarperCollins pay unspecified monetary damages.
HarperCollins would not comment on the lawsuit, but a company representative said, “HarperCollins Publishers firmly stands behind Freakonomics and its authors, Steven Levitt and Stephen Dubner.”
The ABC show 20/20 featured Freakonomics in an hour-long special on April 14. However, there was no mention of the lawsuit, and Levitt has yet to comment on it publicly. The book’s website, Freakonomics.com, which has Levitt’s and Dubner’s weblogs, includes a brief mention of the pending litigation.
“While we were away [in London promoting the paperback edition of Freakonomics], the economist John Lott filed a lawsuit claiming that Freakonomics has libeled him,” wrote Dubner on his blog.
Lott’s website made no mention of the lawsuit.
The lawsuit has opened up discussion on the veracity of Levitt’s claims and whether a lawsuit is an appropriate forum for an academic debate.
The litigation has also shed light on what can happen when an academic book attains blockbuster status.
“Most academic debate is so trivial no one would care,” Moody said. “If the book had appeared and no one had bought it, it wouldn’t be an issue. But Levitt is accusing this guy of falsifying his results in front of millions of people.”
Here is an earlier story from the Chicago Tribune:
A scholar known for his work on guns and crime filed a defamation lawsuit Monday against University of Chicago economist Steven Levitt, co-author of the best-seller "Freakonomics."
John Lott Jr. of Virginia, a former U. of C. visiting professor, alleges that Levitt defamed him in the book by claiming that other scholars had tried and failed to confirm Lott's conclusion that allowing people to carry concealed weapons reduces crime. Publishers Weekly ranked "Freakonomics" eighth this week for non-fiction hardcover books.
According to Levitt's book: "When other scholars have tried to replicate [Lott's] results, they found that right-to-carry laws simply don't bring down crime."
But according to Lott's lawsuit: "In fact, every time that an economist or other researcher has replicated Lott's research, he or she has confirmed Lott's conclusion."
By suggesting that Lott's results could not be replicated, Levitt is "alleging that Lott falsified his results," the lawsuit says.
Lott is seeking a court order to block further sales of "Freakonomics" until the offending statements are retracted and changed. He is also seeking unspecified money damages.
Lott acknowledged in the suit that some scholars have disagreed with his conclusions. But he said those researchers used "different data or methods to analyze the relationship between gun-control laws and crime" and made no attempt to "replicate" Lott's work.
The lawsuit alleges that Levitt and his publisher, HarperCollins Publishers Inc., made the statements with reckless disregard for whether they were true and that the book damaged Lott's reputation.
Neither Levitt nor HarperCollins officials could be reached Monday.
According to the lawsuit, Levitt also defamed Lott in an e-mail that Levitt sent to an economist in Texas last May. The e-mail described work that Lott published in an academic journal in 2001. It falsely stated that Lott's work had not been peer-reviewed and that Lott had blocked scholars with opposing views from appearing in the same issue of the journal, the lawsuit said.
Lott's books include "More Guns, Less Crime: Analyzing Crime and Gun Control Laws," published in 1998. Levitt won the John Bates Clark Medal for economists younger than 40 from the American Economic Association in 2003.
The case was assigned to U.S. District Judge Ruben Castillo.
From the Chronicle of Higher Education:
Last week Mr. Lott filed a defamation lawsuit against Steven D. Levitt, a professor of economics at the University of Chicago and a co-author of the best-selling Freakonomics: A Rogue Economist Explores the Hidden Side of Everything (William Morrow, 2005). (A copy of the complaint can be found on the blog Overlawyered.) Mr. Lott charges that in the book and in private e-mail messages, Mr. Levitt spread lies about the quality and integrity of Mr. Lott's work (The Chronicle, April 13). Much will hinge on exactly what Mr. Levitt meant by the words "replicate" and "peer refereed."
UPDATE: For the conclusion of part of the case, please see this link.
By Kim Velsey
April 21, 2006 in News
John Lott, Jr., a former visiting professor at the University filed a defamation lawsuit on April 10 against economic professor Steven Levitt, co-author of the New York Time bestseller Freakonomics
Lott said the book misrepresents his work on guns and crime, according to court documents. The lawsuit does not name journalist Stephen Dubner, though he co-wrote the book with Levitt.
Freakonomics, which melds Levitt’s economic essays with Dubner’s flowing prose, remains high on the bestseller list. The book’s success, however, may have prompted the legal action, as the lawsuit references the popularity of Freakonomics as a factor contributing to Lott’s damaged reputation.
The lawsuit states that the book “damages Lott’s reputation in the eyes of the academic community in which he works, and in the minds of hundreds of thousands of academics, college students, graduate students, and members of the general public who read Freakonomics.”
The contested material is on pages 133–134 of Freakonomics, in which Levitt writes that researchers have been unable to confirm Lott’s conclusion that right-to-carry gun laws actually reduce crime.
Freakonomics states, “Then there was the troubling allegation that Lott actually invented some of the survey data that supports his more-guns/less-crime theory. Regardless of whether or not the data was faked, Lott’s admittedly intriguing hypothesis doesn’t seem to be true. When other scholars have tried to replicate results, they found that right-to-carry laws simply don’t bring down crime.”
According to the lawsuit, Lott acknowledges that his findings have come under scrutiny in the academic community, but he maintains that he used “different data or methods to analyze the relationship between gun control laws and crime.”
The lawsuit states that scholars who have replicated Lott’s work have achieved the same results. “Every time that an economist or researcher have tried to replicate [Lott’s] results, he or she has confirmed Lott’s conclusion.”
Carl Moody, a professor of economics at the College of William and Mary, said he successfully replicated Lott’s findings and published the results in 2001. Moody said Levitt’s accusation is wrong.
The lawsuit, which also names Levitt’s publisher HarperCollins, states that the publisher acted with malice by failing to verify the statements. It seeks a court order to halt sales of Freakonomics until the statements are retracted or amended and also demands that Levitt and HarperCollins pay unspecified monetary damages.
HarperCollins would not comment on the lawsuit, but a company representative said, “HarperCollins Publishers firmly stands behind Freakonomics and its authors, Steven Levitt and Stephen Dubner.”
The ABC show 20/20 featured Freakonomics in an hour-long special on April 14. However, there was no mention of the lawsuit, and Levitt has yet to comment on it publicly. The book’s website, Freakonomics.com, which has Levitt’s and Dubner’s weblogs, includes a brief mention of the pending litigation.
“While we were away [in London promoting the paperback edition of Freakonomics], the economist John Lott filed a lawsuit claiming that Freakonomics has libeled him,” wrote Dubner on his blog.
Lott’s website made no mention of the lawsuit.
The lawsuit has opened up discussion on the veracity of Levitt’s claims and whether a lawsuit is an appropriate forum for an academic debate.
The litigation has also shed light on what can happen when an academic book attains blockbuster status.
“Most academic debate is so trivial no one would care,” Moody said. “If the book had appeared and no one had bought it, it wouldn’t be an issue. But Levitt is accusing this guy of falsifying his results in front of millions of people.”
Here is an earlier story from the Chicago Tribune:
John Lott Jr. of Virginia, a former U. of C. visiting professor, alleges that Levitt defamed him in the book by claiming that other scholars had tried and failed to confirm Lott's conclusion that allowing people to carry concealed weapons reduces crime. Publishers Weekly ranked "Freakonomics" eighth this week for non-fiction hardcover books.
According to Levitt's book: "When other scholars have tried to replicate [Lott's] results, they found that right-to-carry laws simply don't bring down crime."
But according to Lott's lawsuit: "In fact, every time that an economist or other researcher has replicated Lott's research, he or she has confirmed Lott's conclusion."
By suggesting that Lott's results could not be replicated, Levitt is "alleging that Lott falsified his results," the lawsuit says.
Lott is seeking a court order to block further sales of "Freakonomics" until the offending statements are retracted and changed. He is also seeking unspecified money damages.
Lott acknowledged in the suit that some scholars have disagreed with his conclusions. But he said those researchers used "different data or methods to analyze the relationship between gun-control laws and crime" and made no attempt to "replicate" Lott's work.
The lawsuit alleges that Levitt and his publisher, HarperCollins Publishers Inc., made the statements with reckless disregard for whether they were true and that the book damaged Lott's reputation.
Neither Levitt nor HarperCollins officials could be reached Monday.
According to the lawsuit, Levitt also defamed Lott in an e-mail that Levitt sent to an economist in Texas last May. The e-mail described work that Lott published in an academic journal in 2001. It falsely stated that Lott's work had not been peer-reviewed and that Lott had blocked scholars with opposing views from appearing in the same issue of the journal, the lawsuit said.
Lott's books include "More Guns, Less Crime: Analyzing Crime and Gun Control Laws," published in 1998. Levitt won the John Bates Clark Medal for economists younger than 40 from the American Economic Association in 2003.
The case was assigned to U.S. District Judge Ruben Castillo.
From the Chronicle of Higher Education:
UPDATE: For the conclusion of part of the case, please see this link.
Interesting conference on the abortion and crime link
Abortion Crime SteveLevittToday we just had an interesting conference on the abortion and crime link. There is a video of the event. We invited all the Americans who had finished work on this topic by the middle of February, so while it might appear that almost everyone who had finished a paper thought that the link between abortion and crime was nonexistent or possibly even positive. There was one paper that claimed to find evidence in favor of Donohue and Levitt, but that when the issues raised by Chris Foote were dealt with the significance of the effect went away. Video is available of the two sessions at the top right of the page.
The Economist Magazine on Steve Levitt and Abortion
Abortion Crime JohnDonohue SteveLevittBut what is most amazing is that despite all the statistical significance being eliminated from their panel data set being eliminated when the results are done Donohue and Levitt's way with the the data they want to use, John Donohue now says what they wanted to do:
There was no warning that these were the wrong tests at all in either of their previous papers or in Levitt's book. Levitt now refers to the “collage of evidence," but the panel data was the only test that really amounted to anything worthwhile. The rest of the data was merely cross sectional or time series.
Incorrect claims about the data in More Guns, Less Crime
ConcealedCarry Crime IanAyres JohnDonohue SteveLevittI have received an email comparing my work to that of Steve Levitt's regarding claimed coding errors in More Guns, Less Crime. Of course, this discussion is not very accurate.
1) There are no coding errors in the data used in “More Guns, Less Crime.” The book used crime data from 1977 to 1996, and as far as I know, no academics have claimed that there were any coding errors in that data. An interesting and useful Stanford Law Review article by Plassmann and Whitley added an additional four years to the data and the problem arose in this additional data. Overall, less than 200 cells out of 7.5 million cells were accidentally left blank. More important, the results that Plassmann and Whitley noted were the results which they thought were the correct ones were not affected at all by this minor change in the data.
2) I had put the data set on my website as a favor to Plassmann because it was very large and he did not have the ability to put it up for people to download. The data was corrected as soon as the missing cells were discovered and it was available for people to download. A note was added to the site to alert people to the correction. While I had helped them out on their paper, I let Plassmann and Whitley make their own statements about their results. My website reported the regressions in the same way that I had done them in all my previous estimates.
3) In statements here and here, I have discussed what was involved in Levitt's errors. The regressions where they left out the fixed effects were essentially the only ones that were at all useful in testing their hypothesis because they were the only ones that didn't require large degrees of aggregation to get to their "effective abortion rate" that were related to total murder rates in a state. The work that I did with Whitley directly related the age of the murder to the number of abortions when that murderer was born.
Thanks to Bob Thomas for his email.
1) There are no coding errors in the data used in “More Guns, Less Crime.” The book used crime data from 1977 to 1996, and as far as I know, no academics have claimed that there were any coding errors in that data. An interesting and useful Stanford Law Review article by Plassmann and Whitley added an additional four years to the data and the problem arose in this additional data. Overall, less than 200 cells out of 7.5 million cells were accidentally left blank. More important, the results that Plassmann and Whitley noted were the results which they thought were the correct ones were not affected at all by this minor change in the data.
2) I had put the data set on my website as a favor to Plassmann because it was very large and he did not have the ability to put it up for people to download. The data was corrected as soon as the missing cells were discovered and it was available for people to download. A note was added to the site to alert people to the correction. While I had helped them out on their paper, I let Plassmann and Whitley make their own statements about their results. My website reported the regressions in the same way that I had done them in all my previous estimates.
3) In statements here and here, I have discussed what was involved in Levitt's errors. The regressions where they left out the fixed effects were essentially the only ones that were at all useful in testing their hypothesis because they were the only ones that didn't require large degrees of aggregation to get to their "effective abortion rate" that were related to total murder rates in a state. The work that I did with Whitley directly related the age of the murder to the number of abortions when that murderer was born.
Thanks to Bob Thomas for his email.
Subscribe to:
Posts (Atom)
POPULER
