Dear Signers of the Open Letter,<\/p>\n
There is an “open letter” going around that is critical of my coauthored study on non-citizen voting. \u00a0If the open letter was accurate, I would sign it too. \u00a0And so I can well imagine why a number of colleagues around the country did sign. \u00a0But I’m not signing it because it contains several critical distortions and mistakes. \u00a0Below I will quote from the letter, and then explain its errors.<\/p>\n
\n“We are professional political scientists. We write to clarify the evidence in our field for<\/div>\nPresident Trump\u2019s claim, which he has repeated several times, that millions of non-citizens voted in the 2016 general election.”<\/div>\n<\/blockquote>\nThis part I completely agree with. \u00a0I’ve been trying to clarify the evidence for months as well. \u00a0My study DOES NOT support Trump’s claim that millions of non-citizens voted in the 2016 election. \u00a0My initial response was here<\/a>\u00a0and I have also pushed back<\/a> against subsequent attempts to use that response to make somewhat more modest but still not data based claims.<\/div>\n\n“The president has cited a 2014 article by Jesse Richman, Gulshan Chattha, and David<\/div>\nEarnest, published in the peer-reviewed journal Electoral Studies, as evidence for this claim. \u00a0In that study, Richman and his colleagues used data from the 2008 and 2010 iterations of\u00a0the Cooperative Congressional Election Study (CCES), a large-scale, regular survey that contained more than 30,000 and 55,000 respondents, respectively. The researchers leveraged questions about respondents\u2019 citizenship status and voting to argue that \u201cbetween 7.9% and 14.7% of non-citizens voted in 2008.\u201d”<\/div>\n<\/blockquote>\nThis quote is taken out of context. \u00a0Our full confidence interval \u00a0concerning non-citizen voting was bounded at the lower end by 0.2 percent. \u00a0Quoting only this part gives a very misleading impression of our conclusions and the precision we claimed for them. \u00a0It gives the impression that the range of our estimates was much higher and much more precise than it was. \u00a0The point that needs to be made is that some in the Trump team have taken to reporting the high end of the confidence interval for the highest and least certain estimate from the paper as if it is actually a point estimate.<\/div>\n\n“Given the non-citizen population of about 19.4 million, the authors concluded, \u201cthe number of non-citizen voters. . . could range from just over 38,000 at the very minimum to nearly 2.8 million at the maximum.\u201d The higher bound in this statement is the one that appears to have shaped the president\u2019s rhetoric on the issue.”<\/div>\n<\/blockquote>\nI agree with the authors of the letter that the upper end of this interval\u00a0may have played an unfortunate role in the presidents rhetoric. \u00a0I have, as noted above, attempted to push back against this. \u00a0I will continue to do so as I think it is important that people not get fooled by an extreme upper end estimate that is almost certainly way way way too high. \u00a0The high figure is based upon the confidence interval around an estimate of voting that counts every self-reported non-citizen with some indication of having voted — including those who said they voted but had a validated non-vote, and those who cast a validated vote but said they didn’t vote. \u00a0This estimate is itself almost surely too high. \u00a0And the upper end of a confidence interval is intended to be a point so high that there is a 97.5 percent chance that the true value is lower.<\/div>\n\n“The analysis in this paper has been shown to be incorrect. In a survey as large as the<\/div>\nCCES, even a small rate of response error (where people incorrectly mark the wrong item on a survey) can lead to incorrect conclusions. Importantly, the findings in Richman et al. rest on a sample of only 339 respondents who claimed to be non-citizens in 2008, out\u00a0of about 30,000 CCES respondents. Stephen Ansolabehere, Samantha Luks, and Brian\u00a0Schaffner demonstrated in a 2015 paper (also published in Electoral Studies) that response\u00a0error explains nearly all of the supposed non-citizens in Richman et. al\u2019s sample who voted.\u00a0The underlying intuition is relatively straightforward: Given the dynamics of the CCES (a very large overall sample with a very small subpopulation of non-citizens), even with a very low error rate of 0.1%, we would expect roughly 10% of the people in the \u201cnon-citizen\u201d\u00a0category would actually be citizens. If those people then voted at a high rate, it would appear as if a low (but consequential) percentage of non-citizens were voting, which is precisely the result Richman et al. describe.<\/div>\n<\/div>\n“Indeed, Ansolabehere and colleagues leverage a key feature of the CCES to investigate this possibility. When they examined the responses from people who were asked the citizenship\u00a0question at two different points in time, they found inconsistencies. The citizenship status\u00a0of 56 respondents changed in two years, and 20 people reported moving from citizen to\u00a0non-citizen status (which is not even a plausible change). Of those who we can be more confident are non-citizens, there was not a single voter in 2010. In 2012, there is just one\u00a0person who may have voted, but even in that case the evidence suggests that the respondent\u00a0was actually not a voter.”<\/div>\n<\/div>\n<\/blockquote>\nThese critical tests lack statistical power. \u00a0That is, the likelihood of finding exactly these null results with the sample of 85 individuals who twice repeated that they were non-citizens is quite high if the actual rates of voting were precisely what was observed in the CCES cross-sectional studies analyzed in our 2014 paper. \u00a0Here the distortion above that made it sound as if our estimates were much higher than they were lends this argument credence and apparent statistical power that it lacks.<\/div>\n<\/div>\nThe primary analysis in the Ansolabehere et. al. 2015 paper is of validated voting in the 2010 midterm election. \u00a0All of the evidence points to a low turnout by non-citizens in 2010. \u00a0In the 2010 midterm CCES cross-sectional file 7 non-citizens cast validated votes.\u00a0 Excluding Virginia (no record checks were possible because of state law) this implies that 1.3 percent of non-citizens cast verified or validated votes in 2010.\u00a0 Ansolabehere et. al. examine validated voting in 2010 by 85 non-citizens who twice confirmed their status as non-citizens.\u00a0 A simple exercise with the binomial distribution shows that with 85 trials and a probability of success on each trial of 1.3233 percent, the probability of finding no successes is 32.2 percent. Thus, the observed outcome is one that is entirely plausible in the context of the frequency of non-citizens casting verified votes in 2010.\u00a0 It is an outcome that would occur nearly one third of the time if the actual non-citizen voting rate was precisely what the larger 2010 cross-sectional survey identified.\u00a0 \u00a0Indeed, if one supposes that this estimate was a bit too high \u2013 that the true value was for instance the 0.65 percent Richman et. al. estimate would have been sufficient to account for Franken\u2019s 2008 MN win \u2013 one would get a null value when running 85 trials more than half (57.4 percent) of the time. \u00a0But in a footnote of their paper the authors dismiss as likely invalid the presence of a validated voter in the 2012 85 person sub-sample of respondents who twice confirmed her citizenship status because the individual said she was not registered to vote, contrary to the rest of their analysis which relies exclusively on the validated voting measure alone. \u00a0If we apply the strict standard that one ought to only consider an individual to be a potential voter if they both say they voted and cast a validated vote, then the non-citizen voting rate indicated by the 2010 CCES cross-sectional survey is only 0.38 percent. \u00a0If, as seems reasonable, one assumes that the rate of non-citizen voting in the CCES panel study for 2010 was similar, then there is a 72.4 percent probability (based upon the binomial distribution) that one would get a null result from the 85 person sample in the panel study. \u00a0That is, nearly three quarters of the time if the analysis of the cross-section was entirely unbiased by the issue raised by Ansolabehere et. al. (2015) one would get precisely the result they obtain.<\/div>\n<\/div>\n<\/div>\n\nIf the key analytical results of the Ansolabehere et. al. (2015) paper could have occurred with a high probability if its authors’ claim concerning response \u00a0bias was completely wrong, then additional analysis is needed to further probe the issue. \u00a0My colleagues and I probe these issues \u00a0extensively here<\/a>. \u00a0To sum up our key points: \u00a0after we first point out that their tests lacked statistical power, the second section presents evidence that the citizen status variable in the CCES is more accurate than Ansolabehere et. al. (2015) claim it is, with much of the error accounted for by intentional or unintentional errors made by non-citizens who claim to be citizens. \u00a0We also show that numerous hypotheses that follow from the claim that apparent non-citizen voters are in fact citizens fail. The third section sets aside the evidence from the first and second sections and assumes that Ansolabehere et. al. (2015) were in fact correct about response error.\u00a0 We show that even if their response error argument is correct, there is still significant evidence of non-citizen participation in the U.S. electoral system.<\/div>\n<\/div>\n\n\u00a0“Thus, we believe that the findings in Richman et. al. are driven by measurement error in the CCES, and do not accurately reflect the rates of non-citizen voting in the United States. We agree with Ansolabehere et al. that \u201cthe likely percent of noncitizen voters in recent U.S. elections is 0.\u201d”<\/div>\n<\/blockquote>\nWhile my coauthors and I have always acknowledged the possibility that our findings were biased by measurement error, we took a number of steps to investigate that possibility in the original paper. \u00a0Our response<\/a>\u00a0to Ansolabehere et. al. has gone even further, investigating multiple additional lines of evidence which weigh against their claim that the likely percent of non-citizen voters in recent U.S. elections is 0. \u00a0Our view is that our original analysis was largely valid. \u00a0But we have encouraged people to read Ansolabehere et. al. as well as our original paper.<\/div>\n\n\n“The scholarly political science community has generally rejected the findings in the Richman et al. study and we believe it should not be cited or used in any debate over fraudulent\u00a0voting.”<\/div>\n<\/blockquote>\n<\/div>\nWe have consistently in our communications about our study sought to mention its critics as well. \u00a0We hope that upon a careful and thoughtful weighing of the evidence readers will reach their own thoughtful conclusions. \u00a0We have said things like<\/a> “We stand by our study, but we encourage people to read the critiques too.” \u00a0Ultimately I believe that the debate over fraudulent voting can best advance through a thoughtful exchange of views rather than an attempt to discourage citation or consideration of any study. \u00a0An attempt to excommunicate our study from public debate on the basis of a single under-powered study that would have found precisely what it found if we were completely right somewhere between nearly 1\/3 \u00a0and nearly 3\/4 of the time is premature. \u00a0I share your concern with the way the Richman et. al. 2014 study has been abused by those who would like to cherry pick upper end estimates, but there are more measured and accurate responses available than this one.<\/div>\n","protected":false},"excerpt":{"rendered":"Dear Signers of the Open Letter, There is an “open letter” going around that is critical of my coauthored study on non-citizen voting. \u00a0If the open letter was accurate, I would sign it too. \u00a0And so I can well imagine… Continue Reading →<\/a><\/p>\n","protected":false},"author":817,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","wds_primary_category":0},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/posts\/134"}],"collection":[{"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/users\/817"}],"replies":[{"embeddable":true,"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/comments?post=134"}],"version-history":[{"count":5,"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/posts\/134\/revisions"}],"predecessor-version":[{"id":146,"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/posts\/134\/revisions\/146"}],"wp:attachment":[{"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/media?parent=134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/categories?post=134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fs.wp.odu.edu\/jrichman\/wp-json\/wp\/v2\/tags?post=134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}