In my read of the opinion by Judge Robinson in the Kobach trial, I was disappointed to see that the judge’s interpretation of the data was riddled with omissions and inaccuracies. Let me give you a few examples:
Throughout the entire opinion Judge Robinson ignored half of the evidence I brought to the questions at issue – all of the evidence concerning nationwide rates of non-citizen registration to vote, and all of the evidence concerning non-citizen registration to vote in other states. But surely such evidence is relevant. In the absence of a distinctive legal framework, one would expect Kansas and national non-citizen voter registration rates to be similar. I made this argument explicitly in my reports and to the best of my memory the expert witnesses for the plaintiffs did not take issue with it. None the less, the Judge Robinson ignored all of this evidence entirely.
The judge appeared to lack basic knowledge of the interpretation of statistical estimates. For instance, in a discussion of my analysis of a survey of individuals on the suspense list, Judge Robinson wrote: “Dr. Jesse Richman estimates that only 2.2% of the applicants on the suspense list lack access to DPOC, based on a survey he conducted of individuals on the suspense list. Yet Dr. Richman’s results are not statistically distinguishable from zero, as the margin of error is 2.7.” If the results are not distinguishable from zero, then then this implies that we CANNOT REJECT the null hypothesis that ALL citizens on the suspense list have access to DPOC. This would indicate the possibility that no citizen was excluded from registering to vote due to lack of DPOC, which would actually be a very strong result in favor of the argument made by the state of Kansas if true.
The lack of basic knowledge of statistics allowed the judge to be led astray by trust in the plaintiffs experts even when they were patently and obviously wrong. For example, Professor Ansolabehere combined the analyses in my initial report inappropriately. Not only did he merely average them (not appropriate as it ignores variation in sample sizes) but he calculated the standard error of this estimate completely the wrong way. I demolished Dr. Ansolabehere’s calculation of a confidence interval for his ‘meta analysis’ of the various Kansas results. But the court sided with him none the less. Thus the court refused to recognize even basic statistical facts such as the appropriate calculation of the standard error of the mean, as I pointed out in paragraph 22 of my supplemental report. Ansolabehere incorrectly calculated the standard error of the mean as the standard deviation when it is in fact the standard deviation divided by the square root of the number of observations, as any basic statistics primer will tell you. See for instance: http://davidmlane.com/hyperstat/A103735.html.
And Judge Robinson also relied upon incorrect calculations by the plaintiffs experts concerning the confidence intervals for proportions for each of the estimates in my report. But don’t take my word for it. Let me walk you through it. One of the estimates I examined involved the proportion of newly naturalized citizens who registered to vote in Sedgwick County Kansas who had a voter registration history dating from before their naturalization. The basic numbers are that 8 out of 791 had a prior registration history. Now take this to the first calculator that pops up when you search Google for “confidence interval for proportions calculator” which is http://www.sample-size.net/confidence-interval-proportion/. Enter the sample size as 791 and the number x = 8. When you hit calculate you will see that both confidence intervals do not include zero, contrary to the assertion made by Judge Robinson concerning this estimate. What this means is that we do have statistically significant evidence of prior registration — it is unlikely to be zero. The decision asserted otherwise.