Scotland loses 150,000 votes October 23, 2007
Posted by Richard Carback in : Voting Problems , add a commentYet another problem that could be found and/or avoided with the likes of Punchscan or Scantegrity:
The results process was besieged by delays and saw some 140,000 ballot papers rejected.
From the report:
The voter cast a valid vote on both the regional ballot paper and the constituency ballot paper for 96% of the Scottish parliamentary ballot papers counted. The remaining 4% of voters had one or both parts of their ballot papers rejected. The main categories for rejected ballot papers are:
• 50% of these voters cast a valid vote on the regional ballot paper, but left the constituency ballot paper unmarked (about 2% of all voters). In this case, the valid vote was accepted and the blank paper rejected;
• 25% of these voters cast a valid vote on the constituency ballot paper, but left the regional ballot paper unmarked (about 1% of all voters). Again, only the blank paper was rejected;
• Thus, 75% of these voters marked one cross only, on one or the other side of the combined parliamentary ballot papers (about 3% of all voters);
• Of the remaining 25% of rejected ballot papers, over half of voters ‘over-voted’ – casting two or more votes on the regional ballot paper (about 0.6% of all voters).For the 3% that voted for only one side the report offers three possible explanations including a deliberate choice to vote on only one side of the ballot paper; not understanding that there were two votes; and confusion caused by the appearance of named individuals on the regional list.
While this is inherently a usability problem, had they used Punchscan these voters would have gotten feedback from the scanner, rejecting over voted ballots, indicating what races had not been voted, and posting a receipt voters could use to check that the machine was honest online. DREs and other systems can also provide this kind of feedback, but there is no way to check that they are being honest with you.
The content of posts to the Punchscan blog belong to the author and do not necessarily reflect the thoughts, feelings, or opinions of the Punchscan voting project.
How not to do Logic and Accuracy Testing October 18, 2007
Posted by Richard Carback and in : Voting Problems , add a commentI woke up to a new story in my inbox. I was shocked to see the following:
At least 10 ballots are used for each machine for the test.
Wow. Just wow.
The content of posts to the Punchscan blog belong to the author and do not necessarily reflect the thoughts, feelings, or opinions of the Punchscan voting project.
The Importance of Usability October 17, 2007
Posted by Richard Carback and in : Voting Policy , 2 commentsI came across this story from the seminal today. It is the first segment of a multi-part piece interviewing the election reform activist Dan McCrea, and he had some interesting things to say.
He points out the conflicts of interest in Florida in 2000 and Ohio in 2004, talks about how HAVA made things worse, and points out some other interesting things. However, this caught my eye:
While Florida, and to a greater extent Ohio, remain electoral mysteries, election issues in Sarasota, Florida in 2006 seemed to offer election activists the best chance they had yet had of using the legal process to obtain greater transparency in elections.
While I understand that the premise is that seeing the code might reveal something interesting I am not sure how it could ever achieve any level of election transparency. On the other hand I do think that they should have just shown the code. There might have been some unrelated problems in the code, and it would have been a minor problem to fix flaws found in the software compared to the PR disaster of not revealing the code. From what I have seen there is clear evidence that the problem was a ballot design problem, and revealing the code would have put the flawed software idea to rest.
The Herald Tribune did do an analysis for which I am unaware of a good refutation. Michael Shamos also gave a talk at UMBC about the analysis that he and a team performed on the system. While he admitted that there were some flaws none of them could have caused that particular error, and he also indicated that it was a ballot design problem. At WOTE 2007 I had the chance to meet with Ted Selker and he basically said that it was clearly a case of bad ballot design.
What I have not seen, however, are the results of a real-world test of this idea. Ted indicated to me that he was, at least, planning on it, but I’ve not seen or heard about anything since. To me it seems like a highly plausible hypothesis and it would be interesting to see the results of such a test.
If it turns out to be true it would underscore the importance of usability in a voting system — Just because it is on a computer doesn’t mean it is automatically easier to use. There should be some minimal requirements for testing each ballot design before it can be used in an election.
I look forward to reading the next segment of the interview.
The content of posts to the Punchscan blog belong to the author and do not necessarily reflect the thoughts, feelings, or opinions of the Punchscan voting project.
Complexity and Transparency are not the same
Posted by Richard Carback and in : Voting Goals , add a commentOften when we attempt to explain Punchscan to activists and others we hear something like the following:
Darn it. If my 80-year-old grandma can’t understand it, then we shouldn’t use it. Elections should be transparent.
I have been meaning to address this issue in a blog post for some time, but today I see that Ben Adida has already done a pretty good job. While people may have a point that Punchscan and E2E in general is somewhat complex, this is not the same as transparency. While not everyone will take the time to understand it, the fact remains that a normal citizen could do it if they so desired, and that is the key distinction between E2E and many other systems. There are no experts, closed sources, closed designs, or NDAs. Ben’s definition of transparency is as follows:
A system is transparent if, given a reasonable amount of time and effort, a person with a college education can understand it. Then, those without the education, time, or willingness to understand it can consult with someone they trust who does understand it.
I sort-of agree with this statement, although I would use high-school instead of college. In reality, I think that transparency is fundamentally different, and it has to do with the level of observance people can exert on the election. Anyway, we have been saying something similar to this for a long time now. Here’s what we say on our FAQ about it:
The actual system could, we expect, with such a mock election as introduction, be taught in advanced high school science or college classes. This is many times simpler than convincing anyone of how the software in current voting systems works—even if people were allowed to see it!
I see no reason why anyone with a minimal education would not be able to understand Punchscan with the help of a 1-2 week (or day) course.
The only thing I find even a little objectionable about Ben’s post is the following:
…to anyone familiar with quality control processes, the ballot chain-of-custody is a reliability nightmare: how does one check that no one has tampered with a ballot box full of de-identified ballots that no one can look at during the 24 crucial hours where low-wage, minimally trained election workers are entirely responsible for them?
This doesn’t exactly do justice to the solution most activists are trying to promote. They want semi-translucent boxes in full view, and counting done on the same day at the same place in a specific way in front of anyone who is able to observe. There are still many problems with this but it is a bit different than centralized counting or the DREs we have today.
Kudos to Ben on an excellent post.
The content of posts to the Punchscan blog belong to the author and do not necessarily reflect the thoughts, feelings, or opinions of the Punchscan voting project.
CPSR Election Wrap-up October 12, 2007
Posted by Richard Carback and in : Elections , add a commentWe are now finishing off the 2007 board member election for Computer Professionals for Social Responsibility (CPSR) with the Punchscan software. They have a history of being active about voting issues and have worked with the Verified Voting Foundation. One particular accomplishment of CPSR is EIRS, the web-based Election Incident Reporting System that lets people report problems on election day.
Overall the election went well, and it was a joy to work with CPSR. Over 2/3 of ballot casting voters appear to have had no problem with the system, but there were some issues stemming from the fact that Punchscan was being used over the internet, outside of its usual environment. This provided numerous learning opportunities.
Punchscan over the Internet
Before I talk about what we learned, I want to talk a bit about how the election was conducted because it was not conducted in the usual or intended way. We sent each voter an e-mail with a link to a website where he or she could download a ballot by serial number. The link used SSL with a random unique id, and voters chose from a randomly generated set of ballot serial numbers. After downloading a ballot, the voter could not download a subsequent ballot without requesting a reauthorization code that would invalidate the first ballot the voter downloaded and allow the voter to choose a new ballot.
Each ballot (sample) was a generated PDF with both sets of letters that used javascript in Adobe Reader to let each voter mark it by clicking on candidate names. After a ballot is loaded, instructions pop up explaining how to use the ballot. To vote, a voter clicks on the letters next to the names of the candidates.
After voting, a voter clicks on “Proceed to ballot casting”, is given the choice to remove the top or bottom sheet letters (creating a top/bottom sheet receipt), and is able to print out the ballot. To cast the ballot, the voter could mail or fax it to us. When we received a ballot image, we uploaded it with appropriate marks to the website. Auditing, etc, proceeds as normal at this point.
What We Learned
Fundamentally, the problem that Punchscan encountered was that there were no poll workers to answer voter questions and help them scan the ballots. While we did not receive many questions, about 1/3 of the submitted ballots had problems. Half of the voters did things outside of the directions. The other half of these problems can be explained by the voter not having or using Adobe to mark his or her ballot.
The fact that some voters did not follow directions is not surprising, and in most optical scan systems ballots that are not marked properly are simply not counted. However, in Punchscan a poorly marked ballot does not mean the voter’s vote will not count because the scanner tells the voter what it saw. A poll worker can help the voter clean up the marks and rescan, or give the voter a new ballot to use if it cannot be saved. Because each voter sent his or her ballot over Fax, we could not easily provide such a feedback mechanism. Instead, we requested that any problematic ballots be resent. Unfortunately, this did not work, so CPSR told us to accept and count any ballot for which we could interpret voter intention.
We were surprised that so many people had trouble with Adobe. We chose Adobe because it works on Linux/Mac/Windows, and it has some accessibility features that people might use. In hindsight, it was probably not wise to use it for this particular audience. I do not use Adobe. I consider it kind of bloated, and most of my computers simply don’t have enough power to use it without a huge slowdown. While in the group I am alone in this opinion, we probably should have expected that there would be at least a couple people like me out there who were trying to vote. We might have provide some sort of java-based executable that would run on windows/linux/mac, but I am not sure if I like that idea. I do not know what else we could have used to fix this situation.
Another problem that I do not think we foresaw is that voting by fax is inconvenient in this context. As far as I know, the last CPSR election was done via internet. They told us to expect approximately 70 voters (20% of the total membership), and 68 downloaded a ballot while only half that many cast a ballot. I suspect that these voters who downloaded but did not cast were expecting a web form to cast their ballots. When you think about it, fax machines are found in offices, and are not a piece of technology usually found in a house. I do not have one, none of my family members have one, and only a select few people I know do.
Mistakes
Outside of people having trouble with Adobe, we made a few mistakes of our own. The download screen said “You downloaded ballot.” and not “You downloaded a ballot.” People thought it was supposed to say “You downloaded ballot <number>”. Eventually, we actually edited it to do that.
When we first started, we did not realize our SSL certificate was not universally available, and people saw errors saying that the identity of the server could not be verified. When someone asked about it, we got a new one and were able to replace it by the third day.
There was a problem with IE downloading ballots. We had to work with a voter to resolve it (thanks James).
In the Adobe directions that pop up on the ballot, there was a typo. Between two of the directions, there was no newline, so some people were confused, thinking directions skipped from 2 to 4. In general, we could have spent more time thinking about how to present directions on the ballot.
Statistics
For those interested:
Spoiled ballots: 4 by voters, 4 by me.
Total ballots downloaded (minus spoiled ballots): 68
Total ballots submitted: 33
Total ballots read and counted: 31
There were 2 ballots we could not read. One did not have any information on it that we could use (blank), the other was a copy of the sample ballot and did not have the signature of an eligible voter.
Congratulations to the winners!
The content of posts to the Punchscan blog belong to the author and do not necessarily reflect the thoughts, feelings, or opinions of the Punchscan voting project.