We are now finishing off the 2007 board member election for Computer Professionals for Social Responsibility (CPSR) with the Punchscan software. They have a history of being active about voting issues and have worked with the Verified Voting Foundation. One particular accomplishment of CPSR is EIRS, the web-based Election Incident Reporting System that lets people report problems on election day.

Overall the election went well, and it was a joy to work with CPSR. Over 2/3 of ballot casting voters appear to have had no problem with the system, but there were some issues stemming from the fact that Punchscan was being used over the internet, outside of its usual environment. This provided numerous learning opportunities.

Punchscan over the Internet

Before I talk about what we learned, I want to talk a bit about how the election was conducted because it was not conducted in the usual or intended way. We sent each voter an e-mail with a link to a website where he or she could download a ballot by serial number. The link used SSL with a random unique id, and voters chose from a randomly generated set of ballot serial numbers. After downloading a ballot, the voter could not download a subsequent ballot without requesting a reauthorization code that would invalidate the first ballot the voter downloaded and allow the voter to choose a new ballot.

Each ballot (sample) was a generated PDF with both sets of letters that used javascript in Adobe Reader to let each voter mark it by clicking on candidate names. After a ballot is loaded, instructions pop up explaining how to use the ballot. To vote, a voter clicks on the letters next to the names of the candidates.

After voting, a voter clicks on “Proceed to ballot casting”, is given the choice to remove the top or bottom sheet letters (creating a top/bottom sheet receipt), and is able to print out the ballot. To cast the ballot, the voter could mail or fax it to us. When we received a ballot image, we uploaded it with appropriate marks to the website. Auditing, etc, proceeds as normal at this point.

What We Learned

Fundamentally, the problem that Punchscan encountered was that there were no poll workers to answer voter questions and help them scan the ballots. While we did not receive many questions, about 1/3 of the submitted ballots had problems. Half of the voters did things outside of the directions. The other half of these problems can be explained by the voter not having or using Adobe to mark his or her ballot.

The fact that some voters did not follow directions is not surprising, and in most optical scan systems ballots that are not marked properly are simply not counted. However, in Punchscan a poorly marked ballot does not mean the voter’s vote will not count because the scanner tells the voter what it saw. A poll worker can help the voter clean up the marks and rescan, or give the voter a new ballot to use if it cannot be saved. Because each voter sent his or her ballot over Fax, we could not easily provide such a feedback mechanism. Instead, we requested that any problematic ballots be resent. Unfortunately, this did not work, so CPSR told us to accept and count any ballot for which we could interpret voter intention.

We were surprised that so many people had trouble with Adobe. We chose Adobe because it works on Linux/Mac/Windows, and it has some accessibility features that people might use. In hindsight, it was probably not wise to use it for this particular audience. I do not use Adobe. I consider it kind of bloated, and most of my computers simply don’t have enough power to use it without a huge slowdown. While in the group I am alone in this opinion, we probably should have expected that there would be at least a couple people like me out there who were trying to vote. We might have provide some sort of java-based executable that would run on windows/linux/mac, but I am not sure if I like that idea. I do not know what else we could have used to fix this situation.

Another problem that I do not think we foresaw is that voting by fax is inconvenient in this context. As far as I know, the last CPSR election was done via internet. They told us to expect approximately 70 voters (20% of the total membership), and 68 downloaded a ballot while only half that many cast a ballot. I suspect that these voters who downloaded but did not cast were expecting a web form to cast their ballots. When you think about it, fax machines are found in offices, and are not a piece of technology usually found in a house. I do not have one, none of my family members have one, and only a select few people I know do.

Mistakes

Outside of people having trouble with Adobe, we made a few mistakes of our own. The download screen said “You downloaded ballot.” and not “You downloaded a ballot.” People thought it was supposed to say “You downloaded ballot <number>”. Eventually, we actually edited it to do that.

When we first started, we did not realize our SSL certificate was not universally available, and people saw errors saying that the identity of the server could not be verified. When someone asked about it, we got a new one and were able to replace it by the third day.

There was a problem with IE downloading ballots. We had to work with a voter to resolve it (thanks James).

In the Adobe directions that pop up on the ballot, there was a typo. Between two of the directions, there was no newline, so some people were confused, thinking directions skipped from 2 to 4. In general, we could have spent more time thinking about how to present directions on the ballot.

Statistics

For those interested:

Spoiled ballots: 4 by voters, 4 by me.
Total ballots downloaded (minus spoiled ballots): 68
Total ballots submitted: 33
Total ballots read and counted: 31

There were 2 ballots we could not read. One did not have any information on it that we could use (blank), the other was a copy of the sample ballot and did not have the signature of an eligible voter.

Congratulations to the winners!

This entry was posted on Friday, October 12th, 2007 at 3:51 pm and is filed under Elections. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.