I had meant to post this yesterday, but I woke up not feeling well and spent the day in bed. I see that Ben Adida and Avi Rubin have already posted their experiences. Aleks Essex also posted his experiences a few weeks ago when he was a worker for the most recent Canadian election.

The chief judges invited all the site judges to meet up at the fire house the night before at 6:30 to make room for everything. We also plugged in the DREs, which you can conveniently do without opening the units. I didn’t ask, but the average age was likely in the 40s. We had 1 high school student who was 17, and my guess is that there was an even split between the 20-40 crowd and the 40+ crowd. There were 3 republicans, and the rest were democrats except for the unaffiliated high school student.

After we finished, the Chief Judges explained what would happen in the morning, and asked if there were any questions. The biggest concerns were potential turnout and if we’d get the extra machines we had up in time with the same number of judges as before. They also expressed concern about a technician showing up and staying for the whole day.

Getting Started

I woke up @ 4:30 and arrived a little after 5:30. Myself and another judge set up each DRE while 1-3 judges watched and recorded information. I recall it being a fairly simple process and working pretty well with the exception that the chief judge sometimes had trouble opening the printer compartment. I was a little concerned that the DREs, despite being plugged in all night, all registered only a 60-80% charge.

The electronic poll books were equally easy to set up although some instructions were confusing, telling the user not to plug the poll books into the UPS unit. I think what they meant is to plug a power strip into the UPS unit and then to plug the poll books into that strip, which is what the other judges decided to do.

Voting

Our longest line happened when we first opened the polls, which went all the way out the door and around the corner. People at the end of that line were still cheery, as they only ended up waiting approximately 15-20 minutes. We had a steady stream of people until about 10:30, which picked right up again at 11:00, and stayed mostly steady for the rest of the day with occasional 5-10 minute breaks in flow. Check-in time only took about a minute if you were listed in the poll book. I’m not totally sure, but I believe voting varied from 1-10 minutes, although some voters probably took longer.

I served as a voting unit judge for the whole day, which is all standing (my legs were killing me that night)! This involved taking a voter authority paper, initialing it, writing the unit number you gave the voter, getting the voter started on the machine, explaining how it worked (if necessary), and dropping the the paper into an envelope on the side. The papers are tallied every hour, and the chief judges make rounds to verify the counts match the machine totals so far.

A majority of our 8 machines were in use for almost the whole day, and at the beginning and towards the end, we had periods of about an hour and a half where all of them were constantly in use. No one used the headset or keypad, although we did have a few voters who requested assistance or needed a chair. Many voters used the large text and apparently the button was not big enough as this was our most asked question.

Glitches

We had no major equipment failures during the voting day. The biggest equipment problem was that the smart cards were sometimes difficult for voters to get into the machine, which we solved by starting it in the machine for the voter, and letting him or her push it in until it snapped.

The other problems all involved the electronic poll books, which sported a confusing user interface to navigate. The judges managed and came up with troubleshooting steps. I was later told that a couple voters would not come up on the local polling site search, but when you went to the statewide search, they did and were registered at our local polling site.

I don’t know a specific number, but at least 15 people or so came in and had trouble during the check-in process. About half voted provisionally, and the rest either went to the correct polling place or admitted they knew they were not registered. I believe there might have been one person who walked out on us, which is really unfortunate. There needs to be a better way to deal with registration problems.

Although it mostly worked well for the vast majority of voters, a few voters were tripped up by the interface. At least one person asked me a question after having pressed cast ballot. Another couple asked why the judges question, a vote for 2 race, was red (because they had not voted for two candidates).

Ballot question wording was also an issue, and understandably so due to the legal language. A number of people called us over to ask me the meaning of various passages, and a few even to ask which answer was “yes”, and which was “no.” It would be best if the language were simple and clear.

Early in the day 1 voter complained about the privacy of the machines. We flipped down the screen for him so it faced the ceiling and told him we could angle the machine however he wanted. It turned out to be a more general problem because there was no curtain. That explains what happened later, when we got a call that someone complained that people were being watched as they voted, and someone from the central office came by to check in on us.

Lots of voters complained about the security of the machines. Some were curious why others did not think they were secure. I just tried to stay away from that discussion as much as possible.

Cleaning Up

At the end of the day, we had 1045 DRE voters, and 8 provisional ballot voters. The chief judges told us that the last major election had 300 out of about 1500-1600 registered voters for our precinct overall (about 200 voters per machine), which is a significant improvement in turnout over previous years.

We finished around 9:30. Tearing down was much worse than getting started, as you had to navigate through a bunch of menus that really didn’t need to be there. An “end election” button that printed the necessary information would have sped things up significantly. The accumulator machine did not recognize the modem, so results were phoned in and then the memory cards were driven back by both the chief judges.

Results

Poll-tape Results for our poll site (the English Consul Volunteer Fire Department, 13-8), were recorded by me as follows:

I hope I didn’t make a mistake. You may want to look at the ballot.

2008 Baltimore County Specimen Ballot

You can also download it in PDF. Only 2^44 unique marking patterns this time. Anyone want to determine how many legal patterns there are using my handy guide?

I’m also serving as an election judge. I’m looking forward to it.

Happy voting!

Our friends at the Ottawa Canada Linux Users Group kindly invited Rick and I to give a talk at their annual general meeting in which we demonstrated Scantegrity II. We conducted a mock election of their board of directors in part to demonstrate our invisible ink printing capability as well as to gather some preliminary feedback about the voting experience.

Their response seems promising. Though not a statistically significant sample, all survey respondees indicated they were “confident that I filled out my ballot correctly.” This obviously cannot be taken as an indication of overall usability. I do however think it demonstrates at least that we’ve moved past the usability restrictions of the Punchscan ballot and are on to other topics of discussion.

Also of note was that most respondees would consider using (and/or authoring) an open source software tool for the cryptographic integrity check. I think this is the more academically interesting question. The voting process itself is a bare minimum in terms of overall usability requirements. However it is the people performing the E2E verification steps that give this technology its raison d’être, so we’d hope they would be reasonably able to understand the purpose of, and use, the receipt check/audit tools.

Photos of the event can be seen here.

Aleks and Rick speaking at the OCLUG AGM. (Photo courtesy of Richard Guy Briggs)

Voting with Scantegrity II. (Photo courtesy of Richard Guy Briggs)

Invisible ink print setup: Off-the-shelf inkjet printer, continuous-feed ink system (for bulk specialty invisible ink).

A marked OCLUG mock ballot showing confirmation codes and voter generated receipt.

We are now finishing off the 2007 board member election for Computer Professionals for Social Responsibility (CPSR) with the Punchscan software. They have a history of being active about voting issues and have worked with the Verified Voting Foundation. One particular accomplishment of CPSR is EIRS, the web-based Election Incident Reporting System that lets people report problems on election day.

Overall the election went well, and it was a joy to work with CPSR. Over 2/3 of ballot casting voters appear to have had no problem with the system, but there were some issues stemming from the fact that Punchscan was being used over the internet, outside of its usual environment. This provided numerous learning opportunities.

Punchscan over the Internet

Before I talk about what we learned, I want to talk a bit about how the election was conducted because it was not conducted in the usual or intended way. We sent each voter an e-mail with a link to a website where he or she could download a ballot by serial number. The link used SSL with a random unique id, and voters chose from a randomly generated set of ballot serial numbers. After downloading a ballot, the voter could not download a subsequent ballot without requesting a reauthorization code that would invalidate the first ballot the voter downloaded and allow the voter to choose a new ballot.

Each ballot (sample) was a generated PDF with both sets of letters that used javascript in Adobe Reader to let each voter mark it by clicking on candidate names. After a ballot is loaded, instructions pop up explaining how to use the ballot. To vote, a voter clicks on the letters next to the names of the candidates.

After voting, a voter clicks on “Proceed to ballot casting”, is given the choice to remove the top or bottom sheet letters (creating a top/bottom sheet receipt), and is able to print out the ballot. To cast the ballot, the voter could mail or fax it to us. When we received a ballot image, we uploaded it with appropriate marks to the website. Auditing, etc, proceeds as normal at this point.

What We Learned

Fundamentally, the problem that Punchscan encountered was that there were no poll workers to answer voter questions and help them scan the ballots. While we did not receive many questions, about 1/3 of the submitted ballots had problems. Half of the voters did things outside of the directions. The other half of these problems can be explained by the voter not having or using Adobe to mark his or her ballot.

The fact that some voters did not follow directions is not surprising, and in most optical scan systems ballots that are not marked properly are simply not counted. However, in Punchscan a poorly marked ballot does not mean the voter’s vote will not count because the scanner tells the voter what it saw. A poll worker can help the voter clean up the marks and rescan, or give the voter a new ballot to use if it cannot be saved. Because each voter sent his or her ballot over Fax, we could not easily provide such a feedback mechanism. Instead, we requested that any problematic ballots be resent. Unfortunately, this did not work, so CPSR told us to accept and count any ballot for which we could interpret voter intention.

We were surprised that so many people had trouble with Adobe. We chose Adobe because it works on Linux/Mac/Windows, and it has some accessibility features that people might use. In hindsight, it was probably not wise to use it for this particular audience. I do not use Adobe. I consider it kind of bloated, and most of my computers simply don’t have enough power to use it without a huge slowdown. While in the group I am alone in this opinion, we probably should have expected that there would be at least a couple people like me out there who were trying to vote. We might have provide some sort of java-based executable that would run on windows/linux/mac, but I am not sure if I like that idea. I do not know what else we could have used to fix this situation.

Another problem that I do not think we foresaw is that voting by fax is inconvenient in this context. As far as I know, the last CPSR election was done via internet. They told us to expect approximately 70 voters (20% of the total membership), and 68 downloaded a ballot while only half that many cast a ballot. I suspect that these voters who downloaded but did not cast were expecting a web form to cast their ballots. When you think about it, fax machines are found in offices, and are not a piece of technology usually found in a house. I do not have one, none of my family members have one, and only a select few people I know do.

Mistakes

Outside of people having trouble with Adobe, we made a few mistakes of our own. The download screen said “You downloaded ballot.” and not “You downloaded a ballot.” People thought it was supposed to say “You downloaded ballot <number>”. Eventually, we actually edited it to do that.

When we first started, we did not realize our SSL certificate was not universally available, and people saw errors saying that the identity of the server could not be verified. When someone asked about it, we got a new one and were able to replace it by the third day.

There was a problem with IE downloading ballots. We had to work with a voter to resolve it (thanks James).

In the Adobe directions that pop up on the ballot, there was a typo. Between two of the directions, there was no newline, so some people were confused, thinking directions skipped from 2 to 4. In general, we could have spent more time thinking about how to present directions on the ballot.

Statistics

For those interested:

Spoiled ballots: 4 by voters, 4 by me.
Total ballots downloaded (minus spoiled ballots): 68
Total ballots submitted: 33
Total ballots read and counted: 31

There were 2 ballots we could not read. One did not have any information on it that we could use (blank), the other was a copy of the sample ballot and did not have the signature of an eligible voter.

Congratulations to the winners!