On the bubble: Artificial intelligence could help elections count hard-marked ballots with greater speed and accuracy

Backed with a $1.2 million grant from the National Science Foundation, researchers from three universities including Texas A&M University will investigate the robustness and security of election systems that read hand-marked paper ballots.

Their goal is to build a system that uses artificial intelligence to read ballots more quickly and accurately.

Nitesh Saxena, professor in the Department of Computer Science and Engineering at Texas A&M University, will lead the team, which includes Dan Wallach, professor at Rice University, and Chengcui Zhang, professor at the University of Alabama at Birmingham.

The Texas A&M team received nearly $400,000 for its part in the project.

Across the country, nonpartisan election officials face a demand to produce the results of local and state elections quickly and accurately, and sometimes, accuracy can suffer because of this.

While modern machine-learning and deep-learning techniques have revolutionized how computers “see” the world, even the most sophisticated systems used to scan and process hand-marked ballots primarily look at the average darkness across bubble targets on a sheet.

As a result, traditional paper ballot scanners can miss partially filled-in marks or misclassify stray marks and scanner noise as filled-in bubbles, depending on how they’re configured. In addition, existing systems are not inherently designed to identify potentially fraudulent voting cases where a single voter has filled out multiple ballots.

The proposed research aims to respond to these issues by building an artificial intelligence system called Bubble Aid. By training on data from millions of actual ballots, the system will use modern computer vision and learning techniques to recognize hand-marked bubble targets more effectively than existing systems.