Volusia County, FL, 2020 General Election

• Errors Identified, but No Impact on Election Outcomes.
  Volusia County had to deal with a number of errors that happened due to poor tracking by ES&S (Election Systems & Software, the voting system vendor) of USB drives uploaded and what data is cleared when reset. (See more detail below.)

• Found 4,904 ballots with two images each in the election records.
  Just prior to the close of polls, the Volusia election office started uploading data from the early voting sites. The first USB drive was loaded correctly but the second drive failed from one polling place. They cleared the election and rescanned all paper ballots from that first site. Unfortunately, not all data was actually cleared, and the 4,904 CVR records and ballot images from the first USB drive were still in the system. This suggests a potential defect in the voting system, as results were not cleared appropriately.

  As rescans, the additional ballot images could not easily be located within the images or CVRs, as the order of the new scans will be in a different order and the images and CVR records are shuffled by the election management system (EMS). The images may be identical to the eye, but differ digitally, and that makes them difficult to locate.

• No machine failure was reported and no mention of the rescan.
  Even though the rescanning was done in public, this fact was not documented in their election report and the check box for device failure was unchecked. The failure to document the rescan or mark the checkbox for device failure highlights a procedural oversight.

• Found 537 ballot images and CVRs were missing.
  The USB drive from one of two machines from one voting site was not uploaded, even though the aggregated totals were wirelessly transmitted. The EMS should know if all USB drives are properly uploaded but does not notify the user if any mistakes are made. Election officials became aware of the missing upload only after we identified it in our report. This issue suggests a shortcoming in the voting system software as it should be able to track all the uploads and provide an alert if they are not all uploaded.

• This evidence shows that ES&S maintains two independent tabulations and they are not compared by the EMS.
  The polling place totals are transmitted by “secure” cell modem to the central office. Although this is fast, ES&S claims these are unofficial “tentative” results. Later, the aggregated totals, CVRs and images are uploaded from USB drives, to confirm the results sent in earlier.

  We proved from this election, however, that the final results for polling places are generated ONLY from the totals received by modem. The CVRs are NOT used to generate the final results, and although they are consistent inside any single voting machine, they can grow to differ due to rescanning or missing uploads.

• This evidence shows that the two independent tabulations are not compared by the ES&S voting system.

  Based on this evidence, we were able to conclude that ES&S never compares the two tabulations (one from the CVR and one from the aggregated totals) for consistency, and they don’t require the data from the USB drives. They rely only on the unofficial tentative cell modem transmitted results if the USB is missing.

• The AuditEngine Team Conducted a “Poll Tapes Audit”

  A “Poll Tapes Audit” consists of reviewing the printed cash-register-like printouts from voting machine scanners in polling places. In this case, we used digital reproductions of those tapes. This additional audit component covered only a portion of the results, those from in-person polling places.

  The Poll Tapes Audit confirmed that the one thumb drive had not been uploaded, but the other results were consistent with the reported results, including the polling place where the repeated ballot scans were included.

• Disagreement Rate

  Overall we found an individual contest disagreement rate of 0.237%, or about 24 disagreements per 10,000 ballots between the result by AuditEngine and the voting system across all contests. An individual contest is a single contest on one specific ballot sheet.

• Narrative Report For additional details, refer to the AuditEngine Narrative case study report on three Florida counties: Case Study Report: Three Counties in Florida

In summary, while these discrepancies did not affect the overall outcome of the election, the identified issues point to several procedural and software deficiencies that should be addressed to enhance election integrity.

St. Lucie County, FL, 2020 General Election

• Lucie did not generate a ballot-level Cast Vote Record (CVR), making it impossible to perform a ballot-by-ballot comparison.

• Received 8 image archives with one repeated inside another combined archive. This took a lot of hand work to resolve.

• There were 526 more ballot images than the number of ballots officially cast. This was fewer than the closest vote margin. The discrepancy of 526 more ballot images than officially cast ballots could not be further investigated due to the lack of comprehensive reporting from the election system.

• AuditEngine found an additional 135 ballots cast with 123 additional votes for Biden and three fewer for Trump. Again, we could not diagnose this further due to the lack of CVR.

• Narrative Report: See the Three-Florida Counties Case Study Report: Case Study Report: Three Counties in Florida

In conclusion, the St. Lucie County audit was limited by the lack of ballot-level CVRs and comprehensive reports, making it difficult to investigate key discrepancies, but the images were still fully retabulated and some comparisons were possible.

Collier County, FL, 2020 General Election

• Clean data: Of the three counties we audited in FL, Collier County had the cleanest and most complete data, allowing for smoother analysis with minimal discrepancies. We used fully automated mapping using image processing but it was the last time we used this method. Although we employed fully automated image processing to map the targets on hand-marked paper ballots, we later discontinued this method in favor of using interactive tools or automated mapping using “Ballot Style Masters” that induce fewer recognition errors.

• Two Sheet ballot for all voters: They had a two-sheet ballot for all voters, but these two sheets are combined to only one BMD (Ballot Marking Device) summary page when BMDs are used.

• Repeated BMD images and CVR records.
  Here, because they had a two-sheet ballot for all voters and BMD ballots contain all contests on one sheet, they repeated the ballot images for any BMDs, using two different numbers, and created two CVR records for each single BMD sheet, with only part of the contests in each CVR record. Thus, by looking at the CVR, you would not know that the ballot is a BMD.

• At first, it seemed the ballot inventory was off by 39, with more images than ballots cast. However, after accounting for the repeated BMD images and CVRs, the ballot image and CVR counts matched exactly. (See full report for more detail).

• There were 324 ballot sheets with disagreements out of the 417,777 images processed (0.077%).

• Narrative Report: See the Three-Florida Counties Case Study Report: Case Study Report: Three Counties in Florida

Overall, Collier County’s clean data and low discrepancy rate indicate a well-managed election with minimal issues.

Bartow County, GA 2020 General Election

• Excellent clean data: Bartow County provided accurate, well-organized data, allowing for a smooth audit process. 50,678 ballots were cast in the official election results and the same number of ballot images were provided and processed.

• Low Disagreements:

  Of 1,035,141 individual contests, 328 individual contests were evaluated as disagreed (0.031%) spread across 27 contests.

• There were about 40,000 BMD ballots and all but one was read and was consistent. That one ballot had curved text that could not be accurately processed by our automated systems, but can happen if the ballot sheet creeps to one side as it is fed through the scanner.

• Bartow reclassified overvotes that were reviewed as undervotes.
  Bartow County reclassified overvotes (when more than one candidate is selected in a contest) as undervotes (where no vote is cast in a contest) after they were reviewed. While this doesn’t change election results, it affects the accuracy of vote statistics by inflating undervotes and reducing overvotes to zero. However, this procedure is one way to track whether all overvotes have been reviewed.

• All overvotes and write-ins were adjudicated by County staff.

• There was one incorrect adjudication which was detected and was probably just a mouse-clicking error. This did not affect the election outcome.

• Narrative Report: Bartow County, GA Narrative Report

Overall, Bartow County’s election data was clean and accurate, with only minimal issues that did not impact the overall results.

Fulton County, GA, 2020 General Election

• 72% of the images were missing:
  Due to 72% of ballot images being unavailable (overwritten for the January 2021 runoff election), we were only able to audit 28% of ballot images. This significantly limited the scope of the audit.

• Although the ballots were rescanned and tabulated in “Machine Count 2” (MC2), the absence of corresponding Cast Vote Record (CVR) data prevented a ballot-level comparison, so these images were excluded from the audit.

• AuditEngine processed 148,580 images out of a total of 528,777 ballots. This included 3,923 BMD ballots. The result of the audit was 2,018 disagreed votes (individual contests) among 2,797,303 individual contests—a disagreement rate of 0.072%. This means that over 99.928% of the individual contests were fully agreed upon.

• AuditEngine “read” the human-readable text on 3,923 Dominion BMD summary sheets with no disagreements.

• Ballots from adjacent DeKalb County were mixed in and processed as if they were Fulton County ballots (5 total). Because the ballot size and style numbers overlap between the two counties, the voting system misinterpreted down-ballot contests, leading to incorrect interpretations.

• Ballots from the May 19, 2020 primary election were mistakenly included but were not processed. It’s notable that the mismatched ballot lengths weren’t caught earlier in the process.

• Federal-Only ballots can create a privacy hazard, but can be fixed. Many federal-only ballots are classified to an appropriate style for that voter if they receive the full-length ballot, and this results in only one ballot per style in many cases. These ballots are all the same in terms of the contests on them. Classifying the 650 federal-only ballots into 199 different ballot styles increases the likelihood of identifying individual voters. Consolidating these into fewer styles would mitigate this privacy risk.

  We are still researching why these federal-only ballots are being used as election codes says that the appropriate ballot for the most recent residence address should be used for military and civilian overseas voters.

• Narrative Report: Fulton County, GA 2020 General Election Audit Report

In summary, while 28% of the ballots were successfully audited, the majority of Fulton County’s election data was unavailable. Among the ballots processed, the vast majority of contests were consistent, with only minor discrepancies identified.

DeKalb County, GA, 2022-05-24 Primary Election

• Misconfiguration: Donald Broussard, a candidate for ‘County Commissioner D2 - Dem,’ dropped out before the election. However, the voting system wasn’t reconfigured correctly, and his name still appeared on ballots, leading to incorrect results for the contest.

• The contest was subsequently fully hand-counted and was consistent with AuditEngine results.

• QR Code Misread:

  • We detected this misconfiguration issue early in the audit process, during the mapping of the contests. If AuditEngine were used by the county in the cooperative workflow, this issue would have been detected prior to the election.

  • Hand-marked and BMD ballots still included Broussard, although the Cast Vote Record (CVR) did not.

  • The hand-marked ballots were incorrectly printed with the dropped candidate and interpreted consistent with the misprinted ballots, except that any votes for the dropped candidate (which did exist) were ignored.

  • The BMDs would allow voters to vote for the dropped candidate, and the text summary had the name of the dropped candidate printed. The QR code seemed to match the text, but we couldn’t verify this without more complex reverse engineering.

  • The QR codes use bit fields to represent votes, and the voting system’s scanners misread these fields, changing the outcome of the contest, as the scanners were programmed as if the BMD device would produce ballots with the newly defined QR code encoding.

  • The BMD devices, which generated the ballots, were not updated to exclude Broussard. However, the scanners that read the ballots were reconfigured, leading to the mismatch.

• In total, 135 votes were cast for Broussard, who had dropped out, but applying these votes in any scenario wouldn’t have changed the outcome of the contest.

• After correcting for these errors, Michelle Long Spears gained 3,141 additional votes, moving from third place to first, ultimately winning the contest.

• 453 CVR records were (and still are) missing from our set. These missing records may have impacted the final audit results.

• One BMD ballot from the November 2, 2021 election was mixed in and overprinted over the barcode, a curiosity that would not affect any outcome.

• Several BMD ballots had text that was unreadable because they were lightly printed even though the QR Code was read by the machine. Some were printed so darkly that they were very difficult to read. This raises concerns about voter verification and transparency.

In summary, significant errors, such as the inclusion of a dropped candidate and misinterpretation of QR codes, led to incorrect election results in DeKalb County’s 2022 Primary Election. A hand count of BMD ballots corrected these discrepancies, with Michelle Long Spears gaining 3,141 votes to win the contest. These issues underscore the vulnerability of QR-coded ballots, which can be misread by scanners, highlighting the importance of more transparent and reliable voting methods.

Dane County, WI, 2020 General Election

• Dane County uses ES&S voting systems and has posted the ballot images online for many elections. There were 344,347 unique ballot images in the archive with 3,069 ballot images repeated with the same ballot identifier. There were 26,358 BMD ballots (7.6%). There were 67 styles in this election with 36 different patterns of contests and 33 different contests.

• Excellent Consistency: There were 363 sheets with at least one individual contest categorized as “disagreed”. The two most discrepant contests (as a fraction of the margin of victory) had only 1 or 2 disagreements (0.85% and 0.59% of the margin). We must note that the statewide margin of victory in the Presidential contest was very tight at only 0.63% and 20,682 votes. Overall, the audit found an excellent consistency, with 99.964% of individual contests fully agreed. Only 1,057 contests were classified as “disagreed,” reflecting a mere 0.0358% discrepancy.

• Withheld Ballot Images: 1,298 ballots cast were withheld from the archives due to a law in WI that requires that the voter ID is written on the back of the paper ballot in certain circumstances. We believe this law should be repealed or changed to avoid writing voter ids on the ballots, and the election official in Dane County agrees with repealing this law. This law creates unnecessary barriers to transparency and could undermine voter confidence.

• County Overlap: 7,211 ballot sheets had images but did not have CVRs. This was because those ballots were from an adjoining county and they had a cooperation agreement with that county to process some of the ballots from Dane County, while Dane processed one area from that adjoining county. While cooperation agreements can enhance efficiency, they complicate data integrity and should be avoided if possible to ensure transparency.

• A total of 6,186 ballot sheets had CVR records but associated images were not available for the reasons above and a few other reasons.

• 191 ballot images were corrupted and could not be processed due to images that could not be aligned. This is a relatively high number but not a concern when compared with margins of victory in this county and across the state.

• Verification images were provided by a local activist who independently manually scanned ballots from 4 precincts for comparison. These images were processed by AuditEngine as verification images but it was difficult to compare 1:1 due to the way the ballots were scanned and how the precincts are grouped by the voting system, which we must note is a problem any time a separate scan is compared. It is important to scan the batches of ballots with knowledge of how they will be compared, which was not the case in this example. The discrepancies in scanning methods highlight the need for standardized procedures when comparing separate scans to ensure accuracy.

• AuditEngine found no discrepancies on BMD ballots by reading voter-verifiable text.

• Data can be cleaned up: Overall, the correspondence was very high 99.1% of individual contests were fully agreed with no variants. However, we hope the data can be cleaned up so we don’t have to process all of the missing ballots to make sense of it, which took a lot of time and effort. Of the 2,955,032 individual contests on 350,583 ballot sheets, 1,057 individual contests were classified as “disagreed” (0.0358%) with 99.964% of the individual contests agreed.

• Narrative Report: Dane County, WI 2020 General Election Audit Report

In conclusion, while Dane County’s 2020 General Election demonstrated high data integrity with only minor discrepancies, several issues warrant attention, such as the withholding of ballot images and the complexities introduced by inter-county agreements. Addressing these matters will enhance transparency and confidence in the electoral process.

Monmouth County, NJ, 2022 General Election

• Monmouth County uses ES&S ExpressVote XL in landscape mode using a grid-format. The ballot images are provided in large PDF files with perhaps up to 10,000 pages each rather than in ZIP archives with one PDF for each sheet (2 pages each), which is the normal format from ES&S. The grid format was new to AuditEngine and required additional checks for the orientation. The back of ES&S ballots has no orientation hints without looking at the content of the ballot. The transition to large PDF files with extensive pages introduced complexities in processing, necessitating additional verification for ballot orientation.

• The ES&S ExpressVote XL system is a hybrid system that is a mix of DRE (direct recording electronic) and BMD (Ballot Marking Device) system and poses issues for voter confidence.

• 53 PDF files containing 232,197 sets of images of ballot sheets representing 194,494 BMD ballots and 36,628 nonBMD (hand-marked) ballots were processed. There were 74 more ballot images than cast vote records.

• AuditEngine detected 977 repeated ballot images and cast vote records, and was able to remove the repeated ballots because they were digitally identical. These ballot images were repeated due to 7 USB drives that were loaded into the EMS twice. The fact that AuditEngine was able to remove the repeated images speaks to the effectiveness of the system.

• The county was alerted to this problem after the election was certified and they hand-counted those contests that were directly affected by the repeated ballots. AuditEngine’s independent results of those hand-counted contests were within +/-3 votes of the hand count.

• AuditEngine detected configuration errors in 3 other contests. After we asked them to explain in a letter, the County Clerk said this was due to a “printing error” as those ballot styles in those contests were bilingual. However, only 2 of the ballot styles were actually bilingual. Analysts tried to make sense of the explanation by shifting the page down or up, and could not. We believe this was not a printing error, but a configuration error, which is a common source of errors in election systems. It is an important hazard because a configuration “error” can award votes to the opposite candidate and could later be covered by saying it is a clerical mistake. However, any notion of malfeasance cannot be the case in the second two cases, because the contests in error were uncompetitive.

• In one of the three contests with the configuration errors, the Longbranch BOE contest was hand-counted by election staff that was not disclosed on their website. After we sent a letter to the County Clerk and asked them to explain where they got the results of the election, they admitted that they did perform a hand count. The hand count DID change the outcome and altered ~500 votes for each candidate. If AuditEngine was performed early in the process, it would have detected this issues and removed the identical ballot images prior to certification.

• The two other contests detected by AuditEngine as being incorrectly reported were not hand-counted by the county. These contests, “Avon School District”, and “Freehold Borough BOE” had vote counts that were incorrect by 94 and 107 votes each. However, as they were uncompetitive contests (too few candidates to make them competitive), there was no change in the outcome. The election office explained that the reason for the misconfiguration was due to misprinting the lingual ballots, but one of these was bilingual and one was not.

• There were a total of 1,374,725 individual contests with 16,723 disagreed contests (1.22%) and 98.7% agreed contests. This high discrepancy rate highlights areas for improvement in the election process.

• See Narrative Report of Ballot Image Audits of Burlington, Mercer and Monmouth Counties (2022) New Jersey

In conclusion, while Monmouth County’s 2022 General Election demonstrated a high level of accuracy in most contests, the identification of repeated ballot images and configuration errors serves as a reminder of the importance of rigorous auditing and verification processes to maintain election integrity.

Mercer County, NJ, 2022 General Election

• Mercer County utilized Dominion Voting Systems for the 2022 General Election, reporting 100,140 ballots cast. Significant issues arose, including:
• Last-Minute Style Changes: These caused many ballot scanners to malfunction on election day, necessitating a central rescanning of all hand-marked ballots.
• Missing Data: There were 9,405 ballot IDs missing from CVR records and 6,372 images unaccounted for, totaling 15,777 ballot sheets that could not be compared.
• Inconsistent Contest Results: The Board of Education contest showed discrepancies. Official results named ‘Stacy BYRNE’, ‘Dina THORNTON’, and ‘Jason McSHEENE’ as winners, while the audit suggested ‘Stacy BYRNE’, ‘Dina THORNTON’, and ‘Monica QUASTE’ were likely the true winners, indicating potential errors or misconduct.
• Disagreement Rate: Out of 344,806 individual contests compared, 896 were disagreed (0.25%), highlighting a high overall agreement rate of 99.74%.

The integrity of the election results is called into question due to these discrepancies and the lack of comprehensive data. For further details, refer to the Narrative Report of Ballot Image Audits of Burlington, Mercer and Monmouth Counties (2022) New Jersey

Somerset County, MD, 2022 General Election

Somerset County’s 2022 election was part of Maryland’s Pilot project, involving a total of 6,958 ballots, of which 1,802 (25.8%) were BMD ballots. Key points include:

• Redundant Ballot Images: The county intentionally provided 70% redundant ballot images to facilitate grouping for Early, Election Day, Mail-In, and Provisional ballots. While this was done for organizational purposes, it may not be necessary for comprehensive processing.

• Data Cleanliness: The data was notably clean, and the election followed a “Cooperative Workflow” model. This allowed for pre-election configuration and mapping, resulting in independent tabulation within 24 hours for comparison with CVRs.

• Minimal Discrepancies: Few discrepancies were observed, primarily arising from voter intent evaluations.

• Weak Logic and Accuracy Test (LAT) Data: The LAT used for mapping had limitations, as it did not cover critical failure modes. Specifically, it did not differentiate options on the ballot, which is essential for accurate mapping. Additionally, BMD ballots were not adequately tested, and no test results were provided.

• Lack of BMD String Data: Current exports do not provide sufficient BMD string data necessary for pre-election mapping. Future improvements are needed to source this data effectively.

Overall, while Somerset County demonstrated a clean election process with minimal discrepancies, enhancements in testing protocols and data availability are crucial for improving future election integrity.

Rockville City, MD, 2023 Municipal Election

• Rockville had 12,637 ballots with 199 BMDs (1.57%)
• There was no evidence of significant inconsistencies that would cast any contest into doubt.
• 15 disagreements in voter intent were detected, and 95% of these were correctly interpreted by AuditEngine.
• This was conducted as a trial run of the Cooperative Workflow methodology in a live election. Contest and option names were very consistent, so we were able to configure AuditEngine and “map” the ballot styles using an automated approach. Initial results were provided within ~24 hours. With some improvement in BMD information available prior to the election, the turnaround time can be improved.
• Simplified Totals Report was added to AuditEngine to provide an easy way to review the results when there are no significant discrepancies.

Overall, the Rockville City 2023 Municipal Election demonstrated a smooth process with high accuracy in interpreting voter intent, while the successful application of the Cooperative Workflow methodology suggests potential for further efficiency in future elections.