Volusia County, FL, 2020 General Election

• Not enough errors found to change any winners
  Volusia County had to deal with a number of errors that happened due to poor tracking by ES&S software of USB drives uploaded and what data is cleared when reset.
• Found 4,904 ballots with two images each in the election records.
  Just prior to the close of polls, they started uploading data from the early voting sites. The first USB drive was loaded correctly but the second drive failed. They cleared the election and rescanned all paper ballots from that first site. Unfortunately, not all data was actually cleared, and the 4,904 CVR records and ballot images from the first USB drive were still in the system.
  As rescans, the additional ballot images could not easily be located within the images or CVRs, as those are shuffled by the election management system (EMS).
• No machine failure was reported and no mention of the rescan.
  Even though the rescanning was done in public, this fact was not documented in their election report and the check box for device failure was unchecked.
• Found 537 ballot images and CVRs were missing.
  The USB drive from one of two machines from one voting site was not uploaded, even though the aggregated totals were wirelessly transmitted. The EMS should know if all USB drives are properly uploaded but does not notify the user if any mistakes are made. The election officials were unaware of this missing upload until our report.
• This evidence shows that ES&S maintains two independent tabulations and they are not compared by the EMS.
  The polling place totals are transmitted by "secure" cell modem to the central office. Although this is fast, these are supposed to be only unofficial tentative results. Later, the aggregated totals, CVRs and images are uploaded from USB drives, to confirm the results sent in earlier.
  
  We proved from this election that the final results for polling places are generated ONLY from the totals received by modem. The CVRs are NOT used to generate the final results, and although they are consistent inside any single voting machine, they can grow to differ due to rescanning or missing uploads. ES&S also never compares these two tabulations for consistency, and they don't require the data from the USB drives and will rely only on the unofficial tentative cell modem transmitted results if the USB is missing. It is unclear if the results on the USB drive would be used instead of the modemed results, if they differed.
• AuditEngine conducted a "Poll Tapes Audit" (which covered a portion of the results, only from in-person polling places), confirming that the one thumb drive had not been uploaded, but the other results were consistent.
• Overall we found a disagreement rate of 0.237%, or about 24 disagreements per 10,000 ballots between the result by AuditEngine and the voting system across all contests.

St. Lucie County, FL, 2020 General Election

• Lucie did not generate a ballot-level CVR,and so we were unable to compare on a ballot-by-ballot basis.
• Received 8 image archives with one repeated inside another combined RAR archive. This took a lot of hand work to resolve.
• There were 526 more ballot images than the number of ballots officially cast. This was fewer than the closest vote margin. We could not find the issue due to the lack of comprehensive reports.
• AuditEngine found an additional 135 ballots cast with 123 additional votes for Biden and three fewer for Trump.

Collier County, FL, 2020 General Election

• Clean data: Of the three counties we audited in FL, they had the cleanest and most complete data. We used fully automated mapping using image processing but it was the last time we used this method.
• They had a two-sheet ballot for all voters, but these two sheets are combined to only one BMD summary page, when BMDs are used.
• Repeated BMD images and CVR records.
  Here, they repeat the ballot images for any BMDs and create two CVR records, one for each of the two sheets of a hand-marked ballot.
• Thus, although we thought the ballot inventory was off by 39 -- there were 39 more images than ballots cast -- while we had a total of 78 BMD images, once we understood that the BMD images and CVRs were repeated, then the ballot image and CVR counts exactly matched the number of ballots cast. (See full report for more detail).
• There were 324 ballot sheets with disagreements out of the 417,777 images processed (0.077%).

Bartow County, GA 2020 General Election

• Excellent clean data: 50,678 ballots were cast in the official election results and the same number of ballot images were provided and processed. 328 votes out of a total of 1,035,141 votes on those sheets were evaluated as disagreed (0.031%) spread across 27 contests.
• There were about 40,000 BMD ballots and all but one was read and was consistent. That one ballot had curved text that could not be read.
• Bartow reclassified overvotes that were reviewed as undervotes. This will not affect the outcome, but it does alter the statistics of overvotes, which will appear to be none at the end, and having clarity on each category is important for quality improvement measures.
• All overvotes and write-ins were adjudicated by County staff.
• There was one Incorrect Adjudication which was detected and was probably just a mouse-clicking error.

Fulton County, GA, 2020 General Election

• 72% of the images were missing: only 28% of the ballots were audited and 72% were not, since the ballots images were not available. This was due to the ballot images being overwritten when they scanned the runoff election, which occurred on January 5, 2021.
• Although the ballots were rescanned and tabulated in "Machine Count 2" (MC2), we did not have the corresponding CVR and could not compare on ballot level. So, we did not use those images.
• AuditEngine processed 148,580 images out of a total of 528,777 ballots, with 3,923 BMD ballots, resulting in 2,018 disagreed votes among 61 contests. More than 99% of the individual contests were fully agreed.
• AuditEngine "read" the human readable text on 3,923 Dominion BMD summary sheets with no disagreements.
• Ballots from adjacent DeKalb County were mixed in and processed as if they were Fulton County ballots (5 total). The ballot was the same size and the two counties use the same overlapping style numbers. Down-ballot contests were incorrectly interpreted.
• Ballots from the May 19, 2020 primary election were mixed in but were not processed. These were longer-length paper so it is remarkable that they were not pulled out.
• Discovered that multiple "cards" can exist in the JSON CVR for one ballot. An initially unexplained discrepancy was found between the CVR and the image and official results. This was traced to the use of multiple "cards" in the JSON format, if there is a jam or other problem, the first card must be ignored and only use the second card.
• Federal-Only ballots can create a privacy hazard, but can be fixed. Many federal-only ballots are classified to an appropriate style for that voter if they receive the full-length ballot, and this results in only one ballot per style in many cases. These ballots are all the same in terms of the contests on them. 650 ballots exist from all UOCAVA voters and they are classified into 199 ballot styles. These should be classified into only 1 or a few ballot styles, to avoid this privacy hazard.

DeKalb County, GA, 2022-05-24 Primary Election

• Misconfiguration: One candidate (Donald Broussard) dropped out of the contest "County Commissioner D2 - Dem". This resulted in a misconfiguration of the machines and the outcome of the contest was incorrect.
• The contest was subsequently fully hand-counted and was consistent with AuditEngine results.
• QR Code Misread: We detected this misconfiguration issue early in the process of mapping the contests. The hand-marked and BMD ballots DID include the dropped candidate, but the CVR did not. The hand-marked ballots were correctly printed and interpreted (except for dropping the candidate), while the BMD ballot had the name of the dropped candidate printed, but the QR Code data was misread by the voting system when it was scanned, changing the outcome of the contest.
• 135 Votes for Donald Broussard were "wasted" because he was not running but was still on the ballot, even in BMD machines. However, those votes applied in any combination would not have changed the outcome.
• Votes for Michelle Long Spears changed by 3,141 additional votes moving her from 3rd to the winning position.
• 453 CVR records were (and still are) missing from our set.
• One BMD ballot from the November 2, 2021 election was mixed in and overprinted over the barcode.
• Several BMD ballots had text that was unreadable because they were lightly printed even though the QR Code was read by the machine. Some were printed so darkly that they were very difficult to read.

Dane County, WI, 2020 General Election

• Dane County uses ES&S voting systems and has posted the ballot images online for many elections. There were 344,347 unique ballot images in the archive with 3,069 ballot images repeated with the same ballot identifier. There were 26,358 BMD ballots (7.6%). There were 67 styles in this election with 36 different patterns of contests and 33 different contests.
• Excellent Consistency: There were 363 sheets with contests categorized as "disagreed" across all contests. The two most discrepant contests had only 1 or 2 disagreements at 0.85% and 0.59% of the margin. We must note that the statewide margin of victory in the Presidential contest was only 0.63% and 20,682 votes.
• Withheld Ballot Images: 1,298 ballots cast were missing from the archives due to a law in WI that requires that the voter ID is written on the back of the ballot in certain circumstances. We believe this law should be repealed or changed to avoid writing voter ids on the ballots.
• County Overlap: 7,211 ballots which had images did not have CVRs. This was because those images were from an adjoining county and they had a cooperation agreement with that county to process some of the ballots from Dane County, while Dane processed one area from that county. The result is that these ballots with images did not have the CVRs. It will be helpful to avoid this sort of cooperation agreements that affect the data.
• A total of 6,186 ballot ids were in the CVRs but images were not available for the reasons above and a few other reasons.
• 191 ballot images were corrupted and could not be processed due to bad images that could not be aligned. This is a relatively high number
• Verification images were provided by Peter Bernegger who independently manually scanned ballots from 4 precincts for comparison. These images were processed by AuditEngine as verification images but it was difficult to compare 1:1 due to the way the ballots were scanned and how the precincts are grouped by the voting system, which we must note is a problem any time a separate scan is compared. It is important to scan the batches of ballots with knowledge of how they will be compared, which was not the case in this example.
• AuditEngine found no discrepancies on BMD ballots by reading voter-verifiable text.
• Data can be cleaned up: Overall, the correspondence was very high 99.1% of these contests were fully agreed with no variants. However, we hope the data can be cleaned up so we don't have to process all of the missing ballots to make sense of it, which took a lot of time and effort.

Monmouth County, NJ, 2022 General Election

• Monmouth County usesES&S ExpressVote XL in landscape mode using a grid-format. The ballot images are provided in large PDF files with perhaps up to 10,000 pages each rather than in ZIP archives with one PDF for each sheet (2 pages each), which is the normal format from ES&S. The grid format was new to AuditEngine and required additional checks for the orientation. The back of ES&S ballots has no orientation hints without looking at the content of the ballot.
• 53 PDF files containing 232,197 sets of images of ballot sheets representing 194,494 BMD ballots and 36,628 nonBMD (hand marked) ballots were processed.There were 74 more ballot images than cast vote records.
• AuditEngine detected 977 repeated ballot images and cast vote records, and was able to remove the repeated ballots. These ballots were repeated due to 7 USB drives that were loaded into the EMS twice.
• The county was alerted to this problem after the election was certified and they hand-counted those contests that were directly affected by the repeated ballots.Audit engine's independent results of those hand-counted contests were within +/-3 votes of the hand count.
• AuditEngine detected configuration errors in 3 other contests. After we asked them to explain in a letter, the County Clerk said this was due to a "printing error" as those ballot styles in those contests were bilingual. However, only 2 of the ballot styles were actually bilingual. Analysts tried to make sense of the explanation by shifting the page down or up, and could not. We believe this was not a printing error, but a configuration error, which is a common source of errors in election systems. It is an important hazard because a configuration "error" can award votes to the opposite candidate and could later be covered by saying it is a clerical mistake. However, any notion of malfeasance cannot be the case in the second two cases, because the contests in error were uncompetitive.
• In one of the three contests with the configuration errors, the Longbranch BOE contest was "secretly" hand counted by election staff that was not disclosed on their website. They admitted that they did perform a hand count after we sent a letter to the County Clerk and asked them to explain where they got the results of the election. The hand count DID change the outcome and altered ~500 votes for each candidate.
• The two other contests detected by AuditEngine as being incorrectly reported were not hand-counted by the county. These contests, "Avon School District", and "Freehold Borough BOE" had vote counts that were incorrect by 94 and 107 votes each. However, as they were uncompetitive contests (too few candidates to make them competitive), there was no change in the outcome. One of these was bilingual and one was not.

Mercer County, NJ, 2022 General Election

• Mercer uses Dominion Voting Systems. There were 100,140 ballots cast according to the official report, with 36 ballot image archives and 2 CVR files. There were no BMD ballots.
• There was a last-minute change in the styles which resulted in many ballot scanners not working on election day. These ballots were all hand-marked and were rescanned centrally.
• There were 9,405 ballot_ids missing from CVR records and they had to be synthetically generated, and could not be correlated with the images. However, the number of CVR records is consistent with the official count.
• Missing 6,372 Images. There were 93,768 unique images, 6,372 fewer than the published total of 100,140 ballots cast.
• Inconsistent Contest: One of the most divergent contests according to the ballot images available was the "Members of the Board of Education - Hamilton Township (Vote For=3)". The official results had the 3 winners as 'Stacy BYRNE', 'Dina THORNTON', and 'Jason McSHEENE' with a margin of victory between the 3rd and 4th place winners at 278 votes. The audit found the winners are most likely 'Stacy BYRNE', 'Dina THORNTON', and 'Monica QUASTE', with Quaste getting 68 more votes than McSheene. However, this is based on an incomplete set of ballot images (6,372 ballots missing). Was the win for McSheene an error or a malicious act which was introduced using the records related to the missing ballot images and missing CVR records (and that is why they are being withheld)?

Somerset County, MD, 2022 General Election

• Somerset County 2022 election was processed as part of the Maryland Pilot project. It had very few ballots, only 6,958 ballots of which 1,802 (25.8%) were BMD ballots.
• Somerset intentionally provided 70% redundant ballot images. This was intentional to provide for grouping snapshots for Early, Election Day, Mail-In Ballots and Provisionals. This grouping data can be provided without sending so many redundant images but we don't really need the grouping data because we process all images, including all BMDs, unlike other BIA providers.
• Maryland data was very clean and we operated in "Cooperative Workflow" where we configure and map prior to election day and then produce our independent tabulation within 24 hours and then compare with the CVRs after that.
• There were very few discrepancies and those were due only to voter intent evaluation.
• Logic and Accuracy Test (LAT) data "weak": These data were used for Cooperative Workflow mapping. The LAT has ballots that are marked to test failure modes of the configuration. We noted that the test cases used by Maryland are not able to cover some important failure modes, such as whether the target is mapped to the wrong contest or wrong option. To fix this, every option on the ballot should have a different grand total of votes so they can be differentiated. BMD ballots are not apparently tested, and no test results were provided, only the test data and reference results.
• There is no good source of BMD strings in the current exports we are getting. We need this for cooperative workflow, because we do not have the images prior to the election and need to do the mapping prior to the election. We hope to find a source for this data in the future.

Rockville City, MD, 2023 Municipal Election

• Rockville had 12,637 ballots with 199 BMDs (1.57%)
• There was no evidence of significant inconsistencies that would cast any contest into doubt.
• 15 disagreements in voter intent were detected, and 95% of these were correctly interpreted by AuditEngine.
• This was conducted as a trial run of the Cooperative Workflow methodology in a live election. Contest and option names were very consistent, so we were able to configure AuditEngine and "map" the ballot styles using an automated approach. Initial results were provided within ~24 hours. With some improvement in BMD information available prior to the election, the turnaround time can be improved.
• Simplified Totals Report was added to AuditEngine to provide an easy way to review the results when there are no significant discrepancies.