TargetMapper App

Purpose

The TargetMapper App provides a browser-based user interface to review ballot styles to create a "Target Map", which can be used by AuditEngine to evaluate votes on each hand-marked ballot image. In many cases, if "Ballot Style Masters" are available in searchable PDF form, then those can be parsed and the location of each oval linked to the contest and option in that contest without needing this mapping step. However, of those are not available, then TargetMapper can be used to create the map.

Basic Concepts

Please review the following concepts in the Glossary:

• Audit Job
• Ballot Style
• Cast Vote Record (CVR)
• Contest Name
• Contest Options
• Election Record
• Target
• Target Map

Preparation

Prior to using TargetMapper, there are a number of steps that must be taken:

1. The District Record, Election Record and Audit Job must be set up in the AuditEngine browser app.
2. The Job Settings for this audit must specify 'workflow' as 'public_oversight', and 'ballot_style_masters' must not be specified.
3. User Permissions for the user that will be using TargetMapper must be set up in the user permissions panel to include this audit.
4. The initial stages of AuditEngine must be run to prepare the data for processing, through the stage 'build_target_mapper_package', and 'update_system_status_stage'.
5. The ballots will be parsed to discover all the style numbers, and then clean style masters will be generated by combining many images of the same styles.

TargetMapper Layout

There are three major panes in the layout of the application

1. Ballot Image pane -- This pane will show one side of one ballot style template.
   1. The image can be zoomed in, panned, and rotated using the mouse, scroll wheel and dragging gestures.
   2. The magnified lock icon can be used to lock the zoom, pan and rotation for that side.
   3. There is a second set of rotation buttons to rotate the red text that is added to the image.
2. Style List -- This pane lists the styles to be mapped, with one line item per side.
   1. To work on just one side or the other, use the Page 1, Page 2 check boxes.
   2. The check boxes are use to indicate that you have verified that the side is completed.
   3. A clock face is used to indicate whether all contests have been mapped among both sides.
3. Contest/Options List -- This pane provides the contests and options that can be mapped to the style selected.
   1. "Show All" -- means that all contests will be shown regardless of the contests defined for this ballot style.
   2. "Expand All" -- means the options will be shown for each contest. The plus symbol "+" can be used to selectively expand one contests and show the options, and "-" can be used to minimize the contest.
   3. The edit button can be used to edit the order of the options.

Ballot Pane Detail

1. Map target by clicking when option is highlighted in option list to map a target. Drag oval to adjust. These will always snap to the grid. Double-click to delete. If the oval is the wrong size, use Settings - Allow Target Area Resizing to adjust with the mouse, then disable when it is correct. The new size will be used in all future mapping.
2. Name of the option will be shown near the oval to allow for verification.
3. Pan and Zoom: Use mouse to drag the ballot in the display to pan or use sliders. Use Control Scroll to zoom in and out.
4. Save All: to save the current map. It is good to do this frequently.
5. Copy: If the mapping can be mostly used for another style, click "copy" to copy the mapping to the clipboard.
6. Paste: Click paste to paste the mapping onto the current style. Normally, you can use Copy, then move to another style that is similar, then Clear and Paste. Check that the mapping is correct, and either reposition or delete and re-map those that are incorrect.
7. Clear deletes the mapping for the current style and page.
8. Paste Rest will paste the currently copied map to the pages that are selected (via Page 1, Page 2) in the style list, to all styles that are listed after this one. If style_to_contests is known, it will not paste a contest that does not exist.
9. Clear Rest acts similarly, but clears all the rest.
10. Paste Similar is available if style_to_contests data is available. It will paste all contests to the first and second page of this style, using a prior style that is the most similar to this style, and it will stop when there is any difference between the two styles. This will usually do a very good job of completing most of the mapping of a style, but you must check it carefully.
11. Map Rest will attempt to map all the rest based on similar existing styles. (Not generally used and it is slow.)
12. Auto Paste Similar -- Save one mouse click by automatically implementing Paste Similar when any new style is selected, and if it has not been mapped yet.
13. Freeze Job -- Use this to lock a job so it will not be modified, and to allow the map to be used in the audit pipeline.
14. Rotate Page -- Use this to rotate the page in 30 degree increments. Generally used to flip or rotate to landscape orientation.
15. Zoom Lock -- Use Zoom Lock to set where you want the page to start when mapping, such as to skip the instructions and to zoom at a convenient level to start each page. There is a separate zoom lock for each side (Page 1 and Page 2).
16. Rotate Text -- this will rotate the text when it is hard to read, esp. when mapping grid styles where the text will otherwise overprint.

Style List Detail

1. Green Checkmark -- this is shown when TargetMapper knows what contests are on this style, and all contests and options have been mapped.

2. Incomplete Pie Symbol -- the incomplete pie chart symbol means that TargetMapper detects that some contests and options have been mapped, but not all have been.

3. No Leading Symbol -- No contests and options have been mapped.

4. Highlighted Style -- The style highlighted in Yellow indicates that it is currently being shown in the Ballot Image Pane.

5. Verified Checked -- The blue checkmark on the right is checked by the user after the ballot is fully mapped and checked by the user.

6. Verified Unchecked -- If the verified checkbox is unchecked, the the user has not verified it. If another user is to verify the work again, the verified checkmarks can be all cleared by using Actions - Clear All Verified.

7. Page 1 Checkbox -- indicates that page 1 styles will be shown. Uncheck Page 2 and check Page 1 to show only Page 1 of all styles (all styles will have a page 1).

8. Page 2 Checkbox -- indicates that only styles that include Page 2 will be shown, and only page 2 will be shown. Uncheck Page 1 and check Page 2 to show only Page 2.

9. Total Sides -- shows the total number of sides to be mapped.

10. Started -- shows the total number of sides that have been mapped or partially mapped.

11. Completed -- Shows the number calculated based on the number of contest and options on each style, and whether all those have been mapped. If the style_to_contests data is not provided, then this will never change from 0.

12. Verified -- The number of sides that also have a verified checkbox checked.

Contest/Option List Detail

1. Expand Options: Click on Plus-sign to the left of the contest name to expand the option list for the contest.
2. Minimize Options: Click on minus-sign to the left of the contest name to suppress the option list for a contest.
3. Blue "Mapped" check is shown when all options in the contest have been mapped.
4. Black "Mapped" check is shown next to an option when the target for this option has been mapped.
5. Mapped Coordinates where target has been mapped.
6. Unmapped Contest No Blue checkmark means not all targets have been mapped for this contest.
7. Unmapped Option: No black checkmark means option has not been mapped.
8. Selected Option: Body of option box will be yellow when the option is selected for placing a target.
9. Show All will show all contests even if they are not defined for a style, to allow for mapping to contests when style_to_contests data is not available (such as if there is no CVR).
10. Expand All will expand all options for all contests. If unchecked, then it minimizes all.
11. Edit icon (pencil) allows user to edit primarily the order of the options if they are not in the correct order. This can speed up the mapping process. Sometimes, states will vary the order of the options and then this can be reordered to match. When clicked, this will open up a dialog box.
12. Link icon is used for linking to parsed ballot styles and is no long needed.
13. Reorder icon is used to change the order of contests if needed (rare).

Primary Flow

There are a number of nuances and details for special case, but primarily, the workflow is as follows:

1. Process Audit Pipeline to build_target_mapper_package -- this will generate the files in the /map folder. Also must run the stage 'update_system_status_stage' so TargetMapper can see that the package has been built.
2. Run TargetMapper -- TargetMapper can be run at the link https://mapper.auditengine.org
3. Select the Job -- Select the job to be worked on in the Job List.
   1. Note: Each user must have permissions to work on a mapping project. This is done by an administrator or project manager.
   2. Each worker can be assigned a range of styles to work on and to one page or the other or both.
   3. Select the desired job from the 'Job List' pull down menu. If you don't see the job there, then click the refresh button.
   4. If the job still is not in the list:
   5. Check that the election and audit job have been defined in the engine.auditengine.org browser application.
   6. Check that the user has sufficient permissions for this specific job.
   7. Check that the stages of the audit have been run through "build_targetmapper_package" and "update_system_status_stage".
4. Select The Next Style in the Style List -- Click on the style and page to be mapped. That will bring up the style template image.
5. Select the Next Contest in the Contest/Options List -- Click on the next contest to be mapped. All contests for both sides of the template will be listed.
6. Find the oval for that contest and option -- Use the mouse to click the oval to place the target indicator on the image.
   1. Double click to delete the target indicator
   2. Drag the target indicator to adjust. Normally the snap-grid will be provided and it will snap to the closest allowed location.
   3. Use Settings-Allow Target Area Resizing to adjust the size and offset of the target by dragging the controls on the oval. Disable adjusting when it is an appropriate size for the specific vendor type.
7. Move to the Next Option -- There are two ways to select the next option for mapping. When selected for mapping, the option will change to "Yellow".
   1. Click the next option in the Contest/Options list.
   2. Press "n" for "next"
8. Continue from 3.

If the options are in the right order, the mapping process can proceed very quickly. If they are not in the correct option, then click the "Edit (pencil on paper)" icon near the contest name in the Contest/Options list and edit the order of the options.

When the style is completed, double check your work by comparing the red notations with the contest and options on the ballot, and then click the "verified" checkbox to the right of the style.

At the end of editing, use "Save All" to save the mapping.

Optimizations

Although the primary flow will normally work, there are a number of optimizations that can help to accelerate the process.

1. Copy and Paste -- Very frequently, one ballot style will be identical to or very much like the next. In such a case, you can use "Copy" and the "Paste" to paste the copied style on this template. It will only paste the current page. Note that if a contest does not exist on a template, it will not be pasted, but if the contest is in a different location, it may be mapped to the wrong location.
2. Clear -- this control will simply clear the mapping from this page. Commonly, this is used before a paste.
3. Paste Rest -- Sometimes, all the templates of one page will be exactly the same, such as Page 2. If you use "Paste Rest" then it will paste the copied mapping to the rest of this page.
4. Clear Rest -- Clear all the rest of the current page.
5. Paste Similar -- This is a very handy operation, because it will look back to a prior template that was mapped with the same or many of the same contests, and then it will map the contests that are the same up to the point when the two styles diverge in the contests on it.
   1. Auto Paste Similar -- Use the Checkbox "Auto Paste Similar" to automatically click the Paste Similar button each time a new unmapped style is selected.
   2. Paste Similar is not available if the style_to_contests_dol is not provided in advance.
6. Map Rest -- Automatically apply "Paste Similar" to all the rest of the styles in the list. This may take significant time.

Completion

1. Save All -- to save the current mapping.
2. Click "Freeze Job" so that the job cannot be accidentally changed using this app.
3. In AuditEngine app, under Audit Operations, Import the mapping using the stage "import_targetmap"
4. Check the map report for any errors. These can normally be resolved by editing the map for the style of concern.
5. Generate Redline Proofs by running the stage: gen_all_redlined_proofs
6. General Option Proofs by running the stage: gen_option_proofs_report
7. Run the stage 'gen_style_report' to create the styles report and proof the redline proofs for each style.
8. If there are any mapping mistakes, reopen the job and complete the mapping.

Menu Functions

The menu bar has a number of functions:

File:

• Reload Project Files -- Normally, project files are loaded when the Job is opened. (Need to update functionality)
• Reload and Delete All Work -- This will restart the current job and delete all work. Careful!
• Open Backup File -- Each time "Save All" is clicked, a backup is created of the Target Map being built. This function will provide a list of backup files that can be reopened.
• style_map_dolod -- This is an alternative realization of the result of mapping. (Need to update functionality --what does it do?) [Propose delete this.]
• Import Map -- If there is an existing targetmap.json for this project, it can be manually imported.
• Export Map -- Export the map to a given file name (Need to test, will hang if no job is opened.)
• Export Target Position Spec -- The target position can vary based on the vendor, and may not be centered at the snapgrid. This is set in "Settings - Edit Target Spec"
• Export Code Extraction Spec -- The code extraction spec allows the user to set a location on the ballot face which can be defined to be extracted. Can be edited in "Settings - Edit Code Spec".
• New Project -- (Need to update functionality. Does it make sense to have this? Probably should call this "Close Job?")
• Update Database -- If a job was reopened after have some changes to the list of templates and styles, then the database will need to be updated. -- (Need to update functionality.)

Project Information

Simply provides the File Name - targetmap.json, and the job folder. There is no functionality to the pull down items.

[This should be moved to Settings.]

Job List

Use this list to select the job to be mapped. The items will be shown in the list if the job has built the stage 'build_targetmapper_package' has been completed. The job will show "locked" if the job has been hard locked in the audit actions, and will show "frozen" (snowflake) if the job is frozen. [Note: this should be move under "File" and rename it "Open Job"]

Settings

• Edit Code Spec - This provides the definition of a location on the face of the ballot which will be extracted using OCR and provided in a variable. Currently, the following can be defined:

• pstyle - printed style

• precinct

This functionality should allow using the mouse to define the region.

• Edit Target Position Spec -- This function provides a means to adjust the exact position of the target with respect to the snapgrid. Select the default spec and adjust if needed.

• Edit Style Contest Map -- Allows for the manual adjustment of the contest included in any style.

• Allow Target Area Resizing -- Check this box and then use the handles on the oval to change its size and location vs. snapgrid.

User Lock

To avoid users stepping on each-other's work, the application will be locked for a specific user. There are two functions:

• Request Lock - If the job is locked by another user, you can request that they release their lock.
• Release Lock - If you have a lock on the job, you can release your lock so another user can work on it.

Actions

• Clear All Verified -- will clear all the verified checkmarks to allow the job to be re-verified.
• Add Contest -- Use this to add a contest to the election, so it can be placed in the style contests map. [This could be moved to Settings]
• Show/Hide Value Entry -- (Update the functionality for this menu item.)

Special Operations

There are a number of special operations that deviate from the primary flow and the defined operations.

No styles-to-contests available

For normal operations, the list of contests is defined for each style. This will limit the contests in the contest-options pane to only those that are defined. However, if the contests are not defined for each style, then it is most convenient to obtain this information from the mapping process. This can commonly occur when no CVR exists. Please note: Running in this mode will require more diligence because one extremely helpful piece of information is missing.

1. The difference in operation is that the next contest must be more carefully checked each time and not just use "Next" to go to the next contest.
2. The pie chart symbol in the style list normally shows the completion of each style (both pages) will it not know when the full set of contests has been mapped, so this symbol may not ever show completion (green checkmark)
3. Make sure job settings includes 'define_style_to_contests_from_map' = TRUE
4. When the stage 'import_targetmap' is run, it will also create the style_to_contests data.
5. The redline proofs and option proof report will require careful scrutiny to check that all contests are included for each style.

Change Contests on a style template

If styles-to-contests IS available, then the contests in each style will be predetermined. Normally, you can just go through each contest provided for that style and fill it in. However, sometimes, there will be a mistake in the style-to-contests definition. In that case, we can override the contest that are defined for that style.

1. In the Contest/Option list, checkmark [ ] Show All. This will provide all the contests in the election.
2. Find the correct contest that is on the ballot in the Contest/Option list. Click + (plus sign) to open it.
3. Verify that the options match those options actually on the style, to verify that you are choosing the correct contest.
4. Map the targets for this contest as usual.
5. Repeat from 2.
6. When done mapping the style, checkmark VERIFIED in the Style List, which is the checkbox on the right end of the style item.
7. The progress Icon to the left of the style in the Style List will show partially complete. This is correct because it does not match the contests expected on this style.
8. At this time, this will need to be communicated to the audit process by enabling "define_style_to_contests_from_map" in the job settings file.

Example:

In the case shown here, the ballot style is for TOWNSHIP OF EVESHAM D-12. The contests originally linked to this style were for Hainesport Township. You can see that the contests appropriate for Evesham Township are used instead, and mapped as usual. Note that the progress indicator will not change to a green checkmark, and you should still check the verified checkbox (blue).

Change the styles in the job

It is fairly common for an initial set of styles to be defined, and the mapping completed for those styles, and then later, additional styles are added.

1. Consider the case when ballot_id numbers are used twice in the project for different ballots. Assume that these were initially thought to be repeats of the same ballots, but later, it was determined that the ballots that use the same numbers are of different ballots. Then, the ballot_id values can be modified based on the set so these no longer have the same identifiers. Further, we assume that additional styles are found among the newly renamed ballots.
2. The stages of AuditEngine should be re-run and ballots style templates generated. The new styles should have distinctive style identifiers. Run the stage "build_targetmapper_package" and generate a new package for TargetMapper.
3. Click "File - Update Database". This should then show the new styles in the Style List.

Adopt an existing Targetmap from another job

Sometime a targetmap needs to be adopted from a prior job and then enhanced in the current job. This can occur when the prior job is locked, and we do not want to modify it, and then us it in the current job with enhancements. For this special operation, use the following procedure:

1. In AuditEngine settings, set the 'adopt_from_jobname' to the core job name of the prior job that the targetmap should be adopted from.
2. In AuditEngine settings, set 'enable_adopt_targetmap' to True. This will pull in the target map from the other location and create the file 'adopted_targetmap.json' in the current job.
3. In TargetMapper, select the function File - Adopt Targetmap -- This will the file 'adopted_targetmap.json' as the starting point, along with all other files in the map/ folder, in the operation of TargetMapper.
4. After editing the map, TargetMapper will write the file 'targetmap.json' and will never alter the file 'adopted_targetmap.json'