2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Issued: July 15, 2019 Version: 2.0 Final Prepared by: Decennial Census Management Division [Page intentionally left blank.] Approvals This NRFU Detailed Operational Plan has been reviewed and approved for use. Electronically Signed Dominic Beamer Branch Chief / IPT Lead June 10, 2019 Date Signed Electronically Signed Jennifer Reichert IPT Program Manager June 18, 2019 Date Signed V2.0 July 12, 2019 Final Document Change History Revision # Version Date Description 1 v0.01 06/27/2016 Initial Working DRAFT Version from 2020 Census DOP template. Uses Annotated NRFU BPM Version 4.1. 2 v0.05 07/25/2016 Draft document for NRFU-IPT and ADC review 3 v0.06 08/04/2016 4 V0.07 11/28/2016 Rewrite with new directives from ADC 5 V0.08 12/01/2016 Updated PMO and MITRE Edits 6 V0.09 08/24/2017 7 V0.10 12/07/2017 Incorporated additional comments from DCCO, SMEs. 8 V0.11 01/15/2018 Incorporated additional comments from NRFU IPT. 9 V0.12 02/05/2018 Incorporated comments from DCCO. 10 V1.0 02/08/2018 Final draft submitted to executive review. 11 V1.01 02/14/2019 Updated BPMs, IDEF0, IOD to reflect design changes. Adjusted appendices. Fixed formatting. 12 V1.02 4/15/2019 Updated to reflect changes in contact strategies 13 V2.0 6/18/19 Incorporated comments from NRFU-IPT, DPMO, and MITRE Updated IDEF0 and IOD and inserted updated BPMs for consistency with other operations. Final approved version Note: The fields below update the document version, date, and status in the page footers throughout the document. Document Footer Information Control Table Field Name Version, Date and Status DocVersion: V2.0 DocDate: July 12, 2019 DocStatus: Final V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page i Table of Contents 1 Document Purpose ....................................................................................... 1 2 Operational Overview................................................................................... 2 Operation Purpose ......................................................................................................... 2 Background ..................................................................................................................... 2 2.2.1 Goal of the 2020 Census and How the Census Works .............................................. 3 2.2.2 NRFU Operations Overview ...................................................................................... 4 2.2.2.1 NRFU Universe ...................................................................................................... 4 2.2.2.2 Type of Enumeration Area Descriptions ............................................................... 5 2.2.2.3 NRFU Data Collection ............................................................................................ 6 2.2.2.4 Coverage Improvement ........................................................................................ 8 2.2.3 Why It’s so Important — Most Expensive Aspect of the Census ............................. 9 2.2.4 NRFU Cost Reduction Efforts for the 2020 Census ................................................. 10 2.2.4.1 Reduce the NRFU Workload................................................................................ 13 2.2.4.2 Improve the Efficiency of the Operation ............................................................ 17 2.2.5 Undercount of Young Children ............................................................................... 19 NRFU Components ....................................................................................................... 19 2.3.1 Contact Strategy ..................................................................................................... 19 2.3.2 Administrative Records Modeling and Tailored Contact Attempt Strategy ........... 22 2.3.3 Housing Unit Enumeration ..................................................................................... 24 2.3.3.1 Vacant Units ........................................................................................................ 25 2.3.3.2 Nonhousing Units ................................................................................................ 25 2.3.3.3 Enumeration of a Household .............................................................................. 26 2.3.3.4 Unsuccessful Attempts ........................................................................................ 26 2.3.3.5 Proxy Interviews .................................................................................................. 27 2.3.3.6 Additions to the NRFU Workload ........................................................................ 29 2.3.4 Special Enumerations.............................................................................................. 33 2.3.4.1 Field Verification ................................................................................................. 33 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page ii 2.3.4.2 Multiunit Housing................................................................................................ 34 2.3.4.3 Early NRFU ........................................................................................................... 36 2.3.4.4 SRQA Recollect .................................................................................................... 37 2.3.5 Quality Control ........................................................................................................ 37 2.3.5.1 Evaluation After Initial Training .......................................................................... 38 2.3.5.2 Operational Control Alerts .................................................................................. 38 2.3.5.3 Quality Control Reinterviews .............................................................................. 38 2.3.5.4 Coverage Improvement (CI) Followup ................................................................ 41 2.3.6 Closeout .................................................................................................................. 43 NRFU Operational Timeline .......................................................................................... 43 Field Operations ........................................................................................................... 44 2.5.1 Reengineering Fieldwork ........................................................................................ 44 2.5.2 Reengineering the Field Staff Structure.................................................................. 46 2.5.3 Training ................................................................................................................... 49 2.5.4 Devices and the Instrument .................................................................................... 50 2.5.5 CFS Instructor Kits and Enumerator Kits ................................................................. 51 2.5.6 A Day in the Life of an Enumerator ......................................................................... 51 2.5.7 A Day in the Life of a Census Field Supervisor (CFS) ............................................... 52 2.5.8 A Day in the Life of a Census Field Manager (CFM) ................................................ 53 High-Level Operational Design ..................................................................................... 54 NRFU Operational Context ........................................................................................... 56 2.7.1 NRFU Operational Inputs ........................................................................................ 59 2.7.2 NRFU Operational Controls..................................................................................... 61 2.7.3 NRFU Operational Outputs ..................................................................................... 62 2.7.4 NRFU Operational Mechanisms .............................................................................. 66 2.7.4.1 Staff Resources .................................................................................................... 66 2.7.4.2 Infrastructure Sites .............................................................................................. 67 2.7.4.3 Systems and other Technology Infrastructure .................................................... 68 NRFU Data Flow and Operational Influences ............................................................... 71 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page iii NRFU Design Assumptions ........................................................................................... 80 3 Nonresponse Followup Operation (NRFU) Detailed Process Description ..... 81 NRFU Planning and Preparation [NRFU 18-1] .............................................................. 82 3.1.1 Develop NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules [NRFU 18-1.1] .............................................................................................................. 83 3.1.1.1 Develop NRFU Administrative Records Modeling Requirements [NRFU 18-1.1.1] ............................................................................................................................. 84 3.1.1.2 Develop NRFU Contact Strategies and Business Rules [NRFU 18-1.1.2] ............ 84 3.1.1.3 Develop NRFU Quality Control Plan [NRFU 18-1.1.3] ......................................... 85 3.1.2 Develop and Approve Field Printed Materials and Kit Specs [NRFU 18-1.2].......... 86 3.1.2.1 Provide and Approve Field Enumeration Materials Content [NRFU 18-1.2.1] ... 87 3.1.2.2 Provide and Approve Kit Requirements [NRFU 18-1.2.2] ................................... 87 3.1.3 Identify Areas for Early NRFU [NRFU 18-1.3].......................................................... 88 3.1.4 Develop Requirements for and Approve Systems Supporting NRFU [NRFU 18-1.4] . ................................................................................................................................. 88 3.1.5 Determine Staff Needs and Performance Metrics [NRFU 18-1.5].......................... 89 3.1.5.1 Determine Estimated NRFU Operational Workload by Geographic Area [NRFU 18-1.5.1] ....................................................................................................................... 89 3.1.5.2 Define Operational Performance Metrics and Targets [NRFU 18-1.5.2] ............ 90 3.1.6 Prepare for and Deliver NRFU-Specific Training [18-1.6] ....................................... 91 3.1.6.1 Prepare for NRFU-Specific Training [NRFU 18-1.6.1] .......................................... 91 3.1.6.2 Deliver NRFU-Specific Training [NRFU 18-1.6.2] ................................................. 92 NRFU Operational Workload Management [NRFU 18-2] ............................................ 92 3.2.1 Monitor Progress and Resolve Issues [NRFU 18-2.1] ............................................. 93 3.2.2 Receive and Process NRFU Universe and Updates [NRFU 18-2.2] ......................... 95 3.2.2.1 Perform In-Office Adds [NRFU 18-2.2.1] ............................................................. 95 3.2.2.2 Receive NRFU Universe and Updates from RPO [NRFU 18-2.2.2] ...................... 96 3.2.2.3 Create Work Assignments (incl. Grouped Manager Visit Cases) [NRFU 18-2.2.3] . ............................................................................................................................. 97 NRFU Data Collection [NRFU 18-3] .............................................................................. 98 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page iv 3.3.1 Receive Work Assignment [NRFU 18-3.1]............................................................. 100 3.3.2 Perform Field Verification Data Collection and Quality Control [NRFU 18-3.2] ... 101 3.3.3 Perform NRFU Manager Visit [NRFU 18-3.3] ........................................................ 101 3.3.4 Perform NRFU Case Enumeration [NRFU 18-3.4] ................................................. 103 3.3.4.1 Enumerate Housing Unit [NRFU 18-3.4.1] ........................................................ 104 3.3.4.2 Enumerate Housing Unit via Proxy Interview [NRFU 18-3.4.2] ........................ 107 3.3.5 Perform NRFU Reinterview Data Collection [NRFU 18-3.5] ................................. 108 3.3.6 Closeout Contact Attempt [NRFU 18-3.6] ............................................................ 109 3.3.6.1 Update Case Information .................................................................................. 111 3.3.6.2 Send Data .......................................................................................................... 111 3.3.7 Submit Time and Expense Data [NRFU 18-3.7] .................................................... 113 NRFU Quality Control Adjudication [NRFU 18-4] ....................................................... 113 3.4.1 Receive Automated Quality Control Comparison Results from RPO [NRFU 18-4.1].. ............................................................................................................................... 114 3.4.2 Adjudicate Reinterview Results [NRFU 18-4.2] .................................................... 115 NRFU Operation Closeout .......................................................................................... 117 4 Cost Factors .............................................................................................. 118 Background ................................................................................................................. 118 Cost Factors ................................................................................................................ 118 Relevant IDEF0 Mechanisms ...................................................................................... 119 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) 5 Page v Measures of Success ................................................................................. 121 Appendix A – Acronyms and Terminology ....................................................... 122 Appendix B – References ................................................................................ 130 Appendix C – Activity Tree for Nonresponse Followup Operation (NRFU) ....... 132 Appendix D – Operation-Specific Kits for NRFU ............................................... 134 Appendix E – Administrative Records Determination Methodology ................ 137 Appendix F – NRFU Administrative Records Contact Strategy ......................... 145 Appendix G – NRFU Training Plan ................................................................... 149 List of Figures Figure 1: A New Design for the 2020 Census .................................................................................. 3 Figure 2: Census 2020 Key Innovation Areas ................................................................................ 10 Figure 3: Summary of Reengineering Field Operations ................................................................ 11 Figure 4: Reengineering the Fieldwork ......................................................................................... 45 Figure 5: Reengineering the Field Staff Structure......................................................................... 47 Figure 6: A Day in the Life of an Enumerator................................................................................ 51 Figure 7: A Day in the Life of a CFS ............................................................................................... 52 Figure 8: A Day in the Life of a CFM .............................................................................................. 54 Figure 9: Nonresponse Followup Operation (NRFU) Context Diagram ........................................ 58 Figure 10: 2020 Census Data Collection Integrated Operations Diagram (IOD) .......................... 72 Figure 11: NRFU Operation Context Model .................................................................................. 81 Figure 12: NRFU Planning and Preparation [NRFU 18-1] Constituent Activities .......................... 82 Figure 13: NRFU Operational Workload Management [NRFU 18-2] Constituent Activities ........ 93 Figure 14: Receive and Process NRFU Universe and Updates ...................................................... 95 Figure 15: NRFU Data Collection [NRFU 18-3] Constituent Activities .......................................... 98 Figure 16: Perform NRFU Data Collection .................................................................................. 100 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page vi Figure 17: Perform NRFU Manager Visit..................................................................................... 102 Figure 18: Perform NRFU Case Enumeration.............................................................................. 103 Figure 19: Enumerate Housing Unit............................................................................................ 104 Figure 20: Enumerate Housing Unit via Proxy Interview ............................................................ 107 Figure 21: Perform NRFU Reinterview Data Collection .............................................................. 109 Figure 22: Closeout Contact Attempt ......................................................................................... 110 Figure 23: Update Case Information ........................................................................................... 111 Figure 24: Send Data ................................................................................................................... 112 Figure 25: NRFU Quality Control Adjudication [NRFU 18-4] Constituent Activities ................... 114 Figure 26: Adjudicate Reinterview Results ................................................................................. 115 Figure 27: NRFU Operation Closeout [NRFU 18-5] Constituent Activities ................................. 117 Figure 28: Occupied Probability .................................................................................................. 141 Figure 29: Household Composition Probability .......................................................................... 144 Figure 30: NRFU Administrative Records Contact Strategy ........................................................ 148 List of Tables Table 1: Comparison of 2010 and 2020 Census NRFU Operations............................................... 12 Table 2: Estimated Effects of Workload Reduction ...................................................................... 16 Table 3: NRFU Key Activities and Dates ........................................................................................ 43 Table 4: Reengineered Office Structure, Field Geography, and Staff Role Decomposition Summary ....................................................................................................................................... 48 Table 5: NRFU Operational Inputs ................................................................................................ 59 Table 6: NRFU Operational Controls ............................................................................................. 61 Table 7: NRFU Operational Outputs ............................................................................................. 62 Table 8: Staff Resources Used Within NRFU Operational Activities ............................................. 66 Table 9: Infrastructure Sites for NRFU Operational Activities ...................................................... 67 Table 10: Systems Used Within NRFU Operational Activities ...................................................... 68 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page vii Table 11: Expected NRFU Field Staff Ratios and Nationwide Counts ........................................... 90 Table 12: Acronyms and Abbreviations List................................................................................ 122 Table 13: Glossary of Terms ........................................................................................................ 127 Table 14: NRFU CFS Supply Kit .................................................................................................... 134 Table 15: NRFU CFS Instructor Kit ............................................................................................... 135 Table 16: NRFU Enumerator Supply Kit ...................................................................................... 135 Table 17: NRFU Bulk Supply Kit................................................................................................... 136 Table 18: Employee/Supervisor Training (Day 1) ....................................................................... 150 Table 19: Employee/Supervisor Training (Days 2 – 4) ................................................................ 150 Table 20: Employee/Supervisor Training (Day 5) ....................................................................... 151 Table 21: Employee/Supervisor Post Classroom Online Training .............................................. 152 Table 22: Employee/Supervisor Mid-Operation Online Training ............................................... 152 Table 23: Supervisor Supplemental Training (Days 6 – 7) .......................................................... 152 Table 24: Supervisor Supplemental Training (Day 8).................................................................. 153 Table 25: Supervisor Post Classroom Online Training ................................................................ 153 Table 26: Supervisor Mid-Operation Online Training ................................................................. 153 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page viii [Page intentionally left blank.] V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 1 1 Document Purpose The 2020 Census Detailed Operational Plan for the Nonresponse Followup operation (NRFU) is intended for use by U.S. Census Bureau managers, staff, contractors, and other internal and external stakeholders working on the 2020 Census. The document presents the detailed operational design for the 2020 Census NRFU operation and includes a summary of the operational processes involved, their inputs, outputs and controls, and the basic mechanisms employed to conduct the operational work. The approach for NRFU in the 2020 Census involves a number of significant innovations that directly contribute to meeting cost and quality goals. Section 2 of this document describes these innovations and provides a more detailed description of the design of the operation. Anticipated uses of this document include the following:       Communication – Documents operational design details for internal and external stakeholders. Planning – Documents planning assumptions and key milestones. Staffing – Documents staffing needs and strategies. Design – Describes operations and flows, which inform design of IT systems, manual processes, and training. Development – Identifies business rules and required capabilities to be developed. Testing – Provides a basis for developing integrated test plans for IT systems and processes. This document complements the 2020 Census Operational Plan, which presents the 2020 Census operational design and covers all operations required to execute the 2020 Census, starting with precensus address and geographic feature updates and ending with census data product dissemination and coverage and quality measurement. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 2 2 Operational Overview Operation Purpose The Nonresponse Followup operation (NRFU) serves two primary purposes:   Determines housing unit (HU) status for nonresponding addresses. Enumerates housing units for which a 2020 Census response was not received. In addition, NRFU field operations include:   Field verification of addresses from respondents using self-response that did not contain a Census ID (Non-ID) and that were not on the Master Address File (MAF). Additional cases from other census operations such as Local Update of Census Addresses (LUCA) appeals and through the U.S. Postal Service Delivery Sequence File (DSF) refresh. It is estimated that at least 60 percent1 of the U.S. population will self-respond. The addresses for which the Census Bureau does not receive a self-response form the initial universe of addresses for NRFU. The NRFU universe also contains addresses for living quarters that require field verification of the address. Background It is projected that approximately 40 percent of the HUs in the 50 states, the District of Columbia, and Puerto Rico will not initially self-respond to the 2020 Census, leaving NRFU responsible for determining the status of those HUs, and if occupied, enumerating them. To accomplish this, NRFU recruits, trains, and manages more than 420,000 field staff that conduct nonresponse followup over an approximate 15-week period2 from mid-May to mid-August 2020. The following sections provide an overview of the nonresponse followup approach for the 2020 Census. While many of the activities occur as part of NRFU, some of the activities are actually performed by other operations, such as Local Update of Census Addresses operation (LUCA) and Census Questionnaire Assistance operation (CQA) on behalf of NRFU. In other cases, NRFU 1 The Internet Self-Response Integrated Project Team (IPT) projects a minimum total response rate of 60.5 percent after six weeks. 2 This period does not include early NRFU, which begins in early April 2020. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 3 is performing data collection activities for other operations, such as Non-ID or Self-Response Quality Assurance (SRQA). 2.2.1 Goal of the 2020 Census and How the Census Works The purpose of the 2020 Census is to conduct a census of population and housing and disseminate the results to the President, the states, and the American people. The goal of the 2020 Census is to count everyone once, only once, and in the right place. As shown in Figure 1, the first step in conducting the 2020 Census is to Establish Where to Count, or to identify all addresses where people could live or stay. An accurate address list helps ensure that everyone is counted. For the 2020 Census, the Census Bureau began an inoffice review of 100 percent of the nation’s addresses in September 2015 and continually updates the address list based on data from multiple sources, including the United States Postal Service (USPS); tribal, state, and local governments; satellite imagery; and third-party data providers. The 2020 Census: A New Design for the 21st Century Motivate People to Respond Conduct a nation-wide communications and partnership campaign TELEPHONE AND PAPER SELFRESPONSE NONRESPONSE FOLLOWUP Work with trusted sources to inspire participation Count the Population Collect data from all households, including group and unique living arrangements Make it easy for people to respond anytime, anywhere Maximize outreach using traditional and new media Encourage people to use the new online response option Target advertisements to specific audiences INTERNET SELF-RESPONSE Use the most cost-effective strategy to contact and count nonrespondents Streamline in-field census-taking Establish Where to Count Knock on doors only when necessary Identify all addresses where people could live IN-FIELD IN-OFFICE Release Census Results Count Everyone Once In the Right Place Process and Provide Census Data Deliver apportionment counts to the President by December 31, 2020 Release counts for redistricting by April 1, 2021 Make it easier for the public to get information Conduct a 100% review and update of the nation’s address list Minimize in-field work with in-office updating Use multiple data sources to identify areas with address changes Get local government input Figure 1: A New Design for the 2020 Census Response rates to surveys and censuses have been declining in recent years. To Motivate People to Respond, the 2020 Census includes a nationwide communications and partnership campaign. This campaign is focused on getting people to respond on their own (self-respond), as it costs significantly less to process a response provided through the internet or through a V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 4 paper form than it does to send an enumerator to someone’s home to collect the response. Advertising makes heavy use of digital media, tailoring messages to various audiences. The Census Bureau Counts the Population by collecting information from all households, including people residing in group or unique living arrangements. The Census Bureau wants to make it easy for people to respond anytime and anywhere. To this end, the 2020 Census offers the opportunity and encourages people to respond online and does not require people to enter a unique Census Identification (ID) with their response. Online responses are accurate, secure, and convenient. If people are at the bus stop, waiting at the doctor’s office, or watching TV and do not have their Census ID handy, they can provide their address instead. For those who do not respond, the Census Bureau uses the most cost-effective strategy for contacting and counting people. A goal for the 2020 Census is to reduce the average number of visits by using available data from government administrative records and third-party (AdRec) sources. These data are used to identify vacant households, determine the best time of day to visit a particular household, or to count the people and fill in the responses with existing highquality data from trusted sources. A reduced number of visits leads to significant cost savings. It also allows the Census Bureau to focus its field resources to achieve consistent response rates across geographic areas and demographic groups. Additional cost savings are expected to result from the use of automation to streamline in-field census taking. Fieldworkers use mobile devices for collecting the data. Operations and functions such as recruiting, training, and payroll are also automated, reducing the time required for these activities. New operational control centers rely on automation to manage the work, enabling more efficient case assignment, automatic determination of optimal travel routes, and reduction of the number of physical offices. In general, a streamlined operation and management structure increases productivity and saves costs. The last step in the 2020 Census is to Release the 2020 Census Results. The 2020 Census data are processed and sent to the President (for apportionment) by December 31, 2020, to the states (for redistricting) by March 31, 2021, and to the public beginning in December 2021. 2.2.2 NRFU Operations Overview 2.2.2.1 NRFU Universe The enumeration universe is the complete set of addresses for living quarters that are enumerated for the 2020 Census. The NRFU universe is a subset of that enumeration universe. It comprises the set of addresses for living quarters that are housing units in TEA 1 and TEA 6 for which the Census Bureau has not yet received a response. The NRFU universe size is V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 5 dynamic in nature—addresses that require verification or are discovered in the course of enumeration are added to the universe. The NRFU workload is defined as cases that are being verified or enumerated by field staff. The NRFU operation is characterized through both its universe and its workload. The NRFU universe contains all cases eligible for NRFU, while the workload is the set of cases that are actively being worked in the operation. So, an individual case is always in the universe, but may not necessarily be in the workload. An example of a case that is in the universe but not in the workload is a self-response that occurs after the NRFU universe is created. When self-responses are received, these cases are removed from the workload but remain part of the universe. One of the definitional differences between “universe” and “workload” is that the NRFU universe is never reduced and only increases during the operation, while the workload both increases and decreases. 2.2.2.2 Type of Enumeration Area Descriptions In an effort to ensure the most cost-effective and efficient process to enumerate households, every block in the U.S. is assigned to one specific type of enumeration area (TEA). The TEA reflects the methodology used to enumerate the households within the block, utilizing a variety of information to identify the most cost-effective enumeration approach for all of the United States, District of Columbia, Puerto Rico, and the Island Areas. The 2020 Census TEA values are described in the Geographic Programs Operation – 6. Geographic Delineations Component Detailed Operational Plan. The NRFU universe includes addresses from the two TEAs outlined below:   TEA 1: Self-Response TEA 6: Update Leave TEA 1: Self-Response (SR) is the primary methodology for the 2020 Census. Invitations to participate in the 2020 Census are provided by letter, postcard, or questionnaire. This invitation has a unique ID associated with the housing unit (HU) and is delivered by the USPS. Residents are encouraged to complete their questionnaire on the internet. Residents also have an option to respond by phone or paper questionnaire. Residents who have not responded by a specific date are included in the Nonresponse Followup operation. TEA 6: Update Leave (UL) is conducted in areas where the majority of the housing units do not have either mail delivered to the physical location of the housing unit or the mail delivery information for the housing unit cannot be verified. UL can be found in both urban and rural areas. UL is a method of data collection in which enumerators canvass assigned areas to update V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 6 census address and map information and leave an invitation to participate in the census. Residents who have not responded by phone, paper, or internet by a certain date are followed up with as part of NRFU operations. 2.2.2.3 NRFU Data Collection The NRFU operation gathers the following information:       Identification of vacant units. Identification of units that are not currently housing units (businesses or uninhabitable or destroyed units). Household and person characteristics of people in occupied housing units. Verification that non-ID self-response addresses exist. Quality control reinterviews of all of the above. Quality check reinterviews of self-response. The basic procedures to collect these data in the 2020 Census include:     V2.0 Creation of a workload for all nonresponding households in TEA 1 and TEA 6, field verification cases from non-ID processing, and recollect cases from the Self-Response Quality Assurance (SRQA) activity. Administrative records (AdRec) modeling of the universe to identify potential vacant and non-housing units, to identify occupied housing units for which demographic information can be provided via administrative records, and to model for the best time to visit the household. Grouping of nonresponding HUs that are part of a multiunit (MU), such as an apartment complex, at the beginning of NRFU. This grouping enables an enumerator to conduct a MU Manager Visit (MV) to determine the status of each nonresponding HU within that MU. The manager is asked about each nonresponding HU in the building/complex to identify the vacant or non-housing units. Only the households that are identified as occupied, or that are left unidentified, are then subject to additional visits to be enumerated. Creation of daily assignments of cases to be sent to NRFU enumerators. Case assignment rules vary based on the three operational phases outlined later in this document. Below is a brief description of the case assignment rules during each phase: o In Phase 1, enumerators are assigned cases in an order that optimizes the likelihood of completing the case while also reducing the travel time and distance. Cases not completed are made available for the next day’s assignment and may be assigned to the same or a different enumerator. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 7 o In Phase 2, enumerators are assigned cases semi-permanently. Enumerators will likely receive the same cases each day, with some variation to account for balancing the workload and filling their workday. o In the Closeout Phase, enumerators are assigned cases semi-permanently as in Phase 2. Cases without sufficient data to support apportionment will be assigned daily until sufficient data are collected or the operation ends. Cases assigned to an enumerator will receive the following treatment:      V2.0 Each case receives a set number of attempt days (defined as attempts, but multiple attempts during a day is considered a single attempt day). The default number of attempts for production NRFU, NRFU reinterview (RI), and SRQA recollect cases is six. However, production NRFU cases with sufficient administrative records data available receive only one attempt. Field verification cases receive one attempt. Manager visits receive two attempts. Production NRFU cases are subject to reopening for additional attempts during the Closeout Phase in order to collect sufficient data to support apportionment. After each failed attempt, a Notice of Visit (NOV) is left for the household encouraging self-response. It is expected that up to 10 percent of the NRFU workload will selfrespond because of a NRFU attempt and the NOV. After a NOV is left, the case is placed on hold for a short time before being reassigned to allow time for a self-response. The length of the hold time is dependent on the timing of the attempt. Cases attempted early in the NRFU operation are to be placed on hold for a longer amount of time than cases attempted later in the NRFU operation. On the third failed attempt, a case becomes proxy-eligible. A proxy respondent is a neighbor, landlord, real estate agent, or other knowledgeable person who can provide information about the unit and the people who live there. Proxy responses are an important source of data for the 2020 Census for the hard-to-count population. An enumerator attempts to contact up to three proxy respondents after each noninterview for a proxy-eligible case. On subsequent visits, the enumerator first attempts to enumerate the assigned housing unit before attempting to get a proxy response. Units initially identified as vacant or some other type of nonoccupied unit status are verified by a proxy respondent. If a proxy cannot be found, a second enumerator must verify the status. Contact with housing unit residents is not required for the field verification of non-ID households in the NRFU universe. If the enumerator finds the housing unit with the given address, a GPS map spot is taken by the instrument so that the address can be added to the MAF and the case is closed. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)  Page 8 The enumeration of NRFU RI cases is conducted by a different enumerator from the one who conducted the original interview. The creation of the main NRFU workload occurs in early May 2020. However, before this time, Early NRFU is conducted in blocks surrounding colleges and universities where students live offcampus. Early NRFU aims to enumerate nonresponding students in their Census Day (April 1) housing unit before they leave for their summer break. Early NRFU blocks are subject to NRFU enumeration before all self-response mailings have completed. Administrative Records (AdRec) modeling is not performed for Early NRFU and case assignments during this time frame will not follow the Phased contact strategy outlined above. However, the enumeration procedures will be the same during Early NRFU, including the quality control components (NRFU RI and recollect). 2.2.2.4 Coverage Improvement It is pertinent to the accuracy of the census that only the people who usually live or stay at each address are counted there, and that no one is missed. Coverage improvement methods are used to achieve this goal. This document briefly describes the coverage improvement methods to be used during the 2020 Census, as well as the criteria for the identification of the cases eligible for coverage improvement followup. The goal of the Coverage Improvement (CI) operational activity is to resolve erroneous enumerations (people who were counted in the wrong place or counted more than once) and omissions (people who were missed) from all housing unit data. CI attempts to resolve potential issues identified within the following types of responses:     Paper questionnaire self-response. Internet Self-Response (ISR). Census Questionnaire Assistance (CQA). Nonresponse Followup (NRFU). The Coverage Improvement Project Team performs planning and preparation activities in advance of its 2020 Census operational activities. CI is an outbound telephone interview. The CI Team has developed processing requirements and business rules, which are used to create a workload for the production CI telephone interviews. To select cases for CI, the Self-Response Quality Assurance System runs a code on all respondent-provided housing unit data, creating the CI workload. To be eligible for a Coverage Improvement telephone interview, a case must: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)       Page 9 Have a valid, respondent-provided telephone number. Originate from a paper, ISR, CQA, or NRFU return with data provided by a respondent in the household. Cases identified as NRFU AdRec Occupied or NRFU Proxy are excluded. Be identified as having at least one of the coverage issues designated for inclusion in the Coverage Improvement telephone interview. Contain enough address components to identify the housing unit during a telephone interview (i.e., a city-style address). Identify at least one person by name on the roster who is age 15 or older. Have no previous contact with the NRFU reinterview or Self-Response Quality Assurance re-collect activities. The CI workload is sent to the CQA operation, which assigns the telephone workload, performs case management, conducts telephone interviews utilizing the Coverage Improvement Instrument, and sends collected data back to RPO. 2.2.3 Why It’s so Important — Most Expensive Aspect of the Census With a 2010 Census cost of more than $2.01 billion (Walker et al., 2012), it is clear that improving the efficiency of NRFU is critical to a cost-effective census in 2020. Based on recommendations from 2010 Census studies and reviews, several key innovations are planned for the 2020 Census NRFU. In previous censuses, NRFU enumerators attempted to collect responses from every address that did not return a paper questionnaire. A paper-based operations control system sent assignment areas to crew leaders with the addresses of the nonresponding households. Crew leaders manually divided the assignment areas among their enumerators and instructed them to attempt to contact every housing unit up to six times. Enumerators used paper maps to locate the addresses and paper questionnaires to conduct the interviews. Responses were mailed to a processing center to be scanned. Paper payroll forms also had to be collected and processed. Field costs associated with the NRFU and Address Canvassing operations comprise the most expensive parts of the 2020 Census. With cost reduction in mind, the 2020 Census team focused on four Key Innovation Areas shown in Figure 2. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 10 Figure 2: Census 2020 Key Innovation Areas Specific efforts for reducing the cost of the 2020 Census NRFU operation focused primarily on the last two Innovation Areas – Utilizing Administrative Records and Third-Party Data and Reengineering Field Operations. 2.2.4 NRFU Cost Reduction Efforts for the 2020 Census The use of administrative records and third-party data (AdRec) is intended to reduce the field workload associated with NRFU by reducing the average number of visits to nonresponding housing units. These data are used to identify vacant and nonhousing units and to reduce the number of contact attempts for occupied households if high-quality data exist that could be substituted for enumeration. These data are also used to improve efficiency by tailoring NRFU work assignments based on the best time of day to visit households. Reengineering field operations is intended to increase the efficiency of NRFU by allowing managers and field staff to be more productive and effective. As shown in Figure 3, the 2020 Census streamlined the NRFU field operation and management structure by using automation for increased efficiency. Enumerators use mobile devices to receive assignments and collect census data, allowing for near real-time case status updates and transmission of response data. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 11 Figure 3: Summary of Reengineering Field Operations With the use of mobile devices, census field supervisors are able to work remotely and easily communicate with their staff. The new operational control system enables field supervisors to:    Automatically assign cases using a tailored contact strategy and stopping rules. Optimize enumerator routes in the field. Monitor and manage enumerators’ work with daily tracking of progress and real-time alerts about performance issues. This increased use of technology to more effectively manage fieldwork enables a more efficient office and staffing structure. For example, by making it easier for supervisors to monitor and manage their staff, the number of enumerators for each supervisor can increase. The automation of recruiting, training, and payroll and expense reporting also reduces the need for office staff. The 2010 Census had 494 local census offices (Walker et al., 2012). The new field operations design for the 2020 Census has 248 area census offices. Table 1 summarizes additional NRFU differences between the 2010 and 2020 Censuses that are expected to reduce costs. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 12 Table 1: Comparison of 2010 and 2020 Census NRFU Operations 2010 Census Administrative Records and Third-Party Data: Not used for NRFU in the 2010 Census for the purpose of workload management. 2020 Census Administrative Records and Third-Party Data: These data are used to reduce the NRFU workload by identifying vacant or non-housing units and by using high-quality administrative records data to enumerate occupied households. Enumeration: Paper-based enumeration; used paper maps to locate addresses; mailed paper forms to central processing center to be scanned. Enumeration: Enumerators use mobile devices to collect census data, allowing for near real-time case status updates and transmission of response data. GPS-fed maps are incorporated into the devices to show real-time location and location of the address. Training: In person classroom training. Training: Combination of online, self- study, and classroom training methods. Contacts: Enumerators were advised to attempt an address up to six times until the case was resolved. Contacts: Use an adaptive design contact strategy with a goal of reducing the number of contact attempts on each household. Self-Responses: Limited ability to remove self-responses (manual, laborious process). Self-Responses: Self-responses are removed from the NRFU workload on a near real-time basis throughout the NRFU operation. Field Management Structure: A standard ratio of enumerators to supervisors to enumerators was used and there were 494 local census offices. Field Management Structure: The ratio of enumerators to supervisors is higher, reducing the number of area census offices needed to 248. Multiunits: All multiunits were treated similar to other households. Multiunits: Initial visit to manager to assist with removal of vacant properties, thus reducing NRFU fieldwork. Proxies: Households became proxy eligible after the third contact attempt, however these proxy attempts were not tracked and hard to validate. Proxies: Households become proxy eligible after the third visit and the instrument requires the input for three proxies for each additional attempt upon case eligibility. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 13 For more information on:  Administrative Records and Third-Party Data: See Section 2.2.4.1 and Section 2.3.2.  Enumeration: See Section 2.3.3 and Section 2.3.4.  Training: See Section 2.5.3.  Contacts: See Section 2.3.1.  Field Management Structure: See Section 2.5.2.  Multiunits: See Section 2.3.4.2.  Proxies: See Section 2.3.3.5. 2.2.4.1 Reduce the NRFU Workload A primary cost-saving method for the 2020 Census NRFU operation is to reduce the NRFU workload and attempts needed to get quality data for housing units. This is accomplished several ways:      Use of administrative records to identify vacant and nonhousing units in the NRFU universe and reducing attempts to these units to a single verification. Use of administrative records to provide quality data about the household in occupied units, thus reducing fieldwork to one in-person visit for housing units with high-quality administrative records data. Use of multiunit manager visits (MV) before conducting NRFU interviews with HUs to determine the status of each nonresponding HU within that multiunit (MU). Notices of Visit (NOVs) that direct respondents to complete the questionnaire online, allowing additional self-responses, which would be removed from the NRFU workload. Use of proxy respondents after the third attempt to collect data for hard-to-reach households, thus reducing the need for fourth, fifth, or sixth visits to a unit. Administrative Records The primary use of administrative records for NRFU is to reduce fieldwork for follow-up activities. To this end, the Census Bureau uses data from internal and external sources, such as the 2010 Census, the United States Postal Service (USPS), the Internal Revenue Service (IRS), and the Centers for Medicare and Medicaid Services to identify vacant and non-housing units and remove them from the NRFU workload. Administrative record vacant and non-housing units are identified before the start of NRFU and receive a verification visit to confirm the status. The NRFU operational design uses administrative records and third-party data to enumerate occupied housing units where it makes sense and is feasible. Housing units identified as having good administrative record data V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 14 receive a single contact attempt during NRFU, after which administrative records data are compiled for that household if the attempt was not successful at reaching a respondent. For those addresses that are determined to be occupied, have high-quality administrative records data available, and are incomplete after one personal visit attempt, a final letter encouraging self-response is mailed within seven days. This letter lets the household know that while an enumerator may not visit the unit anymore, they can still respond by going online or contacting CQA. Additionally, addresses removed from the NRFU workload as either vacant or delete receive a final mailing that encourages occupants to self-respond to the 2020 Census. Addresses that are determined to be vacant/delete are immediately mailed a final letter encouraging self-response. For more information on administrative records, see Section 2.3.2 Administrative Records Modeling and Tailored Contact Attempt Strategy; Appendix E – Administrative Records Determination Methodology; and Appendix F – NRFU Administrative Records Contact Strategy. Manager Visits One method for reducing the NRFU workload is to conduct multiunit manager visits (MV). When the NRFU universe is initially formed, any nonresponding HUs that belong to a multiunit building (MU) are grouped together. Before attempting to enumerate each individual nonresponding HU that belongs to a MU, the enumerator first tries to interview the manager of the MU. During a manager visit, the enumerator asks the building manager which of the nonresponding units were occupied or vacant on Census Day. The HUs labeled “vacant” or “other” by a manager do not need to be enumerated and are removed from the NRFU workload. The HUs labeled “occupied” by a manager, or left undetermined, are assigned for enumeration. If the MV is unsuccessful after two attempts, then all nonresponding HUs are assigned for enumeration. Generally, the cases are assigned the following day, however if the manager visit is conducted after 4 P.M on a weekday or anytime on Saturday or Sunday, the enumerator will begin enumerating these cases directly after the MV attempt. For more information on multiunit housing units, see Section 2.3.4.2. Self-Response After a NRFU Visit During the NRFU operation a visit by a census enumerator can spur self-response. Some people are well-meaning about completing the census questionnaire but simply put it off and then forget about it. Often, knowing that someone came to the door is enough to encourage them to respond. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 15 When enumerators are unable to complete an interview, they leave a Notice of Visit (NOV) at the housing unit that informs the household that someone has been there and will return at a later date. The NOV provides the household with the information they need to self-respond either online or through CQA. Self-responding units are removed from the NRFU workload in near real-time upon notification of receipt of a completed census questionnaire or contact through the CQA. While this was also a possibility in the 2010 Census (and Census 2000) at the beginning of the operation, it was a paper-based, manual process that was not efficient or timely enough to have a significant impact on the amount of fieldwork. The automation of the 2020 NRFU operation allows for the NOV to be the prompt for self-response that can reduce the number of followup attempts significantly. Proxy Respondents A proxy is a respondent, such as a neighbor, caretaker, etc., who may have sufficient knowledge to enumerate the NRFU household. In the 2010 Census, about one-fourth of the NRFU interviews for occupied housing units were conducted using proxies. For the 2020 Census, after a third failed attempt to collect data from a housing unit, the case becomes proxy-eligible. An enumerator will attempt to collect data from up to three neighbors or other knowledgeable people about the housing unit after the third, fourth, fifth, and sixth attempts at enumerating the unit. See Section 2.3.3.5. Effect of Workload Reduction As shown in Table 2, the estimated effect of these efforts reduces the NRFU fieldwork by resolving cases without the need for a full interview or the full NRFU contact strategy. The methods for reducing the workload include: (a) reducing the number of possible attempts from six to one, (b) using information provided by a proxy to remove a case from the NRFU workload, and (c) removing a case during the NRFU operation because the HU responded by a different mode. Identifying vacant/delete units as quickly as possible is an important driver in keeping NRFU costs down by reducing the number of cases that enumerators need to repeatedly attempt to contact. In addition, the use of AdRec modeling and Notices of Visit to encourage self-response reduces the number of multiple visits to the same occupied units. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 16 Table 2: Estimated Effects of Workload Reduction Description of Reduction Estimated Percentage of Case Workload Reduced Administrative Records Vacant and Non-housing Unit Removals Housing units (HUs) are identified as vacant or non-housing units and may be removed from the NRFU workload after a single verification attempt. 9.6% Administrative Records Occupied Enumeration data exist for an HU through administrative records, and the HU removed from the NRFU workload after a single attempt (not the standard six attempts). 12.9% Manager Visit – Vacant Removals The building manager indicates that an HU is vacant, and it is removed from the NRFU workload. 5% Self-Response after NRFU Visit The HU was a part of the NRFU workload, but a self-response was received through another mode. The HU is removed from the workload when the response is received, regardless of the number of attempts made. 9.7% Method to Reduce Workload Factors that Could Increase the NRFU Workload in the 2020 Census While concerted efforts are being taken to reduce the cost of NRFU and overall workload, several factors cannot be controlled. Identified risks, described below, that could increase the size of the NRFU operation include:      V2.0 Low self-response rates. High number of Non-ID responses that need Field Verification. High number of SRQA recollect cases that require additional fieldwork. Operational improvements that increase address coverage and thus the universe size. Unsuccessful technical innovations. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 17 Many aspects related to the NRFU operational design and the infrastructure necessary to support it are based on workload assumptions. A key input to those workload assumptions is the self-response rate. If the 2020 Census self-response rate falls below expectations, then the initial NRFU workload will be higher than expected, and the infrastructure may be insufficient to support the increased field data collection volume. In this case, additional enumerators would be needed, and costs would rise accordingly. Similarly, a larger than anticipated number of self-responses without Census IDs using addresses not in the MAF would increase the number of cases that need Field Verification. Other operations are striving to develop and improve the coverage and quality of the address frame used for the 2020 Census, which could affect the NRFU workload. While improved coverage would positively impact the overall census, it could result in a larger than expected universe. This would increase the NRFU operational workload, and expected cost savings from NRFU may not be realized. Technical innovations such as assignment optimization are key elements to the operational design for conducting a more efficient NRFU operation. If any aspect of the planned technical innovations does not perform as expected, then the operational design for NRFU may not be as successful as anticipated. Other Factors that Could Increase the cost of NRFU in the 2020 Census Technical innovations are expected to reduce the cost of NRFU, but the cost of the operation can be greatly impacted by economic conditions beyond the Census Bureau’s control. A strong economy can result in a more difficult hiring environment for field staff, resulting in the need to increase hourly wages, pay overtime, etc. If economic conditions are reflective of this at the time of the 2020 Census, then the costs to implement NRFU may prevent the expected cost savings from being realized. 2.2.4.2 Improve the Efficiency of the Operation The reengineered 2020 Census field operations concept includes an updated field operational model and the associated updates to field staff organizational structure and system capabilities to support it. The overall goal is achieving cost savings through efficiency of the field operations and supporting structures and activities. NRFU is using a reengineered field management structure and approach for managing fieldwork, which includes: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)   Page 18 Using new field staff roles and staffing ratios. Using automation for: o Optimization of daily enumerator assignments. o Near real-time operations information for decision making. o Enhanced operational control system. o Payroll submission and approval processing. o Training of enumerators and managers. Enumerators are assigned NRFU addresses daily, based on their home location, work availability, the location of NRFU workload, and other operational business rules. Enumerators work the addresses in a prescribed order to determine the Census Day status of the housing unit and, when occupied, enumerate the housing unit. Enumerators use an automated data collection application on a mobile device to record the Census Day housing unit status and enumeration data. For more information on devices, see Section 2.5.4. If a respondent is not at home, enumerators leave a NOV, directing the respondent to the internet or CQA to selfrespond. Unlike the 2010 Census, the 2020 Census NRFU operation uses an adaptive design methodology that includes a variable contact strategy, decisions about when proxy responses are acceptable, and stopping rules to control the number of attempts made for each address. As detailed in Section 2.2.4.1, nonresponding HUs that belong to a MU are grouped together so that an MV is conducted first to identify vacant units in the MU. This reduces the NRFU fieldwork for that MU. The 2020 Census NRFU operational design will infuse quality throughout the workload management and data collection processes. Examples of aspects of NRFU designed to maintain or improve quality include:    Increasing accuracy and quality checks by using real-time paradata and editing capabilities. Using a Best Time to Contact model in assigning work to increase the likelihood of finding respondents at home. Providing early opportunities to identify and take corrective action in defined situations by capabilities available through an enhanced operational control system with real-time supervisory alerts. In addition, NRFU includes a reinterview component designed to deter and detect enumerator falsification and procedural violations. The 2020 Census will use analytic techniques to better V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 19 focus QC interviews on finding potential quality issues. A check of responses against administrative records identifies cases with known quality data and allows those cases to be excluded from the QA sample of cases to be reinterviewed. Paradata, such as length of interview, are used to flag suspicious results. These changes make the RI cases more efficiently selected. Training of census field enumerators and managers includes both classroom and computerbased training. Computer-based training occurs before classroom training. Computer-based training ensures uniform training of all NRFU procedures across all regions and offices. Enumerators can train at their own pace, which adds to the efficiency of the training. Classroom training enhances the computer-based training by providing an environment for interactive learning through scenarios and for trainees to ask questions about elements they are unclear about. For more information on NRFU Training, see Section 2.5.3. 2.2.5 Undercount of Young Children The NRFU operation for the 2020 Census pays special attention to the importance of accurately representing young children in the census count. Counting young children is always a challenge in the decennial census. The Census Bureau addresses the challenge in the 2020 Census NRFU operation with three initiatives:    The online training for NRFU enumerators includes updated language to specifically mention counting children. For example, changes to phrasing such as “the census counts people/residents” to “the census counts all adults and children.” The NRFU enumerator in-class training includes a case study that highlights the importance of counting young children. This case study involves a respondent who is confused about counting a young child who is staying with the respondent temporarily. All Frequently Asked Questions (FAQs) and job aids for NRFU enumerators and CQA customer service representatives have been updated to address counting young children. NRFU Components 2.3.1 Contact Strategy All cases in the NRFU workload have specific requirements and business rules regarding contact attempts in order to best complete each case in an efficient and cost-effective way. During the 2020 Census, the NRFU Operation has two components – Early NRFU, which begins in early April 2020, and Regular NRFU (also called “NRFU”), which begins in mid-May 2020. The contact V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 20 strategy for Early NRFU is different from the strategy used during NRFU. Additional details about Early NRFU can be found in Section 2.3.4.3 of this document. The content below focuses on the three-phased contact strategy implemented during NRFU. During both components of the operation, each case will receive a maximum number of contact attempts. Since an enumerator may contact a housing unit more than once in a day, contact attempts are counted by day. For example, the first contact may be in the morning and the respondent is not home. However, later in the day, the enumerator notices a car in the driveway and contacts the housing unit again. While the enumerator made two contact attempts that day, only one contact attempt-day has passed. Thus, for a NRFU case that has a default maximum attempt days of six, this scenario would count as only one of those six attempt-days. Phase 1 of the NRFU contact strategy centers on the first four contact attempt-days. This phase focuses on enumerating households quickly during the beginning of the operation. The MOJO optimizer is utilized to assign cases such that enumerator travel is minimized and households are contacted when a respondent is expected to be home. The MOJO optimizer uses a number of inputs to ensure efficient case assignment, which includes variables like the enumerator’s starting address, work availability, the location of open cases, and best time to contact probabilities from administrative record modeling. The assignment of cases is prioritized during this phase to ensure quality control cases are highest priority. Multiunit Manager Visit (MV) cases also have preference to ensure that the individual units within an apartment complex become available for enumeration earlier in the operation. A secondary preference is given to cases that have not received at least one contact attempt to ensure each housing unit is visited early in the operation. Cases are assigned daily based on efficiency of effort; thus, a case may be assigned to different enumerators each day. The default number of contact attempt-days for a NRFU case is six. At the conclusion of the third contact attempt, a NRFU case becomes proxy-eligible. Enumerators will attempt to contact at least three respondents that do not live in the household but may be knowledgeable about the household. Typically, an enumerator will attempt neighbors on either side of the housing unit and across the street. However, enumerators may interview any respondent that has knowledge about the household. Proxy eligibility and enumeration are further detailed in Section 2.3.3.5. As noted above, the MOJO optimizer utilizes information from administrative records to time the assignment of a case with the time that a respondent is expected to be home. These criteria V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 21 are only used during the first and second contact attempt-days. On subsequent attempts, the best-time-to-contact criteria are ignored in favor of maximizing an enumerator’s work day. After each contact attempt, no matter the phase, the enumerator will leave a Notice of Visit (NOV) notifying the household that a visit was made and encouraging them to self-respond. The NOV contains the ID that the respondent should use to self-respond. During Phase 1, the case is put on hold for two days after the first visit. Instead of re-assigning the case the next day, the two-day hold is implemented to allow time for self-response. However, this hold and almost all others will only be used after the first attempt-day. Again, the holds are removed in favor of maximizing an enumerator’s work day and utilizing all available staff at the beginning of the operation. An exception is made for the five-day hold for cases where a respondent requests an envelope to mail a questionnaire they have either completed or plan to complete. This extended hold allows for the mailing and receipt of that package and is only utilized during Phase 1 contact attempts (attempts one, two, three, and four). A case that has completed four contact attempt-days during Phase 1 is placed on hold in preparation for Phase 2 of the contact strategy. A CFS area remains in Phase 1 until 60 percent of cases in that geography complete four contact attempt-days or are resolved. Phase 2 of the NRFU contact strategy focuses on acquiring a response for cases that have been previously attempted but not yet resolved. This phase capitalizes on findings from inter-census testing that denotes one of the key factors of resolving a case is the enumerator. Phase 2 focuses on assigning cases to better performing enumerators for multiple contact attempts. Phase 2 eligibility is determined at the CFS area geography, which, on average, contains around 4,000 cases. When 60 percent of cases in a CFS area are resolved or reach four contact attemptdays, the CFS area is eligible for Phase 2. CFMs can activate Phase 2 for a CFS area starting June 5, 2020, and all CFS areas are moved into Phase 2 by June 17, 2020. While activating Phase 2, a CFM is required to assign an enumerator or a group of enumerators directly to the CFS area. The CFM assigns better performing enumerators to Phase 2 areas. Those enumerators receive similar assignments each day and can capitalize on knowledge gained about the households in their assignment during previous attempts they made. Each day’s case assignment may be slightly different due to a nightly rebalancing of the workload. Each night, the MOJO optimizer splits the assignment of cases in a CFS area between the enumerators assigned to that area by the CFM. The rebalancing accounts for daily closure of cases and daily staffing changes among enumerators in these CFS areas. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 22 The priority and hold logic from Phase 1 continue to be utilized in Phase 2. After reaching the maximum number of attempts (typically six), cases are closed regardless of their resolution status. The Closeout Phase, or Closeout, is the final phase of the NRFU contact strategy and focuses on resolving all unresolved cases by the end of the NRFU operation (July 24, 2020). This includes the resolution of cases that were previously closed but do not have enough data to support apportionment. Like Phase 2, the Closeout Phase eligibility is determined at the CFS area geography. When 85 percent of cases in a CFS area are closed, regardless of the resolution status, the CFS area is eligible for Closeout. CFMs can activate Closeout for a CFS area starting on June 23, 2020, and all CFS areas are moved into Closeout by July 10, 2020. CFMs will activate Closeout in the same way they activated Phase 2, where specific enumerators will be assigned to the CFS area. A key difference between Phase 2 and Closeout is that during Closeout, cases that were previously closed may be reopened. If a NRFU case was closed without enough data to support apportionment, it will be reopened for additional contact attempt-days until the end of the NRFU operation. Some of these cases may remain closed if there is enough administrative record data to support apportionment or confirm the housing unit status of the case (i.e. vacant or not a housing unit). With the focus on completing enumeration, only NRFU cases are subject to reopen as outlined here. Quality control cases such as NRFU Reinterview and Self Response Quality Assurance Recollect will not be reopened during this phase. 2.3.2 Administrative Records Modeling and Tailored Contact Attempt Strategy At certain points in the NRFU operation, Administrative Records (AdRec) Modeling is performed on all NRFU cases. The results of this modeling will give each case one of four modeled status outcomes: 1. 2. 3. 4. Occupied. Vacant. Delete. No Determination. These modeled statuses have direct implications on the NRFU operation in the following ways: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 23 1. All cases modeled as ‘Occupied’ receive only one contact attempt-day. These cases are not proxy eligible. If these cases are not resolved on the one and only attempt-day, a final mailing to encourage self-response is sent. The case is immediately closed out from NRFU workload. No additional contact attempts are made for these cases by an Enumerator. These are referred to as occupied removals. Administrative records data are used to enumerate the household. 2. All cases modeled as ‘Vacant’ are sent a postcard that encourages self-response. a. If the mailing comes back from the USPS as Undeliverable as Addressed (UAA), the case is immediately added into the NRFU workload and receives one attempt. Depending on the outcome of this one attempt, the case either: i. Is closed from the NRFU workload and referred to as a vacant removal. ii. Remains in the NRFU workload and proceeds with normal follow-up attempts. b. If the mailing does not come back as UAA by June 1, 2020 (or an earlier date defined by Census Bureau management), the case is added into the NRFU workload and proceeds with normal follow-up attempts defined in the remainder of this section. c. If an enumerator observes signs that a case modeled as ‘Vacant’ is actually occupied, the case will receive six contact attempts and no longer be subject to the processing described below for AdRec ‘Vacant’ cases. 3. All cases modeled as ‘Delete’ are sent a postcard that encourages self-response. a. If the mailing comes back as UAA, the case is immediately added into the NRFU workload and receives one attempt. Depending on the outcome of this one attempt, the case either: i. Is closed from the NRFU workload and referred to as a delete removal. ii. Remains in the NRFU workload and proceeds with normal follow-up attempts. b. If the mailing does not come back as UAA by June 1, 2020 (or an earlier date defined by Census Bureau management), the case is added into the NRFU workload and proceeds with normal follow-up attempts defined in the remainder of this document. c. If an enumerator observes signs that a case modeled as ‘Delete’ is actually occupied, the case will receive six contact attempts and no longer be subject to the processing described below for AdRec ‘Delete’ cases. 4. All cases modeled as ‘No Determination’ proceed with normal follow-up attempts defined in the remainder of this section. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 24 In addition to setting a modeled status for each case, the AdRec modeling process also models a best-time-to-contact probability. In June, after the IRS provides additional information, all cases that previously had no determination are modeled again using the updated administrative records data. This will increase the number of cases modeled as occupied. The June modeling also identifies addresses for which the administrative data do not meet the thresholds for early removal but do provide adequate data to support apportionment. During closeout, if these cases had at least one attempt-day, they are removed from the workload and the administrative record data are used for enumeration. The June modeling results include:       Occupied: Same as “Occupied” above and cases that have already had one or more attempts are immediately removed from the workload. Occupied Closeout: Administrative records data do not meet threshold for early removal but do provide adequate data to support apportionment. After six attempts, administrative record data will be used to enumerate the housing unit. At the closeout phase, the case is removed from the workload if at least one contact attempt has been made. Vacant: Same as “Vacant” above and cases that have already had one or more attempts are immediately removed from the workload. Vacant Closeout: After six attempts, the case is closed as vacant. At the closeout phase, the case is removed from the workload if at least one contact attempt has been made and one of those attempts resulted in an observed vacancy by the enumerator. Delete: same as “Delete” above and cases that have already had one or more attempts are immediately removed from the workload. Delete Closeout: After six attempts, the case is closed as a delete. At the closeout phase, the case is removed from the workload if at least one contact attempt has been made and one of those attempts resulted in an observed deletion by the enumerator. 2.3.3 Housing Unit Enumeration NRFU Enumeration consists of an enumerator making a physical visit to a housing unit that has not previously responded to the census. There are four basic outcomes from the visit:   V2.0 The unit is vacant. The unit is not a housing unit (and should be deleted from the workload). July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)   Page 25 The unit is occupied, and the enumerator completes the questionnaire with the household. The unit is unable to be enumerated on this visit. Each of these outcomes has variations and particular actions associated with it. 2.3.3.1 Vacant Units When an enumerator encounters a vacant unit, the enumerator needs to verify that the unit was vacant on Census Day. A knowledgeable individual must confirm the vacancy or provide a proxy response in the case where the household moved out after Census Day. The proxy could be a neighbor, a landlord, or the real estate agent on a “for sale” sign. Once the vacancy is confirmed, the case is closed. However, if the vacancy cannot be verified, the case remains open. The case is assigned to a different enumerator for another attempt. The same procedures for enumeration apply to the second visit. If a second enumerator also identifies it as vacant and also cannot get verification, then the case is closed. Requiring a second enumerator to visit the unit reduces the likelihood of an enumerator declaring a unit vacant as an easy completion. Units identified as vacant during self-response are included in the NRFU workload. These cases are treated as regular NRFU cases, but a single vacant outcome closes out the case, even if that outcome is unverified. The visit acts as a verification of the self-response status. Enumerators do not know that the case was previously self-reported as vacant. To the enumerator, it appears as a regular NRFU case. 2.3.3.2 Nonhousing Units Some addresses turn out to be nonhousing units. Nonhousing unit statuses include:     Demolished. Burned out. Uninhabitable. Nonresidential, such as business. These are sometimes referred as “deletes” as they are usually removed from the census address list, but there are exceptions. In addition to the above, addresses within mobile home parks on which no mobile home/trailer was situated on Census Day are coded as “empty mobile home/trailer site.” This code does not V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 26 delete the address from the MAF, but it ensures that the address is not counted in the census counts. Units that have been converted have special instructions. For example, if the given address is expected to be a single-family home but the unit has been converted into separate apartments, each with a unique address, then the original case is deleted and the new apartments are added using the in-field add procedures. Similar to vacant procedures, most nonhousing unit outcomes need verification by a proxy to ensure that unit did not have someone living in it on Census Day. If a proxy cannot be found, the case is reassigned to a different enumerator for another attempt. 2.3.3.3 Enumeration of a Household If the enumerator encounters an occupied housing unit and someone is home, the enumerator completes the census questionnaire with an eligible respondent. The questions presented to the occupant are the same as those used for self-response, but there may be slight variation in data collection and wording because of the mode of the data collection. NRFU data collection differs from self-response as it is collected by an enumerator asking the questions and inputting answers into an automated application on a mobile device. Depending on the size of the household, enumeration should take between five and 15 minutes. Data are collected on every person in the housing unit whether they are related to the respondent or not. Questions are included to prevent undercounting of nonfamily members or family members who may be away temporarily. To prevent overcounting people, there are questions about other places people might live to ensure that people are counted in only one location. A successful enumeration closes a case. 2.3.3.4 Unsuccessful Attempts An unsuccessful attempt occurs when a visit does not result in an outcome that would complete a case. These possible outcomes include:      V2.0 A vacant or nonhousing unit that has not been proxy-verified. No one home. No eligible respondent home. Refusal or inconvenient time. Language or hearing barrier that prevents enumeration. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 27 When an enumerator is unable to complete an interview, in most cases the enumerator leaves a NOV that provides the resident with the information needed to self-respond online. Upon receipt of a sufficient response for that address, the case is removed from the NRFU workload. If enumerators cannot complete the questionnaire from the housing unit or by using a proxy, they are encouraged to obtain at least an indication of the housing unit status (e.g., vacant, occupied), and the number of people in the housing unit from someone. While this is insufficient to close out a case, it does provide the minimum information needed for the census to meet the basic requirements of providing a population and housing unit count. 2.3.3.5 Proxy Interviews Enumerators attempt to locate the address for each case assigned to them from the NRFU workload and contact a household member at the address. However, in certain situations, the enumerator may attempt to interview a proxy. A proxy is a respondent who may have sufficient knowledge to enumerate the NRFU household. In the 2010 Census, about one-fourth of the NRFU interviews for occupied HUs were conducted using proxy respondents. Proxy Eligibility Cases become proxy-eligible on the third unsuccessful personal visit attempt at an HU. However, there are a few cases that immediately become proxy-eligible. First, if a respondent indicates that they did not live at the NRFU address on Census Day, the respondent is an inmover and treated as a proxy. If the in-mover has sufficient information regarding the people who lived at the case address on Census Day, then the enumerator records the information and the case is closed. If not, the case remains proxy-eligible. However, a proxy alert flag on the device will inform all future enumerators not to contact the in-mover again. Second, if an enumerator encounters a dangerous situation at the assigned case address, the case immediately becomes proxy-eligible. Furthermore, if an enumerator determines that a case is a special case status, depending on business rules, then the enumerator may immediately seek a proxy. The special case statuses that result in an immediate proxy attempt include:       V2.0 Vacant by observation. Does not exist. Demolished/Burned out. Nonresidential. Uninhabitable. Empty mobile home. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 28 Lastly, if an enumerator encounters a nonresident at the case address, the case becomes proxyeligible if:    The nonresident is a personal caretaker or family member. The resident at the case address is unavailable or unable to complete the enumeration. The nonresident can provide the appropriate information needed on the resident(s) of the case address. Choosing a Proxy Once a case becomes proxy-eligible, strict proxy procedures must be followed. After an attempt to interview a proxy, the mobile device prompts the enumerator with “What best describes the proxy?” and give the following options to choose from:           Neighbor. Landlord or property manager (owner, rental office manager, etc.). Real estate agent/office. Relative of household member. Caregiver or health provider. In-mover (moved in after April 1, 2020). Government worker (tax assessor, letter carrier, etc.). Utility worker (meter reader, telephone repair, cable/satellite, etc.). Enumerator personal knowledge. Other. The proxy types above can be grouped in the following ways: Nonresident Inside Case Address: The nonresident inside the case address would include someone who answers the door or the phone at the case address who is not a resident of that address. If there is a nonresident at the case address, this case may become proxy-eligible immediately (See Proxy Eligibility). Otherwise, the nonresident may be interviewed as a proxy after three failed personal visit attempts. Neighbor: A neighbor includes someone who lives within a certain vicinity of the case address the enumerator is trying to contact. The enumerator can refer to “Proxy Alerts” to see if any neighbors have previously been failed proxy attempts, and, if noted, avoid these addresses. Nonneighbor Cases: The enumerator may be able to conduct a proxy interview/attempt with someone who is a nonneighbor who is not currently at the case address. For example, if an V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 29 enumerator attempts a case address and sees that the house is vacant, but there is a for sale sign with a contact number on it, then the enumerator can call this number and conduct a proxy interview/attempt for this case address. The enumerator should only conduct a proxy interview/attempt with a nonneighbor if:   The enumerator is unable to conduct a proxy interview/attempt with a neighbor or a nonresident inside the case address. The enumerator has sufficient evidence that the nonneighbor has information on the house or the occupants of the house. Proxy Interview Once a proxy is selected, the enumerator attempts to conduct a proxy interview in person or by phone. In order to be considered a successful proxy interview, a sufficient amount of information must be documented. The device informs the enumerator when this occurs. A proxy interview/attempt could be unsuccessful for a variety of reasons, which is why it is extremely important that enumerators take detailed case notes. Unsuccessful attempt outcomes include:   Unavailable: If an enumerator attempts to contact a neighbor as a proxy and the neighbor is unavailable, this is considered a failed proxy attempt. A NOV form is not left in the case of proxy attempts. This proxy remains eligible for future contact. Don’t Know/Refuse to Answer: If an enumerator attempts to contact a proxy and that proxy respondent does not know the residents of the case address or is not willing to participate in a proxy interview, then this is considered a failed proxy attempt. The “Proxy Alerts” will notify future enumerators that this proxy is not eligible to be contacted again. 2.3.3.6 Additions to the NRFU Workload During the NRFU operation, new addresses are added to NRFU workload that were not in the workload during the initial cut. These include in-field and in-office adds; insufficient responses from paper returns (called reverse check-in); and cases in a supplemental file for new addresses since the initial enumeration universe was created in January. The supplemental file includes addresses from the following Census activities:   V2.0 New Construction in TEA 1. LUCA Appeals. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)    Page 30 Count Review. Delivery Sequence File (DSF) Refresh. Geocode Resolution. In-Field Adds There are rare cases where a NRFU enumerator needs to add a housing unit to the NRFU workload. An “in-field add” can occur when the assigned NRFU address has been subdivided into separate units with multiunit (MU) designations (unique mailing addresses). The enumerator records the original NRFU address as “unable to locate” since the address without a multiunit designation does not exist. This deletes the case. The enumerator then adds the individual units at the address to enumerate. Basement units or garage apartments without a unique mailing address are not considered unique housing units and should be included in the original housing unit enumeration. Similarly, if an enumerator becomes aware of a housing unit or a group of housing units that did not receive the census mailings (e.g., new house or new housing development), the enumerator can perform in-field adds for the missed housing units to ensure those households have the opportunity to be counted in the census. However, it is preferred that enumerator contact the office for in-office add procedures, which includes a check against the Master Address File to prevent duplicating addresses that are already in the universe. Another type of in-field add is for situations where the occupant did not live at the NRFU address on Census Day (an in-mover). If the occupant can provide proxy information about the previous occupant, it is recorded. In addition, the occupant is asked if they completed the census questionnaire at their previous address. If not, then an enumeration is completed for the respondent for their previous address. This case is treated as an add for field purposes but is not an addition to the universe. The former address provided by the respondent is sent to Non-ID Processing to match with the respondent’s Census Day address. In-Office Adds In previous Censuses it has been necessary to add cases to the NRFU universe in the middle of the operation when the Bureau becomes aware of a missed geographic location. When this occurs, the ACO staff first verifies that the addresses are not already included in the enumeration universe. If the addresses are not in the universe, the addresses are checked against the Master Address File to get a MAFID and geocode information. If the address can be geocoded, it is added to the NRFU workload. Insufficient Paper Self-Response V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 31 When a paper census questionnaire is returned, the bar code on the return envelope with the Census ID is scanned for check-in. All Census IDs that are checked-in are removed from the NRFU workload immediately to prevent a NRFU attempt at an address that has already responded to the census. Census questionnaires are then scanned for data entry. Depending on the number of returns received, the scanning can occur several days after the check-in. Each case then undergoes sufficiency validation to determine if there is adequate information about the household. If it is determined to be “insufficient” by the Census Bureau’s standards, the case is added back into the NRFU workload. This has historically been referred to as a “reverse check-in.” An example of this case would include if someone mailed the form in and left it blank. Initially it would receive a paper “check-in,” but once reviewed and deemed insufficient, it would then be added back into the NRFU workload. More information can be found about reverse check-in in the 2020 Census Detailed Operational Plan for: 10. Paper Data Capture. Reverse check-in addresses added back into the NRFU operation are enumerated following standard NRFU data collection procedures, as described above. New Construction in TEA 1 As part of its effort to continuously update its address files, the Census Bureau monitors construction permits in local governments. If newly built units are completed late in the census universe creation process, the new units are assigned to NRFU enumerators for enumeration. LUCA Appeals The Census Bureau relies on a complete and accurate address list to reach every living quarters and associated population for inclusion in the 2020 Census. The Local Update of Census Addresses (LUCA) is a voluntary decennial census geographic partnership operation where tribal, state, and local governments have the opportunity to review and update the Census Bureau’s residential address list. LUCA participants can add new or missing residential addresses, correct addresses, delete addresses, identify addresses not in their jurisdiction, and identify nonresidential addresses. Participants send their updates to the Census Bureau for review during the LUCA operation, which is scheduled to run from January 2017 to September 2019. By September 2019, participants can obtain feedback materials with the results of Address Canvassing. Participants can appeal LUCA feedback results through the LUCA Appeals office within 30 calendar days of receiving LUCA feedback materials. For each appeal, an Appeals Officer reviews the Census Bureau’s feedback materials, the written documentation, and supporting V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 32 evidence submitted by the eligible governmental unit, and considers the quality of the address reference source as the basis for determining the validity of an address (or group of addresses) and (their) location(s). According to the 2020 Census LUCA Integrated Master Schedule Baseline, LUCA Appeal Results are received from the Office of Management and Budget by January 13, 2020. The Master Address File/Topologically Integrated Geographic Encoding and Referencing (MAF/TIGER) System should be updated with LUCA Appeals Results by March 18, 2020. The Appeals Staff’s decision is final. Inclusion of an address on the list does not mean that a living quarters with that address exists or that the address is included in the final 2020 Census Enumeration Universe. LUCA Appeals that result in corrected or added addresses are the Census Bureau’s responsibility to enumerate, and thus, may be added to the production NRFU universe if a self-response was not received. More information can be found about LUCA Appeals in the 2020 Census Detailed Operational Plan (DOP) – 7. LUCA. Addresses added to the NRFU operation from the LUCA Appeals process are enumerated following standard NRFU data collection procedures, as described above. Count Review Similar to LUCA, Count Review is a separate activity in which the Census Bureau works with state governments to ensure a complete and accurate count. Each state reviews the enumeration universe created in January. If a state reports an issue, the Count Review team adds addresses to the Supplement File for inclusion in the enumeration. DSF Refresh The DSF, which may also be referred to as the Delivery Sequence File by the U.S. Postal Service (USPS), is a list of all addresses (and some related data) maintained by the USPS. The DSF, along with other data from USPS, is processed at the Census Bureau through a project known as “DSF Refresh.” The DSF refresh is the common name given to the series of MAF/TIGER System updates utilizing a number of files delivered to the Census Bureau from USPS. In addition to the address update portion of the DSF, Zone Improvement Plan (ZIP) codes on both MAF and TIGER features (address ranges) are updated. Most of the updates that occur from the DSF are very simple in nature. From a MAF update perspective, they are treated as “add actions” that arise external to the database. Historically the vast majority of these records have matched to existing MAF units, and the update consists of adding an entry to the MAF history for the existing address record. However, each semiannual DSF update cycle does result in the creation of approximately 500,000 new MAF units. As the DSF contains no geographic component below the county, the Census Bureau Geography and Decennial IT divisions run additional processing to attempt to geocode these V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 33 new records to census geography within the MAF/TIGER System, primarily utilizing the address ranges present on TIGER street features. If an address range representing the new address does not exist, it remains in the MAF as an ungeocoded record. If a DSF update is applied to an address, it generally is made to the preferred mailing address for that MAF unit. In summary, DSF Refresh is used as a primary source of address updates for the MAF/TIGER System, and thus, addresses may be added to the production NRFU universe from this operation. More information can be found about DSF Refresh in the 2020 Census DOP – 6. Geographic Programs Operation – Geographic Data Processing Component. Addresses added to the NRFU operation from the DSF Refresh process are enumerated following standard NRFU data collection procedures, as described above. Other Operations Adds In an undertaking as complex as Census 2020, there are myriad risks that can occur. For many operations, adding cases to NRFU is a contingency that can be utilized if the risk becomes an issue. Addresses added to the NRFU operation from other operations are enumerated following standard NRFU data collection procedures, as described above. 2.3.4 Special Enumerations In addition to enumeration of a household in the NRFU case universe, there are two other types of data collection that NRFU enumerators perform: field verification and manager visits. 2.3.4.1 Field Verification Cases that had a self-response without a Census ID and could not be matched to the census address frame (by either automated matching or clerical review processes) are included in the NRFU workload for Field Verification (FV). The purpose of this data collection is to verify by observation that the address in question actually exists in the field. It does not require an interview with a respondent. Non-ID procedures identify cases to be included in the FV universe. A field verification case consists of a case ID, the address to be found, and a map spot of the location to begin looking for the address. The enumerator has to start at the location of the pin indicator on the map and attempt to locate the address. The enumerator must select one of the following three options:  V2.0 Yes – The enumerator selects this option if he/she verifies the existence of the address; and, is at the location that accurately represents where the structure is July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)   Page 34 located on the ground, by standing in one of the following locations listed in order of preference: o The front door: If the house is located on more than one street and the location of the front door is not obvious, use the door that faces the street of its address. o Side or back door: Use the side or back door if the front door is not accessible or if using the side or back door is a locally accepted or customary convention in the community. o Garage door: Use the garage door if one exists and the front, side, or back door is not accessible. o Driveway, pathway, or stairways leading to the structure: Only use this option if the others are not accessible. Make certain that you are somewhere on the property and in view of the structure. No: The enumerator selects this option if they are unable to find the address. In the text box that displays when this option is selected, they enter any information about why they cannot find the address (e.g., no housing units in the vicinity or bad map spot). Other: This option is selected for any other result of the search. In the text box that displays when this option is selected the enumerator enters a reason, such as they found the address, but the housing unit was demolished or replaced by a business. A sample of FV cases are selected for an independent quality control check. This FV QC is performed by a different enumerator from the one who conducted the production FV case. The responses are paired and matched to determine falsification or a lack of following procedures. If the responses conflict and neither the production nor the QC enumerator collected valid GPS data, a third enumerator performs a final independent quality check in order to resolve the disparity. 2.3.4.2 Multiunit Housing As mentioned in Section 2.2.4.1, nonresponding HUs that belong to a MU are grouped together so that a manager visit (MV) can be conducted before attempting to enumerate the individual HUs. A MU is a classification of housing where multiple separate housing units for residential inhabitants or households are contained within one building or several buildings within one complex. A common example of a MU is an apartment building. When the initial NRFU universe is formed, the Operational Control System (OCS) groups together nonresponding housing units that have the same basic street address but different unit designations as an MU. The first step V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 35 in collecting information on the nonresponding HUs that belong to a MU is to conduct a MV. MUs that have fewer than twenty-five units, and/or MUs that have fewer than two nonresponding units, do not receive a MV, as the likelihood of an onsite manager is lower. During an MV interview, the enumerator asks the manager about the occupancy of the nonresponding HUs in the MU. Available outcomes include:    Occupied. Vacant: o For rent. o Rented, not occupied. o For sale only. o Sold, not occupied. o For seasonal, recreational, or occasional use. o For migrant workers. o Other. Other: o Does not exist. o Demolished/burned out. o Nonresidential. o Uninhabitable. o Missing unit designation. o Duplicate. o Other. If the MV is successful, and all units have been given a status, then the MV interview is complete. The units that are identified as “vacant” or “other” do not need to be enumerated and are removed from the NRFU workload. The “occupied” units are assigned individually either the same day or in the next day’s assignment. MVs should occur during business hours, which is often not a good time to find respondents at home. However, if the MV is completed after 4 p.m. on a weekday or any time on a weekend, at least a partial set of the HUs are immediately available for enumeration attempts. A MV can be unsuccessful for a few reasons:  V2.0 The unit does not have a manager: For these situations, there are no further manager attempts; the individual units are available immediately or for next day assignment, depending on the day and time that the MV is completed. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)      Page 36 Manager is not available: If the office is closed or there is no one available who can provide the information, the visit is unsuccessful. A second attempt to interview the manager will occur at a later date. Manager refuses to provide the information: There is no further attempt with the manager, and the individual units are available immediately or for next day assignment, depending on the day and time that the MV is completed. Manager asks the enumerator to return at a less busy time: The enumerator will make an appointment for a return visit. Manager provides information on only some of the units: In this situation, where not all the cases are statused, the next question asks for a reason why the MV was not completed. If the manager is too busy but will complete the task at another time, the MV interview will be continued at a later determined time. Cases marked as occupied are released for assignment the next day. Cases without a resolution will remain grouped into the MU case for the return visit. If the break-off is the result of the manager refusing to provide any more information, then the individual units are available immediately or for next day assignment, depending on the day and time that the MV is completed. MU Manager and the enumerator have a language barrier and cannot complete the case. The MU case can be reassigned to an appropriate enumerator to complete. A follow-up attempt may be conducted by phone. This might occur at the request of the manager or because there is an offsite manager. After two unsuccessful manager visits, no more attempts are made, and the individual units are available immediately or for next day assignment, depending on the day and time that the MV is attempted. Case Notes from the MV are transferred to the individual units in the MU. Therefore, if the manager provides a code to enter the building, the information is available to any enumerator assigned to a unit in the MU. MVs are subject to reinterview. A sample of MU MVs are selected for reinterview. The reinterview consists of a confirmation of the MV and a verification of a portion of the units identified by the manager during the MV. For suspected falsification, there is a rework of the MV to ensure that units were not erroneously classified as vacant or delete. 2.3.4.3 Early NRFU Early NRFU occurs in areas where there are high concentrations of college students living in offcampus housing who are unlikely to be present during NRFU that starts in mid-May. During the 2020 Census, Early NRFU will start on April 9. Any Early NRFU addresses that are unresolved V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 37 and have not reached maximum attempts by May 13 will receive additional field attempts during regular NRFU. With the exception of the use of AdRec modeling and phased approach, the same NRFU contact strategies and business rules apply to the Early NRFU universe as to the regular NRFU universe. The MOJO optimizer will assign cases according to the methodology used in Phase 1 of the NRFU contact strategy for all contact attempts, not just the first four. At the time of the Early NRFU component, there are not enough AdRec data available to make a determination about the housing unit status. Instead, only best-time-to-contact probabilities can be generated. Thus, contact attempts cannot be reduced as they are during regular NRFU. Aside from these exceptions, the business rules and strategies of both components are closely aligned. For this reason, training procedures for field staff who participate in Early NRFU are the same as the standard NRFU training procedures. However, field staff for Early NRFU are recruited and trained prior to the start of recruiting and training for standard NRFU. There are quality control reinterviews on a sample of cases selected for Early NRFU. 2.3.4.4 SRQA Recollect As a check on the quality of the self-response data collection, a portion of the Internet SelfResponse (ISR) completions are selected for Self-Response Quality Assurance (SRQA) recollect by field enumerators. These cases are completed using the NRFU RI instrument and procedures. For example, a notice of visit is not left since the HU has already completed the ISR. Cases have a maximum of six attempt days and are not be reopened in the Closeout Phase. SRQA recollect begins during Early NRFU. 2.3.5 Quality Control To ensure the highest quality data, NRFU includes a variety of quality control measures. These measures include:     V2.0 Evaluation after initial training. Operational Control Alerts that are triggered by the system and sent to the Census Field Supervisor (CFS) to resolve with the enumerator. Quality Control reinterviews of a sample of cases to ensure the enumerators followed proper procedures and that results were not falsified by the enumerators. Coverage Improvement followup of cases that have been identified as having a coverage issue such as: count discrepancy or an affirmative response to the undercount or overcount questions. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 38 2.3.5.1 Evaluation After Initial Training As described in Section 2.5.3, training is one of the most vital parts of the entire NRFU operation. If training is not effective, then it could affect the success of the NRFU operation. Ensuring that the material learned during the initial training of enumerators and Census Field Supervisors (CFSs) is understood and retained is critical. At the end of training, enumerators and CFSs are evaluated using automated software that replicates various situations and prompts them for the correct action to take. Not only does this post-training knowledge assessment reiterate what the enumerators and CFSs have already learned, but also more training can be provided for anyone who did not retain or understand the training. For example, the enumerators who score less than 70 percent on this assessment receive additional on-the-job training led by their CFS. 2.3.5.2 Operational Control Alerts Evaluation of the enumerators continues throughout the entire NRFU operation. During the operation, the Enterprise Census and Survey Enabling Operations Control System (ECaSE OCS) monitors field activities and has the ability to track and report the enumerators’ performance and progress. The CFSs are able to monitor enumerators using ECaSE OCS through a series of performance alerts. Enumerators whose results are less successful, such as too many refusals or too many partial interviews, receive more training or have a CFS accompany them for part of their shift. Enumerators suspected of intentionally violating procedures or falsifying data can be immediately terminated, and their previously submitted cases are reworked to ensure accurate data collection. 2.3.5.3 Quality Control Reinterviews One of the most important parts of Quality Control is conducting reinterviews of completed enumerator interviews. By conducting a reinterview (RI), the Census Bureau can check the quality of the work done by enumerators throughout the entire NRFU operation. A sample of NRFU cases are selected to verify that the enumerator conducted the interview and followed procedures. For all selected cases, the original interview and the RI data are compared through computer and clerical matching. This comparison of data is facilitated by the Sampling, Matching, Review, and Coding System (SMaRCS), which provides QC sampling, computer matching, clerical matching, and final coding of matching outcomes. Cases that do not pass the computer matching are deferred to the National Processing Center (NPC) for clerical matching. Using the SMaRCS application, NPC staff compares the original interview and RI data for any V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 39 discrepancies. They can conduct further investigations by reviewing third party data about the household, as well as make Supplemental RI selections when more data are necessary. If deliberate falsification is detected, the regional census centers (RCCs) staff perform an adjudicative review and are the final step in the resolution process. Cases that the RCC staff determine to have been deliberately falsified receive a Hard Fail outcome code. Enumerators receiving Hard Fail outcome codes are removed from further data collection activities. On a daily flow, as enumerators complete NRFU interviews in the field, the data are sent to the Production Environment for Administrative Records Staging, Integration, and Storage (PEARSIS) system for a matching of the roster names collected at each HU to AdRec. This matching of names is used by SMaRCS to determine a level of confidence that the case is not falsified. If the level of confidence is high enough, then no reinterview is needed, and SMaRCS flags the case as not eligible for RI selection. An enumerator interview completed during NRFU production may be selected for RI for one of four reasons: Analytic: Based on statistical calculations, enumerators whose work differs significantly from other enumerators are flagged as outliers. Cases completed by these enumerators are chosen so that an analytic RI can be used to further investigate these enumerators to determine if they are following proper enumeration procedures. Random: Random RI involves reinterviewing a random sample of the eligible cases completed by every enumerator. Supplemental: Supplemental RI allows the National Processing Center (NPC) staff to select additional cases for RI for any enumerator at any time during NRFU, if they suspect an enumerator may not be following procedures. This can be done through manual selection, where the user selects a specific case for supplemental RI, or future selection, where the user selects an enumerator and the next two cases checked in for that enumerator are selected for supplemental RI. Rework: When an enumerator receives a hard fail outcome for a case in random, supplemental, or analytic RI, all of that enumerator’s checked-in eligible cases are placed in Rework RI. Rework RI is conducted by enumerators to ensure that a hard fail enumerator has not falsified data on any of their other work. The rework collection is used for the enumeration for those housing units. In addition, all cases assigned to the hard fail enumerator that have not yet been completed are reassigned to another V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 40 enumerator and assignment of any new cases to the hard fail enumerator are prevented. Once a case is selected for RI, it is assigned to an enumerator who did not complete the original interview. RI cases are clearly labeled so that an enumerator knows when they select the case that it is not a typical production case. Once the enumerator makes contact with the respondent, they specifically ask for the original respondent. For nonproxy cases (i.e., cases where the original NRFU interview was not conducted with a proxy respondent), if the original respondent is unavailable but the respondent knows about the people in the household, then the respondent is eligible to be reinterviewed. For proxy cases, only the original proxy respondent should be reinterviewed. During a RI, if the respondent confirms the original interview took place, the enumerator collects the population count as well as the roster names and then ends the interview. However, if the respondent cannot verify that the original interview took place, then the enumerator conducts a full NRFU interview with the respondent. Once the RI is completed, the original interview and the data collected from the RI are sent to SMaRCS and compared through computer and clerical matching. The final matching outcome codes are:    PASS – The original data are verified with no suspicion of falsification. FAIL – Review of the interview indicates a problem with the data and the original data cannot be verified, or the review shows a violation of procedures by the enumerator, but there is no sign of deliberate falsification or intentional violation of procedures. HARD FAIL – There is sufficient evidence to show there is deliberate falsification or intentional violation of procedures by the enumerator. In addition to setting Hard Fail outcome codes, the RCC staff may assign a Non-RI Fail or PFAIL code to an enumerator. Non-RI Fail codes are assigned to enumerators suspected of falsification for reason other than RI results (e.g., falsification outside of the RI process). PFAIL codes are assigned to enumerators that show a continuous pattern of violation of procedures or unverifiable data. When an enumerator receives two or more FAIL codes, SMaRCS notifies the field staff to allow corrective measures to be taken. If the enumerator continues to receive FAIL codes after having received guidance from the field staff, then the RCC can issue a PFAIL code to the enumerator upon review of the enumerator’s body of work. Both the Non-RI Fail and PFAIL codes mandate that the enumerator is removed from further data collection activities and follow the same procedure as a Hard Fail outcome. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 41 2.3.5.4 Coverage Improvement (CI) Followup Coverage Issue Indicators The universe of cases eligible for CI follow-up consists of housing unit responses that are identified as having potential coverage issues. Group Quarters and Transitory Location cases are excluded from this universe, though they may be affected by the coverage improvement methods. CI identifies potential coverage issues by looking only at respondent-provided data. No auxiliary data, such as administrative records, are used. To create the initial universe of eligible cases, CI identifies cases that have any of the following three coverage issues:    Count Discrepancies: The respondent-provided population count is different from the number of people for whom data are provided; may be classified as a high or low type of count discrepancy, depending on which value is higher. Affirmative Response to the Undercount Question: The undercount question is a household-level question that asks respondents if they forgot to include certain types of commonly overlooked people in the population count. Commonly overlooked people include babies or children, relatives, and unrelated roommates. Affirmative Response to the Overcount Question: The respondent is asked if anyone on the roster usually lives or stays at any of the types of places listed. For each person selected, the respondent is asked to select from a list of the types of places where that person usually lives or stays. Methods to Resolve Coverage Issues Depending on the eligibility of the cases, the methods to resolve potential coverage issues are listed below:     Conducting a coverage improvement telephone interview. Searching for duplicate enumerations and using census residence criteria to resolve cases. Applying a business rule. Accepting the response as is. Conducting a Coverage Improvement Telephone Interview The Coverage Improvement telephone interview operational activity consists of outbound, computer-assisted telephone interviews conducted by Census Questionnaire Assistance operation (CQA) customer service representatives (CSRs). The CSRs attempt to contact the V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 42 respondent from the original census return and collect information using a version of the ISR instrument that is modified for interviewer-assisted data collection. During the call, the CSRs:      Verify the roster provided by the respondent on the original census return. Ask the undercount question to determine who may have been missed. Ask the overcount question for each person to determine who lived elsewhere on Census Day. Ask for the address of an alternate residence and a specific person’s location on Census Day if the respondent answered the overcount question in the affirmative. CSRs also ask for confirmation that the alternate location is where that person usually lives or stays. Collect demographic information for people added to the roster. After the Coverage Improvement telephone interview is completed, additional processing is needed. For cases that include an address of another location where a person usually lives or stays, CI needs to have the additional address geocoded so that when the census residence criteria are applied, CI can determine where the person should be counted. That is, if a person is to be removed from the enumerated address, CI is able to place them in a verified location. Search for Duplicate Enumerations There are cases that indicate a person usually lives or stays elsewhere but that CI is not able to contact because of ineligibility or nonresponse during the Coverage Improvement telephone interview. In addition, there are cases that provide an address during the Coverage Improvement telephone interview that could not be successfully geocoded, as well as cases that remain unresolved. To account for these types of cases, CI plans to conduct a nationwide, probability-based matching activity among census person records to identify possible duplicate enumerations, similar to the methods used in the 2010 Census. If a match is found, CI uses the response to the overcount question on each record and the census residence criteria to determine where the person should be counted. CI would remove the person from the census return of the other address. Note that during the 2020 Census, CI conducts a nationwide search. Apply a Business Rule Some cases identified as having a coverage issue are not eligible for the Coverage Improvement telephone interview. For cases where a count discrepancy remains unresolved, CI applies an algorithm similar to that used in the 2010 Census. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 43 Accept Response For cases that are not eligible for the coverage improvement methods above, or that remain unresolved, CI accepts the response as is. No further action is taken. 2.3.6 Closeout The NRFU closeout phase is mentioned in Section 2.3.1, Contact Strategy. Closeout includes the following activities:    Handling remaining unresolved cases, including identifying cases that need continued follow-up, evaluating closeout contact strategies to close these cases on schedule, and closing out difficult cases. Reducing staff as workload reduces, including collecting materials such as devices. Performing operational assessments such as cost and progress reporting, lessons learned, and data analysis. NRFU Operational Timeline NRFU runs from April 9, 2020, to July 31, 2020. Table 3 and the corresponding data below detail those activities and their timelines. Table 3: NRFU Key Activities and Dates Key Activities Date Census Day April 1, 2020 Coverage Improvement Production Activities Begin April 3, 2020 Early NRFU Begins April 9, 2020 Early NRFU RI Begins April 10, 2020 NRFU Universe Created May 4, 2020 AdRec Modeling (Round 1) May 9, 2020 NRFU Production Begins May 13, 2020 NRFU RI Begins May 14, 2020 AdRec Modeling (Round 2) June 1, 2020 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 44 Key Activities Date NRFU Data Collection Ends July 24, 2020 NRFU RI Data Collection Ends July 31, 2020 Coverage Improvement Production Activities End July 31, 2020 Field Operations This section provides a top-level picture of how the 2020 Census field operations work and provides the new terms and constructs needed to support the 2020 Census operational design. 2.5.1 Reengineering Fieldwork Figure 4 depicts the elements that interact at the top level to implement reengineered 2020 Census fieldwork. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 45 Figure 4: Reengineering the Fieldwork Inside the reengineered fieldwork structure, systems drive the business and provide oversight to managers at all levels to take rapid action. Most important, a robust and modern OCS handles many tasks and makes many decisions traditionally made by individuals. Daily, optimal, and sequenced contact attempt assignments are provided to the enumerators from the enhanced OCS. These assignments exist on a smartphone device that the enumerators can access to guide them through their data collection activities during the workday. The enumerators use the device to collect information about the contact attempts and capture enumeration interview responses, as well as to submit payroll expenses and to validate their future work availability. Field access to enumerator workload and status information within the OCS is provided to the CFS through a tablet device. The CFSs work from their home in shifts to monitor progress and performance of their assigned enumerators and provide support to other enumerators who are working during their shifts. The tablet device also is used to approve enumerator time sheets V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 46 and capture enumerator interaction data (e.g., enumerator John Doe called to report his car broke down and he is not working today). Workload progress within the OCS is monitored by the Census Field Managers (CFMs) through an operational control center within the area census office (ACO). The CFMs use dashboard summaries and key performance indicators to understand areas of concern and shift resources as needed. CFMs use OCS-provided information and CFS-provided local knowledge to make decisions on when to onboard/offboard enumerators/CFSs as fieldwork progresses. Their work in conjunction with other CFMs in their assigned areas ensures completion of all attempts for their entire area. The area census office manager (ACOM), who is ultimately responsible for the completion of workload within the geographic boundary of the ACO, also uses the OCS to monitor overall workload progress and performance of the CFSs. Automated and classroom training are used to prepare all staff for their roles in the operation. To limit classroom time but still provide ready-to-work enumerators, the Census Bureau uses an electronic means for staff to complete independent study modules before a face-to-face meeting to receive needed materials and ensure readiness for the field (e.g., mock interviews, in-person testing). 2.5.2 Reengineering the Field Staff Structure Eliminating paper for data collection and payroll processing and introducing automated enumerator assignments has created different management duties from those of previous censuses. Figure 5 below depicts the reengineered field staff for 2020 Census fieldwork outlined above. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 47 Figure 5: Reengineering the Field Staff Structure An ACO covers a larger geographic area and has more staff than past censuses’ local field offices. Not all ACOs are the same size. In addition, an ACO is dependent on workload and geography. A professional-level ACOM manages an ACO, supervises CFMs, and ensures completion of attempt workload in the time expected for the assigned area. The ACOM is responsible for managing multiple functional managers (including the CFMs and possibly other managers for administration, recruiting, etc.) and the ACO staff. The ACO staff consists of ACO clerks and multiple office operations supervisors (OOSs), who supplement and support the various functional managers. There is an ACO Manager, an Assistant Manager for Administration, an Assistant Manager for Recruiting, an Assistant Manager for IT, and between six and eleven census field managers at each ACO. Multiple CFMs work within the office to manage operational performance (attempt workload for completion) and supervise CFSs assigned directly to them. They are also responsible for monitoring workload progress within their assigned CFM zone, which is a geographic subset within the boundaries of the ACO area. Although each CFM has a primary geography, they work V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 48 as a team to ensure that all area work is completed. The CFMs are supported by designated OOSs and ACO clerks. CFSs supervise and manage enumerators in the field and help to train new staff. CFSs are hired to work a specific set of hours (shift) each day and are responsible for working locally in the field to support any enumerators working during their shift. This “shift” concept allows for better management of CFS hours and provides continual supervisory coverage for enumerators during work hours without requiring a given supervisor to be available all the time. In addition, CFSs serve as primary supervisors for a group of enumerators for administrative purposes. Those administrative purposes include such tasks as validating payroll. CFMs support CFSs with the initial one-day in-class training for their enumerators. Table 4 shows the relationships of these reengineered field staff roles to the reengineered office structure and field geography. Table 4: Reengineered Office Structure, Field Geography, and Staff Role Decomposition Summary Geographic Span National Work Location/ Role Estimated Number of Staff Managed (by level per manager) Census Bureau headquarters N/A Regional census center (RCC) 1 or more RCC per region. ACO has ACO with multiple CFMs. 248 ACOs to cover the US. 1 ACOM per ACO. CFM works in the ACO and manages a CFM zone within the ACO area. 3 to 5 CFMs per ACO. RCC Area Exists within census field region boundaries. Contains a given number of ACO areas. ACO Area ACO boundaries divide the country into nonoverlapping ACO areas within regions. CFM Zone One per CFM based on workload density with core coverage areas (primary V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Geographic Span Page 49 Work Location/ Role Estimated Number of Staff Managed (by level per manager) responsibility) and possible overlap of daily workload areas with other CFM zones for optimization. CFM Zone CFS works in the field. Manages assigned enumerators and supports nonassigned enumerators working during their shift. 20 CFSs per CFM. CFM Zone Enumerator works in the field. Does cluster of daily workload optimally assigned based on factors. Approximately 15 enumerators per CFS. Note: This table contains the preliminary estimates for the number of staff managed per level. These estimates will be subject to refinement during the operational modeling and testing process. 2.5.3 Training After applicants are selected and they have completed the appropriate documentation, they have their fingerprints taken and are sworn in as Census Bureau employees. Enumerators receive their mobile devices and instructions for online training. Training for field staff includes both online training and classroom training. The enumerators take 8 hours of online training and then have one day of classroom training. During the classroom training, they participate in role-playing exercises to help them become more comfortable with interviewing. After classroom training, they take a post-classroom online Enumerator Challenge and training evaluation. The CFSs take all of the enumerator training plus an additional three online modules, and an additional two days of classroom training, which includes a two-hour facilitation session. The facilitation session provides guidance on how CFSs are to conduct enumerator classroom training. The additional training covers the OCS system and their supervisory responsibilities. CFMs conduct the classroom training for their CFSs, with V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 50 the exception of the facilitation training, which is conducted by a headquarters employee. The CFSs in turn conduct training for the enumerators. The OOS and a pool of clerks support the preparation and execution of training. Besides conducting NRFU, preparing for training is one of the largest activities the CFMs complete. Where logistically possible, CFS training is held at the ACO. Enumerator trainings, and all other CFS trainings, are held mostly offsite, preferably in areas close to where the enumerators live. To train offsite, the CFMs obtain suitable free space. Recruiting assistants help with locating space, as locations where they have held recruiting testing sessions are to be used. All training sites are required to have adequate cellular connectivity, because it is required in order to initialize the enumerator’s smartphones during training. Online training is part of an effort to make the census more cost efficient by utilizing modern technology and training methods. All CFSs and enumerators must complete online training before attending classroom training. Clerks in the administrative area access the completion information in the Learning Management System (LMS) and transfer it into the Decennial Applicant, Personnel, and Payroll System (DAPPS), which enable them to enroll the enumerators in classroom training. Their responsibility belongs to the Assistant Manager for Administration’s (AMA) area, but CFMs also monitor the completion reports. The CFMs conduct three days of classroom training for the CFSs, which includes the one-day enumerator training and two additional days of CFS training. 2.5.4 Devices and the Instrument Data collection for NRFU is conducted on mobile devices. The Census Bureau has a contract to rent devices that are configured specifically for NRFU. The devices have all the software necessary for the enumerator to conduct the census, record their time and expenses, contact their supervisor, and perform any other work needs. The census enumeration program includes a Mobile Case Management (MCM) system that provides the enumerator their case assignment for the day in the order that they should work the cases. This order is created by an optimizer that takes into account the number of attempts the case has received, the time of day to maximize response, travel time and mileage from case to case, and the hours the enumerator is available for that day. The data collection instrument guides the enumerator through the data collection process, whatever the possible outcome. It is designed to capture every outcome that might occur in the field. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 51 2.5.5 CFS Instructor Kits and Enumerator Kits The Field Division provides the other materials needed in kits. Each CFS receives a NRFU CFS Supply Kit and a NRFU CFS Instructor Kit. Each CFS and enumerator receives a NRFU Enumerator Supply Kit. The NRFU Enumerator Supply Kit includes NOVs that are handed out to nonresponding units. The contents of the kit are listed in Appendix D. In addition to the kits, each enumerator and CFS receive a participant manual for their training class. The CFSs also receive an Enumerator Instructor Manual for teaching their enumerator sessions. 2.5.6 A Day in the Life of an Enumerator Figure 6 depicts the elements that interact during a typical day in the life of an enumerator. Enumerators work in the field with daily assignments, depending on the day’s casework and the enumerator’s skills/availability. The enumerator’s key responsibilities include making contact attempts at nonresponding housing units (HUs) and, if contacts are successful, collecting interviews using an automated data collection instrument. Figure 6: A Day in the Life of an Enumerator V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 52 On a daily basis, enumerators use the OCS on a mobile device to receive their caseload, conduct their work, enter payroll information, and indicate times when they are available to work. At the end of each day, they enter or verify the times they are available to work for the upcoming five days. In addition, they enter/verify their payroll information, including hours worked, mileage, and other expenses incurred during their shift. Each night, the control system generates an optimized, sequenced assignment for each enumerator who said they could work the next day. At the beginning of their shift, the enumerator receives their optimized caseload. The enumerator proceeds through the list in sequence conducting the different types of NRFU interviews described in Section 2.3. Performance indicators for enumerators include quality measures and a work score that is compared with other enumerators in the area (a work score is comprised of reliability of work assigned, availability, attempts completed, and interviews completed). Enumerators have a primary CFS who acts as the line manager for Human Resources (HR) purposes. If the enumerator happens to work hours when their primary CFS is not on duty, they are able to contact another CFS, should they need help, through use of a support line manned during daily operational hours by other on-duty CFSs. 2.5.7 A Day in the Life of a Census Field Supervisor (CFS) Figure 7 depicts the elements that interact during a typical day in the life of a CFS. The setting, key characteristics, and a representative scenario for this role are outlined below. Figure 7: A Day in the Life of a CFS V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 53 The CFSs work from their homes, and their work schedule is an assigned daily work shift (e.g., daytime shift, afternoon-evening shift). Primary CFS shift times correspond to the main availability time for their assigned enumerators. CFSs are assigned to supervise teams of approximately 15 enumerators. CFSs’ key responsibilities are working designated shifts to support active enumerators and coach assigned enumerators. In addition to managing and monitoring enumerators, CFSs monitor enumerator performance alerts, a variety of reports and information about their enumerators, other enumerators, and CFSs, and validate and approve payroll. CFSs conduct the enumerator training for their assigned enumerators and review and monitor performance during training and throughout the operation. If enumerator training or operational performance indicates the potential need for additional assistance, CFSs may conduct observations, where they go with the enumerator and monitor performance. In rare cases, CFSs may be asked to conduct observations for enumerators assigned to other CFSs. CFSs communicate with other CFSs regarding interactions or issues with enumerators through the operational control system or supervisor support line. When performance or conduct warrants it, CFSs discipline or terminate enumerators, after discussion with the CFM, and may be expected to perform other Human Resources-related supervisory activities for their assigned enumerators. They also interact with CFMs to help manage the ongoing field data collection activities and staffing needs. 2.5.8 A Day in the Life of a Census Field Manager (CFM) Figure 8 depicts the elements that interact during a typical day in the life of a CFM. The setting, key characteristics, and a representative scenario for this role are outlined below. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 54 Figure 8: A Day in the Life of a CFM CFMs work at the ACO. They work an assigned daily work shift (e.g., daytime shift, afternoonevening shift), with potential for after-hours on-call work depending on need. The key responsibilities of the CFMs are working designated shifts to manage and support ongoing field operations work. They interact with designated CFSs to help manage the ongoing field data collection activities and staffing needs, and coordinate activities and progress with the ACOM at the ACO. CFMs use approved procedures and tools to hire, train, and supervise CFSs. They ensure adequate staffing levels and completion of work for the geographic area assigned (CFM zone) and assist other CFMs as needed to ensure overall success of the office and the geography it covers. They sign termination actions, support other administrative activities related to the fieldwork as needed (e.g., overtime approvals, observation requests), and handle any “realtime” case assignment needs that might occur. High-Level Operational Design While the section above describes the individual components of the NRFU operation, the remainder of this section and Section 3 cover the details of NRFU’s management life cycle. The design of NRFU for the 2020 Census includes five major operational activity areas: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)      Page 55 NRFU Planning and Preparation. NRFU Operational Workload Management. NRFU Data Collection. NRFU Quality Control Adjudication. NRFU Operation Closeout. Each of these major activity areas is summarized below. Together, these activities represent the complete set of work that needs to be performed to conduct this operation. NRFU Planning and Preparation NRFU conducts the planning and preparation needed for its 2020 Census operation. Informed by the analysis of the results from census site tests, NRFU develops effective administrative record modeling requirements, contact strategies and corresponding business rules, and the quality control (QC) plan for the 2020 Census. Based, in part, on these contact strategies and business rules, NRFU systems requirements are developed and approved. NRFU also identifies locations requiring early NRFU activities. NRFU determines staff needs and performance metrics for all NRFU field activities and, working with the Field Infrastructure operation (FLDI), prepares for and delivers NRFU-specific training. NRFU Operational Workload Management The initial universe consists of nonresponding addresses and addresses from other operations requiring fieldwork. Administrative records modeling and contact strategies then are applied with business rules to set parameters for each address, such as the maximum number of contact attempts. Updates to the universe include reinterviews for QC, the removal of selfresponses, removal of cases marked “vacant” or “delete” from administrative records modeling, reduction of contact attempts for cases marked “occupied” from administrative records modeling, added HUs from the field, and additions from the supplemental NRFU universe. ECaSE OCS will work in conjunction with the Assignment Optimizer to create work assignments based on this universe, optimizing the assignments to enumerators, and tracking case parameters such as the count of contact attempts. NRFU monitors progress based on the performance metrics set as part of NRFU preparation and resolves issues that affect the processing of the operational workload. NRFU Data Collection Based on their daily work assignments, enumerators attempt to collect data for specified addresses in the field. This data collection includes enumerating households, performing MVs, and conducting reinterviews and field verification. Enumerators attempt to contact a household V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 56 member, or if appropriate a proxy, to collect response information. The attempt is closed out with the recording of any case notes and the classification of the attempt to determine future action. NRFU Quality Control Adjudication For quality control of its data collection, NRFU conducts reinterviews for a sample of production cases as well as those determined to be potential falsifications. The identification of potential falsification occurs during the AdRec modeling that is performed for potential NRFU QC cases. NRFU Clerical Resolution Staff research any cases identified by computer matching as having anomalies between the reinterview and the original case. This staff recommend if further research is required by field staff to confirm falsification. When falsification is confirmed, the enumerator’s current cases and future assignments are stopped, and based on business rules, the enumerator’s past cases may be selected for rework. NRFU Operation Closeout When its field operation nears completion, NRFU begins closeout activities. As the number of cases decreases, NRFU works with FLDI to reduce staffing levels to meet the current caseload. Closeout includes an operational assessment and an analysis of the data and paradata collected, as well as lessons learned to be applied to the 2030 Census. The full hierarchy of activities for NRFU is provided in Appendix C in the form of an Activity Tree. In the Activity Tree, each major operational activity area listed above is numbered and then decomposed into a numbered set of subactivities, some of which are further decomposed into more detailed numbered subactivities or steps. For a full description of the operational subactivities that comprise NRFU, see the Detailed Process Description discussions in Section 3. NRFU Operational Context The NRFU operational activities described above are conducted within the context of other 2020 Census operations and other programs or data sources that are external to the 2020 Census Program. One way to depict an operational context is by using a “Context Diagram,” which shows the boundary of the operational process, the operational activities it contains, and the information exchanged with its neighbor operations (or other entities) as well as the resources (mechanisms) needed to conduct the operational work. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 57 Figure 9 is a top-level context diagram for NRFU represented as an Integrated Definition, Level 0 (IDEF0) model. An IDEF0 Model of a process (or operation) shows the Inputs, Controls, Outputs, and Mechanisms of the process. These IDEF0 model elements are summarized below and described further in the sections that follow. The yellow box in the center of the IDEF0 model lists the major operational activity areas for the operation, numbered as given in the NRFU Operation Activity Tree in Appendix C. Specific Information Exchanges (IE) are shown in different colored boxes to represent the Inputs (green boxes on left side), Outputs (orange boxes on right side), Controls (purple boxes on top), and Mechanisms (blue boxes on the bottom). Boxes to the left of the Inputs indicate the Provider of the inputs to the operation (typically another 2020 Census operation or an external source). The Provider of the Controls is noted in the box itself. Boxes to the right of the Outputs indicate the Receiver of the outputs (typically another 2020 Census operation or external entity). Each Information Exchange has a name and a unique number for identification purposes. For detailed descriptions of the Inputs, Controls, Outputs, and Mechanisms used by NRFU, see the sections that follow. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 58 Controls: Data to Guide Operation (1. PM) Program Controls: incl. Budget, Operational Plans and Schedule (3. SPC) Security, Privacy, and Confidentiality Controls Inputs: Data Consumed by Operation 32. FLDI • IE241: Draft and Final Operation-Specific Training Materials (for NRFU) 19. RPO • IE112: NRFU Case Universe and Updates (including AdRec Modeling Results) • IE119: QC Computer Match Results • IE120: Enumeration Hold Notification 6. GEOP • IE066: Spatial Data (for assignment) • IE067: Geographic Data Products (for assignment) 31. DSC • IE360: IT Service Outcome Notification • IE577: Service Event Notification Respondent • IE183: Response Data • IE242: NRFU Contact Information Staff: HQ Staff, RCC Staff, ACO Staff, Field Staff, NPC Staff (32. FLDI) IE367: Online and Classroom Training Schedules (4. CFD) IE226: Content Specifications for Automated Instruments (5. LNG) IE206: 2020 Census Language Support Plan Outputs: Data Produced by Operation 18. Nonresponse Followup Operation (NRFU) 18-1. NRFU Planning and Preparation 18-2. NRFU Operational Workload Management 18-3. NRFU Data Collection 18-4. NRFU Quality Control Adjudication 18-5. NRFU Operation Closeout • IE250: Estimated Operational Workload • IE251: Operation-Specific Training Content and Requirements • IE252: Approved Operation-Specific Training Materials • IE615: Time and Expense Records • IE616: Administrative Actions 32. FLDI • IE391: Operation-Specific Kit Requirements 33. DLM • IE209: NRFU Field Support Materials Content • IE049: Geographic Products Requirements • 6. GEOP IE254: Self-Response Case IE Hold Flagging Rules IE255: AdRec Modeling Requirements IE256: Contact Attempt Stopping Rules IE257: QC Plan (for NRFU) IE258: Areas for Early NRFU IE104: Response and Status Data for Cases • IE259: Reinterview Anomaly Outcome • IE131: Address Data Updates from Field Ops • • • • • • IE260: Request for Response • IE261: Notice of Visit 31. DSC • IE404: NRFU Paradata Systems: ECaSE-ENUM, ECaSE-OCS, ECaSE-FOCS, SMaRCS, MaCS, MOJO: BLQ, MOJO: Optimizer; MOJO: HERMES, UTS, LMS 1. PM Other: Census Networks, Mobile Networks, Mobile Devices Mechanisms: Resources Employed by Operation 19. RPO HU or Respondent • IE380: IT Service Requests Sites: HQ, RCC, ACO, Field Area, NPC, Training Venues 4. CFD Other: HQ Office IT Infrastructure, Operation-Specific Kits Version v07f – July 11, 2019 Figure 9: Nonresponse Followup Operation (NRFU) Context Diagram V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 59 2.7.1 NRFU Operational Inputs Inputs are the data that are consumed by the operation. The inputs define the amount of operational work that needs to be performed. Table 5 lists the inputs to NRFU. Table 5: NRFU Operational Inputs Provider Information Exchange Description 32. Field Infrastructure operation (FLDI) IE241: Draft and Final Operation-Specific Training Materials (for NRFU) All materials needed to conduct the online and classroom training for NRFUspecific operational activities. This includes course content, instructor materials, and student handouts. 19. Response Processing Operation (RPO) IE112: NRFU Case Universe and Updates (incl. AdRec Modeling Results) The set of NRFU cases to be enumerated and the parameters associated with those cases (e.g., mode of collection, Administrative Record modeling results, maximum number of contacts) The initial NRFU case universe is provided when NRFU begins. Updates to the universe are sent on a regular basis to reflect changes in case status (e.g., self-response and address changes). The NRFU case universe also includes cases selected for QC reinterview, SRQA recollect and field verification cases. Administrative record modeling results are used to remove cases from the NRFU workload as described in detail in Section 2.3.2. Administrative record modeling is also used to provide the best time of day to contact a particular housing unit. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 60 Provider Information Exchange Description 19. Response Processing Operation (RPO) IE119: QC Computer Match Results Differences found by SMaRCS during an automated comparison of response data collected for reinterview cases against response data collected for production cases. 19. Response Processing Operation (RPO) IE120: Enumeration Hold Notification Notification that a self-response has been received or is in the mail and therefore should be removed from the NRFU workload (i.e., the field staff’s daily assignments). This notification is forwarded to the NRFU field operational control activity as soon as RPO receives it. If the self-response is found to be insufficient, it is added back into the follow-up workload at that time. 6. Geographic Programs operation (GEOP) IE066: Spatial Data (for Assignment) Information regarding Basic Collection Unit (BCU) boundaries that is used for field staff assignments. 6. Geographic Programs operation (GEOP) IE067: Geographic Data Products (for Assignment) Information about the number of addresses in BCUs. Used during planning to determine the number of materials needed to support the operation. 31. Decennial Service Center operation (DSC) IE360: IT Service Outcome Notification Notifications and status updates of IT service results provided to the users (requesters). 31. Decennial Service Center operation (DSC) IE577: Service Event Notification Notifications to all users of a service event such as routine operations updates/changes, system problem/incident status, or privacy/security incident status. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Provider Page 61 Information Exchange Description Typically, these are provided as part of the standard message users receive when they call in for help. Respondent IE183: Response Data Data provided by the respondent in response to census questions. For NRFU, the respondent may be a household member or a proxy. Respondent IE242: NRFU Contact Information Contact information provided by respondents for follow-up. For NRFU, the respondent may be a household member or a proxy. 2.7.2 NRFU Operational Controls Controls are the data that guide the behavior of the operation. They are not consumed by the operation, but rather they provide guidance, models, limits, criteria, cutoff dates, or other information that controls the way in which the operational work is performed. Table 6 lists the controls for NRFU. Table 6: NRFU Operational Controls Provider Information Exchange Description 1. Program Management operation (PM) Program Controls Program Control information including: 3. Security, Privacy, and Confidentiality operation (SPC) Security, Privacy, and Confidentiality Controls V2.0  Budget  Operational Plans and Schedule July 12, 2019 Laws, policies, regulations, and guidelines related to physical security, IT security, data security and privacy and confidentiality impacts, analyses, and processes. These include but are not limited to Title 13, Title 26, and other laws and policies related to protection of personally identifiable information. Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Provider Page 62 Information Exchange Description 4. Content and Forms IE226: Content Design operation Specifications for (CFD) Automated Instruments Content (English and non-English) for the automated instruments and additional content-related guidance such as response option wording, skip patterns, and help text. 32. Field Infrastructure operation (FLDI) IE367: Online and Classroom Training Schedules Schedules for NRFU-specific online and classroom training classes. 5. Language Services operation (LNG) IE206: 2020 Census Language Support Plan Document that specifies the number of languages and level of support for each language to be included in the 2020 Census. 2.7.3 NRFU Operational Outputs Outputs are the data produced by the operation. The outputs constitute the results of operational work that has been performed. Outputs produced may be used as inputs or controls to other operations. Table 7 lists the outputs from NRFU. Table 7: NRFU Operational Outputs Consumer Information Exchange Description 32. Field Infrastructure operation (FLDI) IE250: Estimated Operational Workload Estimate of number of NRFU attempts by geographic area. This information is used by FLDI along with the NRFU contact strategies to create a model for estimating the staffing needs by location. 32. Field Infrastructure operation (FLDI) IE251: Operation-Specific Training Content and Requirements Course content and other training requirements for the NRFU-specific classroom training materials and online training modules developed by FLDI. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 63 Consumer Information Exchange Description 32. Field Infrastructure operation (FLDI) IE252: Approved Approval for training content and Operation-Specific Training materials used to conduct online and classroom training for NRFU-specific Materials operational activities. 32. Field Infrastructure operation (FLDI) IE615: Time and Expense Records Information regarding staff time and reimbursable expenses. Used by FLDI to pay employees. 32. Field Infrastructure operation (FLDI) IE616: Administrative Actions Decisions regarding administrative changes for field and office staff made by field operation management. For example, field operations may decide to transfer an employee to another location. FLDI is responsible for documenting these actions in the appropriate systems. 33. Decennial Logistics Management operation (DLM) IE391: Operation-Specific Kit Requirements A list of the contents that should be included in the operation-specific kits provided to the field staff in support of NRFU. Includes the number of kits required and the count of each item in the kits. 4. Content and Forms IE209: NRFU Field Support Design operation Materials Content (CFD) Content for the nonquestionnaire materials required by NRFU to support field enumeration and other NRFU field activities. 6. Geographic Programs operation (GEOP) IE049: Geographic Products Requirements Requirements for the geographic products that will be needed to conduct NRFU. 19. Response Processing Operation (RPO) IE254: Self-Response Case Hold Flagging Rules Rules for determining which cases to temporarily withhold from the NRFU field follow-up workloads pending receipt of sufficient self-responses for those cases. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Consumer Page 64 Information Exchange Description For example, postal tracking data indicates paper responses are arriving for cases in the NRFU follow-up workload. 19. Response Processing Operation (RPO) IE255: AdRec Modeling Requirements All requirements for performing the administrative record modeling used to reduce NRFU workload. This includes the types of AdRec to be used, the methodology for determining the status of the housing unit, and the contract strategy based on the unit status. 19. Response Processing Operation (RPO) IE256: Contact Attempt Stopping Rules The number of attempts required before a case is closed as “AdRec resolved”. The number of attempts vary depending on the results of the AdRec modeling. 19. Response Processing Operation (RPO) IE257: QC Plan (for NRFU) Plan for conducting NRFU quality control (QC) including sample selection methodologies and rates for reinterviews of original cases, thresholds for automated detection of anomalies, systems used, and procedures such as comparing and resolving reinterview cases and determining required rework. 19. Response Processing Operation (RPO) IE258: Areas for Early NRFU Areas such as those with high concentrations of off-campus college housing where nonresponse follow-up must be performed early because the residents are unlikely to be there during the standard NRFU time frame. 19. Response Processing Operation (RPO) IE104: Response and Status Data for Cases Data that result from enumeration of cases and the associated status information. For NRFU, the response data are the data captured through interviews V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Consumer Page 65 Information Exchange Description conducted by the enumerators. Case status can include not attempted, attempted, completed with household, completed with proxy vacant, and delete. 19. Response Processing Operation (RPO) IE259: Reinterview Anomaly Outcome Results of the clerical and field staff reviews of reinterviews where the automated comparison indicated anomalies between the reinterview response and the production NRFU response. The outcome indicates whether or not the analysis resulted in a hard fail that requires the stoppage of all pending work for the enumerator in question and possible rework of previous cases. 19. Response Processing Operation (RPO) IE131: Address Data Updates from Field Ops Address data updates provided by the field operations. This could include newly identified addresses, deletes when an address listed in the MAF is determined to no longer exist, and changes in living quarter type (e.g., housing unit to group quarter). For NRFU, newly identified addresses include hidden units, in-movers, or respondents with a Usual Home Elsewhere. Housing Unit (HU) or Respondent IE260: Request for Response Request for respondent/proxy to respond to the 2020 Census. For NRFU, requests for responses are made by a telephone call or in-person visit. Housing Unit (HU) or Respondent V2.0 IE261: Notice of Visit July 12, 2019 Paper notice of visit left at the HU directing the respondent to the internet Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Consumer Page 66 Information Exchange Description or Census Questionnaire Assistance to self-respond. 31. Decennial Service Center operation (DSC) IE380: IT Service Requests Requests for information or advice, or for a standard change (a preapproved change that is low risk, relatively common and follows a procedure) or for access to an IT service. 1. Program Management operation (PM) IE404: NRFU Paradata Status and progress data related to the data collection process. This includes administrative and procedural data describing the data collection and management process in all automated systems. 2.7.4 NRFU Operational Mechanisms Mechanisms are the resources (people, places, and things) that are used to perform the operational processes. They include Staff Resources, Infrastructure Sites, Systems, and other Technology Infrastructure. 2.7.4.1 Staff Resources Table 8 identifies the Staff Resources employed for NRFU. Figure 4 in Section 2.5.1, Reengineering Fieldwork, shows some of the increased efficiency provided by this reengineered staffing structure. Table 8: Staff Resources Used Within NRFU Operational Activities Staff Resources Description/Role Headquarters (HQ) Staff HQ staff who manage NRFU and coordinate activities with the Census Questionnaire Assistance (CQA) contractors, regional census center (RCC) staff, area census office (ACO) staff, field staff, and National Processing Center (NPC) staff. RCC Staff RCC staff who manage all NRFU field operations within their designated census region. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 67 Staff Resources Description/Role ACO Staff ACO staff who manage all NRFU field operations within their designated area. The Area Census Office Manager (ACOM) manages multiple ACOs with staff that includes the Census Field Managers (CFMs) and support staff comprising Assistant Managers (technical and administration), office clerks, and office operations supervisors. Field Staff Enumerators and Census Field Supervisors (CFSs) who perform NRFU operational activities in the field. NPC Staff Clerks at the NPC who perform the initial analysis and research of reinterview cases where the automated matching indicated anomalies between the reinterview responses and the production NRFU responses. 2.7.4.2 Infrastructure Sites Table 9 identifies the Infrastructure Sites employed for NRFU. Table 9: Infrastructure Sites for NRFU Operational Activities Infrastructure Site Description/Role Census Headquarters (HQ) HQ site for office work conducted in support of NRFU. This permanent site in Suitland, MD, manages the operation throughout the country. Regional Census Center (RCC) Regional centers that manage all operations within their assigned geographic area. The RCCs oversee the activities of the ACOs. Each RCC manages several ACOs. Area Census Office (ACO) ACOs that are responsible for managing all of the field operations and support activities within their designated area. Field Area The geographic area within which field staff perform data collection activities. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 68 Infrastructure Site Description/Role National Processing Center (NPC) The NPC in Jeffersonville, Indiana, which provides clerical support for the NRFU QC activities. Training Venues Sites where field staff receive classroom training on both general administrative topics and NRFU-specific topics. These sites are coordinated by the ACO staff in FLDI. 2.7.4.3 Systems and other Technology Infrastructure Table 10 identifies the Systems employed for NRFU. Table 10: Systems Used Within NRFU Operational Activities System Description Enterprise Censuses and Surveys Enabling Platform (ECaSE) Enterprise solution that supports 2020 Census operational work. For NRFU, ECaSE is used for field enumeration (ECaSEEnum), for work assignment (ECaSE – Survey Operational Control System), and for schedule and case management (ECaSE – Field Operational Control System) Sampling, Matching, Application supporting QC for field operations during the Reviewing, and Coding System 2020 Census Research and Testing program and the 2020 Census. The SMaRCS application specifically supports QC (SMaRCS) operations designed to determine whether field Listers and Enumerators are using validated procedures and collecting accurate data. The SMaRCS application facilitates QC operations by providing a mechanism for selecting QC samples, validating production interview data against AdRec sources, and by providing a tool for automated and clerical matching to compare the production interview data against reinterview data. SMaRCS also serves as a major control component for QC operations by managing the selection of QC samples for V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) System Page 69 Description field follow-up related to both census and PostEnumeration Survey (PES) operations and tracking the progress of the reinterview work through the matching, field, and resolution processes. Matching and Coding System (MaCS) MaCS is a geography system that is used for adding cases. It provides the MAF and/or geocode information needed to be able to send enumerators to the correct location. MOJO Enterprise solution that supports 2020 Census field assignment work. NRFU uses MOJO: BLQ (Browse Living Quarters) to identify all addresses in a geographic area that is in the enumeration universe and if the address has responded to Census. MOJO: Optimizer to optimize assignments based on routing and determine the most efficient order in which to enumerate a caseload. MOJO: HERMES provides analytic data to help field staff manage the data collection operation. Unified Tracking System (UTS) A data warehouse that combines data from a variety of Census Bureau systems, bringing the data to one place where the users can run or create reports, which allows them to analyze survey and resource performance. This role-based system provides case-level tracking across modes, drill down capability, and pulls cost data into the same system for more efficient cost impact assessment. The UTS extracts data from data collection and operational control systems daily, providing the users near real-time data. UTS has two components: Paradata Repository and Cost & Progress. NRFU uses reports from the Cost & Progress component to manage and monitor NRFU. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) System Page 70 Description Learning Management System System that hosts online training modules for the field (LMS) staff. Other Technology Infrastructure employed for the NRFU operation includes:      V2.0 Census networks for data transmission between operational systems and operational sites. This connectivity is provided by the IT Infrastructure operation (ITIN). Mobile networks for data collection activities using the mobile devices. This network is provided by ITIN. Mobile devices used by the field staff to record the HU status and enumerate occupied HUs. These devices are provided by ITIN. HQ Office IT Infrastructure for conducting NRFU operational work. This infrastructure is provided by the IT Infrastructure operation (ITIN). Operation-Specific Kits that field staff use for data collection activities. These kits are provided by DLM. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 71 NRFU Data Flow and Operational Influences Figure 10 is an Integrated Operations Diagram (IOD), which describes the design concepts for the response data collection operations for the 2020 Census (stateside and Puerto Rico). This diagram assumes that the frame has been developed and address canvassing operations are complete. The diagram shows the Response Processing Operation (RPO) as the hub of data collection and RPO’s interactions with all the other 2020 Census operations that have a role in data collection. The discussion below walks the reader through the diagram, using the circled numbers to help the reader follow the flow. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 72 2020 Census Data Collection Integrated Operations Diagram (IOD) 6. Geographic Programs Operations (GEOP) Geographic Data Processing (GDP) Initial Address and Spatial Data Address Data for MAF Updates 1 (with TEA Designators) Updated Address and Spatial Data 6 Final Address & Block (BCU) Data 6 9 19. Response Processing Operation (RPO) 19-1. RPO PRE-DATA COLLECTION PHASE 19-2. RPO DATA COLLECTION PHASE AdRecs Initial Mailing Workload 9. Forms Printing and Distribution Operation (FPD) 30. Evaluations and Experiments Operation (EAE) Mailing Initial Enumeration Case Universe 3 4 3 13. Non-ID Processing Operation (NID) Post Real-Time Non-ID Processing MAF 2 Paper Questionnaire Packages 7 4 Non-ID Requests/ Results AdRecs 2 10. Paper Data Capture Operation (PDC) 2 12. Internet SelfResponse Operation (ISR) 13. Non-ID Processing Operation (NID) Non-ID Addresses 2 Real-Time NonID Processing CI Cases CI Results 6 17. Census Questionnaire Assistance Operation (CQA) 5 Coverage Improvement (CI) File Review Outcomes 9 EAE Response Data 7 Field Data Collection (including rework) Paper Returns Respondent 4 4 35. Update Leave Operation (UL) 30. Evaluations and Experiments Operation (EAE) 14. Update Enumerate Operation (UE) 15. Group Quarters Operation (GQ) 16. Enumeration at Transitory Locations Operation (ETL) 18. Nonresponse Followup Operation (NRFU) Housing Unit 17. Census Questionnaire Assistance Operation (CQA) 4 5 30. Evaluations and Experiments Operation (EAE) 9 DRF-2, CUF and CEF 23. Count Review Operation (CRO) 21. Data Products and Dissemination Operation (DPD) 9 MDF KEY Production Self-Response and Other Paper Data Capture 2 6 4 2 1 2 3 3 4 3 3 4 6 1 Conditional Mailing Workload Printing Paper Questionnaires and Mailing Materials Telephone Interviews 2 Decennial Response Processing (DRP) Listing QC and Reinterview Results; Adjudication Outcomes EAE Sampling Criteria 1 Decennial Response Processing (DRP) Listing QC and Reinterview Cases; NRFU Anomaly Results 1 Quality Control Management (QCM) Response Data Integration (RDI) Listing Results, Address Data Updates, NRFU Response/Status, and GQ eResponses/Status (Includes AdRec Modeling for NRFU) AdRecs Geographic Data Integration (GDI) Initial Universe/Workload and Updates (includes Production, Rework, and Field Verification) Universe Management (UM) Initial Enumeration Case Universe Universe Creation (UC) Response and Status Data for SelfResponse Cases and Paper Returns from UE, GQ, ETL, and EAE Language Stratification Data 19-3. RPO POST-DATA COLLECTION PHASE USPS Tracing Data Mailing Contact Strategy 4 Office Space, Staff, Kits, Devices, Administrative, Technical, and Other Support 7 Quality Control (including Reinterviewing, Relisting, and Adjudication) Operations 8 Functions Field Infrastructure Support 32. Field Infrastructure Operation (FLDI) 33. Decennial Logistics Management Operation (DLM) 31. Decennial Service Center Operation (DSC) The 2020 Census Data Collection Integrated Operations Diagram depicts the major interactions among the response data collection operations for stateside and Puerto Rico. It assumes the frame has been developed and address canvassing operations are complete. See the accompanying narrative to understand the sequence and flow. RPO Phases 34. IT Infrastructure Operation (ITIN) # Narrative Reference Point Version v7c 06-24-19 Figure 10: 2020 Census Data Collection Integrated Operations Diagram (IOD) V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 73 Pre-Data Collection 1 Before the start of data collection, the Geographic Data Processing (GDP) component of the Geographic Programs operation (GEOP) sends initial Address and Spatial Data, including the Type of Enumeration Area (TEA) designations, to RPO so it can create the Initial Enumeration Case Universe. RPO also receives the mailing contact strategy (i.e., strategy for self-response stratification) so it can identify which housing units receive which kinds of mailings, language stratification information so it knows which language to use, and experimentation stratification data so it knows which housing units are to be included in what types of experiments. The creation of the Initial Enumeration Case Universe and application of the stratification data are done as part of the RPO Universe Creation function. Based on the stratifications, the RPO Universe Management function creates the initial mailing workload and sends it to the Forms Printing and Distribution operation (FPD), which prints and then mails the appropriate materials to mailable housing units for the Self-Response (SR) TEA. The first two of the five potential mailings for the SR TEA are sent unconditionally to all housing units in this TEA. These mailings are sent in English or English and Spanish based on the language stratification data and may include letters or―based on the self-response stratification―questionnaires. During Data Collection 2 Once the RPO Universe Creation work is complete, the Initial Enumeration Case Universe is managed by the RPO Universe Management function, which tracks changes to the enumeration universe for future mailings and for the data collection operations. People living in housing units are encouraged to self-respond through a partnership and communications campaign (not shown on this diagram), through mailings sent by FPD, and through paper questionnaires left at housing units as part of the Update Leave operation (UL). To make it easy for people to respond and to reduce the paper workload, the Census Bureau is using an Internet First strategy for most housing units. Respondents can go to the internet and enter their response using the internet instrument as part of the Internet Self-Response operation (ISR). The internet option offers additional flexibility and allows people to respond in multiple languages. If a respondent calls the Census Questionnaire Assistance operation (CQA), a customer service representative may offer to collect the respondent’s information by telephone. The information collected from these telephone interviews is entered by a customer V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 74 service representative using an ISR instrument similar to the public-facing instrument used by respondents. Respondents can also mail paper questionnaire forms. These forms are received by the Paper Data Capture operation (PDC), which uses scanning and imaging technology to capture the information from these forms. ISR receives the Initial Enumeration Case Universe from the RPO Universe Management function and uses the Enumeration Case Universe to link responses provided through the internet instrument to the appropriate case. If respondents do not have their unique Census ID available, they are still able to complete the census questionnaire as a Non-ID response using the ISR instrument. The Non-ID Processing operation (NID) first attempts to match the address entered by the respondent or customer service representative to a known census address in real-time. For addresses that do not match, the response is still collected and is subject to later Non-ID Processing. 3 Response and status data collected through the various self-response data collection operations are sent (in digital format) to RPO’s Response Data Integration function. Any responses collected through PDC or ISR that are submitted in languages other than English or Spanish are translated by staff at the Tucson call center on behalf of these operations before being sent to RPO. RPO’s Universe Management function uses the response status data to determine the appropriate actions for the case. During the self-response data collection time-period, reminder mailings are sent to housing units in the SR TEA. The first reminder is sent to all housing units in the SR TEA using the initial mailing workload as discussed above. Subsequent reminders are conditional and are only sent to those housing units that have not yet responded. The RPO Universe Management function sends a Conditional Mailing Workload to the FPD operation for these nonresponding units. FPD also receives from the RPO Universe Management function a list of mailable housing units in the UL TEA and mails two reminders to these housing units. Any remaining Non-ID cases are sent by RPO to NID for post real-time Non-ID processing, which attempts to match addresses provided by respondents to known addresses in the Master Address File (MAF) using automated and clerical procedures. As needed, administrative records (AdRecs) are used to supplement the matching process. Most of these Non-ID cases are from internet responses that could not be matched during real-time Non-ID processing. In addition, post data capture Non-ID processing required for paper forms for which the Census ID could V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 75 not be read during data capture. The results of post real-time Non-ID matching are sent back to RPO. Based on predefined business rules, some of the responses that are not able to be matched through NID are sent to the field for verification as part of the Nonresponse Followup operation (NRFU). 4 The discussion above covers self-responses for people living in housing units. Special operations also exist to collect data from people living in other types of living quarters or for whom self-response is not a viable option:  The Group Quarters operation (GQ) enumerates people living in group quarters (e.g., dormitories, correctional facilities, and nursing/skilled-nursing facilities) as well as people experiencing homelessness and receiving services at service-based locations such as soup kitchens. GQ also enumerates people living on maritime vessels and people living in the Military TEA, which includes both on-base group quarters and on-base housing units.  The Enumeration at Transitory Locations operation (ETL) enumerates people who are living in special locations such as recreational vehicle parks, campgrounds, racetracks, circuses, carnivals, marinas, hotels, and motels and who do not have a Usual Home Elsewhere.  The Update Enumerate operation (UE) lists and enumerates housing units in areas that pose unique challenges to the standard self-response data collection operations. These housing units are in the UE and Remote Alaska TEAs, which cover remote areas of the country and other small selected areas. NRFU is another special operation whose primary purposes are to determine the housing unit status of addresses in the SR and UL TEAs for which a self-response was not received and to enumerate those that are believed to be occupied. As mentioned in number 3 above, NRFU also performs a field verification activity to verify selected addresses for Non-ID self-responses that could not be matched to known addresses through NID. Based on the universe case type (derived from TEA and living quarter type), RPO sends the Initial Enumeration Case Universe/Workload to the GQ, ETL, and UE operations. GQ uses this universe to perform an advance contact activity to collect general information and determine the preferred method of enumeration. ETL also performs an advance contact activity to schedule appointments for enumerating its universe of cases. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 76 NRFU does not require advance contact activities. For NRFU, the RPO Universe Management function creates an Initial Case Universe/Workload comprising all housing units in the SR and UL TEAs for which a self-response was not received. The NRFU contact strategy is dependent on the results of an AdRec modeling activity. Four possible status outcomes result from this modeling for a given address:     AdRec Vacant: No one lives there. AdRec Delete: There is no housing unit at that address. AdRec Occupied: There is a high probability that someone lives there and that the Census Bureau has high-quality data about that housing unit. AdRec No Determination: Administrative data are not sufficient to help determine the housing unit status, and a full contact strategy is required. An initial attempt to enumerate is made for all addresses in the NRFU workload. NRFU sends to RPO information regarding the success of this and any subsequent enumeration attempts as part of the response status data. AdRec Vacant and AdRec Delete housing units also receive an additional mailing from FPD. The RPO Universe Management function provides this AdRec mailing workload to FPD (as another type of Conditional Mailing). RPO removes from the follow-up workload any AdRec Occupied cases that cannot be successfully resolved during the first attempt and triggers one final mailing (from FPD) to these addresses to encourage these households to self-respond. RPO also removes AdRec Vacant and AdRec Delete cases that cannot be successfully resolved during this first attempt, provided that those cases do not appear occupied and that information from the United States Postal Service (USPS) about the additional mailing indicates that the unit is either vacant or nonexistent. For all other cases, NRFU continues to attempt to enumerate the housing unit until the attempt is successful or the maximum number of attempts has been reached. Additional attempts are made for selected units during the NRFU Closeout phase based on a reassessment of the AdRec modeling results using a relaxed, lower quality threshold. Self-responses can continue to arrive at any time during NRFU. Accordingly, RPO flags housing units in the follow-up workload for which RPO has received a self-response or a notification of tracing information from the USPS that indicates that a return is on its way to one of the paper data capture facilities. NRFU is notified about these flagged households as soon as the information is available so that it can remove the housing units from the daily workload, if possible. Any self-responses that are flagged but later found by RPO to have insufficient V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 77 enumeration data are added back to the NRFU workload for continued enumeration attempts. The RPO Universe Management function tracks this information and uses it to determine what to include in the next day’s follow-up workload. Housing units that have been successfully enumerated are not included in subsequent follow-up workloads. For NRFU, field data are collected by electronic devices. The electronic data are sent to the RPO Response Data Integration function, which subsequently provides this information to the Decennial Response Processing function for further processing. Paper questionnaires are used to enumerate at living quarters during UE and at housing units during ETL. These paper questionnaires are checked-in at area census offices (ACOs) and then sent to the paper data capture facilities, where they are scanned and imaged by PDC. PDC sends the captured data and case status information to RPO in digital format. GQ is primarily a paper operation. Group quarter responses collected on paper questionnaires are checked-in at the ACOs and sent to PDC for scanning and imaging before being transmitted to RPO. Case status updates are sent to RPO as part of the check-in process. GQ data provided in electronic files (eResponses) require additional processing to prepare the data before they are electronically transmitted to RPO. GQ data collected on paper rosters are entered by ACO clerks into the same file format that is used for eResponses. These response data are then sent electronically to RPO. As part of the Evaluations and Experiments operation (EAE), the Census Bureau may test different questionnaire content and data collection methodologies during the 2020 Census to help evaluate content and modes for the 2020 Census and inform design changes for the 2030 Census. Addresses that are selected to be part of these experiments are identified in the initial universe (see number 1 above). For those addresses, the EAE operation sends households questionnaires with various experimental items, packaging, etc. to assess the impact made on the response or lack thereof. Respondents receiving EAE questionnaires and notices will respond via ISR, CQA, PDC or NRFU. 5 RPO’s Decennial Response Processing function performs quality assurance activities as well as coding and other preparation steps on incoming self-response data. The RPO Universe Management function also supports a Coverage Improvement (CI) activity, the goal of which is to ensure a high-quality census by conducting telephone followup for households where there could be coverage issues on submitted responses. CI is a followup activity and is therefore considered a component of NRFU; however, the CI telephone V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 78 interviews are performed by CQA. CQA receives from the RPO Universe Management function a set of cases with potential coverage issues and provides the results of these cases back to RPO’s Response Data Integration function. 6 As noted above, universe and address updates occur during field operations. Census Bureau field staff may uncover changes to addresses as they perform their daily assignments in any field operation. For example, a UL or UE lister may add an address or find an error in the address or geographic data based on the listing activities, or a NRFU enumerator or a UL lister may go to an address and find an additional unit such as a garage apartment located on the premises. All listing results and other address changes are sent to the RPO Geographic Data Integration function, which passes the information on to the GDP function in GEOP. Changes to the address list may also come from other sources such as appeals from the Local Update of Census Addresses operation (LUCA), the review of addresses performed by the Count Review operation (CRO), and updated files from the postal service. The GDP function within GEOP updates the address data and sends these RPO’s Universe Management Function, which provides these cases to the appropriate operation. Depending on the timing, living quarter type, and TEA designation, RPO may initiate one or more mailings to these new addresses through FPD to encourage self-response. 7 All field operations (GQ, UL, ETL, UE, and NRFU) include quality control (QC) functions. For GQ, the RPO Quality Control Management function creates and sends a sample of the field enumeration cases to ACO staff, who conduct telephone reinterviews for this sample set of cases to confirm that a GQ enumerator visited the site and that the total population count is correct. For UL, the RPO Quality Control Management function selects a sample set of basic collection units (BCUs) for relisting. The QC Listing Results are sent back to the RPO Quality Control Management function for further processing. RPO does not send any changes resulting from UL listing to GEOP until the lister has passed the QC check. Further, if the QC activities result in a hard fail, BCUs already worked may require relisting. RPO includes this rework in subsequent UL production workloads. QC methods for ETL and UE are performed primarily in the field, tailored to meet the circumstances of these unique paper-based operations. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 79 NRFU includes multiple methods for ensuring high-quality data collection. Several of these are integrated into the staff management activities. In addition, samples of field followup cases are selected for reinterview (RI), a process whereby the response data are collected again and compared with the original collected data. The RPO Quality Control Management function creates the RI workload and sends it to the NRFU operation. The RI cases are handled by NRFU field staff. The RI results are sent to the RPO Quality Control Management function, which performs an automated comparison of the RI data against the original data. Anomalies are sent back to NRFU, where additional research is conducted to determine how these cases should be handled. The results of this review (adjudication outcomes) are sent back to the RPO Quality Control Management function. In some cases, the adjudication requires that prior cases performed by the enumerator at fault be reworked. RPO puts these cases back into the NRFU workload as appropriate. 8 NRFU, UE, UL, ETL, and parts of the GQ operation are performed in the field. Several operations provide the support for these field data collection activities. The Field Infrastructure operation (FLDI) recruits, hires, onboards, and trains the staff needed to conduct these operations and also operates the field offices during production. The Decennial Logistics Management operation (DLM) provides the space and logistics support (e.g., supplies and kits) for the offices and the field staff. The Decennial Service Center operation (DSC) provides technical support for field and field office staff. Finally, the IT Infrastructure operation (ITIN) provides the hardware and software used by the field staff and field offices. Post-Data Collection 9 Once data collection is complete, additional processing occurs to prepare the counts for use in apportionment and the data used by the Data Products and Dissemination operation (DPD) to create data products for redistricting and other purposes. The RPO Decennial Response Processing function handles this post-data collection processing, which includes multiple activities:  Supplementing response data with administrative records (AdRec) for those cases that had been identified as AdRec Occupied but for which a nonresponse followup attempt was unsuccessful and no subsequent self-response was received.  Determining the final enumeration universe by reconciling or applying final address and block data from the GDP component of GEOP. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 80  Determining the returns of record for situations where multiple responses have been received for the same housing unit.  Performing count and status imputations.  Performing consistency editing and characteristic allocation supplemented with administrative records (AdRec) data.  Applying tabulation geography.  Performing disclosure avoidance (Note: This is done by RPO for Island Areas Censuses data only. Disclosure avoidance for stateside/PR data is handled by DPD). Similar processing occurs for responses from group quarters. Responses collected through the EAE operation may require slightly different activities. Through these processing activities, the RPO Decennial Response Processing function creates multiple files for Stateside/PR response data, including the Decennial Response File (DRF), the Census Unedited File (CUF), and the Census Edited File (CEF). RPO also creates an Island Areas CUF (IA-CUF), Island Areas CEF (IA-CEF), and Island Areas Microdata Detail Files (IA-MDF) for the Island Areas Censuses (IAC) response data. Each of these files is reviewed within the Census Bureau before the data are sent to the next stage of processing: Some of these reviews are done as part of the CRO. The CUF, the CEF, the IA-CUF, and the IA-MDF are sent to DPD via the Census Data Lake (CDL). DPD uses these files as inputs for data products creation and also creates the Stateside/PR MDFs using the CEF as input. RPO also sends data collected as part of EAE back to the EAE operation for further analysis. NRFU Design Assumptions Assumptions upon which the NRFU operational plan is based are listed below.     V2.0 There is an automated NRFU data collection instrument. There is an automated field operational control system. Administrative records (AdRec) modeling is used to reduce the NRFU fieldwork. A reengineered field structure is used to streamline NRFU operations. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 81 3 Nonresponse Followup Operation (NRFU) Detailed Process Description Figure 11 is a top-level Business Process Model (BPM) showing the Level 1 activity areas within NRFU. BPMs for the 2020 Census follow industry-standard Business Process Model Notation (BPMN). An explanation of how to read the BPMN notations and a full-sized copy of all of the BPMN diagrams for this operation are provided under separate cover. This top-level BPM serves as the Context Model for NRFU. A BPMN Context Model displays the high-level activities within the operation and relationships between them, whereas the IDEF0 Context Diagram shown earlier depicts the boundaries of the operation or activity and the interfaces between the operation or activity and other operations and activities with which it is associated. 2020 Census 18. Nonresponse Followup Operation (NRFU) Purpose: To determine housing unit status for nonresponding addresses and to enumerate housing units for which a 2020 Census response was not received. Business Rules & Contact Strategies NRFU Planning 18-1 and Preparation Develop 10 NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules NRFU Operational Management and Data Collection 18-1.1 18-2.2.1 Develop and Approve Field Printed Materials and Kit Specs Monitor Progress and Resolve Issues 18-1.2 Create Work Assignments (incl. Grouped Manager Visit Cases) Receive NRFU Universe and Updates from RPO 18-2.2.3 18-2.2.2 Business Process Modeling & Analysis Toolkit 18-1.3 Case Status/ Response/ Address Data to RPO Perform NRFU Data Collection 14 Develop Requirements for and Approve Systems Supporting NRFU NRFU Begins 18-1.5 18-3.1 Receive Work Assignment 18-3.2 Perform Field Verification Data Collection 18-3.3 Perform NRFU Manager Visit 18-3.4 Perform NRFU Case Enumeration 18-3.5 Perform NRFU Reinterview Data Collection 18-3.6 Closeout Contact Attempt 18 Prepare for and Deliver NRFUSpecific Training 55 Receive Automated Quality Control Comparison Results from RPO 18-1.6 RI QC Has Begun Page-1 70 Submit Time and Expense Data Operation Closeout Criteria Met? Paradata to PM 18-1.4 16 Determine Staff Needs and Performance Metrics Includes: Early NRFU NRFU Production Field Verification Re-collect Manager Visits In-field Adds 45 50 Start 18-2.1 18-2.2 Receive and Process NRFU Universe and Updates 35 12 Identify Areas for Early NRFU Perform In-Office Adds NRFU Universe/ Workload Updates 11 Early NRFU Areas 38 Case Status Data to RPO (e.g. Rework Needed) 30 18-2 NRFU Operational Workload Management 18-4.1 QC Computer Match Anomaly? Yes 18-3.7 60 No 18-4.2 80 Yes 18-3 NRFU Data Collection Adjudicate Reinterview Results 18-5 NRFU Operation Closeout Closeout NRFU Operation End No (Repeat Daily) 18-5.1 RI Decision Outcome to RPO 18-4 NRFU Quality Control Adjudication 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Page 1 of 10 Figure 11: NRFU Operation Context Model NRFU is subdivided into the following Activity Areas: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)      Page 82 NRFU Planning and Preparation [NRFU 18-1]. NRFU Operational Workload Management [NRFU 18-2]. NRFU Data Collection [NRFU 18-3]. NRFU Quality Control Adjudication [NRFU 18-4]. NRFU Operation Closeout [NRFU 18-5]. The business processes for each of these Level 1 activity areas are discussed along with their inputs and outputs in the following subsections. NRFU Planning and Preparation [NRFU 18-1] Figure 12 shows the BPM for the NRFU Planning and Preparation [NRFU 18-1] activity area (area within the gray rounded rectangle) and its constituent activities within the overall context of the NRFU operation. 2020 Census 18. Nonresponse Followup Operation (NRFU) Purpose: To determine housing unit status for nonresponding addresses and to enumerate housing units for which a 2020 Census response was not received. Business Rules & Contact Strategies NRFU Planning 18-1 and Preparation Develop 10 NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules NRFU Operational Management and Data Collection 18-1.1 18-2.2.1 Develop and Approve Field Printed Materials and Kit Specs Monitor Progress and Resolve Issues 18-1.2 Create Work Assignments (incl. Grouped Manager Visit Cases) Receive NRFU Universe and Updates from RPO 18-2.2.3 18-2.2.2 Business Process Modeling & Analysis Toolkit 18-1.3 Case Status/ Response/ Address Data to RPO Perform NRFU Data Collection 14 Develop Requirements for and Approve Systems Supporting NRFU NRFU Begins 18-1.5 18-3.1 Receive Work Assignment 18-3.2 Perform Field Verification Data Collection 18-3.3 Perform NRFU Manager Visit 18-3.4 Perform NRFU Case Enumeration 18-3.5 Perform NRFU Reinterview Data Collection 18-3.6 Closeout Contact Attempt 18 Prepare for and Deliver NRFUSpecific Training 55 Receive Automated Quality Control Comparison Results from RPO 18-1.6 RI QC Has Begun Page-1 70 Submit Time and Expense Data Operation Closeout Criteria Met? Paradata to PM 18-1.4 16 Determine Staff Needs and Performance Metrics Includes: Early NRFU NRFU Production Field Verification Re-collect Manager Visits In-field Adds 45 50 Start 18-2.1 18-2.2 Receive and Process NRFU Universe and Updates 35 12 Identify Areas for Early NRFU Perform In-Office Adds NRFU Universe/ Workload Updates 11 Early NRFU Areas 38 Case Status Data to RPO (e.g. Rework Needed) 30 18-2 NRFU Operational Workload Management 18-4.1 QC Computer Match Anomaly? Yes 18-3.7 60 No 18-4.2 80 Yes 18-3 NRFU Data Collection Adjudicate Reinterview Results 18-5 NRFU Operation Closeout Closeout NRFU Operation End No (Repeat Daily) 18-5.1 RI Decision Outcome to RPO 18-4 NRFU Quality Control Adjudication 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Page 1 of 10 Figure 12: NRFU Planning and Preparation [NRFU 18-1] Constituent Activities V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 83 The NRFU Planning and Preparation activity area is subdivided into the following operational subactivities:  NRFU Planning and Preparation [NRFU 18-1]. o Develop NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules [NRFU 18-1.1]. o Develop and Approve Field Printed Materials and Kit Specs [NRFU 18-1.2] o Identify Areas for Early NRFU [NRFU 18-1.3]. o Develop Requirements for and Approve Systems Supporting NRFU [NRFU 181.4]. o Determine Staff Needs and Performance Metrics [NRFU 18-1.5]. o Prepare for and Deliver NRFU-Specific Training [NRFU 18-1.6]. NRFU performs planning and preparation activities in advance of its 2020 Census operation. These activities include the following:     Developing processing requirements, contact strategies and other business rules (e.g., proxy-eligibility) that are used by RPO and the operational control systems for case management and control. Providing input on the content of field materials as well as information on the contents and quantity of items in each of the kits provided to the NRFU field staff during training. Identifying geographic areas requiring early NRFU.  Providing requirements for systems supporting NRFU and, once the systems are developed, approving these systems. Providing NRFU-specific training requirements and content and delivering this training. Determining the needs for field staff at the tract level and defining performance metrics for field staff as well as for the entire NRFU field operation. The subsequent sections describe the NRFU Planning and Preparation operational subactivities in detail. 3.1.1 Develop NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules [NRFU 18-1.1] The “Develop NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules” operational subactivity is subdivided into the following constituent activities: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)  Page 84 Develop NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules [NRFU 18-1.1]. o Develop NRFU Administrative Records Modeling Requirements [NRFU 18-1.1.1]. o Develop NRFU Contact Strategies and Business Rules [NRFU 18-1.1.2]. o Develop NRFU Quality Control Plan [NRFU 18-1.1.3]. NRFU provides the following to RPO to use for delivering and managing the NRFU universe and workload:    Administrative record and third-party data (AdRec) modeling sources and methodologies for the production of the administrative record modeling results that RPO applies to the NRFU universe. Contract strategies and associated business rules that RPO applies to the NRFU universe. Quality Control plan that RPO uses to determine the selection by RPO of cases for reinterview. 3.1.1.1 Develop NRFU Administrative Records Modeling Requirements [NRFU 18-1.1.1] As described in Section 2.2.4, the use of AdRec is intended to reduce the field workload associated with NRFU by reducing the average number of visits to nonresponding HUs. During planning and preparation, NRFU develops the processing requirements sent to RPO to be used for AdRec modeling. See Appendix E for a description of the sources and methodology used for this modeling. The results of the modeling are then applied to the NRFU universe to set the status of each NRFU case as AdRec Occupied, AdRec Vacant, AdRec Delete, or AdRec No Determination. See Section 3.2.2 for more information about the receiving of the NRFU universe from RPO. 3.1.1.2 Develop NRFU Contact Strategies and Business Rules [NRFU 18-1.1.2] The NRFU Integrated Project Team (IPT) develops the contact strategies and associated business rules used for conducting NRFU, such as:     V2.0 Maximum days of personal visit attempts for each case. Defined rules for when each case becomes eligible for proxy enumeration. Defined rules for a phase approach of case assignment. Criteria for ECaSE Field OCS to alert a Census Field Supervisor (CFS) of an enumerator’s performance. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)   Page 85 Criteria for ECaSE Field OCS to alert a Census Field Manager (CFM) of a CFS’s performance. Criteria for ECaSE Field OCS to initiate a managerial review of a case. These contact strategies and business rules are sent to RPO to be applied to the universe that RPO provides for NRFU to create case assignments. The strategies and rules are used in the following activities:    NRFU applies these business rules in the development and approval of systems requirements, for example, the requirement for proxy-related screens developed in Enterprise Censuses and Surveys Enabling Platform (ECaSE) Enumeration (ENUM). Response Processing Operation (RPO) applies these business rules when creating the NRFU universe, for example, for setting the number of household contact attempts based on administrative record modeling. ECaSE Field Operational Control System (FOCS) tracks the parameters defined in these business rules when creating case assignments, for example, determining that the required household contact attempts have been met so the assigned case is proxyeligible. See Appendix F for details on the contact strategies resulting from the AdRec modeling. See Section 3.2.2 for more information about the receipt of the NRFU universe from RPO. 3.1.1.3 Develop NRFU Quality Control Plan [NRFU 18-1.1.3] The NRFU IPT develops a plan for selecting the quality control (QC) sample of cases for reinterview as described in this subsection. The majority of completed NRFU cases (by either census household member or proxy) are candidates for reinterview selection, including cases with a vacant or delete status through proxy interview. To select cases for reinterview, RPO samples the completed cases using the Sampling, Matching, Reviewing, and Coding System (SMaRCS). RPO makes the following selections from the sufficient or completed NRFU production interviews:  V2.0 Random: Random selection from the eligible cases completed by every NRFU enumerator. For each enumerator, SMaRCS randomly selects one of the first three eligible cases completed and thereafter every nth eligible case, where n depends on the percentage of cases NRFU wants to select for random reinterview for each enumerator. For example, if approximately 5 percent is selected for random reinterview, then n=20. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)    Page 86 Analytic: Selections based on a variety of outlier tests, such as GPS distance, interview length, and missing phone number that focus the sample on cases that appear suspicious. Supplemental: Selection of an enumerator’s future cases or manual selection of previously completed cases based on a case having computer matching anomalies and the NRFU clerical resolution staff requesting more cases for further investigation of potential falsification. Rework: Selection of an enumerator’s previously completed cases when the matching of a reinterview case with the enumerator’s original case resulted in a determination of falsification. NRFU Quality Control plan includes information for RPO on:    Reinterview Sample Completion Rules. Reinterview Anomalies Determination Rules (thresholds). Reinterview Rework Rules (selection of cases for rework). Automation allows for a smarter QC sample using paradata from ECaSE ENUM. See Section 3.2.2 for a description of processing of NRFU universe updates including reinterview selections and Section 3.3.5 for a description of the NRFU reinterview data collection process. Besides sample selection, the QC plan defines the procedures for comparing and resolving reinterview cases that are described in Section 3.4.2. 3.1.2 Develop and Approve Field Printed Materials and Kit Specs [NRFU 18-1.2] The “Develop and Approve Field Printed Materials and Kit Specs” operational subactivity is subdivided into the following constituent activities:  Develop and Approve Field Printed Materials and Kit Specs [NRFU 18-1.2]. o Provide and Approve Field Enumeration Materials Content [NRFU 18-1.2.1]. o Provide and Approve Kit Requirements [NRFU 18-1.2.2]. NRFU provides Content and Forms Design operation (CFD) with the content for field enumeration materials and approves the final materials. NRFU provides DLM with requirements for assembling field staff supplies and materials into kits and approves the NRFU kit specifications. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 87 The subsequent sections describe the Develop and Approve Field Printed Materials and Kit Specs operational subactivities in detail. 3.1.2.1 Provide and Approve Field Enumeration Materials Content [NRFU 18-1.2.1] NRFU will provide CFD with the content of the following field enumeration materials:     Information Sheet: For an enumerator to refer to during an interview to help the respondent understand the Census Bureau confidentiality policy and the criteria of who should be counted on Census Day. The Information Sheets will be available in multiple languages. Language Identification Card: For an enumerator to show the respondent during an interview attempt when a language barrier is evident. The card lists several languages and is primarily a tool to allow the enumerator to identify the language that the respondent speaks. For each language, a short paragraph in that language asks if there is an English-speaking person available to help them complete the interview or if there is a phone number to allow someone from the Census Bureau to contact them. Notice of Visit (NOV): To be left at a HU by an enumerator after an unsuccessful personal visit attempt. It directs the household to respond to the Census, through either Internet Self-Response (ISR) or a centralized telephone interview service provided by Census Questionnaire Assistance (CQA) staff. Administrative record removal postcard: To be mailed as determined by RPO to any addresses that have been removed from the NRFU workload as a result of having sufficient AdRec data to enumerate the household. This mailout gives the household a final chance to self-respond. CFD creates the materials (which are approved by NRFU) and then provides the print files for the field enumeration materials to DLM after NRFU specifies the kit requirements. RPO provides the list of cases for the administrative record removal postcards to Forms Printing and Distribution. 3.1.2.2 Provide and Approve Kit Requirements [NRFU 18-1.2.2] NRFU provides DLM with the requirements for assembling the materials for the following types of kits:  V2.0 Census Field Supervisor (CFS) Instructor Kit: Contains materials needed to perform enumerator training, including forms such as for a Security Incident Report, Special July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)    Page 88 Sworn Status, personnel and payroll updates, and a Thank You Letter for donated space. Also includes supplies such as a Census Bureau bag, name cards, pens, and a power supply kit. CFS Supply Kit: Contains materials CFSs need to conduct their work, including forms such as for a Security Incident Report and Special Sworn Status, documents such as the CFS Handbook, and an accident reporting kit. Enumerator Supply Kit: Contains materials enumerators need to conduct their work, including the Official Business sign, NOVs, Information Sheets, Language Identification Cards, a Census Bureau bag, pens, a clipboard, and documents such as the Enumerator Handbook. Bulk Supply Kit: Includes extra sets of materials needed for the CFS and enumerators to conduct their work. These are typically delivered to the ACOs for distribution to the staff. DLM creates detailed kit specifications based on these requirements, which are approved by NRFU. Appendix D provides more insight into the contents of these kits. Note: ITIN acquires mobile devices for FLDI to distribute to NRFU field staff during training. CFSs receive tablets to use ECaSE Field OCS to monitor case assignments and enumerators, as well as smartphones to have access to ECaSE ENUM. Enumerators receive smartphones for receiving and performing their data collection assignments with ECaSE ENUM. 3.1.3 Identify Areas for Early NRFU [NRFU 18-1.3] Some areas of the country present special challenges to NRFU, where the residents of those areas are likely to vacate their housing units after Census Day (April 1) but before the start of NRFU, planned for May 13, 2020. The most common scenario occurs where certain colleges and universities close before NRFU begins. To address this challenge, NRFU starts earlier in these areas, to increase the chance of interviewing the residents at their Census Day address. Early NRFU occurs from April 9 through mid-May. Locations requiring early NRFU are identified with the help of field managers who are knowledgeable about the local areas. With input from the Geography Division (GEO), NRFU confirms and provides these requirements to RPO to create the universe for early NRFU. 3.1.4 Develop Requirements for and Approve Systems Supporting NRFU [NRFU 18-1.4] NRFU works with several stakeholders, including the Decennial Census Management Division (DCMD), Population Division (POP), Decennial Statistical Studies Division (DSSD), and Field V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 89 Division (FLD) to develop and review requirements for the systems that support NRFU operations. These systems include ECaSE, MOJO, CQA Systems (for Coverage Improvement), MaCS, UTS, and LMS. For the major systems, requirements are developed through user stories and specifications. The Content and Forms Design operation (CFD) maintains the specifications that describe the requirements for the enumeration instrument, including content (English and non-English) and additional content-related guidance such as response option wording, skip patterns, and help text. 3.1.5 Determine Staff Needs and Performance Metrics [NRFU 18-1.5] The “Determine Staff Needs and Performance Metrics” operational subactivity is subdivided into the following constituent activities:  Determine Staff Needs and Performance Metrics [NRFU 18-1.5]. o Determine Estimated NRFU Operational Workload by Geographic Area [NRFU 18-1.5.1]. o Define Operational Performance Metrics and Targets [NRFU 18-1.5.2]. NRFU estimates the operational workload by geographic area to enable FLDI to determine staffing needs in all locations and plan recruiting efforts. Decennial Logistics Management operation (DLM) uses these staffing estimates to determine office space needs. NRFU also defines performance metrics and targets to be used to monitor and manage the staff and the operation as it progresses. These performance metrics are different from the parameters around ECaSE Field OCS alerts described above in Section 3.1.1.2. The subsequent sections describe the Determine Staff Needs and Performance Metrics operational subactivities in detail. 3.1.5.1 Determine Estimated NRFU Operational Workload by Geographic Area [NRFU 181.5.1] Based on estimates of the operational workload (NRFU cases) for geographic areas and staff ratios from NRFU, FLDI determines staffing needs. Table 11 lists the expected staffing ratios and national counts for NRFU field staff. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 90 Table 11: Expected NRFU Field Staff Ratios and Nationwide Counts Staff Expected Ratio Estimated Nationwide Count Census Field Manager (CFM) Approximately 8 CFMs located in each area census office (ACO) 1,401 Census Field Supervisor (CFS) Approximately 10 CFSs per CFM 15,000 Enumerator Approximately 20 enumerators per CFS 260,000 Geographic area estimates are on a national, RCC, ACO, block group, and tract level. NRFU estimates the workload based on input from stakeholders, results from previous censuses and census tests3, identified self-response rates, and geographic characteristics. FLDI uses the NRFU contact strategies and business rules to develop staffing models and recruiting requirements. FLDI provides the staff requirements to DLM for determining the number of kits and to ITIN for procuring devices. 3.1.5.2 Define Operational Performance Metrics and Targets [NRFU 18-1.5.2] During planning and preparation, NRFU defines metrics and targets for different administrative levels of NRFU. Through Unified Tracking System (UTS) reports, MOJO Hermes reports and ECaSE OCS progress tracking, headquarters (HQ) staff measure the performance of the RCCs for progress and cost of the operation in their regions. The RCCs measure the performance of the ACOs in their region. An example of an ACO performance metric is the overall case closure rate, or the rate of resolution of noncompletion situations such as language barriers and respondent refusals. The CFMs measure the performance of their CFSs. Through ECaSE Field OCS reports, CFSs measure the status and progress of the enumerators. 3 Estimates from census site tests will take into account the lack of census advertising in the test environment. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 91 3.1.6 Prepare for and Deliver NRFU-Specific Training [18-1.6] The “Prepare for and Deliver NRFU-Specific Training” operational subactivity is subdivided into the following constituent activities:  Prepare for and Deliver NRFU-Specific Training [NRFU 18-1.6]. o Prepare for NRFU-Specific Training [NRFU 18-1.6.1]. o Deliver NRFU-Specific Training [NRFU 18-1.6.2]. The subsequent sections describe the Prepare for and Deliver NRFU-Specific Training operational subactivities in detail. 3.1.6.1 Prepare for NRFU-Specific Training [NRFU 18-1.6.1] The “Prepare for NRFU-Specific Training” operational subactivity is subdivided into the following constituent activities:  Prepare for NRFU-Specific Training [NRFU 18-1.6.1]. o Provide NRFU-Specific Training Content and Requirements [NRFU 18-1.6.1.1]. o Review and Approve NRFU-Specific Training [NRFU 18-1.6.1.2]. The subsequent sections describe the Prepare for NRFU-Specific Training operational subactivities in detail. 3.1.6.1.1 Provide NRFU-Specific Training Content and Requirements [NRFU 18-1.6.1.1] To prepare for NRFU-specific training, NRFU provides FLDI with the requirements and course content for the training. FLDI develops the online and classroom training on NRFU-specific topics based on these requirements and the content. FLDI also develops online and classroom training on general topics such as administrative functions. NRFU provides training content and requirements to FLDI for the NRFU-specific training of field staff such as:     V2.0 Enumerators. Census Field Supervisors (CFSs). Census Field Managers (CFMs). Area Census Office Managers (ACOMs). July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 92 FLDI uses these requirements and content to develop NRFU-specific classroom training materials and online training modules. FLDI also develops and approves the general field staff training that is provided in conjunction with the NRFU-specific training. 3.1.6.1.2 Review and Approve NRFU-Specific Training [NRFU 18-1.6.1.2] NRFU reviews the training materials that FLDI develops, provides the venues for the training based on the training schedule that FLDI provides, and delivers the actual classroom training for the NRFU-specific topics. 3.1.6.2 Deliver NRFU-Specific Training [NRFU 18-1.6.2] FLDI establishes nationwide training dates to ensure the staff are trained before NRFU begins. NRFU seeks free public space to deliver classroom trainings. In accordance with the training schedule and using the training materials developed by FLDI and printed by DLM, NRFU conducts training that includes classroom instruction for:    Enumerators using ECaSE ENUM to record the status of assigned addresses and, if occupied, enumerating those households. CFSs using the automated ECaSE Field OCS to support and monitor enumerators and complete the enumeration workload. CFMs using the automated ECaSE Field OCS to support and monitor CFSs as they support the Enumerators and complete the enumeration workload. CFSs first receive enumerator training as part of their training. They then are responsible for administering that training to enumerators. FLDI is responsible for administering the general and NRFU-specific online training that is a prerequisite of the NRFU classroom training. See Appendix G for more information on the NRFU training plan. NRFU Operational Workload Management [NRFU 18-2] Figure 13 shows the BPM for the NRFU Operational Workload Management [NRFU 18-2] activity area (area within the gray rounded rectangle) and its constituent activities within the overall context of NRFU. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 93 2020 Census 18. Nonresponse Followup Operation (NRFU) Purpose: To determine housing unit status for nonresponding addresses and to enumerate housing units for which a 2020 Census response was not received. Business Rules & Contact Strategies NRFU Planning 18-1 and Preparation Develop 10 NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules NRFU Operational Management and Data Collection 18-1.1 18-2.2.1 Develop and Approve Field Printed Materials and Kit Specs Monitor Progress and Resolve Issues Develop 14 Requirements for and Approve Systems Supporting NRFU Create Work Assignments (incl. Grouped Manager Visit Cases) Receive NRFU Universe and Updates from RPO 18-2.2.3 18-2.2.2 Business Process Modeling & Analysis Toolkit 18-1.3 Case Status/ Response/ Address Data to RPO Perform NRFU Data Collection NRFU Begins 18-1.5 18-3.1 Receive Work Assignment 18-3.2 Perform Field Verification Data Collection 18-3.3 Perform NRFU Manager Visit 18-3.4 Perform NRFU Case Enumeration 18-3.5 Perform NRFU Reinterview Data Collection 18-3.6 Closeout Contact Attempt 18 Prepare for and Deliver NRFUSpecific Training 55 Receive Automated Quality Control Comparison Results from RPO 18-1.6 RI QC Has Begun Page-1 70 Submit Time and Expense Data Operation Closeout Criteria Met? Paradata to PM 18-1.4 16 Determine Staff Needs and Performance Metrics Includes: Early NRFU NRFU Production Field Verification Re-collect Manager Visits In-field Adds 45 50 Start 18-2.1 18-2.2 Receive and Process NRFU Universe and Updates 35 18-1.2 12 Identify Areas for Early NRFU Perform In-Office Adds NRFU Universe/ Workload Updates 11 Early NRFU Areas 38 Case Status Data to RPO (e.g. Rework Needed) 30 18-2 NRFU Operational Workload Management 18-4.1 QC Computer Match Anomaly? Yes 18-3.7 18-4.2 Closeout NRFU Operation End No (Repeat Daily) 18-5.1 RI Decision Outcome to RPO 60 No 80 Yes 18-3 NRFU Data Collection Adjudicate Reinterview Results 18-5 NRFU Operation Closeout 18-4 NRFU Quality Control Adjudication 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Page 1 of 10 Figure 13: NRFU Operational Workload Management [NRFU 18-2] Constituent Activities The NRFU Operational Workload Management activity area is subdivided into the following operational subactivities:  NRFU Operational Workload Management [NRFU 18-2]. o Monitor Progress and Resolve Issues [NRFU 18-2.1]. o Receive and Process NRFU Universe and Updates [NRFU 18-2.2]. NRFU receives its initial universe from RPO, as well as continuous updates to the universe throughout the operation. As cases are worked, NRFU monitors progress and resolves operational and performance issues. The subsequent sections describe the NRFU Operational Workload Management operational subactivities in detail. 3.2.1 Monitor Progress and Resolve Issues [NRFU 18-2.1] Monitoring progress and resolving issues includes the following subactivities: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)     Page 94 HQ, RCC, and ACO review of cost, progress, and status. This review includes comparison of current versus projected cost, progress, and status; identifying offices that require follow-up; and inquiry through UTS, MOJO Hermes and ECaSE FOCS reporting to find operational outliers, identify areas of concern, and compare progress toward defined performance metrics such as: o CFM review of noncompletion conditions such as a language or hearing barrier, respondent refusal, and enumerator-identified dangerous situation. For respondent refusals and enumerator-identified dangerous situations, the CFM may determine that the best action is to send another enumerator to make an attempt or to close the case. In a language barrier situation, ECaSE Field OCS reassigns the case to the closest enumerator in the ACO who speaks that language, if one exists. Otherwise, the CFM determines the best course of action to enumerate the household. If an enumerator does not exist in the ACO with that language capability, the CFM may ask the Area Census Office Manager (ACOM) to escalate the request to the RCC. o CFS review of enumerators using ECaSE Field OCS alerts displayed on their mobile device (i.e., tablet). Alerts are generated when performance and conduct metrics are not being met by the enumerator. Alerts include performance and conduct indicators such as too many refusals and potential overcharge of mileage. o CFM review of CFS using ECaSE Field OCS alerts. Alerts include such performance and conduct indicators as low alert resolution rates, and performance indicators in the previous bullet, aggregated to all of the staff reporting to the CFS. Integration with FLDI about staffing needs and employee performance and conduct, including terminating poor performers, and identifying quality enumerators and assigning them to enumerate difficult cases. Resolution of potential apartment mixups. Monitoring progress and resolving issues relating to ongoing Coverage Improvement (CI) activities. Delivery of data to RPO includes the following:    V2.0 Response data of completed enumeration. Event data of the results of every NRFU attempt. Paradata associated with NRFU enumeration. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 95 NRFU provides paradata and NRFU operational workload information to the Program Management operation (PM) for it to use in managing performance across the 2020 Census. 2020 Census 18. Nonresponse Followup Operation (NRFU) 3.2.2 Receive and Process NRFU Universe and Updates [NRFU 18-2.2] o determine housing unit status for nonresponding addresses and to enumerate housing units for which a 2020 Census response was not received. A detailed view of the constituent activities that make up the “Receive and Process NRFU lanning 18-1 NRFU Operational Management and Data Collection Universe and Updates” operational subactivity is given in Figure 14 below. paration 30 Case Status 18-2 NRFU Operational op 10 dRec ing ents, ct , and Rules 18-1.1 op 14 nts for rove ms NRFU Includes: Early NRFU NRFU Production Field Verification Re-collect Manager Visits In-field Adds 45 Create Work Assignments (incl. Grouped Manager Visit Cases) Receive NRFU Universe and Updates from RPO 18-2.2.3 18-2.2.2 Business Process Modeling & Analysis Toolkit 18-1.3 Case Status/ 18-1.4 Address Data to RPO Submit Time and Operation Data Universe Collection Expense Data is subdivided into the Closeout The Receive and Process NRFU and Updates activity area Criteria Met? Paradata following operational subactivities: to PM 18-3.7  NRFU Begins 18-1.5 18-3.1 Receive Work Assignment 18-3.2 Perform Verification Collection[NRFU 18-2.2]. Receive and Process NRFUField Universe andData Updates 18-3.3 Perform NRFU Manager Visit 18-3 NRFU Data 18-3.4 Perform NRFU Case Enumeration Collection o Perform In-Office [NRFU 18-2.2.1]. 18-3.5 Perform Adds NRFU Reinterview Data Collection 18-3.6 Closeout Contact Attempt QC Computer No (Repeat Daily) RI Decision 55 Match Anomaly? 60 Outcome to ReceiveAssignments Automated o Create Work (incl. Grouped Manager Visit Cases) [NRFU 18-2.2.3]. RPO Quality Control 18-1.6 Comparison Results from RPO Yes Adjudicate Reinterview Results The subsequent sections describe the Receive and Process NRFU and Updates 18-4Universe NRFU Quality RI QC Has Begun No Control operational subactivities in18-4.1 detail. 18-4.2 Adjudication 3.2.2.1 Perform In-Office Adds [NRFU 18-2.2.1] 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v5.2b.vsd It is expected that some geographic areas or new construction might not be included in the original enumeration universe. Should this situation occur, in-office adds is the mechanism to include these addresses into the NRFU workload so that an enumerator can visit the HU. V2.0 July 12, 2019 8 Yes o Receive NRFU Universe and Updates from RPO [NRFU 18-2.2.2]. 18 or and RFUaining NRFU Operation Closeout Figure 14: Receive and Process NRFU Universe and Updates Response/ 50 70 Perform NRFU 16 Staff and ance s 18-2.1 18-2.2 Receive and Process NRFU Universe and Updates 35 18-1.2 12 eas for RFU Perform In-Office Adds NRFU Universe/ Workload Updates 11 and Field aterials pecs 38 18-2.2.1 Data to RPO (e.g. Rework Needed) Monitor Progress and Resolve Issues Workload Management Final Closeout NRFU Operation 18-5 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 96 3.2.2.2 Receive NRFU Universe and Updates from RPO [NRFU 18-2.2.2] The enumeration universe is the complete set of addresses for living quarters that are to be enumerated for the 2020 Census. The NRFU universe is a subset of the enumeration universe. It comprises the set of addresses for living quarters that are HUs in the Self-Response Type of Enumeration Area (TEA) and the Update Leave TEA for which the Census Bureau has not yet received a response. The NRFU workload is initially the same as the NRFU universe and declines as cases are worked. Specific cases may also be added to the workload as they are detected during other operations, for instance:      Field Verification cases from Non-ID Processing operation. Housing units identified in the Group Quarters and Enumeration at Transitory Locations field operations. Late adds to the NRFU universe such as from Local Update of Census Addresses operation (LUCA). Blank or insufficient forms received from Paper Data Capture operation. Adds within NRFU such as hidden units discovered during NRFU. ECaSE Field OCS receives the NRFU universe from RPO. For Early NRFU, which begins on April 3, the universe includes nonresponding addresses for college and university students and staff living off-campus who require early enumeration before they leave their housing units for the semester. NRFU contact strategies and business rules are applied to that universe. In mid-May, RPO sends the full production universe of addresses to NRFU, which includes the results of administrative records modeling. The application of the administrative records modeling sets a modeled status for each NRFU case as Occupied, Vacant, Delete, or No Determination. The NRFU production universe includes the following types of cases:     Manager visit cases [See following subsection.]. Early NRFU and NRFU production cases [See Section 3.1.3]. QC-related cases (reinterview and rework) [See Section 3.1.1.3]. Field verification cases [See Section 3.3.2]. RPO updates the universe to include: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)     Page 97 Removal of self-responding addresses and other work status changes, for example, stop works and temporary holds on cases. More current administrative records modeling information. Characteristic updates for cases, for example, language requirements for enumeration, dangerous situations, etc. Reinterview cases and cases identified for rework based on QC results. A reinterview case is assigned to a different enumerator from the one who conducted the original interview. Updates to the universe including the application of AdRec modeling outcomes are made on a recurring basis throughout the course of the operation. 3.2.2.3 Create Work Assignments (incl. Grouped Manager Visit Cases) [NRFU 18-2.2.3] ECaSE Field OCS, in conjunction with the assignment optimizer, creates work assignments from the NRFU universe, based on the following inputs:         Applicable operational business rules, for example business rules that determine whether a reinterview case is assigned to the call center provided by CQA or to the field. Field staff profile that includes starting address and language capability. Work availability entered by field staff. Geographic location of all eligible cases. Case statuses from previous interviews. Information collected from previous contact attempts such as language spoken by the household. Previously collected best time to contact the household. Administrative records modeling of best time to contact. The optimizer is an algorithm designed to minimize cost in terms of time and mileage required to complete the operation. Multiunits (MUs) require special assignment processing. To make the enumeration of MU structures more efficient, nonresponding units in applicable multiunit structures are grouped for a manager visit. This occurs for both early NRFU and production NRFU. For MUs, an enumerator attempts the manager visit before any individual unit attempts. During a manager interview, an enumerator attempts to determine the Census Day status (occupied, vacant, or V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 98 delete) of each nonresponding unit in the building. Section 3.3.3 describes the results of performing a manager visit. ECaSE Survey OCS determines whether to group the units into a manager visit case depending on both the total number of units in the multiunit structure and the number of NRFU production cases in a multiunit. Each day, the assignment optimizer assigns groups to an enumerator based on the enumerator’s availability and the projected time it takes to complete the nonresponding cases within the group. Note that Reinterview and Field Verification cases are not included in manager visit grouping. NRFU Data Collection [NRFU 18-3] Figure 15 shows the BPM for the NRFU Data Collection [NRFU 18-3] activity area (area within the gray rounded rectangle) and its constituent activities within the overall context of NRFU. 2020 Census 18. Nonresponse Followup Operation (NRFU) Purpose: To determine housing unit status for nonresponding addresses and to enumerate housing units for which a 2020 Census response was not received. Business Rules & Contact Strategies NRFU Planning 18-1 and Preparation Develop 10 NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules NRFU Operational Management and Data Collection 18-1.1 18-2.2.1 Develop and Approve Field Printed Materials and Kit Specs Monitor Progress and Resolve Issues Create Work Assignments (incl. Grouped Manager Visit Cases) Receive NRFU Universe and Updates from RPO 18-2.2.3 18-2.2.2 Business Process Modeling & Analysis Toolkit 18-1.3 Case Status/ Response/ Address Data to RPO Perform NRFU Data Collection 14 Develop Requirements for and Approve Systems Supporting NRFU NRFU Begins 18-1.5 18-3.1 Receive Work Assignment 18-3.2 Perform Field Verification Data Collection 18-3.3 Perform NRFU Manager Visit 18-3.4 Perform NRFU Case Enumeration 18-3.5 Perform NRFU Reinterview Data Collection 18-3.6 Closeout Contact Attempt 18 Prepare for and Deliver NRFUSpecific Training 55 Receive Automated Quality Control Comparison Results from RPO 18-1.6 RI QC Has Begun Page-1 70 Submit Time and Expense Data Operation Closeout Criteria Met? Paradata to PM 18-1.4 16 Determine Staff Needs and Performance Metrics Includes: Early NRFU NRFU Production Field Verification Re-collect Manager Visits In-field Adds 45 50 Start 18-2.1 18-2.2 Receive and Process NRFU Universe and Updates 35 18-1.2 12 Identify Areas for Early NRFU Perform In-Office Adds NRFU Universe/ Workload Updates 11 Early NRFU Areas 38 Case Status Data to RPO (e.g. Rework Needed) 30 18-2 NRFU Operational Workload Management 18-4.1 QC Computer Match Anomaly? Yes 18-3.7 60 No 18-4.2 80 Yes 18-3 NRFU Data Collection Adjudicate Reinterview Results 18-5 NRFU Operation Closeout Closeout NRFU Operation End No (Repeat Daily) 18-5.1 RI Decision Outcome to RPO 18-4 NRFU Quality Control Adjudication 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Page 1 of 10 Figure 15: NRFU Data Collection [NRFU 18-3] Constituent Activities V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 99 As shown in the figure above, the NRFU data collection activity area has seven operational subactivities, the first six of which fall within the first box (Perform NRFU Data Collection). These six activities are listed below:  NRFU Data Collection [NRFU 18-3]. o Receive Work Assignment [NRFU 18-3.1]. o Perform Field Verification Data Collection and Quality Control [NRFU 18-3.2]. o Perform NRFU Manager Visit [NRFU 18-3.3]. o Perform NRFU Case Enumeration [NRFU 18-3.4]. o Perform NRFU Reinterview Data Collection [NRFU 18-3.5]. o Closeout Contact Attempt [NRFU 18-3.6]. o Submit Time and Expense Data [NRFU 18-3.7]. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 100 A detailed view of the activities that fall within the Perform NRFU Data Collection activity is shown in Figure 16 below. 50 Perform NRFU Data Collection Business Process Modeling & Analysis Toolkit Purpose: To collect responses from respondents. Case notes may be updated at any point. 18-3 NRFU Data Collection (In Part) 18-3.2 Perform Field Verification Data Collection and Quality Control 50.20 Perform Field Verification Data Collection and Quality Control Verification Case Status:  Verified  Delete  Other 18-3.3 Perform NRFU Manager Visit 50.30 Is Individual Unit Eligible for Enumeration? Perform NRFU Manager Visit Determine Case Type FV Grouped Multiunit 18-3.1 Receive Work Assignment Case Status/ Response/ Address Data Paradata 18-3.5 Perform NRFU Reinterview Data Collection Receive Work Assignment Start Sub-process 50.80 Perform Manager Review Process No Yes 50.10 18-3.6 Closeout Contact Attempt This includes any fieldadds 50.60 Early NRFU NRFU Prod NRFU Rework NRFU RI Yes 50.70 Perform NRFU Reinterview Data Collection Is Manager Review Required? Closeout Contact Attempt No 50.40 18-3.4.1 Enumerate Housing Unit Case Complete? Yes End Sub-process No 50.50 18-3.4.2 Enumerate Housing Unit via Proxy Interview 18-3.4 Perform NRFU Case Enumeration Page-1 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Proxy Eligible? No Yes Proxy Available? No Yes Page 1 of 10 Figure 16: Perform NRFU Data Collection 3.3.1 Receive Work Assignment [NRFU 18-3.1] Work assignments sent by ECaSE Field OCS are received by the enumerators in ECaSE ENUM on their smartphones daily. Work assignments for an enumerator on a day can include all types of cases, including production case enumeration, field verification, MVs, reinterviews for QC and SRQA recollect. Enumerators are expected to follow their list of cases in order. Cases are not added once an assignment is made, but cases may be removed if a self-response is received or a case is placed on hold. Enumerators can switch from a view of the case list to a map with the location of each corresponding case, with the highlighted case on the case list highlighted on the map. When V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 101 selecting a production or reinterview case from the case list, enumerators have the opportunity and are expected to view address details, contact history, and any Case Notes already submitted from previous contact attempts before they begin the case. NRFU Case Enumeration is described in Section 3.3.4. 3.3.2 Perform Field Verification Data Collection and Quality Control [NRFU 18-3.2] In addition to a production or reinterview case type, an enumerator may have a Field Verification case on the case list. Cases that had self-responded without a Census ID and could not be matched to the Census address frame (through either automated matching or clerical review processes) may be included in the NRFU workload for field verification. The purpose of this data collection is to verify by observation if the address in question actually exists in the field. It does not require an interview with a respondent. These cases should inherently only receive a single attempt at case resolution. When a Field Verification case has been given status by an enumerator, it should be removed from the operational workload. There is also a quality control element as some Field Verification cases are sent out again for a re-verification. If there is a mismatch between the original and QC (one enumerator could find the location and the other one could not), a second QC attempt resolves it. 3.3.3 Perform NRFU Manager Visit [NRFU 18-3.3] A detailed view of the constituent activities that make up the “Perform NRFU Manager/Manager Visit” operational subactivity is given in Figure 17 below. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 102 Figure 17: Perform NRFU Manager Visit In addition to production, reinterview, and Field Verification cases, an enumerator may have manager visit cases on a case list. Manager visits are intended to improve efficiency by collecting status of nonresponding units from the manager of a multiunit structure before individual unit enumeration is attempted. Before creating work assignments, ECaSE Field OCS groups the units in a multiunit into a manager visit case based on the criteria described in Section 3.2.2. During a manager visit, an enumerator attempts to interview the manager to determine the Census Day status of each nonresponding unit (such as occupied or vacant). For each unit that a manager indicates is vacant, the case for that unit is automatically completed as a vacant by proxy. If the manager indicates that a unit was occupied, the unit is released as a case for an enumeration attempt. If a unit was neither vacant nor occupied, for example, uninhabitable on Census Day, it is automatically completed as “not a housing unit” by V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 103 proxy. Any units that are not completed during the manager visit are released as NRFU production cases for enumeration attempts. When a manager visit attempt fails because the enumerator cannot locate a manager to interview, the enumerator indicates in ECaSE ENUM one of the following situations:   Manager was not available during this particular attempt. Manager does not exist (i.e., will never be available to be interviewed). If no manager exists to be interviewed, ECaSE Field OCS should not group the units in a manager visit in the future. If a manager was just not available for an attempt, the units are eligible for regrouping if all grouping criteria are met. 3.3.4 Perform NRFU Case Enumeration [NRFU 18-3.4] A detailed view of the constituent activities that make up the “Perform NRFU Case Enumeration” operational subactivity is given in Figure 18 below. Figure 18: Perform NRFU Case Enumeration V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 104 The “Perform NRFU Case Enumeration” operational subactivity is subdivided into the following constituent activities:  Perform NRFU Case Enumeration [NRFU 18-3.4]. o Enumerate Housing Unit [NRFU 18-3.4.1]. o Enumerate Housing Unit via Proxy Interview [NRFU 18-3.4.2]. The subsequent sections describe the Perform NRFU Case Enumeration operational subactivities in detail. 3.3.4.1 Enumerate Housing Unit [NRFU 18-3.4.1] A detailed view of the constituent activities that make up the “Enumerate Housing Unit” operational subactivity is given in Figure 19 below. Figure 19: Enumerate Housing Unit When attempting to initiate contact, an enumerator may encounter an address that has a special status. Examples of special case statuses are:   V2.0 Unable to locate. Does not exist. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)       Page 105 Demolished/Burned out. Nonresidential. Uninhabitable. Empty mobile home. Restricted access. Dangerous address. ECaSE ENUM enables the enumerator to indicate this special case status when they encounter it. Depending on business rules, the enumerator may immediately seek a proxy for an address for certain special case statuses such as Unable to Locate and Does Not Exist. Some special cases such as Does Not Exist and Nonresidential are completed as a “Delete” after two different enumerators indicate the same special case status if a proxy interview was not successful. For proxy interviews see Section 3.3.5. If a reinterview is selected for such a case, the reinterview is attempted at the proxy address. An enumerator attempts to locate the address for each case and contact a household member at the address. For a production case, if a household member contact attempt is unsuccessful, based on business rules, the enumerator may attempt to interview a proxy for the household to collect the information. A proxy would be someone who may have sufficient knowledge to enumerate the original household such as a neighbor or a caregiver [see Section 3.3.5]. ECaSE ENUM enables the enumerator to toggle to screens in another language in addition to English, such as Spanish. If the respondent speaks that language and the enumerator has the language capability, the enumerator can toggle to that language to conduct the interview. When an enumerator attempts to collect a response from a contact, one of the following situations may occur that prevents the interview from beginning:    Communication barrier such as a hearing or language barrier. Refusal to respond for example, because of privacy concerns. Dangerous situation, such as the respondent threatens the enumerator. In each of these situations, the case is marked with the reason to trigger the appropriate action, such as reassigning the case to an enumerator who speaks that language. The respondent may also refuse because it is an inconvenient time. In this situation, the enumerator asks the respondent to provide a best day and time of day as contact information for a subsequent V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 106 attempt. In each of these cases, the enumerator provides an NOV to encourage self-response through the internet or CQA. These marked cases undergo manager reviews as part of resolving issues during the NRFU field operation. Leave Notice of Visit or Phone Message: For an unsuccessful personal visit attempt, the enumerator writes the Case ID on a NOV form and leaves the NOV in an inconspicuous place at the entrance of the attempted address. The NOV directs the household to respond to the Census, via either ECaSE ISR or a centralized telephone interview service provided by CQA staff. A screen in ECaSE ENUM asks the enumerator if they left a NOV; some conditions exist when it would not be appropriate. For example, an enumerator would not leave a NOV in the following situations:      A NOV is already present from a previous attempt. The enumerator determines that the address is not able to be located. The enumerator determines that the address is nonresidential or demolished. The enumerator encounters a dangerous situation. The case is a reinterview case. Begin Interview for Enumeration: For a production case, if the address is occupied and an eligible, willing, and able respondent is identified, the enumerator begins the census questionnaire interview in ECaSE ENUM. The enumerator uses the instructions, reads the scripts, and enters the response data on the ECaSE ENUM screens. As part of the interview, the respondent is asked to provide contact information (a phone number and best day and time of day to contact) for a potential reinterview for QC. In-mover: If a respondent indicates that they did not live at the attempted address on Census Day, the respondent is an in-mover. The enumerator asks if the in-mover knows information about the household members who did live at that address on Census Day. If so, the in-mover is interviewed as a proxy for the Census Day household. The in-mover is also asked if someone completed a census questionnaire where the in-mover lived on Census Day. If no one completed the questionnaire or the in-mover does not know if anyone did, the in-mover’s Census Day address is collected, and a new case is created and enumeration attempted for that address. Status is sent to RPO so that the case will be checked later to ensure that it is not a duplicate. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 107 3.3.4.2 Enumerate Housing Unit via Proxy Interview [NRFU 18-3.4.2] A detailed view of the constituent activities that make up the “Enumerate Housing Unit via 50.50 Enumerate Housing Unit via Proxy Interview Purpose: To attempt to collect responses a suitable Proxy Interview” operational subactivity is given infrom Figure 20proxy. below. Business Process Modeling & Analysis Toolkit 18-3.4 Perform NRFU Case Enumeration (In Part) 18-3.4.2 Enumerate Housing Unit via Proxy Interview 50.50.20 Mark Case as Communication Barrier 50.50.30 50.50.60 Mark Case as Refusal Occupied on Census Day? No Determine Unit Status as of Census Day End Sub Process Yes Yes Yes No Yes 50.50.10 Attempt Interview Start Sub Process 50.50.70 50.50.50 No Is There a Communication Barrier? Yes No Refusal? Is Eligible Respondent Contacted? No Begin Interview Complete Enumeration of HU Is there a Special Case Status Condition? Special Case Status being verified as of Census Day includes:  Vacant Unit  Does not Exist  Demolished/Burned Out  Nonresidential  Uninhabitable  Empty Mobile Home Figure 20: Enumerate Housing Unit via Proxy Interview Page-1 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Page 1 of 10 Enumerators attempt to locate the address for each case assigned to them from the NRFU workload and contact a household member at the address. However, in certain situations, the enumerator may attempt to interview a proxy. A proxy is a respondent who may have sufficient knowledge to enumerate the original household. Cases become proxy-eligible on the third unsuccessful personal visit attempt at a housing unit. However, there are a few cases that immediately become proxy-eligible. First, if a respondent indicates that they did not live at the attempted address on Census Day, the respondent is considered an in-mover and treated as a proxy. If the in-mover has sufficient information regarding the people who lived at the case address on Census Day, then the case is closed. If not, the case remains proxy-eligible. However, a proxy alert flag on the device will inform all future enumerators to not contact the in-mover again. Second, if an enumerator encounters a dangerous situation at the assigned case address, the case immediately becomes proxy-eligible. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 108 Furthermore, if an enumerator determines that a case is a special case status, depending on business rules, then the enumerator should immediately seek a proxy. The special case statuses that result in an immediate proxy attempt include:        Vacant by Observation. Does not Exist. Demolished/Burned out. Nonresidential. Uninhabitable. Empty mobile home. Restricted access. Lastly, if an enumerator encounters a nonresident at the case address, the case becomes proxy eligible if:    The nonresident is a personal caretaker or family member. The resident at the case address is unavailable or unable to complete the enumeration. The nonresident can provide the appropriate information needed on the resident(s) of the case address. Proxy Enumeration: except for changes in question wording because of interviewing a person not part of the household, a proxy enumeration attempts to collect all the information of a regular NRFU case. It is expected that proxy data are not as reliable as self-reported. However, proxy data are preferable to relying on imputation. 3.3.5 Perform NRFU Reinterview Data Collection [NRFU 18-3.5] A detailed view of the constituent activities that make up the “Perform NRFU Reinterview Data Collection” operational subactivity is given in Figure 21 below. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 109 Figure 21: Perform NRFU Reinterview Data Collection Conduct Reinterview: A case in an enumerator’s case list may be a reinterview of a production case based on the sample selection described in Section 3.1.1.3. At the beginning of a reinterview case, the enumerator asks for the original respondent. If a respondent states that they, or a member of the household, have been interviewed before, only a roster for the household is collected for comparison with the original interview. If not, a complete interview is conducted. 3.3.6 Closeout Contact Attempt [NRFU 18-3.6] A detailed view of the constituent activities that make up the “Closeout Contact Attempt” operational subactivity is given in Figure 22 below. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 110 Figure 22: Closeout Contact Attempt The “Closeout Contact Attempt” operational subactivity is subdivided into the following constituent activities:  Closeout Contact Attempt [NRFU 18-3.6]. o Update Case Information [NRFU 18-3.6.1]. o Send Data [NRFU 18-3.6.2]. At the end of a production, reinterview, field verification, or manager visit case attempt, the enumerator records any relevant attempt or case information in Case Notes. ECaSE ENUM updates the case information and classifies the attempt based on the results. Status, paradata, and any collected data are sent to the appropriate operations. The subsequent sections describe the Closeout Contact Attempt operational subactivities in detail. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 111 3.3.6.1 Update Case Information A detailed view of the constituent activities that make up the “Update Case Information” operational subactivity is given in Figure 23 below. Figure 23: Update Case Information At the end of a case attempt, the enumerator enters any information about the case in Case Notes for any subsequent enumerators if other attempts are necessary. By selecting a case on the case list, an enumerator can view any previously entered Case Notes for background information. Examples of Case Notes are the description of a site, information about a dog on the premises, etc. In addition, the CFS and CFM may review Case Notes when researching information pertaining to a specific interview when required. The contact attempt is updated with a classification such as not attempted, attempted, completed with household, completed with proxy, vacant, or delete. An interview is determined to be complete if it meets the sufficiency criteria defined by RPO. Based on updated contact attempts, a case may become proxy-eligible. 3.3.6.2 Send Data A detailed view of the constituent activities that make up the “Send Data” operational subactivity is given in Figure 24 below. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 112 Figure 24: Send Data After each completed attempt (including completed interviews), NRFU sends the following types of data to RPO:  V2.0 Status (operational data): Response status describes the result of the attempt, for example, no one home, insufficient partial, or completed interview with the household. These data determine the next action for the attempt, if needed. Other operational data include the language that the interview was conducted in, whether an interpreter was present, proxy eligibility status, and Case Notes. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)   Page 113 Response data: Response data are any questionnaire responses collected in an interview, such as housing unit (HU) status, household characteristics (rent/own, population count), and person characteristics (sex, race, undercount/overcounts questions). Response data also include Don’t Know/Refuse selection and contact information. Paradata: Paradata are auxiliary data that provide information about the data collection process. Paradata include, for instance, time spent on an interview, the path through ECaSE ENUM, what specific keys were pressed, length of time on each question, how often help functionality was used, etc. After a completed attempt, NRFU sends this type of data to PM (to the paradata repository in UTS). The operational data that are sent to RPO determine the next action for the case, for example, another contact attempt or reinterview or whether the case should be removed from future follow-up because the maximum number of attempts has been reached. Status also includes address updates and address verification. This status might include classification as a Group Quarter or Transitory Location that RPO would then route to the appropriate operation. For completed and sufficient partial cases, the response data are sent to RPO for processing. PM tracks paradata for analysis of quality of data and improvement of future census operations. 3.3.7 Submit Time and Expense Data [NRFU 18-3.7] Each day the enumerator must submit the hours they worked and all expenses accrued during the day, including mileage, parking, tolls, or mass transit expenses. This is the last activity of the day that an enumerator performs. Refer to Figure 15 shown earlier to see where this subactivity fits into the NRFU Data Collection activity area. NRFU Quality Control Adjudication [NRFU 18-4] Figure 25 shows the BPM for the NRFU Quality Control Adjudication activity area (area within the gray rounded rectangle) and its constituent activities within the overall context of NRFU. Quality Control occurs for both NRFU enumeration and Field Verification cases. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 114 2020 Census 18. Nonresponse Followup Operation (NRFU) Purpose: To determine housing unit status for nonresponding addresses and to enumerate housing units for which a 2020 Census response was not received. Business Rules & Contact Strategies NRFU Planning 18-1 and Preparation Develop 10 NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules NRFU Operational Management and Data Collection 18-1.1 Develop and Approve Field Printed Materials and Kit Specs Monitor Progress and Resolve Issues Develop 14 Requirements for and Approve Systems Supporting NRFU Create Work Assignments (incl. Grouped Manager Visit Cases) Receive NRFU Universe and Updates from RPO 18-2.2.3 18-2.2.2 Business Process Modeling & Analysis Toolkit 18-1.3 Case Status/ Response/ Address Data to RPO Perform NRFU Data Collection NRFU Begins 18-1.5 18-3.1 Receive Work Assignment 18-3.2 Perform Field Verification Data Collection 18-3.3 Perform NRFU Manager Visit 18-3.4 Perform NRFU Case Enumeration 18-3.5 Perform NRFU Reinterview Data Collection 18-3.6 Closeout Contact Attempt 18 Prepare for and Deliver NRFUSpecific Training 55 Receive Automated Quality Control Comparison Results from RPO 18-1.6 RI QC Has Begun Page-1 70 Submit Time and Expense Data Operation Closeout Criteria Met? Paradata to PM 18-1.4 16 Determine Staff Needs and Performance Metrics Includes: Early NRFU NRFU Production Field Verification Re-collect Manager Visits In-field Adds 45 50 Start 18-2.1 18-2.2 Receive and Process NRFU Universe and Updates 35 18-1.2 12 Identify Areas for Early NRFU Perform In-Office Adds NRFU Universe/ Workload Updates 11 Early NRFU Areas 38 18-2.2.1 Case Status Data to RPO (e.g. Rework Needed) 30 18-2 NRFU Operational Workload Management 18-4.1 QC Computer Match Anomaly? Yes 18-3.7 Adjudicate Reinterview Results No 18-4.2 80 Yes 18-3 NRFU Data Collection 60 18-5 NRFU Operation Closeout Closeout NRFU Operation End No (Repeat Daily) 18-5.1 RI Decision Outcome to RPO 18-4 NRFU Quality Control Adjudication 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Page 1 of 10 Figure 25: NRFU Quality Control Adjudication [NRFU 18-4] Constituent Activities The NRFU Quality Control Adjudication activity area is subdivided into the following operational subactivities:  NRFU Quality Control Adjudication [NRFU 18-4]. o Receive Automated Quality Control Comparison Results from RPO [NRFU 18-4.1]. o Adjudicate Reinterview Results [NRFU 18-4.2]. The subsequent sections describe the NRFU Quality Control Adjudication operational subactivities in detail. 3.4.1 Receive Automated Quality Control Comparison Results from RPO [NRFU 18-4.1] As described in Section 3.1.1.3, a sample of production cases with completed interviews are selected for reinterviews. An enumerator attempts the reinterview case, and the status and response data are sent to RPO, which performs an automated comparison (i.e., a computer match) of the response data from the original interview and the response data from the reinterview. If the match falls within the threshold as specified in the NRFU Quality Control plan, the case is resolved positively and is not sent to clerical resolution. However, if the V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 115 computer match falls outside of the threshold, RPO sends the case to NRFU for clerical review and adjudication as described in the next subactivity. Field verification QC is only conducted by computer matching of the event codes. 3.4.2 Adjudicate Reinterview Results [NRFU 18-4.2] Business Process Modeling & Analysis Toolkit A detailed view of the constituent activities that make up the “Adjudicate Reinterview Results” 60 Adjudicate Reinterview Results Purpose: Tois compare data with operational subactivity givenreinterview in Figure 26 production below. data and determine reinterview outcome. 18-4.1 NRFU Quality Control Adjudication (In Part) 18-4.1.2 Adjudicate Reinterview Results RI Outcome Can Include Supplemental RI Selection 60.10 Start Sub-process Hard Fail Recommended? Perform Comparison/ Resolution Research and Make Recommendations (Clerical Resolution Staff) 60.20 No Set Final RI Outcome and Send to RPO End Sub-process Yes Clerical Recommendations:  Pass  Fail  Hard Fail - Enumerator Falsification 60.30 Perform Resolution Research (Field Staff) No 60.40 Yes Stop Work (Field Staff) Hard Fail Determination? Figure 26: Adjudicate Reinterview Results Page-1 For cases that do not pass the automated comparison, NRFU receives the following input from 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd RPO:   The identification of any cases for which RPO computer matching found anomalies. Linkage to any supplemental reinterview cases that were requested by the NRFU Clerical Resolution staff for more research on a previous investigation of a case with anomalies. For each case identified as having an anomaly, the Clerical Resolution staff at NPC compares the original and reinterview cases and performs research to make one of the following recommendations: V2.0 July 12, 2019 Final Page 1 of 10 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)    Page 116 Comparison should pass. Comparison should fail. Not enough information exists. When determining a recommendation, the Clerical Resolution staff may request more (known as Supplemental) cases by the same enumerator to be added to the reinterview selection sample for further investigation. The staff would then wait for the supplemental reinterviews to use for more investigation before making a recommendation. For a fail recommendation, the Clerical Resolution staff makes a further recommendation about whether the anomalies are the result of an unintentional error (Soft Fail) or enumerator falsification (Hard Fail). For a recommendation of Pass, Don’t Know, or Soft Fail, the staff sets the corresponding Reinterview Outcome and sends it to RPO. For a Hard Fail recommendation, the case is sent to the RCC staff for final determination. If the RCC staff determines that enumerator falsification did occur, a stop work status is issued for that enumerator, preventing further cases from being assigned to the enumerator. The Reinterview Outcome is set to Hard Fail for the case and sent to RPO. If the field staff determines that falsifying has not occurred, the corresponding Reinterview Outcome is also sent to RPO. A Reinterview Outcome of Hard Fail indicates to RPO the need to add some of the enumerator’s other production cases, depending on QC factors, into the reinterview sample (known as Rework). Any cases assigned to the Hard-Fail enumerator that have not yet been completed are reassigned to another enumerator. The additional reinterview cases added to the QC sample cause an update (addition of these reinterview cases) to the universe from RPO. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 117 NRFU Operation Closeout Figure 27 shows the BPM for NRFU Closeout [NRFU 18-5] activity area (area within the gray rounded rectangle) and its constituent activities within the overall context of NRFU. 2020 Census 18. Nonresponse Followup Operation (NRFU) Purpose: To determine housing unit status for nonresponding addresses and to enumerate housing units for which a 2020 Census response was not received. Business Rules & Contact Strategies NRFU Planning 18-1 and Preparation Develop 10 NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules NRFU Operational Management and Data Collection 18-1.1 18-2.2.1 Develop and Approve Field Printed Materials and Kit Specs Monitor Progress and Resolve Issues 18-1.2 Create Work Assignments (incl. Grouped Manager Visit Cases) Receive NRFU Universe and Updates from RPO 18-2.2.3 18-2.2.2 Business Process Modeling & Analysis Toolkit 18-1.3 Case Status/ Response/ Address Data to RPO Perform NRFU Data Collection 14 Develop Requirements for and Approve Systems Supporting NRFU NRFU Begins 18-1.5 18-3.1 Receive Work Assignment 18-3.2 Perform Field Verification Data Collection 18-3.3 Perform NRFU Manager Visit 18-3.4 Perform NRFU Case Enumeration 18-3.5 Perform NRFU Reinterview Data Collection 18-3.6 Closeout Contact Attempt 18 Prepare for and Deliver NRFUSpecific Training 55 Receive Automated Quality Control Comparison Results from RPO 18-1.6 RI QC Has Begun Page-1 70 Submit Time and Expense Data Operation Closeout Criteria Met? Paradata to PM 18-1.4 16 Determine Staff Needs and Performance Metrics Includes: Early NRFU NRFU Production Field Verification Re-collect Manager Visits In-field Adds 45 50 Start 18-2.1 18-2.2 Receive and Process NRFU Universe and Updates 35 12 Identify Areas for Early NRFU Perform In-Office Adds NRFU Universe/ Workload Updates 11 Early NRFU Areas 38 Case Status Data to RPO (e.g. Rework Needed) 30 18-2 NRFU Operational Workload Management 18-4.1 QC Computer Match Anomaly? Yes 18-3.7 18-4.2 Closeout NRFU Operation End No (Repeat Daily) 18-5.1 RI Decision Outcome to RPO 60 No 80 Yes 18-3 NRFU Data Collection Adjudicate Reinterview Results 18-5 NRFU Operation Closeout 18-4 NRFU Quality Control Adjudication 2020 Census 18. Nonresponse Followup Operation (NRFU) BPM v7.1.vsd Page 1 of 10 Figure 27: NRFU Operation Closeout [NRFU 18-5] Constituent Activities NRFU Closeout activity area has one subactivity as shown below:  NRFU Operation Closeout [NRFU 18-5]. o Closeout NRFU Operation [NRFU 18-5.1]. The NRFU closeout operation includes the following activities:   V2.0 Interacting with FLDI to reduce staff as workload reduces, including collecting materials such as devices. Performing operational assessments such as cost and progress reporting, lessons learned, and data analysis. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 118 4 Cost Factors Background Cost impacts of this operation on overall 2020 Census costs include the following: The investment in NRFU, which includes administrative records and third-party data (AdRec) usage and field reengineering, is projected to influence (reduce  or increase ) the 2020 Census overall costs in the following ways:  Reduced field workload by:     Using AdRec to reduce the number of contact attempts. Using AdRec to enumerate nonresponding HUs. Removing self-responses on a near-real-time basis. Interviewing managers of multiunit buildings to identify and remove vacant units from the Nonresponse Followup (NRFU) workload.  Improved productivity of field staff by:     Streamlining staffing structure through the use of automation. Automating and optimizing the assignment process. Using language information from the planning database to match enumerator language skills to neighborhood language needs. Using AdRec data to determine the best time of day for contact attempts.  Reducing the reinterview workload through a reengineered quality assurance approach.  Reducing the number of hours devoted to training through the use of automation. Cost Factors A list of major cost factors related to the NRFU operation includes the following:       V2.0 NRFU enumeration workload. NRFU field verification workload. Coverage improvement workload. Self-Response Quality Assurance (SRQA) Workload Duration of the operation. Geographic dispersion of workload (impacts number and location of offices). July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)  Page 119 Field staff: o Frontload rate. o Duration of production activities. o Productivity (cases/hour). o Miles traveled per case. o Supervisor to enumerator ratio. o Mileage reimbursement rate. o Staff wage - Production. o Staff wage - Training. o Resolve rate for each attempt. Relevant IDEF0 Mechanisms The following mechanisms from the IDEF0 Context Diagram represent the resources used to support this operation and therefore impact its cost: Staff      Headquarters (HQ) staff. Regional Census Center (RCC) staff. Area Census Office (ACO) staff. Field staff. National Processing Center (NPC) staff. Sites       HQ. RCC. ACO. Field Area. NPC. Training venues. Systems   V2.0 Enterprise Censuses and Surveys Enabling Platform Enumeration (ECaSE – ENUM). Enterprise Censuses and Surveys Enabling Platform Operations Control System (ECaSE – OCS). July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)         Page 120 Enterprise Censuses and Surveys Enabling Platform Field Operations Control System (ECaSE-FOCS). Sampling, Matching, Review, and Coding System (SMaRCS). Mobile Operations Job Optimizer: Browse Living Quarters (MOJO: BLQ). MOJO: Optimizer. MOJO: HERMES. Matching and Coding System (MaCS). Unified Tracking System (UTS). Learning Management System (LMS). Other      V2.0 Census networks. Mobile networks. Mobile devices. HQ Office IT infrastructure. Operation-specific kits. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 121 5 Measures of Success For the 2020 Census operations, the corresponding Measures of Success are documented in the operational assessment study plans and final reports. The operational assessment study plan documents the criteria that is used to define successful completion of the operation. The operational assessment report provides results on whether the criteria were met. In general, operational assessments report on planned to actual variances in budget, schedules, and production and training workloads. The corresponding Measures of Success (as documented in the operational assessment study plan) include variances that exceed established thresholds. See Preparing for the 2020 Census Operational Assessment Study Plan for the potential scope of assessment. Types of success measures include:    V2.0 Process Measures that indicate how well the process works, typically including measures related to completion dates, rates, and productivity rates. Cost Measures that drive the cost of the operation and comparisons of actual costs to planned budgets. Costs can include workload as well as different types of resource costs. Measures of the Quality of the results of the operation, typically including things such as rework rates, error rates, and coverage rates. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 122 Appendix A – Acronyms and Terminology Table 12 lists the acronyms and abbreviations used within this Detailed Operational Plan document. Table 13 lists a Glossary of Terms used within this Detailed Operational Plan document. Table 12: Acronyms and Abbreviations List Acronym Meaning ACO Area Census Office ACOM Area Census Office Manager ACS American Community Survey ADC Assistant Division Chief AdRec Administrative Records AMA Assistant Manager for Administration BC Bureau of Census (prefix used for paper forms) BCU Basic Collection Unit BLQ Browse Living Quarters BPM Business Process Model BPMN Business Process Model Notation CARRA Center for Administrative Records Research and Applications CDL Census Data Lake CEF Census Edited File CFD Content and Forms Design operation CFM Census Field Manager CFS Census Field Supervisor CI Coverage Improvement CMS Center for Medicare and Medicaid Services CQA Census Questionnaire Assistance operation CRO Count Review Operation V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 123 Acronym Meaning CSR Customer Service Representative CUF Census Unedited File DAPPS Decennial Applicant, Personnel, and Payroll System DCCO Decennial Communications Coordination Office DCMD Decennial Census Management Division DLM Decennial Logistics Management operation DOP Detailed Operational Plan DLM Data Products and Dissemination operation DPD Data Products and Dissemination operation DPMO Decennial Program Management Office DRF Decennial Response File DSC Decennial Service Center operation DSF Delivery Sequence File DSSD Decennial Statistical Studies Division E/S English / Spanish EAE Evaluations and Experiments operation ECaSE Enterprise Censuses and Surveys Enabling Platform ECaSE ENUM Enterprise Censuses and Surveys Enabling Platform Enumeration ECaSE FOCS Enterprise Censuses and Surveys Field Operations Control System ECaSE ISR Enterprise Censuses and Surveys Enabling Platform Internet SelfResponse ECaSE OCS Enterprise Censuses and Surveys Enabling Platform Operational Control System EEO Equal Employment Opportunity ENUM Enumeration ETL Enumeration at Transitory Locations operation EV1 Evaluation 1 EV2 Evaluation 2 EV3 Evaluation 3 FAQ Frequently Asked Questions V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 124 Acronym Meaning FLD Field Division FLDI Field Infrastructure operation FOCS Field Operational Control System FPD Forms Printing and Distribution operation FV Field Verification GDP Geographic Data Processing GEO Geography Division GEOP Geographic Programs operation GEOP/GD Geographic Programs operation / Geographic Delineations component GEOP/GDP Geographic Programs operation / Geographic Data Processing component GPS Global Positioning System GQ Group Quarters or Group Quarters operation HH Household HQ Headquarters HR Human Resources HU Housing Unit IA Island Areas IAC Island Areas Censuses operation ID Identification IDEF0 Integrated Definition, Level 0 IE Information Exchanges IHS Indian Health Service IN Indiana IOD Integrated Operations Diagram IPT Integrated Product Team IRS Internal Revenue Service ISR Internet Self-Response operation IT Information Technology ITIN IT Infrastructure operation V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 125 Acronym Meaning LMS Learning Management System LNG Language Services operation LUCA Local Update of Census Addresses operation MaCS Matching and Coding System MA Massachusetts MAF Master Address File MAFID Master Address File Identifier MAF/TIGER Master Address File/Topologically Integrated Geographic Encoding and Referencing system MCM Mobile Case Management MD Maryland MDF Microdata Detail Files MOJO Mobile Operations Job Optimizer MU Multiunit MV Manager Visits N/A Not applicable NID Non-ID operation No-FEAR Notification and Federal Employee Antidiscrimination and Retaliation Act of 2002 Non-ID Non-Identification Non-RI Non-Reinterview NOV Notice of Visit NPC National Processing Center NRFU Nonresponse Followup operation NRFU-IPT NRFU Integrated Project Team NV Notice of Visit OCS Operations Control Systems OOS Office Operation Supervisor PDC Paper Data Capture operation V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 126 Acronym Meaning PEARSIS Production Environment for Administrative Records Staging, Integration, and Storage PES Post-Enumeration Survey PM Program Management operation PMO Program Management Office POP Population Division PR Puerto Rico QA Quality Assurance QC Quality Control RCC Regional Census Center RI Reinterview RPO Response Processing Operation SMaRCS Sampling, Matching, Review, and Coding System SME Subject Matter Expert SOCS Survey Operational Control System SPC Security, Privacy, and Confidentiality operation SR Self-Response SRQA Self-Response Quality Assurance TEA Type of Enumeration Area TIGER Topologically Integrated Geographic Encoding and Referencing System TV Television UAA Undeliverable as Addressed UE Update Enumerate operation UL Update Leave operation US United States USPS United States Postal Service UTS Unified Tracking System VA Virginia ZIP Zone Improvement Plan V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 127 Table 13: Glossary of Terms Term Meaning Assignment Optimizer An algorithm designed to minimize cost in terms of time and mileage required to complete NRFU by optimizing the routing used by field staff to navigate to and between assigned fieldwork locations (i.e., case locations). The part of NRFU that starts on April 1 in areas such as those with high concentrations of off-campus college housing where respondents are unlikely to be present during the production NRFU time frame. Enterprise solution that provides the following functionality:  Supports field data collection for address listing/mapping and enumeration work.  Creates and manages the universe for all enumeration operations.  Maintains operational workloads as data collection proceeds.  Supports work assignment and schedule management for field data collection operations for in-office and mobile users.  Supports self-response data collection via the internet for survey and census respondents and for call center agents on behalf of respondents.  Supports questionnaire design and metadata maintenance. For NRFU, ECaSE is used for field enumeration and parts of operational control. The function of ECaSE that supports field data collection for enumeration work. ECaSE ENUM allows field users to collect data in both online and offline environments. The function of ECaSE that manages the data collection universe for all enumeration operations. ECaSE OCS creates the initial universe, receives status updates and response data, and maintains operational workloads as data collection proceeds. Early NRFU Enterprise Censuses and Surveys Enabling Platform (ECaSE) ECaSE Enumeration (ENUM) ECaSE Survey Operational Control System (SOCS) V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 128 Term Meaning ECaSE Field Operational Control System (FOCS) The function of ECaSE that leverages the MOJO Optimizer for routing used by field staff to navigate to and between assigned field work locations (i.e., case locations for enumeration and block locations for listing). Leverages computational models and standard deviations for providing certain alerts. Enumeration Universe Field Verification Case In-Mover Manager Visit Nonresponse Followup operation (NRFU) NRFU Contact Strategy NRFU Universe Paradata V2.0 The complete set of addresses for living quarters that are to be enumerated for the 2020 Census. A type of case in the NRFU workload for an address that had originally self-responded without a Census ID and could not be matched to the Census address frame (via either automated matching or clerical review processes). During a NRFU interview, a respondent who indicates that he or she did not live at the address on Census Day. A field interview to collect status of nonresponding units from the manager of a multiunit structure before individual unit enumeration is attempted. The 2020 Census operation to determine housing unit status for nonresponding addresses and enumerate housing units for which a census response was not received. The predefined strategy for contacting housing units that are part of the NRFU universe. This could include mailings as well as a number of attempted visits. A subset of the enumeration universe that comprises the set of addresses for living quarters that either (1) require a verification of the address in the field or (2) are housing units in the Self-Response Type of Enumeration Area (TEA) but for which the Census Bureau has not yet received a response and are assumed to be occupied based on a review of administrative records and third-party data. Auxiliary data collected during a sample survey, census, or other data collection that provide information about the data collection process. Paradata are used to evaluate and manage the survey process and may be used for both research and production activities, including adaptive design, at the Census Bureau. July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 129 Term Meaning Production Case A nonresponse follow-up case in the part of NRFU that starts midMay. A respondent who may have sufficient knowledge to enumerate the original household such as a neighbor or a caregiver. A type of case in the NRFU workload that is an interview at the same address as an original nonresponse interview for comparison for QC. One of a set of predefined status conditions associated with a response. Examples include no response, completed sufficient response, and completed insufficient response. An enumerator’s past cases added to the NRFU universe as reinterview cases because falsification by that enumerator was detected by QC. A set of blocks that are expected to be enumerated via the same methodology. The 2020 Census TEA values are as follows: TEA 1 – Self-Response TEA 2 – Update Enumerate TEA 3 – Island Areas TEA 4 – Remote Alaska TEA 6 – Update Leave Proxy Reinterview Case Response Status Rework Type of Enumeration Area (TEA) V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 130 Appendix B – References Appendix B lists the documents or other resources referenced within this Detailed Operational Plan document. U.S. Census Bureau (2018), “2020 Census Operational Plan,” Version 4.0, December 31, 2018. U.S. Census Bureau (2018), “Preparing for the 2020 Census Operational Assessment Study Plan,” Draft, May 10, 2018. U.S. Census Bureau (2016), Keller, A.; Fox, T.; and Mule, V.T. “2014 Census Test – Analysis of Administrative Record Usage.” U.S. Census Bureau. Available at: https://www2.census.gov/programs-surveys/decennial/2020/program-management/finalanalysis-reports/2020-analysis-2014-census-test-ad-rec.pdf (accessed April 2019). American Statistical Association (2014), Morris, D.S. “A Comparison of Methodologies for Classification of Administrative Records Quality for Census Enumeration,” in JSM Proceedings, Survey Research Methods Section, American Statistical Association, Boston, MA, August 2-7, 2014. Alexandria, VA: American Statistical Association. 1729-1743. Available at: https://ww2.amstat.org/sections/srms/Proceedings/y2014/files/311864_88281.pdf (accessed April 2019). U.S. Census Bureau (2016), “2020 Census Detailed Operational Plan for: 6. Geographic Programs Operation- Geographic Partnership Programs Component,” Version 1.0, September 9, 2016. U.S. Census Bureau (2019), “2020 Census Detailed Operational Plan for: 10. Paper Data Capture” Version 2.0, To Appear 2019. U.S. Census Bureau (2018), “2020 Census Detailed Operational Plan for: 6. Geographic Programs Operation (GEOP) – 6-1. Geographic Delineations Component (GEOP/GD),” Version 2.0 Final, December 31, 2018. U.S. Census Bureau (2016), “2020 Census Detailed Operational Plan for: 6. Geographic Programs Operation (GEOP) – 6-3. Geographic Data Processing Component (GEOP/GDP),” Version 1.0 Final, September 13, 2016. U.S. Census Bureau (2018), “2020 Census Detailed Operational Plan for: 7. Local Update of Census Addresses Operation (LUCA),” Version 2.0, November 21, 2018. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 131 U.S. Census Bureau. (2012), Walker, S.; Winder, S.; Jackson, G.; and Heimel, S. (2012), “2010 Census Nonresponse Followup (NRFU) Operations Assessment” 2010 Census Program for Evaluations and Experiments. April 13, 2012. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 132 Appendix C – Activity Tree for Nonresponse Followup Operation (NRFU) This appendix presents the Activity Tree for NRFU. An Activity Tree uses an outline structure to reflect the decomposition of the major operational activities in the operation. Each activity is numbered according to its position in the outline. For example, for the current operation numbered “18,” the first activity would be numbered 18-1. Subactivities under this activity would be numbered sequentially, starting again with the number one. For example, the first subactivity under the first activity would be numbered 18-1.1 the second subactivity as 18-1.2. The second activity would be numbered 18-2, and so on. NRFU Activity Tree:  V2.0 18-1 NRFU Planning and Preparation o 18-1.1 Develop NRFU AdRec Processing Requirements, Contact Strategies, and Business Rules  18-1.1.1 Develop NRFU Administrative Records Modeling Requirements  18-1.1.2 Develop NRFU Contact Strategies and Business Rules  18-1.1.3 Develop NRFU Quality Control Plan o 18-1.2 Develop and Approve Field Printed Materials and Kit Specs  18-1.2.1 Provide and Approve Field Enumeration Material Content  18-1.2.2 Provide and Approve Kit Requirements o 18-1.3 Identify Areas for Early NRFU o 18-1.4 Develop Requirements for and Approve Systems Supporting NRFU Operations o 18-1.5 Determine Staff Needs and Performance Metrics  18-1.5.1 Determine Estimated NRFU Operational Workload by Geographic Area  18-1.5.2 Define Operational Performance Metrics and Targets o 18-1.6 Prepare for and Deliver NRFU-Specific Training  18-1.6.1 Prepare for NRFU-Specific Training  18-1.6.1.1 Provide NRFU-Specific Training Content and Requirements  18-1.6.1.2 Review and Approve NRFU-Specific Training  18-1.6.2 Deliver NRFU-Specific Training July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU)     V2.0 Page 133 18-2 NRFU Operational Workload Management o 18-2.1 Monitor Progress and Resolve Issues o 18-2.2 Receive and Process NRFU Universe and Updates  18-2.2.1 Perform In-Office Adds  18-2.2.2 Receive NRFU Universe and Updates from RPO  18-2.2.3 Create Work Assignments (incl. Grouped Manager Visit Cases) 18-3 NRFU Data Collection o 18-3.1 Receive Work Assignment o 18-3.2 Perform Field Verification Data Collection and Quality Control o 18-3.3 Perform NRFU Manager Visit o 18-3.4 Perform NRFU Case Enumeration  18-3.4.1 Enumerate Housing Unit  18-3.4.2 Enumerate Housing Unit via Proxy Interview o 18-3.5 Perform NRFU Reinterview Data Collection o 18-3.6 Closeout Contact Attempt  18-3.6.1 Update Case Information  18-3.6.2 Send Data o 18-3.7 Submit Time and Expense Data 18-4 NRFU Quality Control Adjudication o 18-4.1 Receive Automated Quality Control Comparison Results from RPO o 18-4.2 Adjudicate Reinterview Results 18-5 NRFU Operation Closeout o 18-5.1 Closeout NRFU Operation July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 134 Appendix D – Operation-Specific Kits for NRFU This appendix lists the contents of the four types of operation-specific kits that are used for NRFU:     NRFU Census Field Supervisor (CFS) Supply Kit (See Table 14). NRFU CFS Instructor Kit (See Table 15). NRFU Enumerator Supply Kit (See Table 16). NRFU Bulk Supply Kit (See Table 17). Table 14: NRFU CFS Supply Kit NRFU CFS Supply Kit V2.0 Form Number Form Name Quantity BC-1206 Security Incident Report 2 BC-1759 Special Sworn Status 3 D-NV(E/S) Notice of Visit D-IS1-NF(E/S) Information Sheet D-282 Documentation of Conduct and/or Performance Problems D-291 DAPPS Update Form 40 D-1222(NRFU) NRFU Observation Form 40 D-IS2-NF(E/S) Information Sheet - Multi-unit Manager D-LI-NF Language Assistance Sheet D-694 NRFU CFS Job Aid D-ER(E/S) Return Envelope - Questionnaire 1,000 July 12, 2019 700 5 300 30 1 10 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 135 Table 15: NRFU CFS Instructor Kit NRFU CFS Instructor Kit Form Number Form Name D-109 OF-104 Rules of Behavior Governing Information Technology (Acceptable Use Policy) Instructor Power Kit Instructor Kit Letter: Thank You for Donating Space DAPPS Update Form Daily Pay and Work Record Modified Oath of Office Acknowledgement Receipt for Returned U.S. Government Property Data Stewardship Understanding and Protecting Census Bureau Information Brochure Name Card WC- Wallet Card Are You Hurt Z1079 Portfolio, U.S. Census, 16" x 11-1/2 x 3", Black, Nylon Oxford, Briefcase, W/Adj Shoulder Strap KIT-553P KIT-G-4 D-276(L) D-291 D-308 D-110 D-11805R POL/03-66862020 Quantity 40 1 1 3 40 40 20 40 40 40 40 1 Table 16: NRFU Enumerator Supply Kit NRFU Enumerator Supply Kit V2.0 Form Number Form Name Quantity BC-1199 D-ID D-NV(E/S) D-IS1-NF(E/S) D-IS2-NF(E/S) D-LI-NF D-ER(E/S) Official Business Sign/Employee Language Identification Flashcard Notice of Visit Information Sheet Information Sheet - Multi-unit Manager Language Assistance Sheet Return Envelope - Questionnaire July 12, 2019 1 1 400 200 1 10 1 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) D-699 SF-91 DF-28(E/S) Z4010 Page 136 NRFU Enumerator Job Aid Motor Vehicle Accident Report Information Sheet English/Spanish Clipboard 2 2 1 1 Table 17: NRFU Bulk Supply Kit NRFU Bulk Supply Kit Form Number Form Name D-1222(NRFU) D-IS2-NF(E/S) D-LI-NF D-ER (E/S) D-109 NRFU Observation Form Information Sheet - Multi-unit Manager Language Assistance Sheet Return Envelope-Questionnaire Rules of Behavior Governing Information Technology (Acceptable Use Policy) Language Identification Card NRFU Enumerator Job Aid Acknowledgement Receipt for Returned U.S. Government Property Clipboard D-ID D-699 D-11805R Z4010 V2.0 Quantity July 12, 2019 150 2,000 1,000 1,000 100 50 50 100 40 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 137 Appendix E – Administrative Records Determination Methodology This appendix describes a high-level application of the approach to be applied during the 2020 Census. The approach uses predictive modeling and linear optimization to make the determinations of AdRec occupied, vacant and nonexistent addresses. Data and Methodology To begin, we compile a household roster composed of AdRec individuals for all housing units (HUs) in the 2010 NRFU universe in the United States, District of Columbia, and Puerto Rico. We ensure that no one is duplicated within a housing unit (HU). For the 2010 data compiled above, we create a separate person and address-level datasets for modeling. The 2010 vintage AdRec sources used to create household rosters are:  Internal Revenue Service (IRS) Individual Tax Returns (1040)  IRS Informational Returns (1099)  Indian Health Service Patient Database  Center for Medicare and Medicaid Services (CMS) Medicare Enrollment Database  Census Household Key File  Social Security Numerical Identification (Numident) File Information from the Veterans Service Group of Illinois file, a third-party file, is used to inform the models but not for compiling the household roster. In addition, we incorporate 2010 vintage data from the USPS Delivery Sequence File, the American Community Survey (ACS), the Master Address File (MAF), and 2010 Census operational information. We also use the USPS Undeliverable-As-Addressed (UAA) reason codes obtained from the second 2010 Census mailing delivered around April 1, 2010. Identifying Vacant Units To identify vacant units, we develop a multinomial logit model, which estimates the unit status probability as of Census Day. The dependent variable has three possible values for each NRFU address record: occupied, vacant, or delete. Model We have developed a model of housing unit (HU) status (occupied, vacant or delete) for purposes of identifying vacant housing units to be removed from the NRFU workload. The V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 138 dependent variable of interest is a three-level indicator of whether the address was occupied, vacant or delete in the 2010 NRFU operation. We are interested in a predictive model for estimating the probability of each housing status. These probabilities are estimated via a multinomial logistic regression model. The association between administrative record information and 2010 Census housing unit status is captured by including administrative record information as predictors in this model. We do this via three types of predictors: indicators of the presence of each of the administrative record sources at the housing unit (here variables), indicators of the presence of each of the administrative record sources at a different housing unit for a person in the housing unit (elsewhere variables), and characteristics of people in the housing unit as determined by administrative records. For example, in a two-person administrative records housing unit, if at least one of the people had an IRS 1040 filed at that address then the IRS 1040 here indicator variable is equal to one. On the other hand, if at least one person had an IRS 1040 form filed at a different address, then the IRS 1040 elsewhere indicator variable is equal one. These types of variables are created for five administrative records sources: IRS 1040, IRS 1099, IHS, Medicare, and third-party data sources. Another important administrative record source incorporated into this unoccupied model is USPS undeliverable as addressed (UAA). UAAs are flagged by the USPS and assigned a reason code for failed delivery, which include reasons such as vacant, no such number, or unable to forward. The model also includes as predictors block group characteristics from the ACS planning database and housing unit characteristics from the Master Address File (MAF). Each housing unit has an associated predicted probability for each of the three housing status categories. These predicted probabilities are passed to the optimization procedure as an input for determining which units are likely to be vacant. We want to make the AdRec Vacant cases those closest to the highest vacant probability and lowest occupied probability. For this situation, we have a two-dimensional plane with the vacant probability on one dimension and the occupied probability on the other dimension. In this instance, the ideal situation would be for an address to have a vacant probability of one and an occupied probability of zero. As a result, we define the Euclidian vacant distance function for vacant determination as: 𝑑𝐴𝑅_𝑉𝑎𝑐 = √(1 − 𝑝𝑣𝑎𝑐𝑎𝑛𝑡 )2 + (0 − 𝑝𝑜𝑐𝑐𝑢𝑝𝑖𝑒𝑑 ) 2 Based on the research and results of the 2015, 2016, and 2018 Census Tests, we have modified the AdRec usage and contact strategy in the NRFU operation for the 2020 Census. As previously V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 139 mentioned, we include an additional mailing for addresses with AdRec determinations. These addresses will receive one contact. This was added to the 2016 Census Test to help address concerns about missing occupied housing units. Based on results of the 2016 and 2018 Census Tests, we have changed the contact strategy for the 2020 Census. For cases initially determined to be AdRec vacant, we will use the UAA information from the USPS on the additional mailing to make a final determination. For AdRec vacant cases, only those addresses for which the additional mailing is returned as UAA will receive the AdRec determination. If the postal carrier is able to deliver the additional mailing, then the address will be added back to the NRFU field workload and receive the full contact strategy of enumerator visits Identifying Nonexistent Addresses The approach described in the previous section for determining AdRec vacant addresses is used to determine AdRec nonexistent addresses that can receive fewer contacts as well. We take advantage of our multinomial outcome space of addresses being classified as occupied, vacant or delete. We define the nonexistent distance function based on the nonexistent probability and occupied probability in a similar manner to determining vacant addresses. We substitute the nonexistent probability for the vacant probability. In this instance, the ideal situation is for an address to have a nonexistent probability of one and an occupied probability of zero. As a result, we define the Euclidian nonexistent distance function as: 𝑑ℎ𝑛𝑒 = √(1 − 𝑝̂ ℎ𝑛𝑒 )2 + (𝑝̂ ℎ𝑜𝑐𝑐 )2 This continuous measure is assigned to every address eligible for the Nonresponse Followup operation. The cutoff can be adjusted upwards or downwards based on the tradeoff between fieldwork reduction and allowable differences. The same change in contact strategy for the administrative record vacants is being also implemented for the administrative record nonexistent addresses. For cases initially determined to be AdRec nonexistent, Census will use the UAA information from the USPS on V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 140 the additional mailing to make a final determination. For AdRec nonexistent cases, only those addresses for which the additional mailing is returned as UAA will receive the AdRec determination. If the postal carrier is able to deliver the additional mailing, then the address will be added back to the NRFU field workload and receive the full contact strategy of enumerator visits Identifying Occupied Units This section lays out the methodology to determine AdRec occupied addresses. Two models are developed to identify occupied units. The first section describes the person-place model and the second section documents the household (HH) composition model. To integrate information from the two models, we use linear programming to identify occupied units. Person-Place Model The person-place model predicts the probability of enumerating the AdRec person at the sample address if fieldwork were conducted. We compile person-place pairs in AdRec files mentioned above and the 2010 Census person- place pairs to define the dependent variable of interest in the person-place model: 𝑦𝑖ℎ = { 1 if person 𝑖 is found in AdRec and 2010 Census at the same address ℎ 0 otherwise Figure 28 below relates to the occupied probability. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 141 Figure 28: Occupied Probability We are interested in a predictive model for estimating the probability, 𝑝𝑖ℎ = 𝑃(𝑦𝑖ℎ = 1), that the 2010 Census and the AdRec roster data place the person at the same address. These probabilities are estimated via a logistic regression model. Morris (2014) documents initial work on this person-place model and investigates logistic regression, decision tree and random forest methodologies for estimating the models. This research showed little difference in predictive power between the three methodologies. For consistency with the household composition model, the logistic regression is used in this framework. The person-place model is fit at the person-level, but decisions are made at the housing unitlevel. Therefore, the person-level predicted probabilities, 𝑝̂ 𝑖ℎ , are summarized for each address such that the housing unit-level predicted probability for address h is defined as: V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 142 𝑝̂ ℎ = min(𝑝̂1ℎ , … , 𝑝̂ 𝑛ℎ ℎ ) (1) where 𝑛ℎ is the number of people at address h. This minimum criterion assigns to the housing unit the predicted probability for the person in the housing unit for which we have the lowest confidence – a relatively conservative approach. The AdRec household count is defined as the sum of all individuals associated with the AdRec address, and each address has the associated predicted probability of having an AdRec/census address match. These are the predicted probabilities that are passed to the linear programming portion to decide which cases were determined to be occupied. Household Composition Model The HH composition model predicts the probability that the sample address would have the same HH composition determined by NRFU fieldwork as its pre-identified AdRec HH composition. The results from the 2014 Census Test motivated the development of the HH composition model. During that test, Keller et al. (2016) observed that units identified as occupied with AdRec were more likely to be occupied in NRFU if the HH composition of the AdRec unit was a single adult, a two-person adult unit without children, or a two-person adult unit with children. We begin by categorizing each AdRec HH roster in this manner:          No AdRec people 1 Adult, 0 Child 1 Adult, >0 Child 2 Adult, 0 Child 2 Adult, >0 Child 3 Adult, 0 Child 3 Adult, >0 Child Someone with undetermined age in HH Other We then create a dependent variable from the HH composition on the 2010 Census. The categorization is similar except that, in all units, everyone has an age. This is because we are using the Census Edited File as the basis for forming the census HH composition. Since these data have age imputed, there are no missing values for age. We fit a multinomial logistic model with the 2010 Census HH composition as the dependent variable over a sample of the data. The predicted probability for the housing unit is the multinomial probability associated with the V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 143 AdRec HH type. These predicted probabilities are then passed to the linear programming portion to determine which cases are identified as occupied. For our occupied decision, we use two predicted probabilities as inputs. The first is the minimum person-place probability for the address. For the person-place probability, we would like this probability to be as close to one as possible. The second is the household composition probability. This is the probability that given we have observed a certain AdRec HH composition, the census HH would be enumerated with the same HH composition. For this HH composition probability, we would also like that to be as close to one as possible. Based on this, our situation can be thought of as a two-dimensional plane where each probability is on one dimension with values between zero and one. Based on the two probabilities, each address would have a point in this two-dimensional space. The best AdRec cases would be those that have shortest distance to the point where the person-place model equals one and the household composition equals one (i.e., the (1,1) point). Based on this idea, we proceed to use the Euclidian distance the occupied distance function. Hence, we define the occupied distance function as: 2 𝑑𝐴𝑅_𝑂𝑐𝑐 = √(1 − 𝑝𝑝𝑒𝑟𝑠𝑜𝑛−𝑝𝑙𝑎𝑐𝑒 ) + (1 − 𝑝𝐻𝐻 𝑐𝑜𝑚𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛 ) 2 Figure 29 below relates to household composition probability. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 144 Figure 29: Household Composition Probability V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 145 Appendix F – NRFU Administrative Records Contact Strategy One of the four innovation areas for the 2020 Census design is the use of Administrative Records and Third-Party Data (AdRec) to reduce field workload. To this end, the Census Bureau has developed a set of logistic regression models that draw upon a variety of AdRec data to reduce the field workload associated with the Nonresponse Followup operation (NRFU). These models serve two primary roles:   Determine housing unit (HU) status after one visit. Provide household enumeration data for selected HUs for which high-quality AdRec data exist and which cannot be enumerated after one visit. Figure 30 (below, following the narrative explanation) shows the NRFU contact strategy for the 2020 Census and the role of AdRec in this strategy. The numbers shown in the black and gold circles in the figure provide a guide for the discussion points found in the narrative explanation. Note: The black circles are associated with self-response, whereas the gold circles are associated with NRFU. 1 The figure below starts with the self-response phase of the data collection, during which HUs are provided an opportunity to respond. How a HU is contacted during self-response affects how the AdRec models handle that unit. NRFU covers nonresponding addresses in the two Type of Enumeration Areas (TEAs)4 that are expected to self-respond: TEA 1 and TEA 6. Addresses in TEA 1 (the bulk of addresses) receive up to five mailings that include a combination of letters, postcard reminders, and paper questionnaires aimed at encouraging people to self-respond. Addresses in TEA 6, which covers areas where most HUs have limited mail delivery capabilities, are visited through the Update Leave operation, during which the lister leaves a questionnaire packet at the door. HUs in TEA 6 that can receive mail receive two reminder mailings. Together, TEA 1 and TEA 6 cover approximately 99 percent of the addresses in the 2020 Census Master Address File (MAF). 4 The Census Bureau creates TEAs to determine the most effective way to enumerate different parts of the country. Different operations are responsible for listing and enumerating the various TEAs. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 146 2 The AdRec models are run on all addresses, assigning each address one of four possible statuses: AdRec Vacant, AdRec Non-Existent, AdRec Occupied, or AdRec No Determination. The AdRec modeling uses U.S. Postal Service Undeliverable As Addressed (UAA) information based on two mailings. For those HUs in TEA 1, the modeling uses the UAA information from the second and third mailings. For HUs in TEA 6 that can receive mail, the modeling uses the UAA information from both mailings. Any HUs in TEA 6 that cannot receive mail will not be assigned an AdRec Vacant or AdRec Non-Existent status. Although not shown on the diagram, the AdRec models are run at least one additional time during NRFU to reflect the most current AdRec data available. 3 When NRFU begins, the operational control system creates the NRFU Universe (i.e., addresses in TEA 1 and TEA 6 that have not yet self-responded). 44 Nonresponding addresses that have been assigned a status of AdRec Vacant or AdRec Non-Existent are sent one additional mailing (AdRec Mailing). The UAA information from this mailing is used later in the process (see number 6 below). 5 At the same time, an attempt is made to enumerate all HUs in the NRFU Universe. If this first attempt is successful (i.e., the enumerator determines the status of the HU [vacant, non-existent, or occupied] and, if occupied, enumerates the unit), then the case is resolved and considered Resolved with Field Visit. 6 If the first attempt is unsuccessful, the HU does not appear to be occupied, and the address has a status of AdRec Vacant or AdRec Non-Existent, then the UAA information from the AdRec Mailing sent to all NRFU AdRec Vacant and AdRec Non-Existent addresses is checked. If the AdRec Mailing is UAA, then the address is recorded as vacant or non-existent, considered Resolved as Vacant or Non-Existent, and receives no additional visits. If the address is not UAA (i.e., the UAA information indicates that the mailing was successfully delivered, contradicting the AdRec modeling results), then the HU becomes subject to the full NRFU contact strategy and additional attempts are made (see number 8 below). 7 If the first attempt is unsuccessful and the address has a status of AdRec Occupied, then―regardless of whether the HU appears to be occupied―an additional mailing is sent to this address to encourage self-response. In the absence of a self-response, the address is enumerated using AdRec data and considered Resolved with AdRec Data. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 147 8 If the first attempt is unsuccessful, additional attempts are made to enumerate at the HU under three circumstances: (1) if the HU appears to be occupied and does not have a status of AdRec Occupied (i.e., it has a status of AdRec Vacant, AdRec Non-Existent, or AdRec No Determination); (2) if the HU does not appear to be occupied and has a status of AdRec Vacant or AdRec Non-Existent and the AdRec Mailing sent to the address is not UAA (see number 6 above); and (3) if the HU does not appear to be occupied and the address has a status of AdRec No Determination (i.e., it does not have a status of either AdRec Vacant, AdRec Non-Existent, or AdRec Occupied). In all three cases, additional enumeration attempts are made until an attempt is successful or the maximum number of attempts has been reached. A successful attempt is considered Resolved with Field Visit. 9 The last weeks of NRFU are considered the NRFU Closeout Phase. At this point, the AdRec statuses of all addresses remaining unresolved after the maximum number of attempts are reassessed using a relaxed quality threshold. Reassessed addresses with a status of AdRec Vacant or AdRec Non-Existent are recorded as vacant or non-existent and considered Resolved as Vacant or Non-Existent. Reassessed addresses with a status of AdRec Occupied are enumerated using AdRec data and considered Resolved with AdRec Data. 10 Those addresses that―even with the lower threshold―cannot be assigned a status of AdRec Vacant, AdRec Non-Existent, or AdRec Occupied, are subject to additional attempts to enumerate that continue either until an attempt is successful or the NRFU operation has ended. 11 Any addresses that cannot be successfully enumerated by the end of NRFU are considered Unresolved. The results for these addresses are imputed. 12 Self-response data collection continues during NRFU. The response data from the selfresponses received are used in place of the results from the AdRec modeling or AdRec enumeration (provided they meet the criteria for a sufficient response). V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 148 POST DATA COLLECTION DATA COLLECTION SELF-RESPONSE 1 Provide Opportunity to Self-Respond: 6 [Up to 5 Mailings] Send AdRec Mailing [2 Reminder Mailings where Feasible] Send Additional Mailing to Encourage Self-Response NO UAA? UAA Info NONRESPONSE FOLLOWUP Possible Results: AdRec Vacant AdRec Non-Existent AdRec Occupied AdRec No Determination YES AdRec Occupied? 7 6 AdRec Vacant or AdRec Non-Existent? Reassess AdRec Models Results with Relaxed Thresholds YES NO [AdRec Occupied] YES YES 9 NO NRFU Closeout Phase Begins NO 7 9 Resolved with AdRec Data 9 NO NRFU UNIVERSE 7 9 Enumerate Using AdRec Data AdRec Vacant, AdRec Non-Existent, or AdRec Occupied? 9 NRFU ADREC VACANT ADREC NONEXISTENT Resolved as Vacant or Non-Existent 7 6 UAA Info 6 9 9 Record Address as Vacant or Non-Existent YES 4 TEA 6: Leave Packet at Door 2 Resolved with Self-Response Modes: Internet Response Collection, Paper Data Capture, Telephone Interview [for those who call in] TEA 1: Contact by Mail Run AdRec Models on All Addresses 12 12 Collect Self-Responses AdRec Vacant or AdRec Non-Existent? 9 3 Create Universe of Nonresponding Addresses 5 Attempt to Enumerate YES [Unresolved] 8 NO YES 6 Attempt to Enumerate Max Attempts? NO [Additional Attempts] [First Attempt] Appears Occupied? 10 NO NRFU Begins Attempt to Enumerate 11 NO Unresolved [Final Attempts] 10 NO 8 Successful? NO YES 5 11 YES Successful? [6 weeks after Self-Response Begins] End of NRFU? YES 5 8 10 Resolved with Field Visit [Enumerated or identified as vacant or not a HU] YES Successful? LEGEND NRFU Administrative Records Contact Strategy, V2.0 AdRec: HU: NRFU: TEA: UAA: Administrative Records and Third-Party Data Housing Unit Nonresponse Followup Operation Type of Enumeration Area Undeliverable As Addressed Shaded Box: Activity relating to AdRec Modeling AdRec Vacant / AdRec Non-Existent Shaded Diamond: Decision relating to AdRec Modeling AdRec Occupied Hatched Box: NRFU Field Work Multiple AdRec Statuses Case Resolved [not AdRec] Data Flow Case Resolved [with AdRec] Logic Flow Case Unresolved Figure 30: NRFU Administrative Records Contact Strategy V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 149 Appendix G – NRFU Training Plan The tables below (Table 18 through Table 26) summarize the current plans for NRFU Training content and coverage. These plans include:   Employee/Supervisor Training o Initial Classroom and Online Training (Days 1 through Day 5) o Post Classroom Online Training o Mid-Operation Online Training Supervisor Supplemental Training o Supplemental Classroom and Online Training (Days 6 through Day 8) o Supervisor Post Classroom Online Training o Supervisor Mid-Operation Online Training A legend is provided for the table row colors below: Legend Operation Specific Generic (Potentially implemented without change) Potentially Generic/Unknown V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 150 Table 18: Employee/Supervisor Training (Day 1) Day 1 (Classroom) Lesson # NRFU-E1 NRFU-E2 NRFU-E3 Module Name Hrs. Appointment and Orientation 1.0 Introduction to Operation 0.15 Receipt of Device (Smartphone) (Shared) Data 1.0 Stewardship (if required for receipt) NRFU-S1 Receipt of Device (Tablet) (Shared) (Supervisor Only) 1.0 Data Stewardship (if required for receipt) NRFU-E4 Operating the Device (Smartphone) (Shared) 0.5 NRFU-S2 Operating the Device (Tablet) (Supervisor Only) (Shared) 0.5 NRFU-E5 Accessing Training Mode 0.25 NRFU-E6 Entering Work Availability (Shared) 0.45 NRFU-E7 Entering Time and Expense (Shared) 1.0 NRFU-E8 Online Training Overview 0.25 Day 1 Total 6.1 Note: Training takes place on different days for Supervisors and Early Employees than for Employees. Training path identical with the exception of issue and use of an additional device (tablet for Supervisors). Table 19: Employee/Supervisor Training (Days 2 – 4) Days 2 - 4 (Online) Lesson # GT-E8 Module Name History of the Census (Shared) Hrs. 0.5 GT-E9 GT-E10 GT-E11 GT-E12 NRFU-E9 NRFU-E10 NRFU-E11 Data Stewardship (Shared) No FEAR (Shared) EEO for Employees (Shared) Safety (Shared) Census Overview Your Role Your Tools 1.0 0.75 0.5 0.5 0.5 0.5 0.5 V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 151 Days 2 - 4 (Online) Lesson # NRFU-E12 NRFU-E13 NRFU-E14 NRFU-E-EV1 NRFU-G1 Module Name Interview Skills Field Processes Challenging Situations Employee Online Training Evaluation Enumerator Gamelette Hrs. 0.5 0.5 0.75 0.25 0.5 Day 2-4 Total 7.25 Prerequisite: Successful completion of the instructor-led (classroom) admin/device day. Table 20: Employee/Supervisor Training (Day 5) Day 5 (Classroom) Lesson # NRFU-E15 NRFU-E16 NRFU-E17 NRFU-E18 NRFU-E19 NRFU-E20 NRFU-E21 NRFU-E22 NRFU-E23 NRFU-E24 NRFU-E25 NRFU-E26 Module Name Introductions and Online Training Review (includes Getting Ready to Instruct component) Preparing for Your Day Common Barriers to Nonresponse Follow-up Interview Exercises Responding to Refusals and Exercises Language Barriers and Exercises Proxy Process and Exercises Multiunit Process and Exercises Reinterview Scripted Triad Role-Play Exercises Field Verification End of Day Tasks Next Steps for Employees Day 5 Total Hrs. 0.25 0.25 0.5 0.75 0.5 0.75 0.5 0.75 0.75 0.5 0.25 0.25 6.0 Prerequisites:  [Enumerator and Supervisor] Successful completion of the Enumerator self-paced (online) training.  [Supervisor Only] Successful completion of the Supervisor self-paced (online) training. V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 152 Table 21: Employee/Supervisor Post Classroom Online Training POST CLASSROOM ONLINE Lesson # NRFU-E-E27 NRFU-E-EV2 Module Name Employee Final Assessment Employee Post Classroom Evaluation POST CLASSROOM ONLINE Total Hrs. 0.5 0.25 0.75 Table 22: Employee/Supervisor Mid-Operation Online Training MID-OPERATION ONLINE Lesson # NRFU-E-EV3 Module Name Employee Mid Operation Evaluation POST CLASSROOM ONLINE Total Hrs. 0.25 0.25 Table 23: Supervisor Supplemental Training (Days 6 – 7) Days 6-7 – Supervisor Only (Online) Lesson # GT-S3 NRFU-S3 NRFU-S4 NRFU-S5 NRFU-S6 Module Name Your Role as a Supervisor (Shared) People Management Operational Control System (OCS) Orientation Performance Management Alert Resolution Scenarios (OCS) Hrs. 0.5 0.5 1 1 1.5 NRFU-S7 Payroll Management 0.5 NRFU-S8 GT-S9 NRFU-S-EV1 Facilitation Skills EEO for Supervisors Supervisor Online Evaluation 0.45 0.5 0.25 6.2 Day 6-7 Total V2.0 July 12, 2019 Final 2020 Census Detailed Operational Plan for: 18. Nonresponse Followup Operation (NRFU) Page 153 Table 24: Supervisor Supplemental Training (Day 8) Day 8 – Supervisor Only (Classroom) Lesson # NRFU-S10 Hrs. 0.5 NRFU-S14 Module Name Introductions and Training Overview (includes Getting Ready to Instruct component) Online Training Recap Supervisor Support Line Supervisor Conversations and Scenarios Planning Classroom Training Facilitation Practical Exercise NRFU-S15 Next Steps/Training Review 0.5 6.2 NRFU-S11 NRFU-S9 NRFU-S12 NRFU-S13 Day 8 Total 0.5 0.5 2 1 1 Table 25: Supervisor Post Classroom Online Training SUPERVISOR POST CLASSROOM ONLINE Lesson # NRFU-S16 NRFU-S-EV2 Module Name Supervisor Final Assessment Supervisor Post Classroom Evaluation SUPERVISOR POST CLASSROOM ONLINE Total Hrs. 0.5 0.25 0.75 Table 26: Supervisor Mid-Operation Online Training SUPERVISOR MID-OPERATION ONLINE Lesson # NRFU-S-EV3 NRFU-S-CO1 V2.0 Module Name Supervisor Mid-Operation Training Evaluation Supervisor Close-out Module SUPERVISOR POST CLASSROOM ONLINE Total July 12, 2019 Hrs. 0.5 0.5 1.0 Final