Arizona Republic/Morrison/Cronkite News Poll Methodology Report October 2016 The purpose of this document is to showcase the features and choices in the Arizona Republic/Morrison/Cronkite News Poll conducted in October 2016. We hope that this methodological report provides the necessary transparency for readers to compare this poll to other polls of Arizona voters. This poll was approved by the Arizona State Institutional Review Board under protocol number STUDY00004747. Please direct questions about the methodology to the Morrison Institute’s Senior Research Fellow, Dr. Eric Hedberg, at ehedberg@asu.edu. The Arizona Republic/Morrison/Cronkite News Poll questionnaire was written jointly by representatives of Arizona State University’s Morrison Institute for Public Policy, The Arizona Republic, and The Walter Cronkite School of Journalism and Mass Communication at Arizona State University. The Poll team contracted with Behavior Research Center (BRC) to collect the data. BRC attempted a total number of 1,216 live telephone interviews (in English and Spanish), and 1,179 calls were answered. The interviews were conducted between October 10 and 15 2016. The average number of valid responses to the questions was 811. Generally, the response rate for complete interviews was 69 percent. Sample The sampling frame was obtained from “L2 Data” of Bothell, WA, which maintains up-to-date lists of registered voters. By keeping the database up to date, it includes voters who registered for the primary. As the sampling frame uses phone numbers as they are reported by the registered voter, whether the number was a land line or cell phone is unknown. Weighting The sampling frame used only registered voters. The weighting procedure involved a poststratification technique that adjusted the sampling weights1 of the individuals who answered the questions analyzed to match the proportions in the registered voter database of county (determined from sampling frame), party (determined by sampling frame), age (determined by a question at the end of the survey), and gender (determined by interviewer observation at the beginning of the interview). Before using the weights, the total counts of voters were adjusted to compensate for the number of likely voters (based on the survey data proportions) and survey non-response. 1 Sampling weights based on counts by county, party, age (10 Year increments), and gender in the registered voter databased obtained by the Arizona Secretary of State and used with permission. Gender was imputed using the first names and the “genderize.io” website. 1 Sampling Error It is inaccurate to use a single margin of error2 for all answers in a survey. For percents and proportions, the margin of error is directly related to the sample size, the impact of the weighting technique, population sizes, and the proportion or percent estimated. All these factors, except the population size assumed, change from poll to poll. This is because, unlike an average, the variance of a proportion is a function of the proportion itself. For a simple random sample, which this poll is not, the 95% MOE of a percent is about , -., 95%𝑀𝑂𝐸 % = 2×100× / , -., where is the sampling variance and 𝑝 is the proportion (and 100 times the proportion is / the percent). As you can see, holding the sample size constant, the MOE changes with the proportion. Thus, our tables present a margin of error for each answer choice and classification category. In addition, the weighting technique typically increases the sampling variance (which relates to the margin of error) by a factor we call the design effect, which cannot be anticipated. Our weighting technique for the October Poll produced a design effect of about 1.36. That makes the MOE formula 95%𝑀𝑂𝐸 % = 2×100× 1.36× , -., / 0 1 MOE 2 3 4 Thus, the MOE for Clinton’s 39.1 percent is 4.3. If Clinton’s percent was lower, the MOE would be lower, as you can see in the following chart which plots the MOE formula against hypothetical percents 0 20 40 60 80 Percent 100 2 The margin of error is defined in our tables as twice the estimated standard error of the estimate. 2 In the August Poll, the Design Effect was less than 1, 0.789, which decreased the sampling variance further and produced a smaller MOE for Clinton’s 35.1 percent. Again, the design effect cannot be anticipated prior to data collection. Assumptions The Arizona Republic/Morrison-Cronkite Poll used as its sampling frame an updated database of registered voters. This methodology makes the assumption that each campaign is likely to attract newly registered voters (who are not captured in the sampling frame that included primary voters) at the same rate. Demographic Comparisons to Registered Voters Although the weighting procedure compensates for the uneven distribution of respondents compared to the proportions in the sampling frame, it is still useful to access the demographics that comprise the sample. In addition, we also test whether certain demographics were more or less likely to respond to the voting questions. Thus, it is important to employ the weighting adjustments. For example, the sample was generally more female than male. Table A1: Unweighted answers to 'GENDER: ' by 'All respondents' Male Female All respondents (N = 45.8 54.2 1179) (± 2.9) (± 2.9) Notes: 95 percent margin of error below estimates in parentheses. Table A2: Unweighted answers to 'GENDER: ' by 'Did respondent answer voter and age questions' Male Female Did not respond to 42.3 57.7 voter and age questions (N = 466) (± 4.6) (± 4.6) 48.1 51.9 3 Responded to voter and age questions (N = 713) (± 3.7) (± 3.7) Notes: 95 percent margin of error below estimates in parentheses. Pattern of answers is statistically significant based on unweighted Pearson chi-square test. 4 About 60 percent of registered voters reside in Maricopa County, 16 percent of registered voters reside in Pima County, with about a quarter in the rest of the state. The 1,179 interviews that were attempted reflect these proportions. However, the respondents to the voter questions are slightly more representative of Pima and less representative of Maricopa. Table A3: Unweighted answers to 'COUNTY ' by 'All respondents' Maricopa All respondents (N 59.9 = 1179) (± 2.9) Pima 15.7 (± 2.1) Notes: 95 percent margin of error below estimates in parentheses. Rest of Arizona 24.4 (± 2.5) Table A4: Unweighted answers to 'COUNTY ' by 'Did respondent answer voter and age questions' Did not respond to voter and age questions (N = 466) Responded to voter and age questions (N = 713) Maricopa 60.7 Pima 16.5 Rest of Arizona 22.7 (± 4.5) 59.3 (± 3.4) 15.1 (± 3.9) 25.5 (± 3.7) (± 2.7) (± 3.3) Notes: 95 percent margin of error below estimates in parentheses. Pattern of answers is not statistically significant based on unweighted Pearson chi-square test. 5 Registration among parties is about 35 percent Republican, 30 percent Democrat, and the rest with other parties or with no party preference (Independents). In our data, respondents to the voting questions tended to be a less representative of Republicans, but no statistical association between party and response was found. Table A5: Unweighted answers to 'PARTY ' by 'All respondents' republican All respondents (N 23.6 = 1179) (± 2.5) democrat 57.0 (± 2.9) OTHER/INDEPENDENT 19.4 (± 2.3) Notes: 95 percent margin of error below estimates in parentheses. Table A6: Unweighted answers to 'PARTY ' by 'Did respondent answer voter and age questions' Did not respond to voter and age questions (N = 466) Responded to voter and age questions (N = 713) republican 23.6 democrat 55.6 OTHER/INDEPENDENT 20.8 (± 3.9) 23.6 (± 4.6) 57.9 (± 3.8) 18.5 (± 3.2) (± 3.7) (± 2.9) Notes: 95 percent margin of error below estimates in parentheses. Pattern of answers is not statistically significant based on unweighted Pearson chi-square test. 6 Other Demographic Characteristics The following tables present details about the unweighted demographics of the sample. Table A7: Unweighted answers to '1. Generally speaking, do you usually think of yourself as a Republican, a Democrat, an Independent, or what?' by 'All respondents' Republican All respondents (N 24.3 = 873) (± 2.9) Democrat 43.3 (± 3.4) Independent/Other 32.4 (± 3.2) Notes: 95 percent margin of error below estimates in parentheses. Table A8: Unweighted answers to '3. In the upcoming election, do you plan to vote for president?' by 'All respondents' Yes All respondents (N 92.4 = 857) (± 1.8) No 2.0 (± 1.0) Notes: 95 percent margin of error below estimates in parentheses. 7 Haven't decided 5.6 (± 1.6)