Arizona Republic/Morrison/Cronkite News Poll Methodology Report August 2016 The purpose of this document is to showcase the features and choices in the Arizona Republic/Morrison/Cronkite News Poll conducted in August of 2016. We hope that this methodological report provides the necessary transparency for readers to compare this poll to other polls of Arizona voters. This poll was approved by the Arizona State Institutional Review Board under protocol number STUDY00004747. Please direct questions about the methodology to the Morrison Institute’s Senior Research Fellow, Dr. Eric Hedberg, at ehedberg@asu.edu. The Arizona Republic/Morrison/Cronkite News Poll questionnaire was written jointly by representatives of Arizona State University’s Morrison Institute, The Arizona Republic, and The Walter Cronkite School of Journalism and Mass Communication at Arizona State University. The Poll team contracted with Behavior Research Center (BRC) to collect the data. BRC attempted a total number of 1,893 live telephone interviews (in English and Spanish), 1,689 calls were answered. The interviews were conducted between August 17 and 31 2016. The average number of valid responses to the questions was 800. Generally, the response rate for complete interviews was 47 percent. Sample The sampling frame was obtained from “L2 Data” of Bothell, WA, which maintains up-to-date lists of registered voters. By keeping the database up to date, it includes voters who registered for the primary. As the sampling frame uses phone numbers as they are reported by the registered voter, whether the number was a land or cell number is unknown. Weighting The sampling frame used only registered voters. The weighting procedure involved a poststratification technique that adjusted the sampling weights1 of the individuals who answered the questions analyzed to match the proportions in the registered voter database of county (determined from sampling frame), party (determined by sampling frame), age (determined by a question at the end of the survey), and gender (determined by interviewer observation at the beginning of the interview). Before using the weights, the total counts of voters were adjusted to compensate for the number of likely voters (based on the survey data proportions) and survey non-response. 1 Sampling weights based on counts by county, party, age (10 year increments), and gender in the registered voter databased obtained by the Arizona Secretary of State and used with permission. Gender was imputed using the first names and the “genderize.io” website. 1 Sampling Error It is incorrect to use a single margin of error2 for all answers in a survey. For percentages and proportions, the margin of error is directly related to the sample size, the weighting technique and population sizes, and the proportion or percent estimated.3 Thus, our tables present a margin of error for each answer choice and classification category. The margin of error is also impacted slightly by the percentages of the other categories and the size of the registered voter population. For the presidential questions, the two primary candidates have levels of about 40 percent based on 700 responses, so the margin of error is approximately 3 to 4 percentage points. Comparison with CNN/ORC Methodology CNN/ORC4 recently conducted a poll in Arizona and estimated different results, showing Trump ahead. However, the methodology was different, and may explain the difference in results. CNN/ORC obtained interviews with 1,003 individuals from the population (with a mix of cellphone and landlines) between August 18 and 23rd. The number of interview attempts is unknown and thus response rates are not reported. They achieved a total of 842 likely registered voters. CNN/ORC identified registered voters using a question that asked respondents if they were registered to vote. They identified likely voters with a question as well. CNN/ORC then weighted the responses to census counts based on gender, race, age, education and region of the state. In contrast, the Arizona Republic/Morrison/Cronkite News Poll used a database of registered voters and weighted the sample to match the proportions of registered voters. We also only asked the candidate questions to likely voters. Finally, the wording of our questions is slightly different (we ask “planning” to vote fore then leaning to vote for, whereas CNN/ORC asked “likely” to vote for then leaning to vote for). This led to far more respondents reporting that they haven’t decided. Moreover, more Republicans are undecided than Democrats. 2 The margin of error is defined in our tables as twice the estimated standard error of the estimate. 3 This is because, unlike an average, the variance of a proportion is a function of the proportion itself. 4 http://i2.cdn.turner.com/cnn/2016/images/08/24/topaz1.pdf 2 CNN/ORC Poll Excerpt: BASED ON 833 REGISTERED VOTERS AND 9 WHO PLAN TO REGISTER, FOR A TOTAL OF 842 REGISTERED VOTERS -- SAMPLING ERROR: +/- 3.5 PERCENTAGE PTS. Q3. (P5) Now suppose that the presidential candidates on the ballot in your state included Hillary Clinton as the Democratic Party’s candidate, Donald Trump as the Republican Party’s candidate, Gary Johnson as the Libertarian Party candidate, and Jill Stein as the Green Party candidate, who would you be more likely to vote for? (RANDOM ORDER) Q3a. (P5a) (IF NONE OR DON’T KNOW) As of today, do you lean more toward --? Clinton, the Democrat Trump, the Republican Johnson, the Libertarian Stein, the Green party candidate None (vol.) Other (vol.) No opinion RV Aug 18-23 2016 LV Aug 18-23 2016 38% 43% 12% 4% 2% 1% * 38% 45% 12% 4% 1% 1% * Arizona Republic/Morrison/Cronkite News Poll Excerpt: Q4. (S1) If the election for U.S. Senate were held today and the candidates were Ann Kirkpatrick, the 3) In theDemocrat upcoming do youthe plan to vote for president? andelection, John McCain, Republican, who would you be more likely to vote for ---? (RANDOM ORDER) Yes Q4a. (S1a) No (IF NEITHER OR DON’T KNOW) As of today, do you lean more toward --? Haven’t decided [If yes in question 3] Kirkpatrick McCain Other (vol.) Neither (vol.) No opinion 4) Registered Who are you planning to vote for? (Read choices, alternating Trump and Clinton as the first Voters choice) August 18-23, 2016 39% 52% 2% 5% 2% The Republican Nominee, Donald Trump Likely TheVoters Democratic Nominee, Hillary Clinton August 18-23, 2016 39% 52% The Libertarian Nominee, Gary Johnson The Green Party Nominee, Jill Stein Or haven’t you decided 2% 5% 2% [If “haven’t yet decided” in question 4, ask:] 5) Which, if any, of these candidates – Clinton, Trump, Johnson, and Stein are you leaning toward? Hilary Clinton Donald Trump Gary Johnson Jill Stein None/Don’t Know 6) What would you list as the two or three primary reasons you favor (Insert candidate name from question 4 or 5)? August 18-23, 2016 POLL AZ1 3 -3[Interviewer: List reasons stated] 3 7) For whom did you vote for President in the 2012 election: Barack Obama, Mitt Romney, someone else or didn’t you vote in the presidential election in 2012? Assumptions Every poll makes assumptions. The difference in the results of each poll speaks, in part, to the difference in the assumptions made when conducting the poll. The CNN/ORC poll assumes that individuals accurately respond to questions about voter registration (and those planning to register) and the likelihood of voting. By weighting the data to census counts, they rely on the self-report of registration to implicitly generalize to the registered voter population. The Arizona Republic/Morrison/Cronkite News Poll used as its sampling frame an updated database of registered voters. This methodology makes the assumption that each campaign is likely to attract newly registered voters (who are not captured in the sampling frame that included primary voters) at the same rate. Demographic Comparisons to Registered Voters Although the weighting procedure compensates for the uneven distribution of respondents compared to the proportions in the sampling frame, it is still useful to access the demographics that comprise the sample. In addition, we also test whether certain demographics were more or less likely to respond to the voting questions. Thus, it is important to employ the weighting adjustments. For example, the sample was generally more female than male. Table A1: Unweighted answers to 'GENDER: (OBSERVED)' by 'All respondents' Male Female All respondents (N = 1689) 45.3 54.7 (± 2.4) (± 2.4) Notes: 95 percent margin of error below estimates in parentheses. 4 Table A2: Unweighted answers to 'GENDER: (OBSERVED)' by 'Did respondent answer voter and age questions' Male Female Did not respond to voter and age 44.2 55.8 questions (N = 985) (± 3.2) (± 3.2) Responded to voter and age questions (N 46.9 53.1 = 704) (± 3.8) (± 3.8) Notes: 95 percent margin of error below estimates in parentheses. Pattern of answers is not statistically significant based on unweighted Pearson chi-square test. About 60 percent of registered voters reside in Maricopa, 15 percent of registered voters reside in Pima, with about a quarter in the rest of the state. The 1689 interviews that were attempted reflect these proportions. However, the respondents to the voter questions are slightly more representative of Pima and less representative of Maricopa. Table A3: Unweighted answers to 'COUNTY (FROM SAMPLE)' by 'All respondents' Maricopa Pima Rest of Arizona All respondents (N = 1689) 63.7 11.6 24.7 (± 2.3) (± 1.6) (± 2.1) Notes: 95 percent margin of error below estimates in parentheses. Table A4: Unweighted answers to 'COUNTY (FROM SAMPLE)' by 'Did respondent answer voter and age questions' Maricopa Pima Rest of Arizona Did not respond to voter and age 66.9 8.8 24.3 questions (N = 985) (± 3.0) (± 1.8) (± 2.7) Responded to voter and age questions 59.2 15.5 25.3 (N = 704) (± 3.7) (± 2.7) (± 3.3) Notes: 95 percent margin of error below estimates in parentheses. Pattern of answers is statistically significant based on unweighted Pearson chi-square test. 5 Registration among parties is about 35 percent Republican, 30 percent Democrat, and the rest with other parties or with no preference. In our data, respondents to the voting questions tended to be a little more representative of Republicans, but no statistical association between party and response was found. Table A5: Unweighted answers to 'PARTY (FROM SAMPLE)' by 'All respondents' democrat republican OTHER/INDEPENDENT All respondents (N = 1689) 31.4 38.8 29.8 (± 2.3) (± 2.4) (± 2.2) Notes: 95 percent margin of error below estimates in parentheses. Table A6: Unweighted answers to 'PARTY (FROM SAMPLE)' by 'Did respondent answer voter and age questions' democrat republican OTHER/INDEPENDENT Did not respond to voter and age 31.0 37.4 31.7 questions (N = 985) (± 2.9) (± 3.1) (± 3.0) Responded to voter and age 32.0 40.8 27.3 questions (N = 704) (± 3.5) (± 3.7) (± 3.4) Notes: 95 percent margin of error below estimates in parentheses. Pattern of answers is not statistically significant based on unweighted Pearson chi-square test. 6 Other Demographic Characteristics The following tables present details about the unweighted demographics of the sample. Table A7: Unweighted answers to '1. Generally speaking, do you usually think of yourself as a Republican, a Democrat, an Independent, or what?' by 'All respondents' Republican Democrat Independent/Other All respondents 35.7 31.0 33.3 (N = 891) (± 3.2) (± 3.1) (± 3.2) Notes: 95 percent margin of error below estimates in parentheses. Table A8: Unweighted answers to '3. In the upcoming election, do you plan to vote for president?' by 'All respondents' Yes No Haven't decided All respondents (N = 93.0 2.2 4.8 867) (± 1.7) (± 1.0) (± 1.5) Notes: 95 percent margin of error below estimates in parentheses. Table A9: Unweighted answers to '7. For whom did you vote for President in the 2012 election: Barack Obama, Mitt Romney, someone else or didn't you vote in the presidential election in 2012?' by 'All respondents' Someone Didn't Don't Barack Obama Mitt Romney else vote know All 41.7 41.6 4.1 9.1 3.5 respondents (N = 849) (± 3.4) (± 3.4) (± 1.4) (± 2.0) (± 1.3) Notes: 95 percent margin of error below estimates in parentheses. 7 Table A10: Unweighted answers to 'Level of Education' by 'All respondents' Some High School or Less College All respondents (N = 800) 17.9 24.9 (± 2.7) (± 3.1) Bachelor's or more 57.3 (± 3.5) Notes: 95 percent margin of error below estimates in parentheses. Table A11: Unweighted answers to 'Racial Background' by 'All respondents' White All respondents (N = 954) 67.6 (± 3.0) Hispanic 10.4 (± 2.0) Other 22.0 (± 2.7) Notes: 95 percent margin of error below estimates in parentheses. Table A12: Unweighted answers to 'Age' by 'All respondents' All respondents (N = 784) Age 18-35 14.5 (± 2.5) Notes: 95 percent margin of error below estimates in parentheses. 8 Age 3650 16.6 (± 2.7) Age 51+ 68.9 (± 3.3)