!
FOR IMMEDIATE RELEASE
Contact:
• Eric Urbas, available for comment 12-1 by Cell 513-284-8161, and 6-7 at Board of Education, CTMPresident@CliftonCommunity.org
• Malcolm Montgomery, CTM Vice-President available for comment 10-6 by phone
513-961-6062x1, and 6-7 at Board of Education, Malcolm-CTM@EduTech.us
May 9, 2016

Clifton Town Meeting Warns of Flawed CPS Survey
Cincinnati – A recent survey sent by Cincinnati Public Schools (CPS) to some Clifton-area
families is flawed and incapable of providing valid answers according to an analysis by three
experts. Clifton Town Meeting (CTM) asked the experts to review the survey, which asked
questions about interest in locating a neighborhood school in the building that currently houses
the Clifton Cultural Arts Center (CCAC).
At the regular CTM Board meeting last Monday, Trustees presented a critique of the survey
based on a review by professionals that included two UC professors and a statistical consultant. Among other findings, the professionals indicated that the survey deviates from best
practices for creating unbiased questions, and that the responses cannot yield a credible basis
for decision-making.

“Clifton has long been a strong supporter of Cincinnati Public Schools and wants to continue
this support,” said Eric Urbas, President of CTM. “We campaign for school levies, raise money
for schools, volunteer time and resources to enrich students’ experiences, and are passionate
about the value of education for all children. Part of that support includes taking responsibility
for helping head off problems we see coming. Using flawed survey data will not help CPS or
our community find successful solutions to the educational needs for Clifton and CUF area residents.”
CTM cautioned the CPS Board and Administration about their concerns with the survey several times before CPS sent it out. Today, CTM sent a formal letter to CPS officials that included
the results of the professional review. The survey results won't be in until after May 15 when it
closes, but CTM doesn't want to see CPS base major decisions on bad data - whatever it says.

According to CTM, there is a growing concern in the Clifton community that CPS is not engaging the community on educational issues or in assessing the impact of evicting CCAC. CTM
believes that even the best survey is a poor substitute for meeting with the people who live in
the community to assess their educational needs, and has urged CPS to adopt a more collaborative approach for effective planning.
Reference Attachment: “How Would You Like to See a New Neighborhood
School in the former Clifton School Building?” An Evaluation by Research Professionals of the
Cincinnati Public Schools Survey Mailed ~April 15, 2016


 Four examples of observations from the research professionals:
(Reference other attachment for the full list & copy of the Survey)

 1) Survey as a Whole
“It is an exceedingly strange survey… It does seem that this survey was sort of
thrown together and that not much thought was given to pre-testing it with community input.”
- Steven R. Howe, Ph.D., Professor Emeritus of Psychology, University of Cincinnati
2) Survey Title: “How would you like to see a new neighborhood public school in the former
Clifton School building?”
“Biases respondents by presenting them with one side of multi-sided issue. The
questions used and their format also failed to provide the neutrality that is needed to get unbiased feedback.”
- Lee Rafales, Ph.D., Consultant, Business Analyst
3) CPS Survey Question 4: Do your children attend private – parochial or charter Schools?
“The survey assumes that people would make the same decision about all of their
children (i.e., it does not account for the fact that someone might be inclined to
leave their 5th grader in place but would be open to moving their 1st grader.”
- Steven R. Howe, Ph.D., Professor Emeritus of Psychology, University of Cincinnati
4) CPS Survey Question 6: "Would you send your child to a neighborhood school in Clifton
serving Clifton, CUF, and Spring Grove Village, if it involved a shared arrangement with CCAC
reimbursement for improvements?”
“Question 6 is perhaps the most important question and it is very badly written.
First, it is a compound question. There is no way to know which part or parts of it
the respondent is answering. There are four elements:
o 1 "in Clifton"?
o 2 "serving Clifton, CUF, and Spring Grove"?
o 3 "a shared arrangement with CCAC"?
o 4 "reimbursement for improvements"?
How does someone answer if they agree with one or some parts and oppose other parts?
Second, the question fails to define several critical terms. What is the nature of
the "shared arrangement"? Shared space? Shared program? Shared staff? The
respondent might like one, but not the others.
o 2. What is "CCAC"? Don't assume the respondent will know. At the least, you
need the full title and a brief description of what it does (written by CCAC, not
CPS).
o 3. What are the "improvements" and what does it mean to reimburse them? How
much will it cost taxpayers? Most important, what CCAC programs would be cancelled? How can the respondent know if or in what way that would affect their
children?”
- Paul Buckley, Statistical Consultant
# # #


 “How Would You Like to See a New Neighborhood School in the former Clifton
School Building?”
An Evaluation by Research Professionals of the Cincinnati Public Schools Survey Mailed ~April 15, 2016
Sponsored by Clifton Town Meeting
April 28, 2016
A photocopy of the survey form and information about research professionals’ credentials can
be found in the Appendix.
Researcher 1
Question 1: okay - needs a "Don't Know" option
Question 2: Badly written. The question is Yes/No, but the answer is not. Some parents will
misread this question and circle the grades their children are in. Some will want to include older children. Some will circle the number corresponding to the number of children in their family.
What does it mean to "have preschool or elementary children"? Does this include all children in
the household? Does it include non-custodial children who live elsewhere?
Question 3: "CPS" has not been defined. It seems obvious to us, but some respondents will be
confused and answer incorrectly. Again, what does it mean to "have preschool or elementary
children"? 
Question 4: How do you define "your children"? Only natural or adopted children? A niece or
grandson who lives with you? Other children living in the household with their own parents? Is
this asking about the same children as the previous two questions? If so, why the wording
change?
Question 5: It would have been good to define "neighborhood school." Again, it seems obvious
to us, but some respondents may think, for example, that it means a school of any kind that
happens to be located in the neighborhood.
Question 6: This is perhaps the most important question and it is very badly written.
First, it is a compound question. There is no way to know which part or parts of it the respondent is answering. There are four elements:
1
"in Clifton"?
2
"serving Clifton, CUF, and Spring Grove"?
3
"a shared arrangement with CCAC"? 
4
"reimbursement for improvements"?
How does someone answer if they agree with one or some parts and oppose other parts?
Second, the question fails to define several critical terms.
1
What is the nature of the "shared arrangement"? Shared space? Shared program? Shared staff? The respondent might like one, but not the others.
2
What is "CCAC"? Don't assume the respondent will know. At the least, you need
the full title and a brief description of what it does (written by CCAC, not CPS).
3
What are the "improvements" and what does it mean to reimburse them? How
much will it cost taxpayers?

 Most important, what CCAC programs would be cancelled? How can the respondent know if or
in what way that would affect their children?
Question 7: This question assumes a lot of knowledge on the part of the respondent. What
does this mean and what are the implications for the respondent's children? For example,
would it reduce the number of seats available for neighborhood children? Are all children enrolled in the school also in the special program or is it optional?
Question 8: This seems like a simple question, but the true answer will probably depend on
exactly where the school is located, how the school is set up (e.g., how will question 7 be answered by CPS?), etc. It is asking the respondent to make a decision based on little information. They will "fill in the blank" in order to answer and, when you analyze the data, you won't
know what they were assuming.
Question 9: okay, but there should have been at least an assurance of confidentiality - which
should have been included in the opening paragraph. Otherwise, some respondents will worry
that their answers may harm their children. It would have been better to ask for the information
on a separate page and respondent told that it would be entered into a separate data collection
system.
My other concerns are with representativeness of the sample plan (for which I have no information), what kind of weighting will be done to correct for sampling and response characteristics, and the size of the standard errors.

 Researcher 2
To: Board of Education, Cincinnati Public Schools
As you are aware, the former Clifton School building has for the last several years been leased
to the Clifton Cultural Arts Center (CCAC) and is now being proposed as the home for a new
Clifton neighborhood school.
I recently had the opportunity to view a survey that was distributed by CPS to assess the interest of families in Clifton and several adjacent neighborhoods for a neighborhood school which
would be physically located in the former Clifton School building.
I have over thirty years of experience as a business analyst working with people and organizations to understand their needs and help them achieve their objectives. While my activities
have not focused on survey development per se, I have had to identify and eliminate bias
when attempting to understand a group or work culture. In my estimation, the survey I viewed
was flawed and deviated from best practices for creating unbiased questions. The survey will
not only fail to achieve a fair assessment of the wishes of the community, but is likely to misrepresent those wishes.
While it is difficult to entirely remove bias, attempts to do so lead to better questions and better
feedback from respondents. For example, the title of the survey “How would you like to see a
new neighborhood public school in the former Clifton School building?” biases respondents by
presenting them with one side of multi-sided issue. The questions used and their format also
failed to provide the neutrality that is needed to get unbiased feedback. More typical of surveys are questions that provide a rating scale (e.g., not likely, somewhat likely, very likely) to
gauge interest along the entire spectrum. It is also a good idea to include at least one openended question that allows for unconstrained feedback from respondents.
I strongly encourage the board to view the results of this survey, whatever they are, with skepticism. I also suggest that in the future, when gathering information from the community, there
is a closer adherence to the standards and best practices of instruments used for this purpose.
Yours truly,
Lee Rafales, Ph.D.
Consultant, Business Analyst

 Researcher 3
It is an exceedingly strange survey.
 
1.
It assumes familiarity with entities such as CCAC and insight into what “reimbursement”
would entail and who would pay it, and in what amount.
2.
It assumes one’s current constellation of children will be identical to what the constellation would be at the time that a school might realistically be available.
3.
It assumes people’s residential attraction to Clifton would not change as the school becomes a reality; in other words, that people in Clifton looking for better schools now would not
move out, and that potential residents of Clifton who would be attracted to the new school
would not move in.
4.
It assumes that people would make the same decision about all of their children (i.e., it
does not account for the fact that someone might be inclined to leave their 5th grader in place
but would be open to moving their 1st grader.
5.
There’s an insinuation that people interested in the school would have to work at recruiting a student body.
 
But why not ask CPS if you can meet with the person or people who developed the survey and
ask to partner with them on creating a better one? Or at the very least ask them to do some
sort of large group planning event where people could come together and generate this kind of
data and more, and to do so with the opportunity to have CPS folks present. 
It does seem that this survey was sort of thrown together and that not much thought was given
to pre-testing it with community input or to generating community support for participation and
interpretation.
 
 

  Appendix
Researcher Identification
1

Paul Buckley. Social Science research credentials:
• 1985-95 Senior Systems Scientist and Associate Director of the Center for Computing
and Information Systems at the National Opinion Research Center, University of
Chicago - also Survey Director on a number of large-scale surveys
• 1995-98 Senior Associate and Department Manager of the Survey Operations Center
for Abt Associates, Inc. a social science research organization which is headquartered
in Cambridge MA - also served as Survey Director on large-scale surveys
• 1999-2005 Statistical Consultant, specializing in survey design, sample design, and
data analysis.
• “Most of my professional work was for Health & Human Services (principally the CDC)
and the Dept of Education, but also for a variety of other not-for-profits and government agencies.
• Since 2005, I have mostly been doing Quaker history and theology, but have done
some survey work for religiously-affiliated not-for-profits.”

2

Lee Rafales, Ph.D., Consultant, Business Analyst

3

Steven R. Howe, Ph.D., Professor Emeritus of Psychology, University of Cincinnati, and
Fellow of the Society for Community Research and Action
*1980-1993, UC Institute for Policy Research, where he was involved in numerous surveys conducted on behalf of governmental and not-for-profit entities, including the City
of Cincinnati and the Cincinnati Public Schools.
*1993 to 2014, UC Department of Psychology, where his work involved extensive use of
governmental and organizational survey data, including the U.S. Census, the American
Community Survey, the American Housing Survey, the VA’s All-Employee Survey, and
the Federal Employee Viewpoint Survey. He also organized numerous large-group
planning events involving both professionals and community participants.

  

Survey Form

filling?sz How wnuld gnu like to see a new
Neighborhu?d Public in the
fnriner Clifton Schnol Building?

a.de mm

but F?nili-nar

We hm Jamal Iliat why?i?un and {I?ll?un Fnlnim- and
Sprint might-Mind Enid-Ills in: him In trailing: ?Mm-hm pl?lt m.
In P'ub-lil: Schul- nl In such awn-DI.
Human-"plum mud hymn-m blimp: quad-um

1.. Any-nun midthlul': (Flags: thunk ml!)
Dar-n Dani-gauging. Elan:

 

.1.-

 

 

 

 

 

 

1. [ll-Gym Iu-m much-ml ar??mmw El 'I'h Hu-
[circle-all Ilul: mph]
12345??59mnu
3. [lawn turrenth-hmrn-hmEr: uraludantuianP?-dmmurrathad? DH.
ohm]? 
1. an yum-children attend [uh-ale- pumhm mchm?l? sci-null? "a
5. ?I'I'uuld m??pp?rl a n?mhn?md Echo-HI in Elihu; TII Cl lug-b-

??'iTll. Elihu. CUF Il?ld Eprin: Gm Village?

 

15. Um DH: Dingh-
nan-in; Ell?quCUFmd 
arr-ungumnl ECHE and 

 

AM ya: inlrrumt in naighha?'md :cl'nml Idnpling nip-?n] ping-am
udditinn hath-regular turticulum?? Il' ya. marijuana-?r mun.

Du- [jinn-law

 

 

 

5. ?you Imu- childran hm CPS trust-net urban-1 nr p?nlefmhil] In. 
ant-unm- uhml Locum! in the-am.de 
tun. wharf-mud Emuan Eli?m?'

 

Emilia-Hm?

gm?? How wnuld you like. to see a new
Public: School in the
farmer Clifton Schoui Building?

9. Wnukl gm- Built In ?nlnin?milmg [Emilia I'm-sud? Ethan-1'?
[fa-n. plenum-alum gnu: mutant 

 

Jun?milk!

 

Thank Jrlirui'ul' taking,