Hernando County School Board
Superintendent’s 360 Degree Feedback Survey Report
Submitted July 25, 2017

Prepared by:
Center for Research, Evaluation, Assessment, and Measurement
University of South Florida

i 
 

 About the Authors
George MacDonald, Ph.D.
George MacDonald is the Director for the Center for Research, Evaluation, Assessment, and
Measurement (CREAM) in the College of Education at the University of South Florida. George
holds a Ph.D. in Curriculum and Instruction with an emphasis in Measurement and Evaluation.
MacDonald is a Co-Principal Investigator for the National Science Foundation (NSF) funded
study, The Effects of Social Capital and Cultural Models on the Retention and Degree
Attainment of Women and Minority Engineering Undergraduates. He is responsible for a
number of program evaluations, and conducts the Assisted Living Facilities and State-Wide
Public Guardianship certification exams for the Florida Department of Elder Affairs. His
research agenda focuses on cognitive diagnostic assessment employing the linear logistic test
model particularly as it applies to mathematics education.
Reginald Lee, M.A.
Mr. Reginald Lee is a senior social and behavioral researcher at the Center for Research,
Evaluation, Assessment, and Measurement and a Co-Principal Investigator for the National
Science Foundation (NSF) funded study, The Effects of Social Capital and Cultural Models on
the Retention and Degree Attainment of Women and Minority Engineering Undergraduates. Mr.
Lee combines academic and workforce experience using mixed methods to address educational
policy issues. He is a former special education teacher and a candidate to receive his Ph.D. with
dual specializations in educational measurement and research, and special education (2017).

ii 
 

 Table of Contents
ABOUT THE AUTHORS ......................................................................................................................... II 
TABLE OF CONTENTS ........................................................................................................................ III 
CREAM OVERVIEW ..............................................................................................................................IV 
EXECUTIVE SUMMARY ........................................................................................................................ 1 
BACKGROUND ......................................................................................................................................... 2 
PSYCHOMETRICS ................................................................................................................................... 2 
Survey Validity ............................................................................................................................... 2 
Survey Reliability ........................................................................................................................... 3 
Method ............................................................................................................................................ 3 
Item Performance .......................................................................................................................... 4 
Cognitive Diagnostic Assessment of Survey ................................................................................ 4 
RESULTS .................................................................................................................................................... 6 
Sample Results ............................................................................................................................... 6 
School‐based Respondents............................................................................................................. 9 
District-based Respondents ......................................................................................................... 10 
Sample Results by Domain .......................................................................................................... 12 
ANONYMOUS THEMATIC ANALYSIS.............................................................................................. 17 
REFERENCES .......................................................................................................................................... 22 
APPENDICES ........................................................................................................................................... 23 

iii 
 

 CREAM Overview
The Center for Research, Evaluation, Assessment and Measurement (CREAM) is a “hub” of
research activity in the College of Education. It houses all funded research projects in the
College, provides a venue for faculty engagement in interdisciplinary research efforts, and
supports faculty engagement in a broad range of research activities (e.g., qualitative, quantitative,
and mixed methods). In addition, CREAM utilizes the talents of faculty in the College of
Education to provide a broad spectrum of research-related services (e.g., program evaluation,
analytical inquiry, and technical consultation) at the local, regional, national, and international
level.
Mission
To engage in high-quality research, evaluation, assessment, and measurement efforts and
enhance the culture of research productivity in the College of Education
Objectives:







Facilitate the implementation of high quality funded-research projects
Facilitate faculty and student engagement in interdisciplinary research, evaluation,
assessment, or measurement opportunities
Provide consulting services related to research design through the Consulting of Research
in Education (Core)
Build and extend the faculty and student capacity for engagement in high quality
research, evaluation, assessment, or measurement efforts through professional
development opportunities
Showcase faculty and student research, evaluation, assessment, or measurement efforts.

iv 
 

 Executive Summary
The University of South Florida, Center for Research, Evaluation, Assessment and Measurement
(CREAM), was contracted by the Hernando County School Board (HCSC) to develop, administer
electronically, and analyze a Superintendent’s 360 Degree Feedback Survey instrument.
CREAM was provided names and/or email addresses for 3,051 employees. Responses were
received from 942 individuals. Of those who began the survey CREAM received data from 4
school board members, 40 School-based Administrators, 548 School-based Instructional Staff, 185
School-based non-Instructional Staff, 109 District Staff, 26 School-based Administrators, and the
Superintendent.
There were 52 Likert type items on the online Survey and two open ended items. The items were
developed employing Danielle Stufflebeam’s Superintendents Duties (Stufflebeam, 1995). The
items were mapped to the nine domains of the Superintendent Competencies of the American
Association of School Administrators (AASA).
We examined the average respondent rating by item. The results demonstrate the Superintendent
received average scores between 3.01 and 3.49 on 18 of the 52 items and was scored between 2.0
and 2.99 on 33 items. One of the items was exactly on the mean. In sum, the performance of the
Superintendent as reported by employees of the Hernando County School District is 2.86 which
falls between slightly to moderately effective on the survey scale.
Next we analyzed the item responses by AASA domain using cognitive diagnostic assessment
modeling. We employed the approach used by MacDonald (2013). The non-linear fixed
procedure fits the model by maximizing an approximation to the likelihood integrated over the
random effects (SAS PROC NLMIXED). These results demonstrate a similar pattern as seen in
the item means. Results indicated her strongest areas of performance was Values and Ethics of
Leadership and her weakest area was Communication and Community Relations. According to
Cohen (1986) both of these fall into the range of medium sized effects (.50 - .79). Results also
demonstrate that School-based Administrators and District Administrators rate the Superintends
performance higher than Instructional, non-instructional and District Staff. In brief, her overall
performance ratings as reported by employees of the District is average with one domain below
average and another above average.
The Qualitative results as reported in 90 pages of written response to the two open-ended items
indicate some believe she has strong beliefs, is guided by deeply held principles and believes in her
students and their success. They believe she has a vision, establishes high expectations for all staff
and is a data driven decision maker. However, others have strong feelings she leads a fear-based
administration, is vindictive when solving disagreements, communicates poorly with all groups
and retaliates against those who disagree with her. Many are now discouraged with her job
performance and believe she has broken their trust. These people believe it is time for a change in
Superintendents.

1 
 

 Background
The University of South Florida, Center for Research, Evaluation, Assessment and Measurement
(CREAM), was contracted by the Hernando County School Board (HCSC) to develop, administer
electronically, and analyze a Superintendent’s 360 Degree Feedback Survey instrument. The
development process was initiated following a psychometric consultation (Dr. George MacDonald,
2016) to the Hernando County School Board chaired by First Last. In an effort to reduce costs to
the school district the survey was administered electronically.
CREAM consulted with the school board, former Deputy Superintendent Eric Willams, and
Superintendent Lori Romano. Protocols were developed to conduct interviews and focus groups
with teachers, principals, parents, and district staff. Visits to an elementary and a high school
afforded opportunities to learn what the various constituencies considered important in the survey
instrument. Development of the survey was an iterative process with changes made to the
instrument at each step. The survey was piloted with a group of high school teachers in a paper and
pencil format. The survey was presented to Superintendent Romano in a meeting at the district
office the day before Thanksgiving 2016. The Superintendent shared the survey with her cabinet
and solicited their feedback. Their comments were considered and a request to meet with the
School Board was made of the Superintendent. Dr. MacDonald presented the survey to a School
Board Workshop on January 24th, 2017. Following the workshop final changes were made to the
survey as requested by the School Board members.
To assure respondents’ anonymity the final survey was administered through CREAM and
responses secured on a password protected server at the University of South Florida.
Psychometrics
Survey Validity
It is necessary to establish that the survey is reliable or a consistent measure of the performance
domains required by a school district superintendent. It is possible for a survey to be reliable and
not be a valid measure. Evidence of validity is important in determining the appropriate use of the
survey as a tool to add value to the annual evaluation of a superintendent. There are four types of
validity to consider in assessing the validity of the survey instrument:
1. Content
2. Criterion-related
3. Construct
4. Consequential
Content validity considers the alignment between survey questions and performance domains. We
conducted individual and focus group interviews with two school principals, two groups of
teachers (elementary and high school), three school board members, and Superintendent Romano
during the survey development phase to judge content validity. This process included both face
validity in that survey items appear to measure a construct (e.g., policy and governance). Updated
2 
 

 content –related evidence of validity came from the judgments of district staff and school board
members’ feedback following a January Board Workshop. An additional step in determining
content validity evaluated the survey items from the perspective of Stufflebeam’s Evaluation of
Superintendent Performance: Toward a General Model (1995).
Construct validity refers to whether the survey items measure domains that capture the intersection
of superintendent competencies (AASA) and superintendent of a school district superintendent.
Questions permitted respondents to provide their level of agreement or perceived level of
effectiveness to the specific questions mapped to the domains of interest. Appendix B presents
Stufflebeam’s matrix.
Survey Reliability
The Superintendent’s 360 Degree Feedback Survey is a 52-question survey with questions mapped
to nine domains. Responses were dichotomized at the mean to obtain parameter estimates using De
Boeck and Wilsons (2004) extension of Fischer’s Linear Logistic Test Model (1973).
Cronbach’s alpha is a measure of internal consistency. The reliability estimate for the total items
on the scale is 0.99. Values above 0.7 are considered acceptable, and values approaching 0.9 are
considered good (Cronbach, 1951). The Cronbach alpha values for the individual domains range
between 0.80 and 0.98 which provides strong evidence that the survey is reliably targeting the nine
domains.
When considering the use of cognitive diagnostic assessement, reliability estimates stabilize once
the sample size has reached approximately 600 respondents (MacDonald, 2013). A sample size
above 600 per domain provides strong evidence the parameters estimates for this survey are stable.
Therefore we can have confidence the parameter estimates on the nine domains ranging from 0.71
to -0.68 are reliable. Summary statistics for the full sample are found in Tables 4 and 5.
Method
Responses form a row in the person response matrix which is transposed so that person becomes an
item group and item responses are nested in the person (group) and the q-matrix. The person ability
estimate (theta) was generated (approximately distributed) from a normal distribution with a mean
0, and standard deviation 1.
A Q-matrix (a.k.a., weight matrix) was created to provide the initial cognitive component values,
the item-response matrix was developed from survey responses from Hernando County School
District Superintendent’s 360 Degree Feedback Survey.
Sample. A sample of Hernando County School District employees, school board members and the

Superintendent was developed from responses to a USF-CREAM administered online survey (942
respondents, 52 survey Likert-type items, scored dichotomously).
Procedure. The analysis of the survey sample followed the approach used by MacDonald (2013).

SAS’s NLMixed Procedure was employed to fit a non-linear mixed model that maximizes an
approximation to the likelihood integrated over the random effects. When the marginal likelihood
is formed in this way, the undesirable limitation on person inferences is avoided.
3 
 

 Item Performance
Individual survey item responses are presented in Tables 4 and 5 using the percentage responding
above the mean rating as agree and effective. High percentages indicate that for a particular survey
question respondents were more likely to respond favorably (agree, effective) while low
percentages indicate that respondents were less likely to respond favorably (disagree, not
effective). The average rating was above three on 18 of the 52 questions (34.5 %) indicating that
respondents were more likely to agree or view the Superintendent as effective. The average rating
was below three on 33 of the 52 questions (63.4 %) indicating these respondents were less likely to
agree or view the Superintendent as effective. There was one question with a mean rating of three
(1.9 %).
Cognitive Diagnostic Assessment of Survey
The Linear Logistic test Model (LLTM) (Fisher, 1973; MacDonald, 2013), is a cognitive
diagnostic assessment approach, used to derive the estimates to evaluate the validity of the
domains surveyed.
A negative estimate provides evidence that respondents were more likely to endorse an item.
Positive value indicate respondents were less likely to endorse an item.
These estimates allow us to see differences between performances ratings for school-based
administrators, district administrators, school-based instructional staff, school-based noninstructional staff, school board members and district staff.
Before estimation, every item is dichotomized. The Likert items are dichotomized at the mean
with 1 indicating above mean in agree/effective direction, and 0 below mean in disagree/ineffective
direction.
The model that is estimated has the following form
Pr  p  

exp  p  i 

1  exp  p   i 

Pr  p 

where

 

is the probability of the respondent giving a 1 response to item i,

p

is the “ability”

of the respondent, and a i is the difficulty of item i. Domains enter into the model as determinants
of the difficulty of each item, as expressed in the following equation
p

i   ij j  c  
j 1

where

j

j

is the parameter estimate for the cognitive component (CKS) j,

 ij

is the given weight

of
with respect to the difficulty of item i, and c is an arbitrary normalization constant. In this
survey there are nine domains and each item has a weight on each of the nine domains that is either
1 or 0 as determined by researcher judgment and represented in the “Q matrix.”
4 
 

  
If we substitute the second equation into the first we get:
p


exp   p   ij j  c 
j 1

  
Pr  p  
p


1  exp   p   ij j  c 
j 1





From inspection of this equation we can see that a negative parameter estimate for a given j
makes the term in parenthesis larger and so increases the probability of a 1 response to the item. A



positive parameter estimate for a given j makes the term smaller and so decreases the probability
of a 1 response. In the original context of testing where a 1 response means person i was correct
on item j, negative coefficients mean easier items (higher probabilities of superintendent
effectiveness and positive coefficients mean harder items (lower probabilities of superintendent
effectiveness).

5 
 

 Results
Sample Results 
Data were collected from two sources provided by Hernando County School Board staff. We were
provided a list of names and email addresses. These individuals were emailed an individual link to
access the survey. We were also provided a list of names and work units for individuals that did
not have a school district email address. Individuals whose names were provided in this manner
were directed to specific school sites where they could access the survey portal through the school
librarian (See table 1).
Table 1. Hernando Superintendent 360 Degree Feedback Survey Distribution Summary
Email Invitation
Personal Link
Total

Respondent Pool
2549
502
3051

Respondents
910
32
942

Completion Rate
35.7 %
6.4 %
30.9 %

Participants were presented with a letter authorized by the Hernando School Board and
Superintendent Lori Romano describing the purpose of the survey, the intended use of the survey
results, and assurances regarding the confidentiality of their responses.
Respondents were asked to provide the position that best described their role with the Hernando
County School District from among seven options. The roles, number and percent are presented in
table 2.
 

Table 2. Respondent, School District Roles, Number and Percent
Position
School Board Member (1)
School‐based Administrator (2)
School‐based Instructional Staff (3)
School‐based Non‐Instructional Staff (4)
District Staff (5)
District‐based Administrator (6)
Superintendent (7)

Number
4
40
548
185
109
26
1

Percent
0.4
4.4
60.0
20.3
11.9
2.8
0.1

 

The coefficients presented in Table 3 indicate the Superintendent performance as rated by the
respondents. The estimates use a logit scale (-4 to +4).
Principals, assistant principals, teachers, and non-instructional staff comprise the school-based
respondents. In the context of 360 degree feedback survey we might think of the four groups as
school levels each a bit further from direct supervision or influence from the Superintendent. In the
Hernando County School District the Superintendent recommends individuals for the principal role
and these individuals serve at the pleasure of the Superintendent. In turn the principals are
responsible for sharing the Superintendent’s vision with the staff in their respective schools.
Results for school-based administrators suggest they found it easier to endorse the effectiveness of
the Superintendent’s performance when compared to instructional and non-instructional staff for
6 
 

 seven of the nine domains surveyed. With the exception of Leadership and District Culture and
Human Resources Management school-based administrators were more likely to endorse the
effectiveness of the superintendent than instructional or non-instructional staff. Table 3 presents
the results of the Cognitive Diagnostic Assessment for the Superintendent Competencies by sample
size and role group.
 

Table 3. Linear Logistic Test Model Domain Estimates for Sample and Role Groups
Survey Domain
Leadership and District Culture
Policy and Governance
Communication & Community Relations
Organizational Management
Curriculum Planning and Development
Instructional Leadership
Human Resources Management
Values and Ethics of Leadership
Labor Relations

Sample
‐0.13
0.13
0.71
0.07
0.29
-0.07
0.32
‐0.68
0.17

2
0.1
‐0.13
0.11
‐0.35
‐0.32
‐0.36
0.71
‐1.08
‐0.94

3
‐0.03
0.17
0.94
0.22
0.46
‐0.11
0.42
‐0.69
0.26

4
‐0.22
0.07
0.34
‐0.31
0.34
0.22
0.17
‐0.58
0.33

5
‐0.39
0.31
0.26
0.04
‐0.29
‐0.04
0.2
‐0.59
‐0.21

6
‐0.21
0.49
0.38
‐0.31
‐0.05
‐0.56
0.44
‐0.83
‐0.52

Note: 1‐ School Board members (omitted), 2‐School‐based Administrators, 3‐School‐based Instructional
staff, 4‐School‐based Non‐Instructional staff, 5‐District Staff, 6‐District‐based Administrators
 

Table 4 contains the individual items on which the Superintendent was rated above the mean. The
values in table 4 are interpreted as the proportion agreeing, or judging the superintendent to be
effective for individual survey questions.
In brief the respondents rated the superintendent highest for promoting high standards for staff
performance (3.49). In absolute terms, however, the Superintendents highest rating of 3.49
provides evidence she is judged to be slightly above neutral and just below agree on
Superintendent effectiveness for the items contained in table 4.
Table 4. Number, Mean, Standard Deviation & Item-to-total Correlation for Items above the Mean
Item

N

Mean

S.D.

Promotes high standards for staff performance
Delegates responsibility
Promotes high standards for teaching
Ensures use of student data to adapt practices
Promotes the school district's mission and vision
Promotes high standards for learning
Ensures professional development for staff
Participates in community events and activities
Ensures the evaluation of all staff
Ensures use of current research to adapt practices
Ensures school district compliance with federally‐funded
education program requirements
Models appropriate moral leadership
Ensures high quality curriculum
Ensures school district compliance with legal requirements

759
655
743
637
762
750
668
610
627
615
550

3.49
3.49
3.48
3.48
3.44
3.42
3.35
3.34
3.34
3.21
3.17

1.41
1.39
1.40
1.39
1.41
1.43
1.44
1.45
1.44
1.41
1.30

743
652
559

3.14
3.11
3.10

1.48
1.44
1.34

Item‐Total
Correlation
0.74
0.69
0.74
0.77
0.77
0.76
0.75
0.77
0.73
0.82
0.85
0.79
0.80
0.85
7 

 

 Ensures improvement in student achievement
Implements school board policies and practices
Ensures leadership development for staff
Communicates with School Board members
Recognizes others for their professional efforts

648
618
638
443
753

3.09
3.07
3.06
3.03
3.00

1.46
1.29
1.46
1.40
1.48

0.86
0.87
0.85
0.86
0.79

Table 5 contains the individual items on which the Superintendent was rated below the mean. The
values in table 5 are interpreted as the proportion dis-agreeing, or judging the superintendent to be
ineffective for individual survey questions.
In brief the respondents rated the superintendent lowest for communication with school volunteers
(2.00) In absolute terms the Superintendents lowest rating of 2.00 provides evidence she is judged
to be somewhat or slightly effective. When the scale is agreement a low mean suggest the
respondents somewhat disagree the superintendent is effective.
In brief, the respondents rated the superintendent as ineffective in communications with almost
every group.
Table 5. Number, Mean, Standard Deviation & Item-to-total Correlation for Items below the Mean
Item
N
612
Ensures maintenance of district facilities
610
Ensures development of district technology plan
Ensures effective culturally diverse policies and practices 569
524
Recommends policies to the school board
Ensures school district compliance with special education 577
479
Communicates with School Administrators
666
Shares authority
Empowers leadership of others
710
645
Balances community concerns and the best interest of
641
Provides timely information
729
Takes risks to encourage positive change
441
Communicates with Business and community leaders
633
Supports new ideas
670
Leads by example
Shares opportunities to problem solve
617
Communicates about the Information parents need
624
636
Actively listens to diverse opinions and interests
Ensures approaches to attract highly qualified professionals 635
686
Supports independent action
674
Listens to needs and concerns
633
Maintains effective working relationships
Communicates with the groups
544
Explains budget issues and budget decisions to community 608
740
Works effectively to build consensus
538
Communicates with the News media
636
Explains budget issues and budget decisions to school
672
Communicates about the needs of the school district
Communicates about the information staff need
635
675
Communicates about the information teachers need

Mean S.D.
Item‐Total
2.96
1.47
0.78
2.95
1.47
0.83
2.95
1.37
0.87
2.94
1.27
0.88
2.90
1.36
0.83
2.86
1.40
0.91
2.85
1.44
0.76
2.83
1.48
0.84
2.78
1.46
0.92
2.76
1.44
0.87
2.76
1.44
0.83
2.74
1.47
0.85
2.74
1.46
0.88
2.71
1.54
0.91
2.62
1.50
0.91
2.58
1.40
0.88
2.57
1.50
0.88
2.57
1.48
0.87
2.55
1.42
0.84
2.55
1.53
0.87
2.55
1.50
0.90
2.51
1.47
0.87
2.49
1.42
0.86
2.48
1.45
0.87
2.45
1.40
0.86
2.43
1.41
0.87
2.38
1.43
0.89
2.31
1.41
0.91
2.25
1.40
0.89
8 

 

 Communicates with parents and students
Communicates with School Support and Facility Staff
Communicates with teachers
Communicates with school volunteers

596
543
668
445

2.20
2.09
2.03
2.00

1.36
1.40
1.35
1.37

0.86
0.85
0.84
0.81

School-based Respondents
Figure 1 presents graphically the parameter estimates in logits for school-based respondents,
administrators, instructional staff, and non-instructional staff. High negative logits indicate groups
were more likely endorse the Superintendent’s performance on a domain. High positive logits
indicates that respondents were less likely to endorse the Superintendent’s performance on a
domain. The parameter estimates presented in Table 3 are described below.
School administrators were more likely to endorse the Superintendent’s performance on the Values
and Ethics of Leadership (-1.08), Labor Relations (-0.94), Instructional Leadership, (-0.36),
Organizational Management (-0.35), and Curriculum Planning, and Development (-0.33) domains.
School administrators were less likely to endorse the Superintendent’s performance on Human
Relations Management (0.71).
School-based instructional staff were more likely to endorse the Superintendent’s performance on
the Values and Ethics of Leadership (-0.69) domain. School-based instructional staff were less
likely to endorse the Superintendent’s performance on the Communications and Community
Relations (0.94), Curriculum Planning, and Development (0.46), Human Relations Management
(0.42), and Labor Relations (0.26) domains.
Non-instructional staff were more likely to endorse the Superintendent’s performance on two
domains, Values and Ethics of Leadership (-0.58) and Organizational Management (-0.31) and
were less likely to endorse the Superintendent’s performance on the Curriculum Planning, and
Development (0.34), Communications and Community Relations (0.34), and Labor Relations
(0.33) domains.

 

Figure 1. Linear Logistic Test Model Parameter Estimates for School‐based Individuals
9 
 

 District‐based Respondents 
Figure 2 presents graphically the parameter estimates in logits for district-based respondents,
district-based administrators, district staff, and school board members. As in the previous section,
high negative logits indicate that respondents were more likely to endorse the Superintendent’s
performance on the domains. High positive logits indicates that respondents were less likely to
endorse the Superintendent’s performance on a domain. The parameter estimates presented in
Table 3 are discussed below.
District staff were more likely to endorse the Superintendent’s performance on three domains,
Values and Ethics of Leadership (-0.59), Leadership and District Culture (-0.39) and Curriculum
Planning, and Development (-0.29). District staff were less likely to endorse the Superintendent’s
performance on the Policy and Governance (0.31), and Communications and Community Relations
(0.26) domains.
District-based administrators were more likely to endorse the Superintendent’s performance on the
Values and Ethics of Leadership (-0.83), Instructional Leadership (-0.56), Labor Relations (-0.52),
and Organizational Management (-0.31) domains. District-based administrators were less likely to
endorse the Superintendent’s performance on the Policy and Governance (0.48), Human Resources
Management (0.43) and Communications and Community Relations (0.38) domains.

 

Figure 2. Linear Logistic Test Model estimates for District‐based Staff & School Board Members

Figure 3 presents graphically the parameter estimates in logits for the three organizational units,
academic services, business services, and support operations. As mentioned previously, high
negative logits indicate that respondents were more likely to endorse the Superintendent’s
performance on the domains. High positive logits indicates that respondents were less likely to
endorse the Superintendent’s performance on a domain. Table 6 presents the parameter estimates
for the full sample and the organizational units.
10 
 

 Academic Services respondents were more likely to endorse the Superintendent’s performance on
the Values and Ethics of Leadership (-0.35) and Labor Relations (-0.26) domains. Academic
Services respondents were less likely to endorse the Superintendent’s performance on the Policy
and Governance (0.37), and Communications and Community Relations (0.23) domains.
Business Services respondents were more likely to endorse the Superintendent’s performance on
the Values and Ethics of Leadership (-1.08) and Organizational Management (-0.79) domains.
Business Services respondents were less likely to endorse the Superintendent’s performance on the
Labor relations (0.93), Leadership and District Culture (0.52), and Curriculum Planning and
Development (0.45) domains.
Support Operations respondents were more likely to endorse the Superintendent’s performance on
the Values and Ethics of Leadership (-0.77) and Leadership and District Culture (-0.51) domains.
Support Operations respondents were less likely to endorse the Superintendent’s performance on
the Curriculum Planning and Development (1.20) and Human Resources Management (0.52), and
Labor Relations (0.26) domains.

Figure 3.  Linear Logistic Test Model Parameter Estimates for Organizational Units 
 
Table 6. Linear Logistic Test Model Estimates for Total Sample and Organizational Units
Survey Domain
Leadership and District Culture
Policy and Governance
Communication and Community Relations
Organizational Management
Curriculum Planning and Development
Instructional Leadership
Human Resources Management

Sample
‐0.13
0.13
0.71
0.07
0.29
-0.07
0.32

AS
‐0.13
0.37
0.23
0.00
‐0.01
0.01
0.14

BS
0.52
‐0.20
0.03
‐0.79
0.45
‐0.19
‐0.16

SO
‐0.51
‐0.08
0.18
‐0.21
1.20
0.07
0.52
11 

 

 ‐0.68
0.17

Values and Ethics of Leadership
Labor Relations

‐0.35
‐0.26

‐1.08
0.93

‐0.77
0.26

 

Note; 1‐Academic Services, 3‐Business Services, 4‐Support Operations

Sample Results by Domain 
Tables 7 to 15 present the dichotomized variables used in the linear logistic test model analyses.
Items were dichotomized so that any item with a mean rating of 3.25 or higher was coded 1 while
the values less than the mean rating were coded 0. The values in the tables below should be
interpreted as the proportion agreeing, or judging the Superintendent as effective for each survey
item. The item means for the original variables are reported in Tables 4 and 5.
Leadership and District Culture 

The parameter estimate for this domain (-0.13) suggests an average rating for her performance.
Fourteen items mapped to this domain with the proportion greater than 0.50 for nine items. Nearly
sixty percent of respondents agreed that the Superintendent promotes high standards for teaching
the highest rated item on this domain. Similar proportions of respondents agreed that the
Superintendent shares authority (57.3), promotes high standards for staff performance (57.2%),
promotes the school district's mission and vision (56.9%), and promotes high standards for learning
(56.1%). Only 42 % agreed that that the Superintendent recognizes others for their professional
efforts.
Table 7. Leadership and District Culture Items
Domain
Leadership and District
Culture
(N=14, Alpha = 0.94)

Description (Sample Items)















Models appropriate moral leadership
Promotes the school district's mission and vision
Promotes high standards for teaching
Promotes high standards for learning
Promotes high standards for staff performance
Delegates responsibility
Empowers leadership of others
Shares authority
Supports independent action
Recognizes others for their professional efforts
Takes risks to encourage positive change
Works effectively to build consensus
Actively listens to diverse opinions and interests
Supports new ideas

Policy and Governance 

The parameter estimate for this domain (0.13) suggests an average rating for her performance.
Nine items mapped to the Policy and Governance domain with the proportion greater than 0.50 for
three items. Almost two‐thirds of respondents rated the Superintendent’s level of effectiveness
highly in response to the question, recommends policies to the school board (64.88%). Slightly
12 
 

 fewer than 40 percent of respondents rated the Superintendent as effectively communicating the
needs of the school district (38.09%).
Table 8. Policy and Governance Items
Domain
Description (Sample Items)
Policy and Governance  Recommends policies to the school board
(N=9, Alpha = 0.93)
 Implements school board policies and practices
 Ensures school district compliance with federally‐funded education
program requirements
 Ensures school district compliance with special education program
requirements
 Ensures school district compliance with legal requirements
 Ensures effective culturally diverse policies and practices
 Explains budget issues and budget decisions to school district staff
 Explains budget issues and budget decisions to community
members
 Communicates effectively about the needs of the school district
Communications and Community Relationships 

The parameter estimate for this domain (0.71) is statistically significant and a moderate effect. It
provides evidence that this domain is an area of concern for the Superintendents’ performance.
Nineteen items mapped to the Communications and Community Relations domain with the
proportion greater than 0.50 for seven items. Over sixty percent of respondents reported that the
Superintendent effectively communicates information that parents need (61.1%). Only 28 percent
agreed that the superintendent effectively communicates with teachers.
Table 9. Communications and Community Relationships Items
Domain
Description (Sample Items)
Communications and  Promotes the school district's mission and vision
Community Relations  Works effectively to build consensus
(N=19, Alpha = 0.98)  Explains budget issues and budget decisions to community
members
 Communicate with School Board members
 Communicate with School District staff
 Communicate with School Administrators
 Communicate with Teachers
 Communicate with School support and facilities staff
 Communicate with School volunteers
 Communicate with Parents and students
 Communicate with Business and community leaders
 Communicate with News media
 Communicates information parents need
 Communicates information teachers need
 Communicates information staff need
 Communicates needs of the school district
 Provides timely information
 Participates in community events and activities
13 
 

 

Balances community concerns and the best interest of students

Organizational Management 

The parameter estimate for this domain (0.07) suggests an average rating for her performance.
Sixteen items mapped to this domain with the proportion greater than 0.50 for six items. Almost
sixty percent of respondents reported that the Superintendent ensures maintenance of district
facilities (58.5%), ensures development of district technology plan (57.4%), and shares authority
(57.4%). Smaller proportions of respondents report that the Superintendent communicates
information teachers need (34.5%), communicates information staff need (35.9%), or
communicates about the needs of the school district (38.1%).
Table 10. Organizational Management Items
Domain
Organizational
Management
(N= 16, Alpha = 0.96)

Description (Sample Items)

















Delegates responsibility
Empowers leadership of others
Shares authority
Supports independent action
Works effectively to build consensus
Communicates with School District staff
Communicates information teachers need
Communicates information staff need
Communicates about the needs of the school district
Actively listens to diverse opinions and interests
Supports new ideas
Maintains effective working relationships
Shares opportunities to problem solve
Listens to needs and concerns
Ensures maintenance of district facilities
Ensures development of district technology plan

Curriculum Planning and Development 

The parameter estimate for this domain (0.29) suggests an average rating for her performance. Four
items mapped to this domain with the proportion greater than 0.50 for a single item. Respondents
report that the Superintendent ensures use of student data to adapt practices (57.4%), however a
smaller proportion report that the Superintendent ensures improvement in student achievement
(43.7%)
Table 11. Curriculum Planning and Development Items
Domain
Curriculum Planning
and Development
(N = 4, Alpha = 0.90)

Description (Sample Items)





Ensures high quality curriculum
Ensures use of student data to adapt practices
Ensures use of current research to adapt practices
Ensures improvement in student achievement

14 
 

 Instructional Management 

The parameter estimate for this domain (-0.07) suggests an average rating for her performance.
Eleven items mapped to this domain with the proportion greater than 0.50 for five items.
Respondents reported that the Superintendent promotes high standards for teaching (58.0%),
ensures use of student data to adapt practices (57.4%), and promotes high standards for learning
(56.1%). However, less than thirty percent of respondents report that the Superintendent
communicates with teachers (28.1%).
Table 12. Instructional Management Items
Domain
Instructional
Leadership
(N= 11, Alpha = 0.94)

Description (Sample Items)












Promotes high standards for teaching
Promotes high standards for learning
Communicates with School Administrators
Communicates with Teachers
Listens to needs and concerns
Ensures high quality curriculum
Ensures use of student data to adapt practices
Ensures use of current research to adapt practices
Ensures professional development for staff
Ensures leadership development for staff
Ensures improvement in student achievement

Human Resources Management 

The parameter estimate for this domain (0.32) suggests an average rating for her performance.
Seven items mapped to this domain with the proportion greater than 0.50 for three items. Over
sixty percent reported that the Superintendent ensures effective culturally diverse policies and
practices (62.7%). A majority of respondents reported that the Superintendent ensures professional
development for staff (54.8%) and ensures the evaluation of all staff (51.2%). However, close to a
third of respondents reported the Superintendent does not communicate with school support and
facilities staff (30.9%) or communicate with teachers (28.1%).
Table 13. Human Resources Management
Domain
Human Resources
Management
(N=7, Alpha= 0.90)

Description (Sample Items)








Ensures effective culturally diverse policies and practices
Communicates with Teachers
Communicates with School support and facilities staff
Ensures professional development for staff
Ensures leadership development for staff
Ensures approaches to attract highly qualified professionals
Ensures the evaluation of all staff

Values and Ethics of Leadership 

The parameter estimate for this domain (-0.68) is statistically significant and a moderate effect. It
provides evidence that this domain is an area of effectiveness for the Superintendents’
performance. Four items mapped to this domain with the proportion greater than 0.50 for three
15 
 

 items. Respondents report the Superintendent balances community concerns and the best interest of
students (53.6%) and promotes the school district’s mission and vision (56.9%). Less than half of
respndents report that the Superintendent models appropriate moral leadership (46.0%).
Table 14. Values and Ethics of Leadership Items
Domain
Values and Ethics of
Leadership (N=4,
Alpha = 0.84)

Description (Sample Items)





Models appropriate moral leadership
Promotes the school district's mission and vision
Ensures effective culturally diverse policies and practices
Balances community concerns and the best interest of students

Labor Relations 

The parameter estimate for this domain (0.32) suggests an average rating for her performance.
Three items mapped to this domain with the proportion greater than 0.50 for one item. Less than
half of respondents reported that the Superintendent maintains effective working relationships
(44.3%) or listens to needs and concerns (43.9%).
Table 15. Labor Relations Items
Domain
Labor Relations
(N=3, Alpha= 0.80)

Description (Sample Items)




Maintains effective working relationships
Listens to needs and concerns
Ensures the evaluation of all staff

16 
 

 Anonymous Thematic Analysis
The online survey included two open ended questions to obtain information from respondents that
could be used to provide additional feedback to the school board in addition to the quantitative data
shared in the report. The items received 412 responses out of 917 respondents. Using this count
45% of the survey respondents commented. USF will retain the raw responses and condensed
reports from these open ended items but due to the requests for anonymity from Hernando County
School Board and survey respondents we will not be providing direct quotes.
The respondents were asked to respond to the following open-ended item.
In what areas is the Superintendent especially effective?



The most common response to this item was that the Superintendent had no areas in which
she was especially effective.
Many responded they had no intensive interaction or no experience with the
Superintendent.

There were positive responses to the Superintendents effectiveness, which can be summarized as:














Dr. Romano is a person of strong beliefs and principals.
Dr. Romano believes in students and their success in school.
Dr. Romano is efficient and has vision having established high expectations for overall staff
and students.
Dr. Romano is a decision maker who understands policies, procedures, curriculum and
technology.
Dr. Romano is trying to change the district culture to implement a data driven model
scaffolding student success.
Dr. Romano is approachable, professional, remembers names, and is willing to entertain
ideas.
Dr. Romano communicates well with District Administrators and School-based
Administrators.
Dr. Romano is professional, courteous, and willing to listen to ideas.
Dr. Romano is effective in building district leadership and creates an encouraging
environment for employees.
Dr. Romano communicates well with staff, community partners, stakeholders, families and
students.
Dr. Romano conducts on-site visits in schools and is especially effective at not being pulled
into community drama.
Dr. Romano is well-spoken, delegates responsibility well, and has a friendly engaging
presence.
Dr. Romano employs a systems based approach and is therefore strong in analyzing and
solving problems.

17 
 

 








Dr. Romano has established a vision for the School District which impresses some school
based administrators. They trust her capacity as Superintendent and believe she is effective
in her role.
Many feel Dr. Romano is effective at building partnerships with local government agencies,
community stakeholders and business partners for the benefit of the School District.
Some feel she is a has made unpopular but needed changes for the betterment of the school
district.
Many feel she has a positive attitude as she aims the School District toward becoming an
“A” grade District.
Some feel she is excellent at speaking with members of the Press.
Some feel Dr. Romano has a polite and respectful demeanor.
Many feel she is highly effective within the community.

There were mixed responses to the Superintendents effectiveness, which can be summarized as:











Some feel the Superintendent has no tangible effect on their work.
She does a great job listening but does not follow-up on concerns or issues.
She pays attention to the elementary and middle schools curricular needs, but is
disorganized.
Some feel she is good at one-on-one conversation when she makes herself available.
Some believe she cares about the District and its success but takes or initiates actions that
are not helpful to teachers or students.
Many appreciated the Superintendents visits to the schools early in her tenure but rarely see
her in the last few years. They feel her early effective communication with teachers has not
lasted.
Many feel the Superintendent is a good speaker but not a good listener.
Some feel her business background is especially helpful and makes her a force to be
reckoned with in the community but is detrimental to communication with staff.
Many feel she designs, implements and evaluates school programs effectively but without
empowering administrative staff to communicate effectively if at all.

There were negative responses to the Superintendents effectiveness, which can be summarized as:







The Superintendent is effective in making poor decisions.
She never makes it a priority to make the teachers feel appreciated or valued. Evaluations
are biased, and administrators are moved without reason, which breeds job insecurity.
Many feel she has not visited their school.
Many feel she has removed District Administrators, staff, and School-based administrators
who do not support her decisions and goals.
Many believe she has created an atmosphere of fear and retribution for those that speak out
against her or disagree with her positions.
Many feel the Superintendent has created an atmosphere of low morale for employees in
Hernando County Schools.
18 

 

 

















Many believe they are on notice about their job performance and therefore do not disagree
with or offer suggestions to the Superintendent.
Many feel the Superintendent is responsible for those “above the instructional level being
clearly closed to feedback from those of us in the classroom”.
Many feel the Superintendent is a very poor communicator and organizer.
That she is effective in driving down the morale in the School District.
Some report that when serious events are reported to the Superintendent they “disappear for
caring enough to try and trust her or her confidants.”
Many feel the superintendent does not directly engage with teachers, parents and staff.
A few believe the Superintendent is dishonest.
Some feel the Superintendent presents well to those outside the district but does not follow
through on issues or concerns.
Some feel she deceives the public and school employees by presenting information in one
manner but once she achieves her goals she proceeds with her own plan.
Some community members feel the Superintendent does not take part in community events.
Many feel the Superintendent does not serve the needs of students and is selfishly oriented
toward her own career.
Some feel she effectively manipulates data to give the appearance of improvement.
Many feel the Superintendent micromanagement leadership style is adversely affecting the
District.
Many feel the Superintendent fires good employees and that a select few get re-assigned to
these positions with fancy new titles and raises.
Many feel the Superintendent pretends to listen and will do whatever it takes to appease
complaining parents.
Many feel the Superintendent is not concerned about what goes on in the classroom
focusing instead on measures of student success.
Many feel she does not respond to emails, letters or other types of communication.

An open ended question was administered and responded which received 512 responses out of 975
respondents. Using this count 53% of the survey respondents commented. USF will retain the 31
pages of raw responses from this item but due to the requests for anonymity from Hernando
County School Board and survey respondents we will not be providing direct quotes. 512/975
The respondents were asked to respond to the following open-ended item.
In what areas should the Superintendent improve her performance?





Many feel she has eliminated competent staff at the district level for no good reason
because she was threatened by them or disagreed with them.
Many feel she was effective at enabling a considerable raise for herself but teacher only
received a small increase in pay.
Some feel that student programs have been cut without notice or explanation.
Many feel the superintendent hears the concerns of teacher and should listen and address
major issues.
19 

 

 





























Many feel her administration is not transparent and should be improved.
Many feel effective communication from the Superintendent needs improvement.
Many feel Dr. Romano should improve in all areas of her job responsibilities.
Many feel she should lead by example rather than through issuing orders or delegation to
subordinates.
Many feel she should visit schools as she did early in her tenure with the District.
Many feel the Superintendent should listen to the concerns of teachers.
Many feel more money should be re-allocated to the school for instructional purpose.
Many have become discouraged and suggest the only solution for problems with the
Superintendent is her resignation or termination.
Some feel the Superintendent should reach beyond District based and School based
administrators to communicate with District staff, school based instructional and noninstructional staff.
Many no longer trust the Superintendent and do not believe that trust can be regained.
Many feel she should not solve disagreements with vindictive behavior.
Many feel she should not make empty promises.
Many feel she should not lead a fear based administration.
Many want her to put the Districts needs above her own self-interest and career aspirations.
A few suggested she enable more professional development for teachers and staff.
Many feel Dr. Romano should be more honest in her communications.
Many suggest a more targeted purchase of curriculum.
Many suggest that Dr. Romano has become less connected with the schools and its teacher,
and students and more controlling of information as her tenure has advanced.
Many feel there should be more money for maintenance, technology, supplies and
curriculum.
Many feel the superintendent needs to focus on employee morale at all levels.
Many feel the Superintendent favors a few schools within the District and ignores the needs
of the remaining schools.
Many feel the Superintendent is weak in student achievement because she does not listen to
her teachers or support them.
Many feel the Superintendent should stop terminating and reassigning political rivals as a
means of eliminating competition for her Superintendent position.
Some feel she needs to identify and meet the needs of struggling schools within the District.
Some feel the superintendent should be more active in the community.
Some feel the Superintendent should look into the needs of ESE students.
Some think the Superintendent should improve her performance in the area of
Emotional/Behavioral Disability (EBD) students.
Some believe she should be more receptive to ideas and suggestions.

In Sum
In sum, the comments indicate the Superintendent is polite, well presented publically and excellent
when dealing with the Press. She has strong beliefs, is guided by deeply held principles and
20 
 

 believes in her students and their success. She has a vision, establishes high expectations for all
staff and is a data driven decision maker. She is working hard to build effective partnerships with
local government agencies, community stakeholders and business partners.
The comments further indicate she listens without following up on concerns or issues, that she
eliminates competent staff at the district level when feeling threatened and has advocated for her
own salary increase without considering others in the District.
Finally, the responses indicate the Superintendent does not visit the schools as she did early in her
tenure, her administration should be more transparent and she should improve her job performance
in all areas. They feel she does not communicate well and has created an atmosphere of fear and
retaliation for those that speak out against her, which has resulted in low morale within the District.
The teachers within the District feel she does not listen to their concerns. Many are now
discouraged with her job performance and believe she has created a fear-based administration
which has broken trust. That group does not believe the trust can be regained and think the time
has come for her to leave the district or be relieved of duties.

21 
 

 References
MacDonald, G. (2014). The performance of the linear logistic test model when the Q‐matrix is misspecified: A simulation 
study. University of South Florida, ProQuest, UMI Dissertations Publishing, 2014. 3618477.  
 
Stufflebeam, D.L. (1995). Evaluation of Superintendent Performance: Toward a General Model. Studies in Educational 
Evaluation, 21, 153‐225 
 
Stufflebeam, D.L., (1995). A Portfolio for Evaluation of School Superintendents. Center for Research in Educational 
Accountability and Teacher Evaluation (CREATE), Kalamazoo, MI.  
 

22 
 

 Appendices

23

Appendix A – Superintendent’s 360 Degree Feedback Survey
Hernando County School District Superintendent's 360 Degree Feedback Survey – April 2017 
 
 

 
Dear Hernando County School Board employee, 
 
 

 
The University of South Florida is administering a Superintendent’s 360 Degree Feedback Survey on behalf 
of the Hernando County School Board.  By participating in the survey, you will contribute to the Hernando 
County School Board’s annual evaluation of the school district superintendent. The information that you 
provide will assist the Hernando County School Board members in evaluating the Superintendent's 
performance over the past year.  Your individual responses to the survey will be kept confidential.  Only 
members of the USF survey team will have access to your information and responses. Please be assured 
that Individually identifiable responses will not be provided in any survey reports.  To be eligible to 
participate in this survey, you have received a survey link addressed to an email address provided to USF by 
the Hernando County school district. Responses will be encrypted and transmitted electronically to be 
stored on a password protected server housed at the University of South Florida in Tampa.  Please be 
honest in your responses and provide descriptive answers where asked. We thank 
you for responding to the survey, and for participating in the evaluation process. If you have questions 
about your participation in this survey, please call Dr. George MacDonald at (813) 974‐5977. 
 
 

 
Sincerely, 
 

Beth Naverud (District 3) Board Chairperson 
 

Mark C. Johnson (District 1) Board Vice‐Chairperson 
 

Susan Duval (District 5) 
 

Gus Guadagnino (District 4) Linda K. Prescott (District 2) Board Members 
 

 
Lori Romano, Superintendent 

24 
 

 Which one of the following positions best describes your role with the Hernando County School 
District? 
 









School Board Member (1) 
School‐based Administrator (2) 
School‐based Instructional Staff (3) 
School‐based Non‐Instructional Staff (4) 
District Staff (5) 
District‐based Administrator (6) 
Superintendent (7) 

 

 
If you are school‐based employee please indicate school site. 
 






























Brooksville Elementary (1) 
Chocachatti Elementary (2) 
Deltona Elementary (3) 
Eastside Elementary (4) 
J. D. Floyd Elementary (5) 
Moton Elementary (6) 
Pine Grove Elementary (7) 
Spring Hill Elementary (8) 
Suncoast Elementary (9) 
Westside Elementary (10) 
Challenger K‐8 (11) 
Explorer K‐8 (12) 
Winding Waters K‐8 (13) 
D. S. Parrott Middle (14) 
Fox Chapel Middle (15) 
Powell Middle (16) 
West Hernando Middle (17) 
Central High (18) 
Hernando High (19) 
F.W. Springstead High (20) 
Nature Coast Technical High (21) 
Weeki Wachee High (22) 
Catapult Academy (23) 
Endeavor Academy (24) 
Quest Academy (25) 
Home School (26) 
FL Virtual School ‐ Fulltime (27) 
Other Center or Charter (28) 

25 
 

 If you are district‐based employee please indicate your organizational unit. 
 






Schools (1) 
Academic Services Division (2) 
Business Services Division (3) 
Support Operations Division (4) 

 

 
How many years have you lived within the Hernando County School District boundaries? 
 









Less than one year (1) 
One year (2) 
Two years (3) 
Three years (4) 
Four years (5) 
Five or more years (6) 
Do not live within school district boundaries (7) 

26 
 

 Please indicate your level of agreement with the statements below regarding the Superintendent. 
 
 

 

Neither 
disagree  Somewhat 
nor agree  agree (4) 
(3) 

Strongly 
Somewhat 
disagree (1)
disagree (2) 
Models appropriate moral 
leadership (1) 

 




 

Promotes the school district's   

mission and vision (2) 
 
Promotes high standards for   

teaching (3) 
 
Promotes high standards for   

learning (4) 
 
Promotes high standards for staff   

performance (5) 

Shares authority (8) Supports 

 

 

 


 

 


 


 

 

 

 

 

 

 


 

 


 


 

 

 

 

 

 

 


 

 


 


 

 

 

 

 

 

 


 

 


 


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 
























 

 

 

 

 

 

 

 

 

 

 

 

 

 


 

 


 


 

 

 

 

 

 

 


 

 


 




independent action (9) Recognizes 
others for their 
professional efforts (10) 



 

 

(7) 

 



 

Empowers leadership of others   



 

 

 





Not able to 
evaluate (6)

Strongly 
agree (5) 

 

 

Delegates responsibility (6) 



 



Takes risks to encourage positive   

change (11) 
 
 
Works effectively to build 

consensus (12) 

























27 
 

 Please indicate  the Superintendent's level of effectiveness for each of the statements below. 
 

 

 

Not 
Slightly 
Moderately 
effective at effective (2)
effective (3) 
all (1) 
Recommends policies to the school   

board (1) 
 
Implements school board policies   

and practices (2) 
 
Ensures school district compliance   
with federally‐funded education  
 
program requirements (3) 
 
Ensures school district compliance   
with special education program  
 
requirements (4) 

 

Ensures school district compliance 

with legal requirements (5) 
 
Ensures effective culturally diverse   

policies and practices (6) 
 
Explains budget issues and budget   
decisions to school district staff (7)  
 

Explains budget issues and budget   
decisions to community members (8)  






 
Not able to
Very 
Extremely 
evaluate (6)
effective (4)  effective (5) 



 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


 
 

 



 

 
 





 

 

 

 


 
 

 

 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


 
 


 
 









 
 








 

 

 


 
 

 

 


 



 

 
 



 

 

 



 
 

 

 







 

 

 


 



 
 


28 
 

  
 
Not able to 
Mostly 
Very 
evaluate 
effectiv e  effectiv e 
(6) 
(4) 
(5) 

Somewh at 
Not 
effectiv e  Effectiv e 
effectiv e 
(2) 
(3) 
(1) 

 

School Board members (1) School District  











 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

staff (2) School Administrators (3) Teachers  





(4) 





School support and facilities staff (5) 



School volunteers (6) Parents and students  





(7) 





Business and community leaders (8) 





News media (9) 











































 

 
 
 
 
 
 

Please indicate how effectively the Superintendent communicates about the topics listed below. 
 

 

 
Not able to 
Mostly 
Very 
evaluate 
effective  effective 
(6) 
(4) 
(5) 

Not 
Somewhat 
Effective
effective  effective (2)
(3) 
(1) 
Information parents need (1) Information  
 

 

 

 

 

teachers need (2) Information staff need (3) 
Needs of the school district (4) 



 
















 

 

 

 

 

 

 

 

 

 

 

 

















29 
 

  

Please indicate your level of agreement with the statements below regarding the performance of the 
Superintendent. 
 

Neither 
 
Not able to 
 
Strongly  Somewhat  disagree 
Strongly 
evaluate 
Somewhat 
disagree  disagree (2) nor agree
agree (5) 
(6) 
agree (4) 
(1) 
(3) 

 

Actively listens to diverse opinions and 

interests (1) 









 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 

 

 

 

Participates in community events and   

 

Supports new ideas (2) Leads by example  
(3) 
Provides timely information (4) 
activities (5) 




 

Balances community concerns and the   

best interest of students (6) 
 
Maintains effective working relationships   

(7) 
 
Shares opportunities to problem solve   

(8) 

 

Listens to needs and concerns (9) 



 



 






































30 
 

  

 

Neither 
 
Not able to 
 
Strongly  Somewhat  disagree 
Strongly 
evaluate 
Somewhat 
disagree  disagree (2) nor agree
agree (5) 
(6) 
agree (4) 
(1) 
(3) 

 

Ensures high quality curriculum (1) 
practices (2) 


 

Ensures use of current research to adapt   

practices (3) 
 
Ensures professional development for   

staff (4) 
 
Ensures leadership development for staff   

(5) 
 
Ensures maintenance of district facilities   

(6) 
 
 
Ensures development of district 

technology plan (7) 
 
Ensures approaches to attract highly   

qualified professionals (8) 
Ensures the evaluation of all staff (9) 
Ensures improvement in student 
achievement (10) 











 

 

 

 

 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 


 


 


 


 

 

 

 

 

 

 

 



 
Ensures use of student data to adapt 

 

 

 

 

 


































 
 
 

 
Please consider your experience with the Superintendent and respond to the following questions in the 
space provided. 
 
 
 

In what areas is the Superintendent especially effective? 
 
 

 
In what areas should the Superintendent improve his or her performance? 

31 
 

  

Appendix B – Stufflebeam (1995) Duties and Competencies Matrix

Superintendent Competencies (AASA)
Superintendent
Duties
Foster Student
Growth
Foster Equality of
Opportunity
Foster a Positive
School Climate
Lead School
Improvement
Foster Improvement
of Classroom
Instruction
Lead and Manage
Personnel
Manage District
Resources
Foster Positive
Student Conduct
Foster Effective
School Community
Relations
Engage in
Professional
Development
Relate Effectively to
the Board

Leadership
Communications
and District Policy and
and Community
Culture
Governance Relations

Curriculum
Organizational Planning and
Management Development

X

X

X

X

X

X

X

X

X

X

X

X

X
X

X

Human
Values and
Instructional
Resources
Ethics of
Management Management Leadership
X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X
X

X

X

X
X

Source: Stufflebeam, D.L. (1995). Evaluation of Superintendent Performance: Tward a General Model. Studies in Educational Evaluation, 21, 153‐225

32 
 

 1.8

1.4
1.2

0.3
0.6
Ill-1
0.2

-0. 2

-D.E


?l.2
-l.4
?l.6
-l.8

Appendix Superintendent Core Competency Parameter Estimates by Role

Parameter Estimates by Roie

Less Likely to Endorse Superintendent's
Performance

I . I-Ullr I I I

.H-- II.



More Likely to Endorse
Superintendent's Performance
1 luaderrxinp '2 Paine-R. F. 21 ?1 l?lnrn mum in .3 8. Film '1 (If r1 .iinn liiu'nlmn"
District Lnliu-e (Jet-emana- Lummun-ly Management Planning 84 Management Management team-I Ship
Rt?lntiun'. 

IE-uarrlMemuers 

33