UCLA-RFP-FY2017-581-MMACO-0288
GSEIS-Smarter Balanced College Admission Collaboration
Statement of Work
1. Background Information and Overview
1. History and Development of the Smarter Balanced Assessments
The Smarter Balanced Assessment System was developed by the Smarter Balanced Assessment
Consortium currently UCLA GSEIS-Smarter Balanced (“Smarter Balanced” or “the Consortium”) in
response to the 2010 Race to the Top Assessment Program. Current Smarter Balanced members
include 15 states, one territory, and the Bureau of Indian Education.
Smarter Balanced assessment development began with in-depth discussions among many groups of
educators, content experts, and researchers of the high level content identified in the Common Core
State Standards (“CCSS”). High level content was then translated to broad claims for each content
area, assessment targets for specific grade levels, and evidence required for students to demonstrate
their knowledge, skills, and abilities. These components created the building blocks for a
comprehensive test blueprint and provided the framework for item specifications and scoring criteria
that would guide the steps in test development. Within the timeframe defined by the Race to the Top
grant, Smarter Balanced completed assessment development activities including piloting and field
testing of multiple item types, development of a computer adaptive testing (CAT) algorithm for the
summative assessments, development of an open source assessment delivery platform featuring a
variety of embedded usability, accessibility, and accommodations features, and achievement level
setting for the summative assessments.
The resulting Smarter Balanced Assessment System includes interim assessments, formative
assessment resources, and summative assessments in ELA/literacy and mathematics for students in
grades 3-8 and high school. The summative assessments are delivered online and include both a CAT
component that leverages the benefits of multiple item types to provide a precise measure of each
student’s knowledge, skills, and abilities; and a performance task that provides a more in-depth view
of the student’s competencies in a real-world context. The summative assessments yield individual
student scale scores for each content area. In addition, students receive one of four achievement level
designations based on scale score cut points for the content areas, as well as a claim level score
indicating the category of achievement for each claim within each content area. Stakeholders are
provided with achievement level descriptors to clearly link the achievement level designation and
claim score to the content of the assessments. A sample Smarter Balanced summative assessment
Individual Student Report is included in Attachment 1.
A complete overview of Smarter Balanced development activities completed during the grant period
may be found in Attachment 2: Smarter Balanced Assessment Consortium End-of-Grant Report. Test
blueprints, item specifications, ALDs, and Usability, Accessibility, and Accommodations Guidelines are
available for review on the Consortium’s website at www.smarterbalanced.org.
Given this promising outlook for providing relevant and useful information to postsecondary
institutions, Smarter Balanced members have engaged in discussions of how to improve the high
school assessments and seek to develop solutions that address three specific stakeholder concerns.
First, members have expressed that some high school students are not motivated to do well on
Smarter Balanced tests, perhaps because they do not believe that avoiding remedial work offers
sufficient value for their efforts. Second, members indicated a concern that the Smarter Balanced high
school tests compete for time which students perceive may be better allocated to other activities (e.g.,
other tests, additional instructional time). Finally, Consortium members do not currently have a
systematic method of providing to postsecondary institutions the data regarding students’ college and
career readiness.

RFP No. FY2017-581-MMACO-0288
Statement of Work final

Page 1 of 6

 UCLA-RFP-FY2017-581-MMACO-0288
GSEIS-Smarter Balanced College Admission Collaboration
Statement of Work
2. Framework – The Smarter Balanced High School Strategy
Spring 2015 marked the first Consortium-wide operational administration of the Smarter Balanced
summative assessments. For the vast majority of Consortium members, the Smarter Balanced
summative assessments serve as both the state accountability measure and the federal accountability
measure under the requirements of Elementary and Secondary Education Act (ESEA)/Every Student
succeeds Act (ESSA). However, the impact of the Smarter Balanced high school assessments has
reached well beyond the realm of K-12 accountability systems. Historically, acceptance and placement
decisions within the nation’s postsecondary institutions have been made independently of the largescale high school assessments informing most states’ accountability systems. Smarter Balanced, in
contrast, forged a relationship with postsecondary educators and institutions by involving those
stakeholder groups in all aspects of test development and maintaining a focus on college and career
readiness. The result is an unprecedented relationship between the accountability assessments used
in high schools and their contribution to academic decisions in the postsecondary setting. Currently
256 colleges and universities accept the Smarter Balanced high school assessments as evidence of
students’ ability to successfully engage in credit bearing mathematics and English language
arts/literacy coursework. These 256 colleges and universities represent public higher education
systems and private institutions in 10 states. In most of these states, these institutions enroll more
than 80 percent of the state’s undergraduates. This is very significant for the approximately 600,000
high school students who will complete Smarter Balanced assessments as part of their states’ largescale assessment programs.
Given these stakeholder concerns, and the desire to continue providing relevant, valid high school
assessments, Smarter Balanced wishes to engage in collaborative work with developers of college
admissions tests to better meet the needs of students. Smarter Balanced is open to a variety of
solutions to this complex problem. One possible deliverable would be a system that allows a student
to engage in a single testing event resulting in both a Smarter Balanced score and a score that can be
used as part of the college admissions process. However, Smarter Balanced encourages suppliers to
propose innovative solutions and collaborative relationships.
2. Technical Requirements
A. Description of the Body of Work
The supplier’s proposal must provide documentation as appropriate and describe the development
process and technical characteristics of the measure of college and career readiness being used in the
body of work. Specifically, the supplier must identify:
A.
B.
C.
D.
E.
F.

Ownership and use of student data;
The content standards to which the measure is aligned;
Description of the test structure;
Description of the item bank including a description of the metadata available for each item;
Scaling procedures;
Evidence of reliability and validity of the assessment, including evidence of the assessment’s
appropriateness for inclusion in college admissions/placement decisions;
G. The current mode of administration and existing administration window(s);
H. Universal Design characteristics of the assessment;
I. How the assessment incorporates usability and accessibility features, and allowable
accommodations (including what does or does not produce a college reportable score), to provide
access to the largest possible number of students; and
J. Scoring and reporting procedures.

RFP No. FY2017-581-MMACO-0288
Statement of Work final

Page 2 of 6

 UCLA-RFP-FY2017-581-MMACO-0288
GSEIS-Smarter Balanced College Admission Collaboration
Statement of Work
The proposal must outline specific procedures by which the supplier intends to achieve the outcome
of using available empirical data from both the Smarter Balanced summative assessments and the
supplier’s measure of college readiness to provide information pertaining to college readiness for
students completing Smarter Balanced assessments. The Smarter Balanced Assessment Consortium
2013-14 Technical Report—covering initial development, field testing, and achievement level setting—
and the Smarter Balanced Assessment Consortium 2014-15 Technical Report—covering the 2015
operational summative test administration—are available in Attachment 3 and Attachment 4 of the
SOW, respectively, as references. The supplier’s proposal must address each of the following elements
in describing the body of work:
B. Deliverable(s)
The supplier’s proposal should specifically identify and describe the anticipated deliverable(s) resulting
from the work. In particular, the proposal must indicate the specific reporting elements that will
contribute to a student’s college admissions process as a result of the proposed work. The proposal
should include mock-ups of all deliverables.
C. Methodology
The supplier’s proposal must describe the methodology by which the intended deliverable(s) will be
developed. At a minimum, the supplier must identify:
a. All relevant data sources – The supplier must clearly identify all data sources to be provided by
Smarter Balanced and all data sources to be provided by the supplier. The supplier must clearly
identify any item-level, student-level, and/or test-level data needed to complete the work.
b. How data collection will be accomplished – The supplier must clearly identify the processes and
formats that will be used to support any exchange of data between Smarter Balanced and the
supplier. The supplier’s proposal should describe how test administration, scoring, and reporting
systems might interact, and should identify any necessary steps to facilitate secure data flow
between the two, if the supplier’s proposed methodology requires such data exchange. If data
collection requires sampling, the supplier’s proposal must describe the intended sampling
procedures, and must indicate when data will be collected (e.g., during a specific testing window).
Data collection may include, but not necessarily be limited to, individual student data, item-level
data, and aggregate score data.
i.

Test administration modes and windows – If the supplier’s methodology requires
concurrent administration of the Smarter Balanced high school assessment with another
college readiness measure, the proposal must specifically identify when, how, and by what
means the concurrent administration will occur. Smarter Balanced summative assessments
are administered during a spring window; however, specific administration windows and
restrictions vary across members according to their unique policies and procedures.

ii.

Test design– If the supplier’s methodology requires administration of embedded items sets,
the supplier’s proposal must describe the source of the item set(s) and how the items will
be identified. Additionally, the supplier must describe the specific design for embedding the
item set(s).

iii.

Item development – If the supplier’s methodology requires new item development, the
supplier’s proposal must describe how, when, and by whom items will be developed. The
proposal must also address development or identification of item specifications; identify

RFP No. FY2017-581-MMACO-0288
Statement of Work final

Page 3 of 6

 UCLA-RFP-FY2017-581-MMACO-0288
GSEIS-Smarter Balanced College Admission Collaboration
Statement of Work
item types; specify pilot and/or field testing requirements; and describe all review and
revision procedures.
iv.

Accessibility features and accommodations – If the supplier’s proposed methodology
requires concurrent administration of the Smarter Balanced high school assessment with
another college readiness measure, or administration of embedded item sets, the proposal
must address the means by which assessment items can be presented in a manner that is
consistent with current Smarter Balanced usability, accessibility, and accommodations
features, guidelines, and policies.
In addition, the supplier’s proposal should identify roles and responsibilities in the process
of data collection. The supplier’s proposal should address roles and responsibilities of
supplier personnel, Smarter Balanced personnel, and any requirements of Smarter
Balanced members.

c.

Relevant data analysis procedures – The supplier must clearly identify each step in the data
analysis procedures, including scoring. The supplier’s proposal should specify the methodology for
all necessary data analysis procedures, including the rationale for selecting each particular
methodology. Additionally, the supplier should identify and describe the roles and responsibilities
of supplier personnel in completing data analyses.

d. Reporting – The supplier’s proposal must identify
i.
how reportable student information will flow into the Smarter Balanced system,
ii.
in what format data will be presented,
iii.
how Smarter Balanced members may access data for reporting purposes, and
iv.
how Smarter Balanced scores will be transmitted to postsecondary (public and private)
institutions for the purpose of supporting college admission and determining readiness for
entry-level, credit-bearing courses.
e. Anticipated challenges and proposed solutions – The supplier’s proposal should identify any
potential challenges the supplier anticipates in completing each step in the proposed
methodology, and should describe proposed solutions to each anticipated challenge.
D. Technical Reporting
The supplier shall provide a technical report detailing the work completed. The technical report will
provide important evidence in support of the validity argument for the augmented Smarter Balanced
high school assessments. The technical report must be suitable for inclusion in future Smarter
Balanced Peer Review submissions to the US Department of Education. The supplier’s proposal must
include a proposed Technical Report outline consistent with the proposed work.
E. Protection of Student Data Privacy
The supplier shall ensure adherence to all requirements of the Family Education Rights and Privacy Act
(FERPA), as well as all Smarter Balanced data privacy requirements, when handling individually
identifiable student data. The supplier’s proposal shall clearly identify all areas in which individual
student data will be necessary for the proposed work. In each instance, the proposal must specify how
the supplier will ensure protection of student privacy. At a minimum, the supplier shall ensure deidentification of student data, as well as industry standard protection within technology infrastructure
to ensure protection of student data privacy. The supplier’s proposal should specify methodology for
de-identification of student data where applicable, and should also clearly describe security features
RFP No. FY2017-581-MMACO-0288
Statement of Work final

Page 4 of 6

 UCLA-RFP-FY2017-581-MMACO-0288
GSEIS-Smarter Balanced College Admission Collaboration
Statement of Work
within the project’s technology infrastructure. Further, the supplier must disclose whether Smarter
Balanced members may retain ownership of student data and if the supplier retains data ownership,
whether any of the data associated with this body of work might be sold to another entity as part of
the supplier’s commercial activities.
The supplier will be required to agree to the terms and conditions in the UC Appendix - Data Security
and Privacy which is attached to this SOW as Attachment 5.
3. Management Requirements
A. Risk Management
Identify risks to the success of the project. Include effective mitigation strategies. Also describe how
emerging risks will be identified and addressed during project execution.
B. Personnel
The supplier’s management proposal must identify a project manager, as well as necessary project
staff (e.g., content experts, research and psychometric staff, technology staff, accessibility experts,
and support staff) to complete the proposed scope of work. Smarter Balanced recognizes that staffing
levels and necessary expertise will vary based upon the specific deliverable(s) and methodology
proposed; however, it is the responsibility of the supplier to ensure the project is sufficiently staffed
to ensure timely and accurate completion of the work. The supplier’s management proposal should
accurately reflect the proposed scope of work, and must include specific descriptions of each position
within the project, the expected role within the project, and the level of expertise required for each.
The supplier should provide the names of proposed project staff and a summary of their qualifications.
If the supplier is anticipating the use of subcontractors for any part of the scope of work, the proposal
must identify each subcontractor and their anticipated role in the project. The supplier shall be solely
responsible for acceptable completion of all subcontractors’ work.
C. Governance
The RFP is intended to establish a collaborative relationship between Smarter Balanced and the
supplier(s). Therefore, the supplier should propose a governance structure that describes the
anticipated oversight of the project. The proposal should define the roles of Smarter Balanced
leadership and Smarter Balanced membership, the roles of the supplier’s project staff, and the
relationships among them. The supplier should provide an organizational chart that illustrates the
working and supervisory relationships among the supplier’s leadership, Smarter Balanced leadership,
project staff, and any subcontractor staff.
D. Communication
The supplier must designate one individual (i.e., the Project Manager) to serve as the point of contact
for all collaborative work with Smarter Balanced. The supplier should propose a communication
strategy including, but not necessarily limited to, a schedule of regular meetings and conference calls
to ensure the timely and accurate completion of the scope of work. In addition to facilitating internal
communication among collaborators, the supplier will work with Smarter Balanced leadership to
develop appropriate communication for the Smarter Balanced membership. The nature of such
communications will be determined by the specific components of the proposed work plan. The
supplier’s proposal should address the types and methods of communications (e.g., a communications
packet with sample materials for Smarter Balanced members) tha.t will be developed and provided
for the Smarter Balanced membership. The supplier must acknowledge that all communications with
the Smarter Balanced membership must either be formally approved by Smarter Balanced leadership
prior to being disseminated by the supplier, or must be disseminated by Smarter Balanced leadership.
RFP No. FY2017-581-MMACO-0288
Statement of Work final

Page 5 of 6

 UCLA-RFP-FY2017-581-MMACO-0288
GSEIS-Smarter Balanced College Admission Collaboration
Statement of Work
4. Value Proposition Requirements
A. Value proposition for Smarter Balanced for members, local schools, and high school students
The supplier must articulate why the proposed assessment is appropriate for the intended purpose of
adding information in support of college admissions and placement processes to the Smarter Balanced
reporting structure, and the value proposition of a partnership between the supplier’s assessment and
Smarter Balanced for members, local schools, and high school students.
B. Intellectual Property
The supplier’s proposal shall acknowledge that all assessment resources (e.g., items and item
metadata, item calibrations and the associated scale) report formats, technology structures, and
methodologies developed by Smarter Balanced prior to contract award shall remain the intellectual
property of Smarter Balanced for the duration of the awarded contract and in perpetuity. Likewise,
assessment resources developed by the supplier prior to the contract shall remain the intellectual
property of the supplier for the duration of the awarded contract and in perpetuity. The supplier
should describe what approach will be taken for assigning ownership of intellectual property that is
developed jointly as a result of the awarded contract.
C. Intent to Collaborate
Given that the supplier(s) will be privy to technical information and details that are not generally
available, the Consortium has a vested interest in seeking suppliers that will use that information only
to benefit and advance the mission and well-being of all parties, including each party’s intellectual
property/financial interests and goodwill. Therefore, the supplier should describe the manner in which
it will operationalize a collaborative, supportive, and mutually beneficial relationship with Smarter
Balanced that supports the work and goals of each party.
D. Service Levels
If appropriate to the body of work proposed, the supplier should identify service levels to which the
supplier is amenable. The supplier may otherwise describe the process by which service levels may be
negotiated prior to execution of any agreement between the supplier and the Consortium.

RFP No. FY2017-581-MMACO-0288
Statement of Work final

Page 6 of 6