ros ATTORNEYS Daniel King Member 317.237.3957 317.237.3900 dking?fbtiaw.com October 6, 2017 HAND DELIVERED - VIA ELECTRONIC MAIL - protest@idoa.in.gov Vendor Complaint/Protest Coordinator Indiana Department of Administration, Procurement Division 402 W. Washington Street, Room Indianapolis, IN 46204 RE: Protest of Recommendation of American Institute for Research as Successful Respondent for RFS 18-001, ILEARN IREAD-3 Student Assessments To Whom It May Concern: Data Recognition Corporation hereby protests the Indiana Department of Administration?s recommendation of American Institute for Research as the successful respondent for RFS 18-001, ILEARN Student Assessments DRC is an Indiana employer with more than 77,000 square feet of of?ce space, 28 full?time employees and more than 700 seasonal employees who work for DRC in the state of Indiana. DRC provided a competitive proposal at the lowest cost estimate to the State of Indiana (?State?), while meeting the State?s core objective of contracting for cost~appropriate academic assessments that ?will be aligned to Indiana Academic Standards, as adopted by the Indiana State Board of Education.? See RFS, p. 5. In addition, DRC made the strongest effort and was awarded the most points for participation of Minority and Women?s Business Enterprises. AIR made no effort and received negative points for ignoring this important State initiative. During the RFS process, DRC was the lowest cost proposer and received an initial Management Assessment and Quality Score of 45.09. AIR was the third lowest cost proposer and received an initial MAQ Score of 55.45. However, after oral evaluations conducted by a panel, initial MAQ Score was drastically adjusted downwards by 11.23 points to a ?nal MAQ Score of 33.86. The three other shortlisted bidders initial MAQ scores were only adjusted after the oral evaluations. ?nal MAQ Score increased by 0.82 points. Pearson?s ?nal MAQ Score decreased by 3.63 points. Questar Assessment?s final MAQ Score decreased by 6.87 points. The consequence of the inexplicable downward manipulation to MAQ Score is that the State will now potentially spend an additional $6,000,000 for the future academic assessments. This amount represents the difference between best and 201 N. Illinois Street, Suite 1900 I PO. Box 44961 I Indianapolis, IN 46244-0961 I 317.237.3800 Ifrostbrowntodd.com Overnight deliver)! use zip cvde 46204 Offices in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia October 6, 2017 Page 2 ?nal offer price of $37,592,480.69 and estimated contract price of $43,407,87008. For the reasons we explain more fully below, there are substantive and procedural ?aws fatal to proposal. First, prOposal is premised on using Smarter Balanced content as the basis of the assessments, making them a subcontractor fer roughly 17% of estimated three?year contract amount (which translates to approximately $3,700,000 for each of the second and third years of the contract period for a total of roughly $7,400,000 being earned by Smarter Balanced), according to estimates for industry standard pricing provided by Smarter Balanced for use of their items. Yet, AIR, Smarter Balanced, and Dr. Charity Flores (?Flores?) (a former employee of Smarter Balanced and the Director of the Of?ce of Student Assessment for the Indiana Department of Education failed to disclose the existence of her con?ict of interest as a past employee of Smarter Balanced. Flores served a crucial role in the RPS process as a member of the panel conducting the oral evaluations, questioning the short-listed bidders, and in?uencing the ?nal MAQ Scores. It was at this stage in the RPS process that MAQ Score was inexplicably slashed by 11.23 points, causing it to be scored behind AIR, despite its lower price. In addition, the content for the assessments proposed by AIR (as stated in their proposal) from Smarter Balanced was developed to be fully aligned with Common Core standards, ?13; Indiana Academic Standards. As such, proposal violates Indiana?s prohibition of adopting Common Core standards and entering into contracts for assessments and tests produced solely by a consortium of states, such as Smarter Balanced. Finally, IDOA failed to administer the RFS in a manner consistent with its procedures set forth in the RFS, including but not limited to requesting BAFOs prior to the oral evaluations, which jeopardized the integrity of the process and created the Opportunity to manipulate ?nal MAQ scoring to achieve a desired result. Pursuant to 25 IAC IDOA has the authority to reject any particular proposal when the bid is not responsive in that it does not conform in all material respects to the requirements of the solicitation. Alternatively, IDOA can cancel the solicitation in whole or in part under Indiana Code 5~22-18-2 and 25 IAC In this case, proposal should be rejected in favor of an award to the next-highest scoring respondent, DRC, or the contract should be re-procured through a new, expedited RPS process providing a fair competition among respondents thus ensuring the State receives the best-value service provider. PARTICIPATION IN THE ORAL EVALUATION PROCESS VIOLATED CONFLICT OF INTEREST LAWS By participating in the evaluation of the proposals, Flores engaged in conduct that constituted a con?ict of interest. It is our understanding that Flores served as the Deputy Director for Content for Smarter Balanced until sometime in January 2017, at which time she joined the IDOE. As set forth in detail in Technical Proposal, Smarter Balanced is a 201 N. Illinois Street, Suite 1900 I P.O. Box 44961 I Indianapolis, IN 46244-0961 I 317.237.3800 Ifrostbrowntodd.com Overnight delivery use Zip code 46204 Of?ces in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia October 6, 2017 Page 3 signi?cant subcontractor for AIR and is estimated to receive approximately 17% of the proceeds of any contract awarded to AIR, which could amount to roughly $7,400,000 in revenue to Flores? former employer. Flores would have known this fact when evaluating the proposals and it was her duty to avoid the con?ict under Indiana law. Speci?cally, 40 IAC 2-1-9(a) provides as follows: A state of?cer or employee shall not engage in conduct that constitutes a con?ict of interest. It is the state of?cer?s or employee's duty to determine if a potential con?ict of interest exists, to avoid the con?ict, if possible, or, where applicable, to disclose a con?ict as follows: (1) A state of?cer or employee may seek an advisory opinion to determine whether a con?ict of interest exists using the procedure in 40 IAC 2-2. (2) If the state of?cer or employee determines that a conflict of interest or potential con?ict of interest exists, that individual shall, if possible, be screened from participating in the matter and the matter shall be assigned to another individual who does not have a con?ict of interest. (3) The state 'officer or employee may make a disclosure under IC 4-2-6-9 or IC 35-44-1-3 35- 44 was repealed by SECTION 53, effective July 1, 2012.], where applicable. As referenced in subsection (1) above, disclosure of a con?ict may be done in accordance with the procedure provided for in 40 IAC 2-2, which permits the Indiana State Ethics Commission (the ?Commission?) to render advisory opinions with respect to a potential con?ict of interest. An advisory opinion may be requested by any state of?cer or employee. See 40 IAC Such requests shall be in writing and signed by the person making the request. See 40 IAC Also referenced above is the option to make a disclosure under Indiana Code 4- 26-9, which requires the state of?cer or employee to ?le a written disclosure statement with the Commission that: (A) details the con?ict of interest; (B) describes and af?rms the implementation of a screen established by the ethics of?cer; (C) is signed by both: 201 N. Street, Suite 1900 PO. Box 44961 Indianapolis, IN 462440961 317.237.3800 frostbrowntodd.com Overnight die/Every use zip code 46204 Of?ces in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia October 6, 2017 Page 4 the state of?cer, employee, or special state appointee who identi?es the potential con?ict of interest; and (ii) (ii) the agency ethics of?cer; (D) includes a copy of the disclosure provided to the appointing authority; and (E) is filed not later than seven (7) days after the conduct that gives rise to the con?ict. See Indiana Code A written disclosure ?led under this subdivision shall be posted on the Inspector General's website. See Indiana Code After conducting a search of the Inspector General?s website, DRC was unable to locate any disclosure statements ?led by . Flores and there were. no advisory opinions that appeared related to Flores? con?ict of interest arising from her former employment with Smarter Balanced earlier this year and her current role in the RFS process as an employee of IDOE. Based on the foregoing, it is axiomatic that it was Flores? duty to determine if a potential con?ict of interest existed and, after making such determination, to either (I) avoid the con?ict; (2) seek an advisory opinion from the Commission; (3) be screened from participating in evaluating the proposals; or (4) ?le a written disclosure statement with the Commission. At this time and despite all reasonable efforts, DRC has not discovered any evidence that Flores complied with Indiana?s con?ict of interest laws. DRC has requested copies of all public records related to Flores? obligation to avoid the con?ict of interest from both the Indiana Of?ce of the Inspector General and Indiana State Ethics Commission. Copies of Public Records Act requests are enclosed. DRC will provide the information received as a result of its Public Records Act requests. At the very least, Flores should have been ?screened from participating in the process]? as required by 40 IAC As the above statutes and regulations make clear, an actual con?ict need not have existed for Flores to have been required to take one of these steps. 40 IAC 2~l~9(a) merely requires that there be a potential for a con?ict of interest. There can be no doubt that a potential for con?ict of interest existed, as Flores had left Smarter Balanced, a company expected to be paid millions of dollars if the contract was awarded to AIR, only months before being expected to fairly and justly evaluate the proposals. The evidence is clear that Flores violated the above procedural and disclosure requirements. Consequently, pursuant to Indiana Code ?4-2-6-12, the appropriate remedy is rejection of proposal or, alternatively, IDOA can cancel the solicitation in whole or in part under Indiana Code 5-22-18-2 and 25 IAC 201 N. Illinois Street, Suite 1900 I PO. Box 44961 I Indianapolis, IN 46244?0961 1 317.237.3800 Ifrostbrowntodd.com Overnight dis/Every use 2:10 code 46204 Offices in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia October 6, 2017 Page 5 PROPOSAL VIOLATES PROHIBITION AGAINST ADOPTING COMMON CORE STANDARDS proposal for assessment services is fundamentally premised on utilizing Smarter Balanced as the subcontractor to perform its assessments. However, in 2015, Indiana passed a law that prohibited the State from ?adopt[ing] Common Core (Common Core State Standards Initiative) or an assessment or test . . . that is produced solely by the United States government or a consortium of states.? See Indiana Code proposal runs afoul of this statute and, as such, should be rejected. Smarter Balanced is a public agency sponsored by members drawn from a consortium of states including California, Connecticut, Delaware, Hawaii, Idaho, Michigan, Montana, Nevada, North Carolina, Oregon, South Dakota, Vermont, and Washington. Smarter Balanced was awarded a $178 million?dollar federal grant to develop assessments to meet the Common Core standards. AIR proposes to use the Smarter Balanced item pool as its primary basis for ILEARN assessments. Furthermore, AlR?s proposal includes numerous references to the alignment of its assessments to the Common Core standards. For example: 0 Page 23 ?Producing more than 10,000 items annually, our Test Development team has been deveIOping and aligning existing items to Common Core State Standards since its inception.? - Page 44 - ?The Smarter Balanced assessment is one of only four assessments independently evaluated for alignment by the Fordham Institute for which a completely independent alignment study was implemented (Doorey and Polikoff, 2016). This study was independently designed, funded, and executed and found that Smarter Balanced elementary and middle school assessments are well aligned to the Common Core State Standards.? 0 Page 46 4 ?De?ne the domain Organized the Common Core State Standards into domain areas that support the further de?nition of test blueprints and reporting categories.? 0 Page 47 ?For example, item speci?cations provide clear direction to item writers, editors, and reviewers about how each target should be measured. Exhibit 1.8.4~1 provides an excerpt from the grade 6 ELAJliteracy item speci?cations for Claim 1 (Reading), Target 1 (Key Details). It describes critical information about item writing limits, depth of knowledge to be assessed, alignment to the Common Core State Standards, and allowable item types (not shown)? 201 N. Illinois Street, Suite 1900 P.O. Box 44961 i Indianapolis, IN 46244-0961 317.237.3800 Ifrostbrowntodd.com Overnight delivery use code 46204 Of?ces in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia October 6, 2017 Page ,6 Page 55 - Smarter Balanced Metadata Available for Mathematics (Text Table) References the ITS Metadata attributes of ?primary standard? as standard.? I Page 80 - ?The Smarter Balanced assessments were developed to ensure students engage in items and tasks that demonstrate college and career readiness. As noted in the Mathematics Content Speci?cations, one of the guiding principles for the development of the Smarter Balanced mathematics assessments: Assessments include evidence of student performance on challenging tasks that evaluate Common Core standards of 21? century learning. 0 Page 180 ?The evidence provided supports the conclusion that the Smarter Balanced summative assessments accurately measure the knowledge and skills of the Common Core, which aligns substantially with the Indiana Academic Standards. Although Indiana will enact a custom blueprint aligned to the Indiana Academic Standards, which will require Indiana~speci?c item development aligned to standards not represented in the Smarter Balanced item pools, the evidence suggests that the ILEARN assessments in ELA and mathematics, based on the augmented Smarter Balanced item pools, Will meet peer review guidelines.? 0 Page 205 proposes to implement custom ELA and mathematics assessments using the Smarter Balanced item banks as supplemented by Indianawspeci?c items developed to measure Indiana Academic Standards not represented in the Common Core standards.? 0 Page 205 ?Because we propose to deliver the ILEARN ELA and mathematics assessments using the Smarter Balanced item pool, augmented with Indiana-speci?c items developed to measure Indiana Academic Standards not assessed in the Common Core, the validation framework underlying the Smarter Balanced assessments will inform the validity of test score interpretation for the ILEARN ELA and mathematics assessments.? - Page 216 ?The focus on vocabulary and language skills in the assessment of language acquisition and use is documented in the Smarter Balanced ELAIL content specifications ([Content Speci?cationsfor the Summarfve Assessment of the Common Core 201 N. Illinois Street, Suite 1900 I PO. Box 44961 i Indianapolis, IN 46244-0961 I 317.237.3800 Ifrostbrowntodd.com Overnight delivery use zip code 46204 Of?ces in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia October 6, 2017 Page 7 State Standards for English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects]. Page 231 PROJECT EXPERIENCE ?Grade 5 Mathematics Lead, Common Core Internal Bank (2014). Managed the development of over 400 grade 5 mathematics items for the AIR internal Common Core bank.? As noted, proposal repeatedly states that it intends to primarily use Smarter Balanced for its assessments and that those assessments, tests and item pool were speci?cally developed for the purpose of being fully aligned with Common Core standards. Although proposal suggests that it will make some future augmentation or supplementation of the assessments to align with Indiana Academic Standards, any such future effort will never alter the fundamental nature of these assessmentsmalignment with Common Core standards, not Indiana Academic Standards. The legislature?s efforts in 2015 were intended to create the opportunity for the State to develop its own academic standards to be assessed independently of Common Core standards imposed by the federal government or state consortiums. Contrary to the intent of the legislature, proposal ignores the statutory prohibition on adopting Common Core standards and the use of assessments or tests ?produced solely by . . . a consortium of states.? See Indiana Code In short, the proposal of AIR violates Indiana Code 20~ and, as such, must be rejected in accordance with 25 IAC ADMINISTRATION OF THE RFS PROCESS COMPROMISED THE INTEGRITY OF SCORING THE PROPOSALS The recommendation of AIR as the successful reSpondent is being made without observance of the procedures required by the RFS. Speci?cally, the RFS set forth the framework for scoring the proposals, including the submission of BAFOS after the oral evaluation of the short~listed bidders. IDOA made the unilateral decision to require BAP Os before the oral evaluation, which allowed for the more subjective oral evaluation to become the controlling factor in the evaluation process. This jeopardized the integrity of the evaluation process by allowing the potential for manipulation of scoring to achieve a desired outcome as opposed to allowing an oral evaluation to be conducted objectively without the evaluators being prejudiced by knowing the BAFOS of each short-listed proposer prior to ?nal MAQ scoring. Indiana procurement law requires that any award shall be made to the responsible offeror whose proposal is determined in writing to be the most advantageous to the State, taking into consideration price and the other evaluation factors set forth in the RPS. See Indiana Code 5- 22-93. Here, irregular and atypical decision to request BAFOs prior to the oral evaluation controverts sound bidding procedures and the express requirements of the RFS. For the reasons set forth above, DRC protests the recommendation of AIR as the successful respondent for RFS 18?001, ILEARN IREAD-3 Student Assessments. 201 N. Illinois Street, Suite 1900 I PO. Box 44961 1 Indianapolis, IN 46244-0961 I 3112313800 jfrostbrowntoddcom - Overnight delivery use 27,0 code 46204 Of?ces in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia October 6, 2017 Page 8 Please send the written response to this Bid Protest Letter by mail to Data Recognition Corporation, c/o Daniel P. King, Frost Brown Todd, LLC, PO. Box 44961, Indianapolis, IN 46244-0961 and by email to dking@fbtlaw.oom. Sincerely, FROST BROWN TODD LLC Daniel King Attachments 4845-7964-0913V1 201 N. Illinois Street, Suite 1900 I PO. Box 44961 I Indianapolis, IN 46244-0961 I 317.237.3800 [frostbrowntodd.com Overnight delivery use 21:0 code 46204 Of?ces In Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia ros romt?bddm ATTORNEYS Joshua N. Kutch Associate 317.237.3816 317.237.3900 jkutch@fbtlaw.com October 5, 2017 HAND DELIVERED Indiana State Ethics Commission c/o Public Records 315 West Ohio Street, Room 104 Indianapolis, lN 46204 Re: Access to Public Records Request for Dr. Charity Flores To Whom It May Concern: Pursuant to the Access to Public Records Act (Ind. Code I would like to obtain a cepy of the following public records relating to Dr. Charity Flores: - All requests, inquiries or noti?cations seeking an advisory opinion ?our a public agency regarding a potential con?ict of interest. a All written disclosure statements ?led with a public agency regarding a potential con?ict of interest. I All advisory opinions, responses or decisions made by a public agency regarding a potential con?ict of interest. I- All appeals of advisory opinions, responses or decisions made by a public agency regarding a potential con?ict of interest. 0 All documents relating to any con?ict of interest or potential con?ict of interest. I understand that if I seek a copy of these records there may be a copying fee. Could you please inform me of that cost prior to making the copies? I can be reached at jkutch@fbtlaw.corn or 317-237-3816. According to the statute, you have twenty-four (24) hours to respond to this request. Thank you in advance for your assistance on this matter. Very truly yours, Jo shua N. Kutch EN20864.Public-20864 4826-3053-0385vl 201 N. Iilinois Street, Suite 1900 I P.O. Box 44961 I Indianapoiis, IN 46244-0961 13172373800 Ifrostbrowntodd.com defivery use code 46204 Of?ces in Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia r69nint?loddm ATTORNEYS Joshua M. Kutch Associate 317.237.3816 317.237.3900 jkutch@fbtlaw.corn October 5, 2017 HAND DELIVERED Indiana Of?ce of the InsPector General c/o Public Records 315 West Ohio Street, Room 104 Indianapolis, IN 46204 Re: Access to Public Records Request for Dr. Charity Flores To Whom It May Concern: Pursuant to the Access to Public Records Act (ind. Code 5?14?3), I would like to obtain a copy of the following public records relating to Dr. Charity Flores: I All requests, inquiries or noti?cations seeking an advisory Opinion from a public agency regarding a potential con?ict of interest. 0 All written disclosure statements ?led with a public agency regarding a potential con?ict of interest. a All advisory opinions, responses or decisions made by a public agency regarding a potential con?ict of interest. a All appeals of advisory opinions, responses or decisions made by a public agency regarding a potential con?ict of interest. 0 All documents relating to any con?ict of interest or potential con?ict of interest. I understand that if I seek a cepy of these records there may be a 00pying fee. Could you please inform me of that cost prior to making the c0pies? I can be reached at jkutch@fbtlaw.com or 3 17-23 7-3 816. According to the statute, you have twenty-four (24) hours to respond to this request. Thank you in advance for your assistance on this matter. Very truly yours, Joshua N. Kutch BN20868.Public-20868 4844-6035-0529Vl 201 N. Illinois Street, Suite 1900 P.0. Box 44961 I Indianapolis, in 46244-0961 317.237.3300 Ifrostbrowntodd.com Ovemighr delivery use Zip code 46204 Of?ces In Indiana, Kentucky, Ohio, Tennessee, Texas, Virginia and West Virginia