U.S. Department of Education Staff Report to the Senior Department Official on Recognition Compliance Issues RECOMMENDATION PAGE 1. Agency: Accrediting Council for Independent Colleges and Schools (1956/2013) (The dates provided are the date of initial listing as a recognized agency and the date of the agency’s last grant of recognition.) 2. Action Item: Petition for Continued Recognition 3. Current Scope of Recognition: The accreditation of private postsecondary institutions offering certificates or diplomas, and postsecondary institutions offering associate, bachelor’s, or master's degrees in programs designed to educate students for professional, technical, or occupational careers, including those that offer those programs via distance education. 4. Requested Scope of Recognition: Same as above. 5. Date of Advisory Committee Meeting: June, 2016 6. Staff Recommendation: Deny the agency’s petition for renewal of recognition, and withdraw the agency’s recognition”. The staff recommendation represents one step in the overall process of reviewing an agency’s recognition. NACIQI will make its own independent recommendation, and both staff and NACIQI recommendations will be provided for the Senior Department Official for the actual decision. The staff recommendation is to withdraw recognition, which would mean the agency could not remedy its compliance issues. However, in order to provide guidance to NACIQI or the SDO – and ultimately the agency itself – about what the agency would need to do if the SDO were to decide to let it seek to come into compliance, this staff report describes the actions the agency would need to take to correct the issues cited. 7. Issues or Problems: It does not appear that the agency meets the following sections of the Secretary’s Criteria for Recognition. These issues are summarized below and discussed in detail under the Summary of Findings section. -- The agency still needs to address how well graduates of its institutions succeed on those licensing exams that are required for employment, especially by using data specific to each licensing exam. In addition, the agency still needs to provide documentation of continued positive relationships with state licensing-related entities and nurse accrediting agencies, especially by providing current documents. [§602.13] -- The agency’s response did not demonstrate that ACICS reserves and planning can weather highly-probable long-term decreases in budgeted revenue. The agency needs to submit its FY 2015-2016 audit when it becomes available in October 2016. In addition, the agency needs to submit its FY 2016-2017 budget at that time as well. [§602.15(a)(1)] -- The agency needs to document how its revised training program for all volunteers provides more focus on consistently recognizing problems and questionable practices at institutions, particularly concerning student achievement. In addition, the agency needs to document that each volunteer has undergone the improved training process before being permitted to fulfill the tasks assigned to them. Furthermore, the agency needs to document the membership and activities of its new Ethics Review Board. It needs to document the integrity and sufficiency of its data verification regime, both in design and implementation. [§602.15(a)(2)] -- The agency needs to clarify and document its entire process for the recruitment, selection, and verification of the qualifications and experience possessed by those selected to serve on the agency’s evaluation teams and decision-making bodies. [§602.15(a)(3)] -- The agency needs to ensure that the conflict of interest attestation forms required of appeal board members are clearly understood, and consistently interpreted, by those required to sign them. [§602.15(a)(5)] 1 -- The agency still needs to provide clear documentation of its consistent past practices to ensure that members of the Intermediate Review Committee (IRC) were free from conflicts of interest. [§602.15(a)(6)] -- The agency must effectively demonstrate that it has resolved issues of widespread placement rate falsification. The agency must explain its delay in implementing verification it promised to begin performing in 2011. The agency must specifically explain what actions it took with respect to each pending or settled State or federal lawsuit initiated for the benefit of students against ACICS-accredited institutions in the last 5 years to demonstrate that its actions were appropriate and effective. The agency must also demonstrate that it took follow up action on evidence it had that placement rate data submitted by institutions was unreliable. The agency must also provide current documentation policies/practices to address the non-compliant issues. [§602.16(a)(1)(i)] -- The agency needs to fully implement its plans to more consistently review and identify at-risk institutions. In addition, the agency needs to develop and implement its strengthened expectations for visiting teams to more consistently uncover institutional fiscal and administrative problems while on-site. [§602.16(a)(1)(v)] -- The agency needs to fully implement its new and strengthened initiatives regarding misrepresentations to prospective students and abusive recruiting. In addition, the agency needs to regularly verify that each institution’s recruitment process is complying with the new ACICS requirements. [§602.16(a)(1)(vii)] -- The agency needs to continue implementing its strengthened process for obtaining and evaluating the record of student complaints for each institution. In addition, the agency needs to compile evidence that its strengthened process is effective in practice. [§602.16(a)(1)(ix)] -- The agency does not meet the requirements of this section of the criteria. The Department cannot see how the agency could apply these revisions in such a way as to document effectiveness in monitoring in the time it would be given to respond in a compliance report, particularly in view of its weak record in monitoring and failure to document enforcement, and prior lack of cooperation with the Department. [§602.16(a)(1)(x)] -- The agency does not meet the requirements of this section. The agency must clarify its policies and procedures relative to its new data integrity standards and new team reviewer. The agency must also demonstrate that it has applied these standards. (For example, in a site visit report) [§602.17(a)] -- The agency needs to provide evidence that its regular on-site visit process currently and consistently obtains sufficient information to determine if the institution complies with ACICS standards. [§602.17(c)] -- The agency must have a reasonable basis for determining that the information it relies on for making accrediting decisions is accurate. The agency must clarify how the agency holds institutions accountable for ensuring integrity in their data submissions and explain their verification processes. The agency must provide documentation demonstrating that they have applied their standards (such as an onsite visit report). [§602.18(d)] -- The agency does not meet the requirements of this section of the criteria. The agency must demonstrate that it could apply revision in such a way as to document effectiveness in monitoring in the time it would be given to respond in a compliance report, particularly in view of its weak record in monitoring and failure to document enforcement. Also, the agency must demonstrate that it provided the appropriate information and documentation of the use of all monitoring mechanisms included in the agency’s narrative as requested in the staff draft analysis. [§602.19(b)] -- The agency’s response to the documentation requests regarding enforcement of timelines was insufficient. In addition, the agency’s proposed language revisions are insufficient and have not been finalized and implemented, and the agency has been ineffective in past efforts to bring itself into compliance. As a result, the agency cannot be found in compliance with the requirements of this section. [§602.20(a)] -- The agency’s proposed policy language regarding good cause extensions does not include a maximum time period, as requested. In addition, the agency did not provide the requested documentation demonstrating ACICS took immediate adverse action when an institution did not bring itself into compliance within the specified time period. As a result, the agency cannot be found in compliance with the requirements of this section. [§602.20(b)] -- The agency must provide additional documentation to demonstrate that it has a systematic reoccurring process to identify issues occurring at problematic institutions outside of its normal standards review process as noted in the draft analysis. The agency must also provide a completed CEP and evidence of a standards revision as a result of its review of a CEP. [§602.21(a)(b)] -- The agency continues to be out of compliance with this criterion. The agency needs to develop a written policy that would enable it to consistently determine whether a new comprehensive evaluation is required. [§602.22(a)(3)] -- The agency does not meet the requirements of this section. The agency needs to revise its teach out plan policies to include the requirements of this section of the criteria. [§602.24(c)(1)] 2 -- The agency does not meet the criteria in this section. The agency's Accreditation Criteria provided fail to comply with the requirements of this section and as a response for this item are not effective until July 1, 2016. The agency has not complied with this section in practice and has provided no basis for concluding it will do so in the future. It is unclear to Department staff how the agency has held its institutions accountable for fraud and abuse prior to this or how it will following the effective date (July 1, 2016) of the new standards. [§602.27(a)(6-7),(b)] 3 EXECUTIVE SUMMARY P ART I: GENERAL INFORMATION ABOUT THE AGENCY The Accrediting Council for Independent Colleges and Schools (ACICS) is a national institutional accreditor that was founded in 1912. The agency currently accredits approximately 900 campuses (245 main and 674 additional locations) in 47 states and Puerto Rico. The agency’s recognition enables its institutions to establish eligibility to receive Federal student assistance funding under Title IV of the Higher Education Act of 1965, as amended (Title IV). The agency serves as the Title IV gatekeeper for the vast majority of the institutions it accredits. Consequently, the agency must meet the Secretary’s separate and independent requirements. Recognition History The Secretary of Education first recognized ACICS in 1956 under the agency’s former name, the Accrediting Commission for Business Schools. Since that time, the Secretary periodically reviewed the agency and granted it continued recognition. In 1985, ACICS requested an expansion of scope to include the accreditation of master's degree programs. That request was subsequently granted by the Secretary. In 2006, ACICS requested an expansion of scope to accredit institutions that offer programs via distance education. That request was also subsequently granted by the Secretary. The last full review of ACICS took place at the June 2011 meeting of the National Advisory Committee on Institutional Quality and Integrity (NACIQI). After that review the Department continued the current recognition of the agency, and required a compliance report in 12 months on the issues identified in the staff report. As a result of the agency’s review at the June 2013 NACIQI meeting, the compliance report was accepted and the agency’s recognition was renewed for the remaining period of three years. In conjunction with the current review, Department staff reviewed the agency’s narrative and supporting documentation and conducted an observation of the agency’s decision meeting held in Washington, DC in April 2016. Since the agency’s last review, the Department had received three written complaints from students. After reviewing those complaints, it was found that ACICS had not violated its own written policies or the requirements of the Secretary’s Criteria for Recognition. In connection with the agency’s current petition for continued recognition, however, the Department received approximately 40 written third-party comments. Since those comments have raised serious concerns regarding the accrediting activities of ACICS, they have been used to identify several of the issues listed in the “Issues or Problem” section of this analysis. 4 P ART II: S UMMARY OF FINDINGS §602.13 Acceptance of the agency by others. The agency must demonstrate that its standards, policies, procedures, and decisions to grant or deny accreditation are widely accepted in the United States by-(a) Educators and educational institutions; and (b) Licensing bodies, practitioners, and employers in the professional or vocational fields for which the educational institutions or programs within the agency's jurisdiction prepare their students. Traditionally, ACICS has been able to demonstrate its support among educators and practitioners by its varied site visitor list of several hundred experienced educators and practitioners. In addition, the agency has been able to demonstrate its acceptance by educational institutions by the extensive list of ACICS-accredited institutions. Furthermore, the agency has demonstrated its acceptance outside of its own universe through its recognition over the last few years by the Council for Higher Education Accreditation (CHEA). (Department staff noted that CHEA deferred its decision to renew ACICS' recognition in April 2016 and asked the agency to return for review in November 2016.) When it comes to licensing agencies, ACICS appears to have obtained the acceptance of only one entity, the American Registry of Radiologic Technologists. However, ACICS accredits institutions that offer programs in other areas that either require licensure for employment, or at least expect licensure of those graduates who want to advance to higher-paying positions in their chosen fields. Therefore, the agency needs to delineate the programs that its accredited institutions offer where a licensing examination for that field exists. Then the agency needs to address how well graduates of its institutions succeed on those licensing exams that are required for employment, as well as on those licensing examinations that are optional for employment. Also, during its last review for recognition five years ago, ACICS reported that its accredited nurse education programs (within ACICS institutions) had received special consideration from accrediting agencies for nurse education. At that time, ACICS also reported that Texas had recognized ACICS for degree-granting authority and that ACICS accreditation was used by state agencies as an element of their licensure requirements in California, Florida and Utah. However, that evidence of acceptance was not highlighted in the current petition for recognition. Therefore, the agency needs to provide documentation of continued positive relationships with state licensing-related entities and nurse accrediting agencies. Furthermore, the current petition narrative notes the agency's acceptance by employers is demonstrated by the placement performance for graduates reported by its institutions on their annual Campus Accountability Reports (CAR). The average percentage of students placed in their field or related field of study for all ACICS institutions was 66 percent in 2012, 72 percent in 2013, and 74 percent in 2014. In addition, the agency previously reported that the percentage in 2009 was even higher at 75 percent. Department staff is particularly concerned because the demonstration of employer support is based on data that is self-reported by the institutions themselves. This is of concern because little, if any, of that data appears to have been verified by the agency, and because any verification by the agency appears to have taken place during the site team evaluations that only occur approximately every five years, after much of the data is already too old to be of use. In addition, Department staff is concerned about the accuracy of job placement rates because, as shown in Section 602.16(a)(1)(i), there is documentation of a widespread problem with ACICS- accredited institutions providing unverifiable or false data in their annual reports to ACICS. For example, the data from the Corinthian schools included fraudulent job-placement figures that led to financial challenges and ultimately closure, with major disruption for the multitudes of affected students. Furthermore, the job placement figures from CEC, Westwood, Le Cordon Bleu, and Lincoln Tech, among others, were deemed fraudulent as well. The agency needs to provide reliable documentation of its acceptance by employers. Analyst Remarks to Response: The draft staff analysis found that the agency needed to delineate the programs that its accredited institutions offer where a licensing examination for that field exists. As well, the agency needed to address how well graduates of its institutions succeed on those licensing exams that are required for employment. In addition, the agency needed to provide documentation of continued positive relationships with state licensing-related entities and nurse accrediting agencies. In response, ACICS does provide current documentation of its recognition by the American Registry of Radiologic Technologists (ARRT). However, although the agency again notes that it has been recognized by the Council for Higher Education Accreditation (CHEA). Department staff again notes that currently CHEA has deferred its decision to renew ACICS' recognition and asked the agency to return for review in November 2016. In addition, the agency states that it "retains good standing with all state approval agencies, including formal recognition by the Texas Higher Education Coordinating Board," But the letter from the Texas Higher Education Coordinating Board is not current (dated 2008), and without current documentation it is difficult to accept the claim that ACICS retains good standing 5 with "all" state approval agencies, especially in light of the negative third-party comments submitted by the attorneys general of 13 states. In addition, the documentation the agency submitted from the National League for Nursing Accrediting Commission (NLNAC) is from 2008, before that agency underwent massive reform and evolved into the Accreditation Commission for Education in Nursing (ACEN). As well, the documentation from the Commission on Collegiate Nursing Education (CCNE) is a generic template from 2008 and only includes ACICS in a list of existing national accrediting agencies. Regarding other specialized programmatic accreditors (some of which are not recognized by the Department), the fact that ACICS expects its institutions to seek their accreditation in certain cases is not evidence of ACICS's acceptance by those specialized accreditors in return. The agency reports the following programs that require graduates to attain licensure or certification to work: nursing, medical assistant, occupational therapy, massage therapy, cosmetology, veterinary science, radiologic technologists, and surgical technician. The agency states that it requires programs to meet state board requirements and ensures that they do so during the agency's initial and renewal accreditation processes. The agency notes that it requires programs to achieve a minimum of 60% licensure/certification and to comply with that respective state's standard. ACICS notes the average reported placement rate for these programs was 73.72%. These and other statements in the agency's narrative depend on and assert the reliability of student achievement data collected by the agency. They accordingly are addressed in 602.16(a)(1)(i), and as stated there, the agency has failed to establish that the data it collects can be relied upon. In summary, the agency still needs to address how well graduates of its institutions succeed on those licensing exams that are required for employment, especially by using data specific to each licensing exam. In addition, the agency still needs to provide documentation of continued positive relationships with state licensing-related entities and nurse accrediting agencies, especially by providing current documents. §602.15 Administrative and fiscal responsibilities The agency must have the administrative and fiscal capability to carry out its accreditation activities in light of its requested scope of recognition. The agency meets this requirement if the agency demonstrates that-(a) The agency has-(1) Adequate administrative staff and financial resources to carry out its accrediting responsibilities; Although no complaints have been received at the Department indicating that there have been staffing problems, the recent departure of the executive director, with no permanent replacement, seems to indicate that the agency has staffing concerns. In the recognition process, agency staff members have been accessible and have been able to provide requests for information. In addition, when Department staff visited the agency's offices in April 2016 several staff members were interviewed and the administrative processes were observed to be functioning efficiently. However, Department staff is concerned that the lack of effective monitoring approaches for its institutions reflects inadequate staffing and financial resources. Regarding finances, the agency's budget for the current fiscal year indicates that ACICS has managed to maintain revenues adequate to conduct the accrediting activities expected of a recognized accrediting agency. The primary sources of revenue are the dues and fees paid by accredited institutions. The remainder comes from educational activities and investment income. Each institution pays for the costs of its own on-site evaluation. However, the fiscal year 2016 budget indicates that the agency plans to reduce its expenditures by over one million dollars and yet it still projects a deficit of almost 200,000 dollars, primarily due to the "contraction of the private independent career college sector." As the agency also indicated in its budget, ACICS has been "scaling operations to match resource levels." Department staff is concerned that significant reductions in personnel and/or accrediting operations could result in a diminishment of the agency's efficiency when it comes to strictly enforcing its standards and maintaining the heightened monitoring of its institutions that is needed. When Department staff observed the agency's April 2016 decision-making meeting, ACICS expressed goals of strengthening its standards and improving its enforcement of them. If it does so, the agency can expect a contraction in its membership with the resulting decrease in revenues. In addition, a stricter regimen will likely result in an increase in both time and money expended by the agency. There is likely to be an increase in appeals from the adversely-affected institutions, as well as an increase in costly litigation that can follow an unsuccessful appeal. Once the institutions become accustomed to strict enforcement of increasingly clearer agency standards, their performance may improve and the number of appeals and subsequent litigation may diminish, although membership may be reduced. Therefore, the agency needs to provide specifics as to how ACICS plans to operate efficiently and effectively in light of reduced revenue. In addition, the agency needs to submit its audited financial statement for fiscal year 2014-15 now, and its audited financial statement for fiscal year 2015-16 as soon as that audit becomes available. 6 Analyst Remarks to Response: The draft staff analysis found that the agency needs to provide specific details as to how ACICS plans to operate efficiently and effectively in light of reduced revenue. In addition, the agency needs to submit its audited financial statement for fiscal year 2014-15 now, and its audited financial statement for fiscal year 2015-16 as soon as that audit becomes available. In response, the agency documented that the mechanisms are in place to effectively manage any increased litigation by adversely-affected institutions resulting from a stricter enforcement of ACICS requirements. In particular, ACICS has insurance for protracted legal disputes and can recover the cost of litigation from the institutions involved. As requested, ACICS provided its audited financial statements for the period ending June 30, 2015 and that report confirmed that the agency was maintaining sufficient resources to continue operating as it had been. That report was provided to the agency in October 2015 by the firm of Councilor, Buchanan & Mitchell, P.C. (Exhibits 201 and 202). Department staff also requested that the current year’s audit be provided when available since the agency was predicting significant reductions due to the contraction of private career colleges, a trend that is expected to continue. Since that audit is not expected to be available until October 2016, there is still concern that a greater-than-expected contraction could adversely affect the agency’s already questionable effectiveness when it comes to monitoring and enforcement issues. Department staff agrees that the documentation included in the agency’s response to the draft analysis demonstrates that ACICS had the financial reserves capable of handling “unanticipated, short-term ebbs in budgeted revenue.” Department staff also notes, however, that the agency’s circumstances have changed dramatically since October 2015. At that time, the audit included the optimistic statement that “Management believes it is highly probable that ACICS will continue to be recognized as an accrediting body for an indefinite period.” At this time, however, evidence is needed that the agency’s reserves and planning can weather highly-probable long-term decreases in budgeted revenue. The documentation provided by ACICS was unable to demonstrate compliance with the requirements of this section at this time. As a result, the agency still needs to submit its FY 2015-2016 audit when it becomes available in October 2016, and at the same time, the agency needs to submit its FY 2016-2017 budget. (2) Competent and knowledgeable individuals, qualified by education and experience in their own right and trained by the agency on their responsibilities, as appropriate for their roles, regarding the agency's standards, policies, and procedures, to conduct its on-site evaluations, apply or establish its policies, and make its accrediting and preaccrediting decisions, including, if applicable to the agency's scope, their responsibilities regarding distance education and correspondence education; Policy and Decision-Making: The ACICS Council, which is currently comprised of 14 members, is both the policy-making and decision-making body. The agency provided sample resumes for a few commissioners as evidence of their extensive qualifications and experience. In addition, the agency provides all commissioners with training that describes the Council responsibilities and operations, and includes distance education, as well as regular updating during Commission meetings. Furthermore, the agency uses an Intermediate Review Committee (IRC), comprised of trained individuals, to take make an initial review of each institution's materials and site team report in order to present a recommendation to the ACICS Council for their consideration. Typically, the IRC meets the week prior to the Council meeting in order to prepare their recommendations. The agency also maintains a separate Review Board pool from which to select individuals at any given time to hear appeals. As also applies to the Council and IRC, the members of the Review Board must be free of conflicts of interest as defined in the agency's canons of ethical responsibilities and be trained before undertaking their duties. Department staff observed a Council meeting and found the individual members to be thorough in their examination of the materials. However, concerns regarding Council operations are noted at the end of this section. In addition, Department staff is aware that some previous members of the decision-making body were connected to institutions that were themselves the subject of serious concern to the agency for various reasons. The agency was also aware of the appearance of inappropriate self- interest and a lack of competency presented by those previous situations. During its April 2016 decision-meeting, ACICS drafted a proposal to create an ethics review board for the purpose of identifying any decision-makers who should resign or be removed from their positions. That draft proposal was sent to the agency's constituents for comments before adoption by the agency. The agency needs to provide information and documentation on how this ethics review board will specifically prevent the ethical issues faced by ACICS over the past several years. On-Site Evaluators: 7 ACICS does provide for the qualification of its site evaluators by requiring that they receive training, including the evaluation of distance education, and their performance is regularly reviewed. As well, ACICS makes training available on the web and the agency uses a tracking system to ensure that the required persons begin and complete their training. Furthermore, site evaluators must document their competencies in the area they will evaluate and adhere to the agency's canons of ethical responsibility. Evaluators include college faculty, state department of education officials, practitioners, administrators and deans. The agency has developed a large database of site team evaluators that includes those from ACICS accredited institutions and those who have no current connection to an ACICS institution. Sample site visit reports examined by Department staff indicated that the agency's standards were examined by the volunteers while on-site. Department staff is concerned because of the Council's failures (discussed in other sections) in discovering and acting upon questionable and fraudulent behavior at a significant number of larger institutions accredited by ACICS. As well, Department staff is concerned because the materials available to the decision-makers are informed by the volunteer evaluators who conduct the site visits for the agency. The on-site evaluators are the eyes and ears of the Council, and their ability to consistently detect questionable and fraudulent behavior at visited institutions needs to be greatly improved. Therefore, the agency needs to substantially improve its training program for all volunteers, including the on-site evaluators, the IRC members, the decision-makers, and those hearing appeals. The training needs to provide more focus on consistently recognizing problems and questionable and fraudulent practices at institutions, particularly concerning student achievement. In addition, the agency needs to ensure that each volunteer has undergone the improved training process before being permitted to fulfill the tasks assigned to them. Furthermore, the agency needs to report if the draft proposal to create an ethics review board was adopted and implemented as written, or revised in some manner, to ensure decision makers are competent and free of self-interest. Analyst Remarks to Response: The draft staff analysis found that the agency needs to substantially improve its training program for all volunteers, including the on-site evaluators, the Intermediate Review Committee members, the decision-makers, and those hearing appeals. The training needs to provide more focus on consistently recognizing problems and questionable practices at institutions, particularly concerning student achievement. In addition, the agency needs to ensure that each volunteer has undergone the improved training process before being permitted to fulfill the tasks assigned to them. Furthermore, the agency needs to report if the draft proposal to create an ethics review board was adopted and implemented as written, or revised in some manner. In response, the agency noted that it began revising its training in May 2016 that is now being provided to all of its volunteers. The content of that training is still in flux and will need to be documented more thoroughly by the agency. And of course, it is too early to determine the effectiveness of that training and the eventual outcomes that may result. As well, those results will need to be documented fully by the agency. The response focused on the addition of a dedicated data integrity reviewer to each site visit. That person will be specially trained to contact and verify the placement outcomes of all graduates reported as placed in the prior reporting period. However, the documentation does not describe the data integrity reviewer’s qualifications, nor the process ACICS used or resources it called upon in coming up with its verification scheme. The description of the scheme is sketchy and the some of the parameters articulated are vague. There are two other unresolved issues regarding the data integrity reviewer. First, it must be remembered that the prior reporting that is being validated on-site by the data integrity reviewer may be several months old by the time of the site visit, depending on when that information had been reported originally. Therefore, a focus on more recent placements will also be needed. As well, it must be remembered that institutions are normally visited only once during their period of accreditation, typically when a renewal of accreditation is due. Therefore, the institutions that are not scheduled for another site visit for several years will still need special attention from ACICS to verify their reported placements. Related to data integrity, the agency noted its development of a training manual for the student relations specialist. That person is responsible for "the evaluation of the ethical practices in recruitment/admissions, the assessment of satisfactory academic progress for accurate monitoring and evaluation, the determination of the appropriateness of refunds, and the results of federal audits (U.S. Department of Education Financial Student Aid and Veterans' Affairs)." Again, it is too early to evaluate the effectiveness of the training, which is only now being conducted. Regarding the Ethics Review Board (ERB), the agency did decide on its establishment to consist of one commissioner affiliated with an ACICS institution and two independent public member commissioners. (Currently, there are five public members on the commission). The materials indicate that the ERB can review real or perceived conflicts of interest, and that it has the power to direct a commissioner to resign. A commissioner must automatically resign if he or she "is employed by an institution that is deemed to be under sustained and serious scrutiny with regard to ACICS standards and requirements." Whether or not the ERB will exercise its power diligently, effectively and consistently also remains to be seen. The agency had been requested to show that it had substantially improved its training, and that the revised training was being required of all volunteers going forward. In addition, the agency was required to note if it had established 8 its Ethics Review Board as envisioned. ACICS has begun the lengthy process to establish an effective training program, implement it for all volunteers, and to progress with its Ethics Review Board. At this time, only the beginning stages of these processes can be documented. Little documentation is available regarding consistent implementation, and no documentation is available regarding effectiveness and actual results. Therefore, ACICS needs to document how its revised training program for all volunteers provides more focus on consistently recognizing problems and questionable practices at institutions, particularly concerning student achievement. In addition, the agency needs to document that each volunteer has undergone the improved training process before being permitted to fulfill the tasks assigned to them. Furthermore, the agency needs to document the membership and activities of its new Ethics Review Board. (3) Academic and administrative personnel on its evaluation, policy, and decision-making bodies, if the agency accredits institutions; As an institutional accreditor ACICS is required to have both academic personnel, and administrative personnel, on all site evaluation teams, and on all policy and decision-making bodies. The agency’s written policies confirm that is the agency’s intent in assigning the proper persons to teams and committees. However, other than a cryptic list for one site visit, there is insufficient evidence that the agency consistently assigns the proper persons as expected. In addition, it appears that evaluators classify themselves to certain categories without verification by the agency as to the accuracy of their claim. Furthermore, it appears that the information in the agency’s database may be quite old and showing that the evaluators may not have had updated training for several years or more. Therefore, the agency needs to provide evidence that it has maintained adequate representation of both academic personnel, and administrative personnel, on its site teams and decision-making bodies, and that it continues to do so. In addition, the agency needs to document how it has verified that those selected for evaluation and/or decision-making positions actually possess the necessary qualifications to fulfill those roles effectively. Analyst Remarks to Response: The draft staff analysis found that the agency needs to provide evidence that it has maintained adequate representation of both academic personnel, and administrative personnel, on its site teams and decision-making bodies. In addition, the agency needs to document how it has verified that those selected for evaluation and/or decision-making positions actually possessed the necessary qualifications to fulfill those roles effectively. In response, ACICS noted that it requires every full onsite evaluation team to be composed of at least one academic and one administrative evaluator. Department staff notes that although this interpretation of the regulation is minimalist, it nonetheless meets the requirement. In actual practice, the documentation shows that the agency typically has sufficient inclusion of the required designations represented. And as previously noted, the agency’s policy and decision-making body includes adequate representation of the required persons. ACICS reported that evaluators initially self-identify their preferred roles and academic or administrative designation. Then that self-identification vetted and verified by ACICS before the person is assigned to a team. In addition, the agency reported that it reviewed “all current resumes of active evaluators, including the currency of their professional experience, and the relevance of documented experience to the evaluation roles they are assigned.” The agency reported that as a result, of 1138 active evaluators, 405 meet the “academic” designation and 666 meet the “administrative” designation. The agency also reported that it conducts resume reviews, requests for attestations of qualifications, and evidence of licensure/certification/registration in those fields relevant to their evaluator assignments. However, requesting an “attestation” of qualifications is quite different from reviewing “documented experience.” It is not clear the extent to which ACICS relies on attestations versus actual documented experience in assigning its evaluators to specific tasks. The results of the review and purging of the agency’s evaluator pool, while certainly laudable, does raise questions as to the qualifications of those who served as evaluators up to that point. The inconsistent and unreliable process that resulted from the unknown percentage of unqualified or poorly vetted evaluators obviously cannot be repeated. (5) Representatives of the public on all decision-making bodies; and The agency requires at least three public members on the decision-making body (there are currently six on that body) and at least one public member on any appeal panel. Although not a Department requirement, the agency also requires public member participation on each site evaluation team. The agency’s definition of a public member covers the conditions contained in the Secretary’s definition. In addition, ACICS provided documentation that it verifies that a member of the decision-making body fulfills the conditions for being designated as a public member (or a non-public member) by requiring a signed affidavit that enumerates the necessary conditions for both types of members. However, it is not clear if members of any appeal 9 body sign the form designed for the main decision-making body, or a form more specific to their specialized role. Therefore, the agency needs to provide documentation that members of each appeal body have signed the generalized form designed for both public and non-public members of the main decision-making body, or a form more specific to their specialized role. In addition, the agency needs to document that it has been consistently verifying that persons currently classified as public members on any decision-making body fully meet the Secretary's definition. Analyst Remarks to Response: The draft staff analysis found that the agency needs to provide documentation that members of any appeal body sign the generalized form designed for both public and non-public members of the main decision-making body, or a form more specific to their specialized role. In addition, the agency needs to document that it has been consistently verifying that persons currently classified as public members on any decision-making body fully meet the Secretary’s definition. In response, the agency provided documentation that the members of an appeal board do sign a specialized attestation form designed for their circumstances. However, there appears to be some confusion by the members of those appeal boards as to whether or not they are attesting to being a “member” (affiliated with an ACICS school) or a public member, or they are being designated as an academic or an administrator. Some persons check only one box, while others respond to two different categories (cf. Exhibit 210). Department staff understands that when it comes to the full decision-making commission, if a commissioner could qualify to be both an academic and a public member that the person can only serve as a public member. However, that distinction is not clear on the form for persons who hear appeals. The sample materials actually signed by appeals board members indicate there is no apparent consistency. As a result, the effectiveness of the attestation materials being used by the agency is called into question. The agency provided materials showing that it uses cross-referenced materials to verify that persons currently classified as public members on the full commission meet the Secretary’s definition of a public member. However, as noted above, the materials used to verify the status of appeal board members appear to be inconsistently interpreted by those required to sign them, thereby raising questions regarding their effectiveness. As a result, the agency cannot be found in compliance with the requirements of this section. (6) Clear and effective controls against conflicts of interest, or the appearance of conflicts of interest, by the agency's-(i) Board members; (ii) Commissioners; (iii) Evaluation team members; (iv) Consultants; (v) Administrative staff; and (vi) Other agency representatives; and ACICS requires all decision-makers, staff and site visitors to disclose any conflicts of interest before assignments are made to review reports, to participate in site visits and to take action on any institutions. In addition, reminders of the agency's controls against conflicts of interest are part of the training provided to those involved in various aspects of the agency's operations. The agency expects decision-makers, including appeal panel members, to sign an attestation that they are aware of the ethical expectations. And for site visitors, who are identified as volunteer "independent contractors," the agency expects those persons to sign attestations of their awareness of the agency's policies in these matters. Department staff observed an ACICS decision-making meeting and noted that various commissioners were identified and left the room when consideration of an institution where a conflict, or potential conflict, could be involved. There is one issue, however, that concerns Department staff. It is not clear if every member of the Intermediate Review Committee (IRC) signs any kind of attestation connected with conflicts of interest. Although technically not a decision-making body, IRC members make an initial review of each institution's materials and site team report in order to present a recommendation to the ACICS decision-makers for their consideration. As a result, IRC members provide a critical service that could be affected by an undisclosed conflict of interest. The agency needs to provide documentation that all members of an IRC have consistently signed an attestation regarding conflict of interest issues, and if not, to explain why not. Analyst Remarks to Response: 10 The draft staff analysis found that the agency needs to provide documentation that all members of an Intermediate Review Committee (IRC) have consistently signed an attestation regarding conflict of interest issues, and if not, to explain why not. In response, the agency stated that “Individuals who have been assigned to serve on the Intermediate Review Committee have signed the IRC Conflict of Interest Attestation.” In addition, the agency provided documentation that four persons signed the form on May 31, 2016 in preparation for an IRC meeting that had not yet taken place when the agency’s response was submitted to the Department. The draft staff analysis had requested documentation that members of the IRC have been consistently required to sign a conflict of interest attestation before they served on an IRC. Department staff finds the documentation (Exhibit 212 – Sample July 2016 IRC attestations) provided by ACICS as a wholly inadequate response. The original request was to document what the practice of ACICS had been to date. A handful of forms that appeared to be a record of what the new agency practice will be (from this point forward) is not responsive. Therefore, at this point it is unclear what, if anything, ACICS had been doing all along to ensure that its IRC members were free of conflicts of interest. Based on the evidence provided, ACICS must be found in noncompliance with the requirements of this section. §602.16 Accreditation and preaccreditation standards (a) The agency must demonstrate that it has standards for accreditation, and preaccreditation, if offered, that are sufficiently rigorous to ensure that the agency is a reliable authority regarding the quality of the education or training provided by the institutions or programs it accredits. The agency meets this requirement if (1) The agency's accreditation standards effectively address the quality of the institution or program in the following areas: (i) Success with respect to student achievement in relation to the institution's mission, which may include different standards for different institutions or programs, as established by the institution, including, as appropriate, consideration of course completion, State licensing examination, and job placement rates. Section 3-1-110 of the agency’s Accreditation Criteria states that ACICS reviews information on the degree to which the institution meets its own predetermined educational outcomes. Section 3-1-111 states that each institution must have a Campus Effectiveness Plan (CEP) that includes at a minimum the following elements: student retention rates; student placement rates; level of graduate & employer satisfaction; student learning outcomes; & graduation rates. Each institution must identify & describe in the CEP how the data were collected, the rationale for using each type of data, a summary & analysis of the data, & an explanation of how the data have been used to improve educational processes. ACICS provides its ACICS Student Achievement Webpage, which indicates that ACICS has set standards at 60% for both retention & placement. ACICS also states that an institution is required to submit licensing exam rates in its annual Campus Accountability Report (CAR). According to ACICS, student achievement information is reviewed both during the on-site visit& in the CAR. ACICS included documentation of a site team report, CAR, & template self-study. In response to supplemental questions from the Department, ACICS explains the steps it has taken to improve the effectiveness of the agency’s operations & outcomes of 7 multi-campus systems, & explains increased attention & action for placement on heightened monitoring for those institutions. ACICS also included a table of the enhancements made to ensure the integrity of self-reported student achievement data. However, the agency’s Accreditation Criteria are unclear on the agency’s review of the educational outcomes the institution sets, and on how institutions are informed of the outcomes requirements ACICS has set for retention & placement. ACICS has not described how it determined that the rates it set are sufficiently rigorous to qualify the agency as reliable. While it appears the ACICS Student Achievement Webpage was created in furtherance of the agency’s 2013 compliance report regarding program-level student achievement data, the relationship between the prescriptive ACICS Student Achievement Webpage & the agency's Accreditation Criteria, is unclear, so staff cannot determine what, if any, consequence the Webpage has. The site team report & CAR demonstrate that outcome rates are collected, but it is unclear how this information is reviewed & verified, or what & how recommendations are made to the institution. The site team report also does not demonstrate that the team made a judgment about the appropriateness of the measures chosen nor the rigor of the goals established by the institution. The template self-study asks relevant questions, but as it is a template, ACICS has not demonstrated its implementation or effectiveness. ACICS must submit complete self-studies & site team reports the Arlington, VA and Silver Spring, MD campuses of Everest College to verify how the outcomes data submitted by institutions are reviewed & verified, & how this data is used. Although the table in the agency’s response to supplemental questions from the Department describes what outcomes data is self-reported & what data is subject to verification either on-site, via third-party or by ACICS verification, no documentation was provided as how ACICS determines which data to verify or that it has actually implemented these verification processes. In its response, ACICS must submit such documentation. In the agency's 2013 compliance report, ACICS stated that programs that require a license to become employed would need to demonstrate at least a 60% first attempt pass rate on the licensure exams, & to be in compliance with all State or national requirements. But the CEP provided in the petition does not include a reporting requirement for 11 licensing exam rates. ACICS did not provide any documentation regarding its decision not to require reporting on these rates, nor did it provide any documentation as to how, if, or when the licensing exam rate data in the CAR is reviewed by ACICS. Also, licensing exam rate information is not included in the Accreditation Criteria thresholds, so it is unclear if this outcome is being reviewed. If ACICS decided not to include licensure exam rates & compliance with State/national requirements as student achievement measures, ACICS must clearly explain its rationale for this. Such rates are commonly used by the greater accreditation community to measure student success for vocational programs. ACICS previously reported it was expanding efforts to verify placement data. In the agency’s 2013 compliance report, staff noted, "Previously, the agency had been doing spot checks of self-reported placement data during scheduled site visits. However, ACICS is now moving to regular independent verification by pre-approved third party auditors. Initially, 20% of campuses will be randomly selected to have their rates independently verified each year, while all institutions coming up for renewed accreditation will be independently audited for the accuracy of their submitted student achievement data." However, ACICS has not documented implementation of such changes. Documentation indicates that ACICS did not follow through on these commitments & only recently began experimenting with verification. Even more concerning is substantial evidence the agency’s accrediting process is ineffective on student achievement. At least 3 times, oversight agencies acted against ACICS-accredited institutions for falsified or low placement rates where ACICS had irrefutable evidence of the same, often from the agency’s own reviews. ACICS left the institution’s accreditation in place or re-accredited it anyway. Also, many investigations & lawsuits were brought against ACICS-accredited institutions for falsified placement rates in the last 5 years, resulting in many judgments/high-dollar settlements. This included a $2.5M settlement of the NY Attorney General’s (AG) action against CEC; a consent decree worth $4.5M in the CO AG action against Alta/Westwood; a consent decree for $3.75M against Salter College in a MA AG action; settlements against Kaplan Career Inst. ($1.375M) & LTI ($1.1M) in actions by the MA AG; & a default judgment against Corinthian for $1.1B in a CA AG action. Many other cases are pending. ACICS’ materials do not reflect real movement to address the pervasive noncompliance with its student achievement standards. While ACICS recently issued a show cause letter to the ITT multi-campus chain, citing unresolved investigations from 20 different AGs regarding, e.g., placement, & other rates, that it took this many State investigations to result in action by ACICS concerning compliance with its student achievement standards, in a time frame coincident with the agency’s submission of its application for renewal of recognition, shows the agency’s application of those standards is ineffective. An April 19, 2016 ACICS “Memorandum to the Field” proposing changes to agency policies does not go beyond stating that ACICS has discretion to independently review data . In response, ACICS should explain what actions it took with respect to each pending or settled State or federal lawsuit initiated for the benefit of students against ACICS-accredited institutions in the last 5 years. Analyst Remarks to Response: The agency explains that its student achievement measures were based on a review of other accreditors’ measures, outreach to stakeholders, and feedback from member institutions. In response to the draft, the agency provides documentation concerning how they utilize licensure rates. (Section 2-1-809 ). The licensure rate information is reviewed annually through the Campus Accountability Report (CAR). If the CAR reflects performance that does not meet the Council’s student achievement standard for licensure at 60% the institution will be placed on an Improvement Plan as part of its Campus Effectiveness Plan (CEP). The CEP focuses on 6 data elements. While licensure is included, it is not one of the 6 elements specifically mentioned for ‘institutional effectiveness’ in the accreditation manual, so it is unclear whether licensure has ramifications for accreditation status; and there is no explanation of how licensure rates are calculated. If ACICS has a benchmark for licensure and a program is not meeting that benchmark, the agency should take the appropriate enforcement action. The agency has provided documentation of enforcement of licensure rates in Ex. 195, but the exhibit also demonstrates the agency is not enforcing appropriate timelines under §602.20. (More in that section). Moreover, it is unclear whether the licensure rate standards have been actually been approved by the Council. Ex. 214 demonstrates that student achievement standards, including licensure, were discussed by the Council, but the minutes indicate that the Council recommended Table B, which does not include licensure rate standards/benchmarks. Council standards and agency benchmarks (including licensure rates) are posted on the agency’s webpage. Ex. 217 verifies that the Institutional Effectiveness Committee (IEC) voted in Dec. to remove graduation (grad) rates as a benchmark. This is a change from the draft; it is not clear why the change is only being reported now. This appears to be a significant retreat in terms of accountability of the agency’s institutions, since level of graduate satisfaction, even if somehow effectively defined and implemented, does not reflect satisfaction among students who do not graduate, and since grad rates are commonly included in the recognition process by recognized accrediting agencies. While the updated Accreditation Manual removes grad rates, there was not documentation of the updated CAR, nor the demonstration of the application of this change at institutions. It is also unclear how this change will be communicated to all institutions to ensure they’re collecting information about the ‘level of graduate satisfaction.’ Also unclear is the role of the IEC as it relates to the Council. While it appears as if the IEC includes Council members, there is no documentation explaining the IEC’s role or training. The IEC is not recognized as a decision maker for this agency, so the IEC should not make agency policy. The agency states it conducts verification annually by selecting 20% of institutions for review of placement data. Selecting a sample of institutions for verification is not uncommon among accrediting agencies. The agency has included Ex.193 to document this verification. However, it is unclear how this process actually occurs, if backup information is collected from the institution to verify these placements, or by whom and with what qualifications. This regime apparently was put in place only earlier this year. The agency asserts, regarding §602.13, that the regime involves “independent third-party verification of all placement data, at the institution’s expense, when deemed necessary by the Council,” but does not describe the qualifications of the 3rd parties, does not explain when the Council might deem this necessary, and provides no documentation it has occurred. 12 Ex. 206 does not provide the details related to reported changes in data integrity standards and the addition of a new data integrity reviewer for the site team. If information is found to be inaccurate, the agency will require additional intensive verification of the CAR. However, this intensive process is not explained. The new data integrity standard is effective 08/01/2016. It remains unclear how the agency has held institutions accountable for ensuring integrity in their data submissions before adopting this standard, or afterward. The addition of the new reviewer is said to be effective for the spring 16 cycle, but that more guidance will be issued in fall 16, but neither Ex. 215a nor 215b document this change. The narrative states the reviewer has been trained for this role. Documentation for this training was included in section 34 CFR 602.17(a), Ex.203. In its narrative regarding §602.13, the agency contends that the average reported placement rate for ACICS-accredited programs where graduates require licensure is 73.72%; that the agency’s data integrity algorithm resulted in incidences of data integrity problems dropping from 3,000 in 2011 to 150 in 2015; and that ACICS’s placement verification program resulted in review of 1,137 placements for 85 ACICS-accredited campuses in 2016, with a verification rate of 90%. ACICS further asserts that there is “no available evidence to date to confirm fraudulent behavior by member institutions” and that placement rate litigation settlements have not admitted liability. ACICS’ assertions ignore the findings made by the Dept. and the California Attorney General confirming widespread placement rate fraud at institutions nationwide accredited by ACICS well into 2015. https://studentaid.ed.gov/sa/sites/default/files/ev-wy-findings.pdf. Such findings of misconduct, subsequently upheld in a California court http://oag.ca.gov/news/press-releases/attorney-general-kamala-d-harris-obtains-11-billion-judgment-against-predatory, were such as to compel the Dept. to agree to discharge millions of dollars of federal student loans. In addition, it is not plausible to dismiss multiple settlements, that impose very substantial obligations, monetary and otherwise, and that result from claims filed in the public interest by state attorneys general, as if they were materially lacking in substantiation. Overall, in the context of the documentation of widespread placement rate fraud, ACICS’s statements regarding licensure pass rates achieved and the effectiveness of its algorithm cannot be credited. While it is unclear whether the 85 campuses ACICS verified reflected entire institutions, or only portions of institutions, in either case this reflects testing, on average, of only 13 students per campus. The agency has provided documentation as to how the site visit team reviews the CEP, in the form of the site visit reports and the self-study. (Ex. 215a & 215b). This information along with Ex. 43, 44, 46, 52 and 195 demonstrate how the agency reviews its institutions. However, Ex.215a is outside of this review period. The agency also failed to document multiple reviews of different institutions demonstrating implementation of the student achievement standards. (a)(1)(v) Fiscal and administrative capacity as appropriate to the specified scale of operations. The agency’s expectations regarding fiscal and administrative capacity are based primarily in the agency standards on “Organization” and “Administration.” Regarding fiscal matters, the agency requires an institution to have adequate revenues and assets to meet its responsibilities, to ensure continuity of service, and to accomplish its mission that reflects a balanced application of resources to operations. In addition, the amount of expenses and debt must not burden the educational objectives of the institution and the financial profile of the institution must support the intent of its objectives. Furthermore, the agency requires all institutions to submit an Annual Financial Report (AFR) that includes financial statements accompanied by audited financial reports prepared and certified by an independent certified public accountant. If financial triggers warrant extra scrutiny, the institution may be monitored by a special financial committee and the agency’s written policies allow ACICS to request additional financial reports when necessary. The agency’s expectations regarding general administrative capacity include the following aspects of administrative competence. In summary, each institution is expected to have a chief executive officer and a qualified on-site administrator at the main campus and at all branches. Institutional managers are responsible for implementation of the institutional mission as determined by the governing body, and management is expected to continuously evaluate the programs of study, the student activity program, guidance services, financial aid services, instructional procedures and resources. In addition, institutions are expected to maintain records for future planning, for the students for informational purposes, and for evaluations visits. Those administrative records are to cover financial aid activities, admissions, curricula, accreditation and licensure, guidance, instructional resources, supplies and equipment, school plant, faculty and staff, student activities and student personnel. Student records maintained include test scores, initial and periodic academic and career advising, admission rates, completion rates of those enrolled, and general placement rates or specific career placement rates, as appropriate, among other things. The materials viewed by Department staff showed that the agency’s visiting team evaluates evidence of the institution’s fiscal and administrative capacity while on-site. The evidence includes the organizational chart, budgets and audits, as well as staff duties, to see if the overall fiscal and administrative capacity of the institution is adequate to accomplish the stated goals and objectives. However, it is apparent that the actions of the on-site team, together with the materials required by the agency, are insufficient to consistently determine the fiscal and administrative capacity of problematic institutions. The agency’s fiscal and administrative standards have failed to identify institutions that were unable to run their programs efficiently, as evidenced by their poor retention and placement rates, and by their poor stewardship of Title IV funds. As well, the agency’s ability to consistently judge the competence and/or ethical practices of the institutional administrators is also 13 called into question. The agency needs to demonstrate that its standards for evaluating the fiscal and administrative capacity of each institution are sufficiently effective. In addition, the agency needs to provide a comprehensive and workable plan for remedying this deficiency in the agency’s accreditation process, as well as any changes it has made already to ensure compliance with this section. Analyst Remarks to Response: The draft staff analysis found that the agency needs to demonstrate that its standards for evaluating the fiscal and administrative capacity of each institution are sufficiently effective. In addition, the agency needs to provide a comprehensive and workable plan for remedying this deficiency in the agency’s accreditation process, as well as any changes it has made already to ensure compliance with this section. The agency was cited because its on-site teams, following agency materials, were unable to consistently identify institutions that were unable to run their programs efficiently, as evidenced by their poor retention and placement rates, and by their poor stewardship of Title IV funds. As well, the agency’s ability to consistently judge the competence and/or ethical practices of the institutional administrators was also called into question. In response, ACICS summarized its previous requirements and practices and also provided some documentation that it followed up on complaints, but those materials were not responsive to the main concerns. The agency, however, did acknowledge its need to improve methods for discerning financial weakness. As well, the agency acknowledged that it needed to move more quickly and forcefully to take action when an institution did not meet the fiscal and administrative requirements. As a result, the agency established an “At-Risk Institution” committee to review all actions facing an institution, as well as information that calls into question its general operations. The “at-risk” review will include 1) financial stability; 2) student achievement performance; 3) adverse information; 4) complaints; 5) enrollment growth monitoring; 6) excessive substantive changes monitoring; 7) information derived from Title IV compliance audits, inquiries and information exchanges; and 8) findings derived from the most recent comprehensive site visit or quality assurance monitoring visit. If heightened risk is discerned, the agency may conduct an unannounced or short-notice visit, a status change to heightened-monitoring, and a demand for a written response to specific issues. The agency also noted it plans to strengthen the recurring reviews regarding the financial and/or administrative capacity of institutions. As part of that process, the ACICS Financial Review Committee enhanced its financial ratios, and the agency plans to test the “Altman Z-Score” calculation during its August 2016 meeting. That calculation analyzes “five distinctive financial ratios used for determining the odds of bankruptcy among companies.” The agency discussed the paper reviews that it plans to start using, but did not discuss any plans to enhance the focus of its visitors while actually on site as part of the evaluation team. Although most of the agency’s plans are commendable and should improve the agency’s ability to uncover difficulties more expeditiously, at this time the plans have not been fully implemented or produced significant and tangible results. (a)(1)(vii) Recruiting and admissions practices, academic calendars, catalogs, publications, grading, and advertising. The agency's expectations regarding these various areas are primarily contained in a number of standards and explanatory materials. The standards areas include "Admissions and Recruitment," "Standards of Satisfactory Progress," "Tuition and Fees," and "Publications." The explanatory materials include "Appendix C: Institutional Publications Requirements" and "Appendix D: Standards of Satisfactory Progress." In summary, the agency's expectations are that an institution has the ultimate responsibility for the activities of its employees, vendors, contractors, or agents in the referral, recruiting, evaluation, and admissions processes. At a minimum, the institution's recruitment policies must be ethical and the admissions policy must state that students admitted to programs leading to a certificate, diploma, or degree must have graduated from a high school or its equivalent, or demonstrate an ability to benefit from the training offered. Also, the school must have a statement regarding transfer of credit. The institution is expected to ensure that any person or entity engaged in admissions or recruitment activities on its behalf is communicating current and accurate information regarding courses and programs, services, tuition, terms and operating policies. As well, recruitment and enrollment personnel may not be designated as counselors or advisors and may not make final decisions regarding financial aid eligibility, packaging, awarding and disbursement. An institution participating in Federal student financial aid must designate a person to administer student aid and the person who determines the amount of student awards cannot be responsible for disbursing those awards. The guidelines for institutional publications expect that all accredited and applicant institutions publish an acceptable catalog that includes a listing of the full-time faculty members (showing the degrees held, institutions that awarded the degrees and area of teaching specialization); an academic calendar; names and titles of the administrators of the institution; a statement of the programs offered; and if currently accredited, a statement accurately denoting this fact. The agency expects that an institution's policies conform to its mission and demonstrate compatibility with its 14 objectives. In addition, the institution is expected to fully explain the grading system used on the transcript, that it be consistent with the institutional catalog, and that the institution maintains a permanent academic record of the student's progress. The materials viewed by Department staff showed that the agency's visiting team evaluates the self-study and reviews evidence including the transfer of credit policy, recruitment materials and advertisements, copies of catalogs and handbooks, the academic calendar and class schedules, and student records including grades. As a result, the petition narrative notes that ACICS has applied a substantial number of sanctions to large multi-campus systems based on deficiencies in these categories. Nonetheless, it is apparent that the agency's process for implementing its standards in this area is not sufficiently effective. Over the last five years, a significant number of State attorneys general and others have obtained sizeable recoveries against ACICS-accredited institutions based on misrepresentations to prospective students and abusive recruiting. For example, the Illinois Attorney General brought action against Alta/Westwood for misrepresentation, achieving agreement by the institution to forgive $15 million in institutional loans. The U.S. Department of Justice recovered $95.5 million in a settlement of 4 qui tam lawsuits against EDMC Management Corporation. ["Qui tam" has part of the penalty going to the one bringing the action and the rest to a public body.] Daymar College entered into a consent decree for payment of $12.4 million in an action brought by the Kentucky Attorney General. The North Carolina Attorney General withdrew the license for the dental assistant program at Kaplan College's Charlotte, North Carolina campus, based on misrepresentations. A student won $2 million in punitive damages against Vatterott College for violations of this nature. Therefore, it is apparent that the agency's standards in this area, and its implementation of them, are not effective in ensuring academic quality. As a result, a comprehensive and workable plan for remedying this deficiency in the agency's accreditation process is required, as well as any changes it has made already to ensure compliance with this section. Analyst Remarks to Response: The draft staff analysis found that the agency must demonstrate that its process for implementing its standards is sufficiently effective regarding misrepresentations to prospective students and abusive recruiting. In addition, the agency needs to provide a comprehensive and workable plan for remedying this deficiency in the agency's accreditation process, as well as any changes it has made already to ensure compliance with this section. In response, the agency first provided clarifications regarding some of the specific problems cited by the Department relevant to abusive recruiting and misrepresentations. In summary, one of the institutions was not accredited by ACICS; another institution's problems were resolved before it became ACICS accredited. However, although ACICS argues that the four EDMC qui tam lawsuits “originated” at institutions that were not accredited by ACICS, the case litigated pertained to all EDMC campuses; and the settlement, which included claims being investigated by 40 state attorneys general, was corporate-wide. So ACICS is not accurate in suggesting that the litigation and settlement did not pertain to ACICS-accredited schools. Nonetheless, the agency did acknowledge that some ACICS-accredited institutions have been accused of misrepresentation and abusive recruiting, and initiated the following reforms. Those recent reforms include new standards that apply to recruiting practices (cf. Exhibits 125 and 206). Basically, the agency is requiring each institution to document that it is consistently monitoring its own recruitment practices. That involves ensuring that anyone involved in "admissions or recruitment activities on its behalf is communicating current and accurate information regarding courses and programs, student achievement disclosures services, tuition, terms, and operating policies." In addition, the agency is requiring each institution to publish campus-level and program-level student achievement rates. Furthermore, the agency is strengthening its monitoring of at-risk institutions, increasing call for comment opportunities, and investigating all complaints, even those submitted anonymously (Exhibit 222). Department staff agrees that it is important to clarify the institution's culpability for its own recruitment activities. However, the agency's track record related to verifying self-reported information has been problematic. As a result, ACICS should be regularly verifying each institution's recruitment processes, a verification that may include the use of "secret shoppers." In any event, the agency's new and strengthened initiatives have not yet produced the evidence to find ACICS in compliance with the requirements of this section. (a)(1)(ix) Record of student complaints received by, or available to, the agency. The agency's expectations regarding the record of student complaints are found under "Complaints and Adverse Information" in Section 2-3-700, and "Integrity" in Section 3-1-202(d). The first set of expectations primarily covers the processes used by ACICS, while the section under "Integrity" sets forth the minimal grievance-related requirements that each institution must maintain. ACICS requires each institution to establish, publish and implement appropriate grievance policies and procedures for considering complaints received from students, employees and other interest parties. The institution must make its grievance procedures for students accessible by publishing the procedure in the institutional catalog or student 15 handbook, and must also include the ACICS name and address. Regarding the agency's obligations, ACICS written policy is to maintain a record of the complaints it receives. The policy also details the mechanism used to effectively process complaints in a consistent manner. Regarding recent activities, the petition narrative notes that ACICS has reviewed the pattern of student complaints involving some large multi-campus systems and has applied heightened monitoring requirements, including additional site visits, in order to protect student interests. The narrative also notes the most frequently received and resolved complaints from those institutions processed through the agency's complaints resolution program involve financial aid issues. Nonetheless, it is apparent that the agency's process for implementing its standards in this area is not sufficiently effective. The large number of third-party comments that the Department recently received regarding ACICS clearly indicate that what ACICS has been doing since its last review by the Department is insufficient. The evidence indicates that three aspects involving complaints needs to be revamped and greatly improved. First, ACICS in its own complaint processing has not been able to adequately obtain, evaluate and resolve student complaints efficiently. As well, the agency's process and requirements for schools to inform ACICS regarding complaints and their resolution, as well as those for learning of systemic complaints independently of the institution, have not been sufficient. Furthermore, although the materials viewed by Department staff showed that the agency's visiting teams review the written student complaints record, the teams are not sufficiently using that information to successfully evaluate each institution's compliance with ACICS standards. Therefore, it is apparent that the agency's standards in this area, and its implementation of them, are not effective in ensuring academic quality. As a result, a comprehensive and workable plan for remedying this deficiency in the agency's accreditation process is required, as well as any changes it has made already to ensure compliance with this section. Analyst Remarks to Response: The draft staff analysis found that the agency needs to demonstrate that its standards for obtaining and evaluating the record of student complaints for each institution are sufficiently effective. In addition, the agency needs to provide a comprehensive and workable plan for remedying this deficiency in the agency’s accreditation process, as well as any changes it has made already to ensure compliance with this section. In response, the agency noted its online complaints system that is used to solicit, receive and review complaints from students, faculty, staff and others. The agency then examines the complaints for patterns in order to identify systemic problems that will require further investigation. As a result, the agency recently conducted special visits to two schools. The schools are being given the opportunity to respond to the findings of the on-site evaluators. After that is completed, the schools are being brought before the ACICS commissioners for their decision. The agency noted that this approach allows for a “rapid action process” for the resolution of issues and enforcement, as appropriate. In addition to the special process just described, the agency uses the pattern of complaints prior to regular site visits so that the evaluators can make focused inquiries in a timely manner. As noted previously, the agency also processes anonymous complaints, a practice that is not common among accrediting bodies. Furthermore, the agency provided substantial documentation to show that it is implementing its stronger focus on student complaints. It is too early for the evidence of the process’ effectiveness to be compiled. (a)(1)(x) Record of compliance with the institution's program responsibilities under Title IV of the Act, based on the most recent student loan default rate data provided by the Secretary, the results of financial or compliance audits, program reviews, and any other information that the Secretary may provide to the agency; and ACICS’s primary standards relative to Title IV requirements are found in Section 3-1-430. Standards cover institutionally-financed grants & loans; tuition & charges; the refund policy; the administration of student financial aid; & the policy on cash discounts. Also, the following other sections relate: “Complaints and Adverse Information” (Section 2-3-700); “Student Loan Cohort Default Rates (CDR) Review” (Section 2-1-810); “Counseling and Guidance” (Section 3-1-441); and “Guidelines on Disclosure &Notification” (Appendix G). The student services standards require that institutions counsel students on repayment and document this. ACICS requires an institution to publish in its catalog an equitable refund policy. An institution that participates in Title IV programs is expected to be knowledgeable of, and in compliance with, applicable laws and regulations. Each institution must address its default rate annually and in its self-study. The agency is expected to monitor each institution’s student loan CDR. If the institution’s most recent CDR triggers a review, or its default rate improvement plan failed to minimize student loan defaults adequately, the agency will require additional action. ACICS policy indicates that the agency uses information provided by the Secretary to determine whether to investigate an institution. In addition, any time ACICS receives information or has substantive evidence that an institution is in jeopardy of having its eligibility status with another entity or its accreditation status with another agency restricted, ACICS can take further action. However, the agency’s Title IV standards and process for implementing them is ineffective, as was demonstrated 16 when the Department investigated Title IV fraud by an ACICS-accredited institution, Michigan Jewish Institute (MJI). The Department notified ACICS in late 2012 of concerns about MJI as to (i) whether MJI’s programs were designed to educate students for professional, technical or occupational careers as required by the agency’s published policies and by the agency’s scope of recognition, and (ii) whether the institution’s contractual arrangements with foreign institutions met ACICS’s and Department requirements. In response, ACICS instituted a special review of MJI. ACICS notified MJI of its determination that “a vast majority of the institution’s students” were enrolled in a program of religious studies outside ACICS’s scope of recognition, and required those programs to be taught out (April 17, 2013 ACICS letter). ACICS also required MJI to provide it with copies of any contracts with institutions abroad. Under the agency’s published policies, MJI should have provided these contracts to ACICS before initiating them. In the April 2013 letters, ACICS advised MJI of numerous findings of noncompliance and solicited additional information (ACICS letters dated August 19, 2013 and December 30, 2013). In April 2014, ACICS renewed the institution’s accreditation “with admonishment,” including in the letter additional demands for 13 contracts MJI still had not provided (New Grant Approval Letter with Admonishment). Notwithstanding, ACICS wrote to the Department in July 2014, to respond to “questions that we have been advised that the Department of Education representative . . . raised during a recent program review regarding ACICS’s approval of certain education abroad activities of MJI” (ACICS letter of July 23, 2014). In the letter, ACICS said nothing about its numerous findings regarding MJI, about the fact that MJI had not provided prior notice of its international contracts to ACICS, violating published policies, nor about the fact that the religious studies program was outside the agency’s accreditation and recognition. Instead, the letter recited that MJI had been subject to “comprehensive on-site reviews, in which MJI’s international partnership agreements were specifically examined to determine their compliance,” and that “MJI’s international partnership agreements and education abroad activities have been previously been, and . . . remain approved in compliance with applicable Accreditation Criteria.” The letter concealed the agency’s own concerns about MJI, which were directly pertinent to Title IV fraud the Department ultimately uncovered, and, in failing to disclose ACICS’ own finding regarding the religious studies programs, withheld proof of MJI’s liability to the Department for repayment of millions. The Department’s own investigation of MJI later revealed what ACICS knew or should have known: that the institution operated primarily by recruiting and awarding Pell Grants to individuals who sought the funds to finance ineligible religious studies at ineligible foreign institutions rather than using them to support any significant engagement in the programs MJI purported to offer (cf. MJI Letters, dated February 25, 2016 and April 15, 2016). The agency’s conduct with respect to the MJI and the Department indicates that the agency does not effectively apply its standards and policies regarding compliance with Title IV. Therefore, the agency must explain what changes it has made and will make that ensure it is consistently effective with respect to Title IV compliance. Analyst Remarks to Response: In response to the Department Staff draft analysis, ACICS has documented its policy revisions and process to explain changes it has made to improve its effectiveness with respect to Title IV compliance (Exhibit 180, Accreditation Criteria, July 1, 2016, Appendix J, p. 126). The agency's revised policies and procedures concern encountering and sharing information with the Department in advance of a scheduled program review, and increased outreach to the multi-campus team of the Federal Student Aid Program Compliance division. ACICS has added the review of annual FSA compliance audits to the materials it brings before the Council for consideration in evaluating an institution's financial stability. ACICS has also implemented a process by which institutions placed on heightened cash monitoring status by the Department are required to submit letters of credit and must also provide financial improvement plans to the Council for review and approval. If institutions are placed on HCM2 status by the Department they are required to submit a teach-out plan that provides for the orderly cessation of operations in the event of financial catastrophe or a decision by the owners to close the institution. ACICS' new policy of notifying the Department of any institution is found non-compliant with an ACICS standard related to Title IV, including credit hours calculations, Satisfactory Academic Progress, refunds return of Title IV resources, tuition and fees, scholarships, cohort default rates or financial stability is demonstrated in the agency's Exhibit 249: EDMC, Zenith Education Group, ITT, Delta Career Education Show Cause letters 2016. However, the criteria required the agency to have been doing this all along. In addition, requirements for sharing information with the Department are not narrowly limited to Title-IV specific issues, as ACICS’s policies would suggest; instead, they apply whenever a recognized agency has reason to believe an accredited institution has been engaged in fraud or abuse. Contrary to ACICS’s narrative, and as demonstrated in part by the Michigan Jewish site review report ACICS has submitted with its petition, the evidence of abuse on the part of that institution was clear notwithstanding the institution’s use of study abroad and ought to have been disclosed to the Department, even more so since the Department expressly asked the agency about the school in performing a program review. While the agency has provided its revised multi-faceted approach to monitoring which will target and apply increased monitoring/reporting of specific schools that are not meeting agency standards/expectations, the Department is concerned regarding whether the agency will be able to demonstrate and provide documentation that it collects information in the appropriate time lines and follows up on a variety of reporting areas including for example, financial, student achievement, retention, and job placement data along with the institution's Title IV responsibilities. §602.17 Application of standards in reaching an accrediting decision. 17 The agency must have effective mechanisms for evaluating an institution's or program's compliance with the agency's standards before reaching a decision to accredit or preaccredit the institution or program. The agency meets this requirement if the agency demonstrates that it-(a) Evaluates whether an institution or program-(1) Maintains clearly specified educational objectives that are consistent with its mission and appropriate in light of the degrees or certificates awarded; (2) Is successful in achieving its stated objectives; and (3) Maintains degree and certificate requirements that at least conform to commonly accepted standards; ACICS’ standards expect each institution’s objectives to be devoted substantially to career-related education. As well, there are to be reasonable programs of instruction with a mode of delivery and facilities that help students develop skills and competencies to enhance their careers (cf. Section 3-1-100: Mission: Purpose and Objectives, and Section 1-2-100: Minimum Eligibility Requirements). The stated process is for the agency to send a visiting team to review the data collected on the elements of student retention, student placement, graduate satisfaction, employer satisfaction, and student learning outcomes. The focus is to be on each program credential offered in order to see whether the institution has achieved its missions and objectives. The final step is for the agency’s decision-makers to assess the success, or failure, of an institution in achieving its stated objectives. However, based on the widespread placement rate falsification discussed in Section 602.16(a)(1)(i), the agency is not in compliance with this section. The agency accredits institutions that provide vocational programs. Placement in the field of training is the ultimate objective of these programs. Without reliable information on which to determine whether students are placed, the agency is unable to make a reliable determination as to whether the institutions are successful in achieving this objective. Therefore, the agency needs to demonstrate what it will do to consistently ensure the reliability of its decisions as to whether an institution is successful with regard to placement. Analyst Remarks to Response: In response to the draft analysis, the agency has further explained their processes for a systematic review of its institutions relative to mission and objectives.The agency included documentation (Exhibit 229) that demonstrates how they have placed institutions in a probationary status based on their review of student outcomes data. However, Exhibit 206 does not provide enough details related to reported changes in data integrity standards and the addition of a new data integrity reviewer on the site team for Department staff to understand the impact of these changes on the agency’s overall review of the institutions relative to mission and objectives. For example, if information is found to be inaccurate on the institutions Campus Accountability Report (CAR), it is stated that the agency will require additional intensive verification of the CAR. However, the intensive verification process is not explained. Additionally, the new data integrity standard is effective 08/01/2016. It remains unclear how the agency has held institutions accountable for ensuring integrity in their data submissions before adopting this standard, or afterward. A new data integrity reviewer has been added as part of the onsite visit team and is said to be effective for the spring 2016 cycle, but that more guidance will be issued in fall 2016. However, there was not a site visit report (or exhibit) that included information about the new reviewer demonstrating application of this new standard. The narrative states the reviewer has been trained for this role and the agency has included documentation of communication and training for this role in Exhibit 203. Without additional information to clarify the new standards and documents demonstrating the application of these processes, the agency has not demonstrated that it has consistent application of standards in the reliability of its decisions relative to its institutions for mission and objectives. (c) Conducts at least one on-site review of the institution or program during which it obtains sufficient information to determine if the institution or program complies with the agency's standards; The agency conducts on-site reviews of approximately three days duration for institutions seeking either initial or renewed accreditation. The intent of the on-site evaluation is to verify information contained in the self-study and to review institutional compliance with ACICS standards, policies and procedures. The agency provides guidance to the team regarding the conduct of the on-site evaluation and on how to prepare the written team report while on-site. Guidance includes questions that should be asked during interviews with the institution’s director, students, admission director and representatives, the financial aid director and officers, placement director and faculty. 18 In order to help ensure that sufficient information is obtained by the team, a checklist is provided to the institution of materials and documents that must be assembled before the site visit. While on-site, team members review those materials, including student records, and student achievement outcome data on retention and placement rates provided by the institution in its annual report to ACICS. In addition, the team reviews faculty and administrative staff records and publications; classrooms and instructional resources; and equipment and educational facilities. Subject matter specialists evaluate and review each program’s area of study. At the conclusion of the visit the team conducts an exit interview to report its overall draft findings. The team chair prepares the final evaluation report after editing the draft before submitting it to the agency. However, based on the widespread placement rate falsification discussed in Section 602.16(a)(1)(i), and prevalent actions by state and federal authorities for recruiting abuse and misrepresentation discussed in Section 602.16(a)(1)(vii), the agency is not in compliance with this section. ACICS is apparently not obtaining sufficient information to determine if the institution complies with the agency's standards. Therefore, the agency needs to explain what changes it has made and will make to ensure it consistently obtains sufficient information during an on-site evaluation to make a reliable compliance determination. Analyst Remarks to Response: The draft staff analysis found that the agency needs to explain what changes it has made and will make to ensure it consistently obtains sufficient information during a site visit to make a reliable compliance determination. In response, the agency referred to a number of actions that it has taken that are related to the on-site process. Those actions include gathering information from a variety of external sources in a “call for comment” prior to the institution’s visit. Critical information found at any time will put the institution into the “at-risk” category, which heightens the level of investigation both during and outside of the regularly-scheduled on-site process. As well, the agency has expanded its verification of the institutional materials submitted to ACICS, materials also used by the on-site team. The strongest improvement that ACICS has made to the regular on-site process is the additional inclusion of a “data integrity reviewer” on each site visit. That person’s primary role is to test and review the institution’s self-reported data while on-site and following up immediately on any questionable data. The agency also reported that is strengthening the template outline used by on-site teams, especially emphasizing explanatory statements, so that the report is more useful to both the institution and the agency’s decision-makers. In summary, the agency discussed a number of actions that it has taken that are related to, but mostly outside of, the actual on-site process. This criterion requires the agency to conduct at least one on-site review during which it obtains sufficient information to determine if the institution complies with the agency's standards. Although the agency is making strides at improving its overall processes, there was insufficient evidence that its regular on-site visit process currently and consistently meets the requirements of this section. §602.18 Ensuring consistency in decision-making The agency must consistently apply and enforce standards that respect the stated mission of the institution, including religious mission, and that ensure that the education or training offered by an institution or program, including any offered through distance education or correspondence education, is of sufficient quality to achieve its stated objective for the duration of any accreditation or preaccreditation period granted by the agency. The agency meets this requirement if the agency-(d) Has a reasonable basis for determining that the information the agency relies on for making accrediting decisions is accurate; and Up until recently, ACICS relied on traditional methods to obtain reasonably accurate information. It began with the training of its on-site evaluators before they began their work, with occasional updating of that training to meet new challenges. While on-site, the evaluators followed a written process for obtaining the needed information, and for verifying its accuracy, through cross-checking existing records and interviewing numerous parties in different situations. The agency also instructed its evaluators to present their initial findings to the institution while on-site, where the first opportunity was given for the institution to ensure that the team had accurate information. Then, after the draft evaluation report was shared with the institution, another opportunity occurred to ensure that the information was accurate before it went to the ACICS decision-makers. More recently, the agency instituted a “Placement Verification Program” to randomly call employers to see if the information provided by selected institutions was reasonably accurate. Although that program had a valid purpose, it is apparent that the agency had an inadequate process and provided insufficient resources to accomplish its purpose. The same concerns are true for the verification of institutional retention rates. Therefore, based on the widespread placement rate falsification discussed in Section 602.16(a)(1)(i), the agency is not in compliance with this section. The agency does not appear to have a reasonable basis for determining that the 19 information it relies on for making accrediting decisions is accurate. As a result, the agency needs to address what it will do to ensure that its decisions are consistently based on reliable information. Analyst Remarks to Response: In response to the draft analysis, the agency provided additional explanation for clarifying the processes and the use of information it relies on for making accrediting decisions is accurate. The agency states in its narrative that they are not able to conduct verification on every item in the institutional information and the agency relies on the standard that states that “All data reported to ACICS for any purpose is expected to reflect an accurate and verifiable portrayal of institutional performance and is subject to review for integrity, accuracy, and completeness.” (Memo to the Field, pg. 4) Department staff concern is that this is a new standard that is not in effect until July 1, 2016. It remains unclear how the agency has held institutions accountable for ensuring integrity in their data submissions before adopting this standard, or afterward. It has been noted in other section that verification of information has been problematic for the agency. The agency explains that student achievement data are reviewed at three points in time-upon submission by algorithms, by the data integrity reviewer during site visits, and by monthly sampling. If any of these reviews reveal concerns, the data is subject to an independent audit. However, while stated in the agency’s documents that the data integrity reviewers has been added to the site visit team in Spring 2016, Department staff has been unable to confirm the involvement of this individual on a site visit report. Exhibit 203 was included that demonstrates documentation of training for the data integrity reviewer. Additionally with regards to the independent third-party verification of all placement data, (at the institution’s expense, when deemed necessary by the Council) the agency does not describe the qualifications of the 3rd parties, does not explain when the Council might deem this necessary, and provides no documentation it has occurred. The agency has included documentation of a disclosure verification presentation from their 2016 annual conference. While it documents training, it is not clear who attended the session, if the information was made available to those unable to make the conference, or how the information in this training session is communicated to all institutions. It appears that the requirement of the disclosure for verification and comparison of the institutions website against the Campus Accountability Report (CAR) by the institution is a result of a CHEA requirement. While the agency states that they will be systematically verifying the information in these locations for accuracy, they have not provided documentation demonstrating this review. §602.19 Monitoring and reevaluation of accredited institutions and programs. (b) The agency must demonstrate it has, and effectively applies, a set of monitoring and evaluation approaches that enables the agency to identify problems with an institution's or program's continued compliance with agency standards and that takes into account institutional or program strengths and stability. These approaches must include periodic reports, and collection and analysis of key data and indicators, identified by the agency, including, but not limited to, fiscal information and measures of student achievement, consistent with the provisions of §602.16(f). This provision does not require institutions or programs to provide annual reports on each specific accreditation criterion. The agency stated it has multiple monitoring approaches that include the Annual Financial Report (AFR), Campus Accountability Report (CAR), substantive change requests, complaints and other external information, Cohort Default Rate (CDR) reviews, and special on-site evaluations to any institution deemed to need additional oversight. With regard to fiscal information, all institutions must submit an AFR, which includes the annual audited financial statement. ACICS states that an institution that is deemed to have "abnormal financial risk" may be required to submit a financial improvement plan and quarterly financial reports; may be issued a compliance warning, show cause or other negative action; or may be required to host a special or unannounced visit. ACICS states that if an institution does not meet the agency's financial stability requirements, the institution would be notified in writing of the deficiencies, the degree of substandard performance, the required remedies, and timeframes to return to compliance. However, ACICS did not provide any information or documentation to describe its process for reviewing the financial information it collects or the manner by which it identifies "abnormal financial risk" or what its financial stability requirements are. ACICS also did not provide a template or completed example of an AFR. ACICS provided 2 examples of actions it has taken as a result of financial review. One example was in response to the review of an AFR, which required a quarterly financial report, but no further action (such as review of the quarterly financial report or any other action by ACICS) was provided. The other example was in response to action taken by the Department's Office of Federal Student Aid with regards to financial monitoring - not the agency's review - and also did not include any further action by ACICS. With regard to student achievement, all institutions must submit a CAR, which includes retention, placement, and licensure pass rates, as applicable. ACICS states that an institution that is deemed to have "substandard performance" may be required to develop a program-level or campus-level improvement plan as an addendum to its Campus Effectiveness Plan (CEP); and/or may be issued a compliance warning, a show-cause directive or other negative action. Although ACICS provided template response letters, it did not provide any information or documentation to describe its process for reviewing the student achievement information it collects or the manner by which it identifies "substandard performance." In its response to supplemental questions posed by the Department, 20 ACICS states it "has fortified its requirements for the integrity of student achievement data, subjecting the data to a multi-step integrity check upon submittal, enhanced templates for review by evaluators during site visits, 3rd party verification in certain circumstances, and verification through an in-house outreach process," but provided no documentation of the additional processes stated. ACICS provided a completed example of a CAR in Section 602.16(a)(1)(i). ACICS provided 2 examples of actions it has taken as a result of the review of student achievement. One example was an institution's CEP, but no agency action letter regarding the CEP nor further action (such as review of the CEP or any other action by ACICS) was provided. The other example was an agency action letter that noted "a weak and deteriorating level of student achievement," placed the institution on show cause, required an improvement plan and teach-out plan, but also did not include any further action by ACICS. Within its narrative and Accreditation Criteria, Section 2-1-810, ACICS states that an "improvement plan is reviewed by Council for its sufficiency and effectiveness, and must be included in the CEP for review during the next comprehensive on-site visit" and if "the Council determines the institution is out of compliance with the Council's requirement for student achievement, the Council will issue a compliance warning and require the institution to demonstrate compliance with the next year's CAR." The review of improvement actions required by ACICS a year or more in the future does not align with the requirements of Section 602.20. Therefore, the agency's current financial and student achievement monitoring and evaluation approaches have not demonstrated that ACICS is able to identify financial issues or problems with an institution's student achievement measures, based on the agency's recent history and limited action with multiple institutions that have had publicly-reported, financial and student achievement issues. In response, ACICS must include documentation of the financial &/or student achievement monitoring of 2 Sanford Brown campuses in Table C of its response to questions from the Department, & the Arlington, VA & Silver Spring, MD Everest College campuses. Beyond the review of fiscal information and measures of student achievement, ACICS states it monitors institutions via substantive change requests, complaints and other external information, Cohort Default Rate (CDR) reviews, and special on-site evaluations to any institution deemed to need additional oversight. Although ACICS stated that it monitors its institutions using these approaches, it provided exceptionally limited information and documentation on its review process for such approaches. For example, the agency's standard on CDR reviews states ACICS "will monitor an institution's student loan cohort default rates" and an institution "may be subject to additional reports or actions based upon these rates" with no indication on how the review will be conducted or benchmarks for action. ACICS provided committee meeting minutes for the review of "adverse cases", which included a timeline and current status for each institution. However, most of the cases were status reports based on the actions of others (i.e. State agencies, State Attorneys General, the Department, etc.) and were not action items by ACICS. In its response to supplemental questions from the Department, ACICS included a table of the "heightened monitoring" imposed on seven institutions and/or multi-campus groups. Although the table includes when ACICS was first notified of issues by State agencies, State Attorneys General, the Department, etc., no documentation was provided as to what the agency did in response to the receipt of that information. In its response, ACICS must submit documentation of the process and action taken for the Corinthian Colleges campuses located in CA after notification of the State Attorney General action in 2011, and must submit documentation of the process and action taken for the Career Education Corporation campuses in NY after notification of the State Attorney General action in 2011. In addition, the large number of substantial settlements agreed to by ACICS-accredited institutions in qui tam actions and actions by State attorneys general indicate that ACICS is not effective in its monitoring. Its monitoring regime appears insufficient to deter widespread misconduct regarding placement, recruiting and admissions. The actions discussed in Section 602.16(a)(1)(i) also indicate that the agency does not act on the information obtained through its monitoring. The agency must demonstrate how it was effective in its monitoring and provide a comprehensive plan for addressing this pervasive problem. Analyst Remarks to Response: In response to the Department Staff Draft analysis, ACICS via its narrative and supporting exhibits provided information and documentation to describe its process for reviewing data and information it has collected and the manner in which it has identified deficiencies with regards to fiscal information and measures of student achievement. (Exhibit 127: Response to Secretary of Education Letter May 16, 2016 from A. Bieda). The agency states, that the Council monitors and evaluates institutional performance regarding financial stability, student achievement, excessive growth, substantive changes, and adverse information and complaints through its standing committees. The agency's narrative details the process used by the committees involved in the monitoring and evaluation process. With regard to monitoring fiscal information, every accredited institution must submit an Annual Financial Report (AFR) (Exhibit 57: AFR Template), including audited financial statements. The content and purpose of the AFR and audited financial statement are prescribed by the Council (Exhibit 180: Accreditation Criteria, Title II, Chapter 1, Section 2-1-802 and 2-1-803, p. 12)). The agency also provided documentation via its exhibits that are meant to demonstrate that it has collected the information its process require – the QFR, FIP or both. (Exhibit 252: Everest QFR Letter; Exhibit 253: Everest Institute QFR Letter; Exhibit 254: Everest Institute CQFR Letter). In requiring the institution to submit a QFR or FIP, the institution is notified in writing by the Council of the deficiencies, the degree of substandard performance, the required remedies, and timeframes. The agency provided a rubric to show how it arrives at these judgments. However, its documentation does not show any consequence of the filing of such reports, other than one instance of requiring a second report. With regard to student achievement, the agency does not explain its failure to uncover or report Corinthian’s 21 widespread placement rate misrepresentations, other than to imply it does not credit any evidence of noncompliance or misconduct not included in a final judgment of a court of law. As discussed above in response to 602.16(a)(1)(i), Corinthian’s widespread misconduct in this area has been confirmed by a California court and through the Department’s own investigations, resulting in appointment of a special master (Exhibit uploaded by Department staff) and proceedings for discharge of thousands of student loans. The agency also provides only the most general response to the Department’s requests for documentation regarding Sanford Brown and CEC schools. Letters regarding deficient student achievement data (Exhibit 240) do not feature public sanctions and emphasize deferrals, mitigating circumstances, waivers, and rounding errors. Minutes intended to establish effective monitoring (Exhibit 250) reflect approvals of institutional requests, with one non-substantive denial and minimal deferrals . While the agency's revised Accreditation Criteria (Exhibit 180, effective July 2016) documents a revised set of monitoring and evaluation approaches for identifying problems with an institution's or program's continued compliance with agency standards and with this section of the criteria, those criteria have not yet been applied. The Department cannot see how the agency could apply these revisions in such a way as to document effectiveness in monitoring in the time it would be given to respond in a compliance report, particularly in view of its weak record in monitoring and failure to document enforcement. Also, Department staff was not able to verify that ACICS provided the appropriate information and documentation of the use of all monitoring mechanisms included in the agency's narrative as requested in the staff draft analysis. §602.20 Enforcement of standards (a) If the agency's review of an institution or program under any standard indicates that the institution or program is not in compliance with that standard, the agency must-(1) Immediately initiate adverse action against the institution or program; or (2) Require the institution or program to take appropriate action to bring itself into compliance with the agency's standards within a time period that must not exceed-(i) Twelve months, if the program, or the longest program offered by the institution, is less than one year in length; (ii) Eighteen months, if the program, or the longest program offered by the institution, is at least one year, but less than two years, in length; or (iii) Two years, if the program, or the longest program offered by the institution, is at least two years in length. The agency has a policy that meets the requirements of this section, included in Title 2, Chapter 3 of the agency's Accreditation Criteria. What is not clear is that ACICS is enforcing the policy, which seems inconsistent with or at least not referenced in other pertinent agency materials. In addition, many of the issues noted here were noted in the agency's 2011 petition for recognition & 2013 compliance report to the Department. Although the agency's policy states that it would take "prompt adverse action" or provide a time period to return to compliance, ACICS states that an institution would be able to "respond to a deferral, compliance warning, or show-cause directive" before the Council "denies, withdraws, suspends or revokes accreditation," which does not meet the requirements of this section. In addition, the agency provided its "Council Action Process Chart" which does not have an option for an immediate adverse action, as defined by the Department in Section 602.3. The agency's narrative & section 2-3-700 of its Accreditation Criteria, ACICS states that it reviews information submitted by third-parties, to include from "federal or state agencies or other accrediting agencies, or through public media sources." Although ACICS provided limited documentation concerning its review of a complaint & its monitoring & investigation of "adverse" information, ACICS did not provide any documentation on the use of information received from federal or state agencies or other accrediting agencies, or through public media sources to initiate agency enforcement action. In its response, ACICS must submit such documentation regarding enforcement action taken in response to the State Attorneys General investigations in public media sources related to Corinthian Colleges, ITT Technical Institutes, and Westwood Colleges, & notifications received from the Department's Office of Federal Student Aid concerning Corinthian Colleges. Section 2-3-101 of its Accreditation Criteria, ACICS states that it can include an "admonition" with a grant of accreditation action. An admonition is used when the "Council may judge an institution to be generally in compliance with the criteria, but it also may wish to call the institution's attention to one or more deficiencies that are not serious enough to preclude a grant of accreditation but that nonetheless must be corrected." The agency's statement strongly suggests that any admonition would pertain to an institution that is out of compliance with one or more standards. Under 602.20, ACICS does not have the option of granting accreditation while an institution is out-of-compliance with the agency's standards. Section 602.19(b), the annual reporting process described by ACICS - both in the narrative & in the agency's policies and procedures - is not comprehensive to enable ACICS to identify problems with an institution's continued compliance with agency standards. This is supported by the agency's narrative in this section, which states that ACICS will take action based on an annual report that indicates substandard performance with respect to student 22 achievement or financial stability, but does not provide specific documentation as to what the agency considers substandard performance in those areas. The agency included its ACICS Student Achievement Webpage, but it is unclear that this information is provided in the agency's standards or anywhere else besides the Webpage, & whether it is therefore enforceable and enforced by ACICS. Section 602.16(a)(1)(i) is not clear that the agency's review of student achievement meets the enforcement timelines as required by this section. An action letter the agency has provided that it sent to an institution that was not meeting the agency's student achievement standards is concerning. The letter explains that the institution must submit an improvement plan & is placed in the "year 1 category" for not achieving the required student achievement rates. While the letter is linked to the due process information included on the ACICS Student Achievement Webpage, it is not clear in the letter that "year 1" means deferral. In addition, while it is permissible to defer decision when an agency needs additional information to determine compliance, that cannot be done when an institution is clearly out-of-compliance with not only the agency's benchmarks, but its standards, as reflected in this example. Under the requirements of this section, any program of less than one year should face adverse action at the end of "year 1," absent an extension for good cause. Without additional information regarding the length of the institution's longest program, Dept staff cannot tell if ACICS met the requirements for enforcement timelines of this section with respect to this example, since the letter clearly states that the institution is not meeting the required standard. The lack of clarity is supported by the agency's narrative in this section, which states "For the first year below student achievement standards, the campus is required to complete an improvement plan for review by Council. If the campus or program performs below standard for a second consecutive year, the Council issues a compliance warning. Programs or campuses that perform below student achievement standards for a third consecutive year are subject to immediate adverse action, including suspension of accreditation, or program termination." This section requires an agency to take immediate adverse action, or give an institution a specific timeframe for coming into compliance. The agency's narrative, & its ACICS Student Achievement Webpage, clearly indicate that ACICS provides a time period longer than regulations allow to return to compliance for an institution that is not meeting the agency's student achievement standards. Based on the narrative throughout the petition, ACICS appears to be confused with the application of the regulation with regards to its review of student achievement. Although the agency's student achievement standards have a clear bright-line for retention & placement rates, the agency appears to only determine an institution out-of-compliance after given time to improve. Specifically, the agency's narrative states "Institutions with programs out of compliance with student achievement standards are expected to remedy their performance or discontinue the program within one year." However, the information included in the ACICS Student Achievement Webpage clearly states "If a program reports three consecutive years of below-standard retention, placement, or licensure pass rates, it will be required to cease enrollment and terminate the program of study." The agency's interpretation of this regulation in regards to its review of student achievement is incorrect & must be revised to reflect enforcement as soon as the institution or program's rates are reported below the agency's published standard. ACICS provided examples of council action letters on deferral, show cause, & an unannounced site visit. However, as the examples provided did not include the full cycle of the review (i.e. first determination that the institution was out of compliance to final action), notification of the specific time period to return to compliance, nor any examples of adverse actions, the agency has not demonstrated that it initiates immediate adverse action or enforces the required time period for an institution found to be out-of-compliance. Analyst Remarks to Response: The draft staff analysis found that the agency needs to revise its enforcement action policies to meet the requirements of this section. The agency must also provide the specific documentation requested regarding the agency's use of information received from federal or state agencies or other accrediting agencies, or through public media sources to initiate agency enforcement action. Furthermore, the agency must provide complete documentation to demonstrate that it has initiated immediate adverse action or enforced the required time period for an institution found to be out-of-compliance with any standard. In response, the agency noted that it has begun the process to revise its pertinent criteria so that the related problems noted in the draft analysis will be remedied. As noted in the draft analysis, the agency did publish a chart that contained the necessary timeframes for coming into compliance that coincides with the Department’s required timelines. The agency’s proposed revisions should help clarify somewhat earlier language that tended to confuse and hamper a consistent application of the agency’s standards, particularly with regard to the maximum enforcement timeframes. However, ambiguities and inconsistencies remain, such as the timeframes for enforcing student achievement standards, as well as the significance of agency “admonitions” and their consistency with this criterion. In addition, the revisions do not address the action of “accreditation deferred,” and include exceptions to enforcement for “significant progress.” These exceptions appear to enable multiple extensions based on unquantified improvement, negating the effectiveness of the standard. All of this precludes a determination that the revised policies would comply even if other policies were conformed with them. The student achievement website information has a heading for “withdrawal or suspension of a program,” but the only enforcement options it discusses are probation or other unspecified conditions on accreditation. Similarly, under the heading “Immediate Adverse Action,” the website lists only probation or other conditions on accreditation. Probation is not an “adverse action” within the meaning of the Criteria. The website also indicates that probation/”conditions” may be imposed for deficient rates of student achievement only “absent any other evidence” and only if the rates are “significantly below” the standard and the institution’s capacity to improve is “limited.” Taken together, these website policies do not appear to provide ACICS with a basis for taking adverse action as defined in the Criteria based on noncompliance with the agency’s student achievement benchmarks. The proposed revisions are to be considered by the ACICS decision-makers at their August 2016 meeting. Those 23 proposed revisions should reduce confusion somewhat. However, further revisions are likely before final approval, and they will need to be circulated to the agency’s constituents before final adoption and implementation. With the various scenarios envisioned in the ACICS proposal (cf. Exhibit 121) Department staff believes that it is still not entirely clear when each enforcement timeframe actually begins, or in other words, when “the clock starts ticking.” The agency has many interlocking policies, including the repeated granting of deferrals, which tend to delay and/or muddle the enforcement calculations. As well, the letters that the affected institution will actually receive regarding the beginning and length of the enforcement period will need to be clear and precise, which has not been demonstrated. As well, ACICS was asked to provide specific documentation regarding the agency's use of information received from federal or state agencies or other accrediting agencies, or through public media sources to initiate agency enforcement action. In response, the agency provided a summary (Exhibit 244) that highlighted the increased monitoring that ACICS initiated as a result information received from outside sources. However, initiating heightened monitoring, and initiating an enforcement action, are two entirely different responses to a critical situation. In addition, ACICS was asked to provide complete documentation to demonstrate that it initiated immediate adverse action or enforced the required time period for an institution found to be out-of-compliance with any standard. However, ACICS did not provide the requested documentation. It did not provide documentation requested with regard to ITT, Westwood, and agency response to state and federal investigations, nor did it provide any of the other documentation requested in the draft. The agency was cited on this criterion in 2011 and 2013, and the fact that it remains out of compliance undercuts confidence in the its documentation of initiatives to revise its policies. Since the agency’s response to the documentation requests was insufficient and since the proposed language revisions do not remedy the deficiencies and inconsistencies and have not been finalized and implemented, and in view of the agency’s ineffectiveness in addressing this Criterion in the past, ACICS was unable to demonstrate compliance with the requirements of this section. (b) If the institution or program does not bring itself into compliance within the specified period, the agency must take immediate adverse action unless the agency, for good cause, extends the period for achieving compliance. The agency provided a written policy that meets the requirements of this section, in which the agency can take immediate adverse action if an institution does not bring itself into compliance within the specified time period. However, the agency did not provide any documentation to demonstrate that it has ever taken such action, as required by the regulations and the agency's policies. As discussed in Section 602.20(a), the agency’s written policies regarding student achievement on their face violate this requirement since they establish a timeframe for coming into compliance that may exceed the time periods provided in Section 602.20(a). The agency also has a written policy that includes parameters under which a good cause extension could be granted. However, the policy does not include a maximum length of any good cause extension. (The agency stated that no institutions have been granted a good cause extension in the previous five years, and therefore the agency could not provide documentation of its granting of good cause extensions to demonstrate compliance with this section.) Until the agency satisfactorily addresses these matters, a finding of compliance cannot be made Analyst Remarks to Response: The draft staff analysis found that the agency needed to provide documentation to demonstrate that it had taken immediate adverse action if an institution did not bring itself into compliance within the specified time period. In addition, the agency needed to revise its policy on good cause extensions to include a maximum time period. In response, the agency noted that it has adopted a revised policy (effective July 2016) that allows a good cause extension at the discretion of the Council, provided there is evidence that the area of deficiency is improving significantly, and that the standard maximum timeframe does not afford the institution sufficient time to demonstrate full compliance. The agency has incorporated a variant of that language into its template letter that is only used in situations when student outcomes fall below the ACICS expectations. In those circumstances, the institution is informed that “The Council has the discretion, in rare circumstances, to consider an extension of the compliance interval for good cause. However, the institution/program will be required to 1)demonstrate “significant improvement” in the student achievement metric since the previous report, and 2) how the additional time will enable it to meet the requirement. Without those two elements manifest, a good cause extension will not be considered by the Council or available to the institution or program.” It should be noted that the draft staff analysis expressly noted that ACICS needed to revise its policy on good cause extensions to include a maximum time period. The revised policy does not include a maximum time period. Furthermore, the revised policy may actually undermine the ability of ACICS to remove accreditation from an institution since the policy applies when “the standard maximum timeframe does not afford the institution sufficient time to demonstrate full compliance.” In addition, the draft staff analysis asked the agency to provide documentation to demonstrate that it had taken immediate adverse action if an institution did not bring itself into compliance within the specified time period. However, ACICS did not provide the requested documentation. 24 However, ACICS did not provide the requested documentation. In summary, the agency’s proposed policy language regarding good cause extensions (to take effect in July 2016) does not include a maximum time period, as requested. As well, the agency’s response did not provide the requested documentation to demonstrate ACICS took immediate adverse action when an institution did not bring itself into compliance within the specified time period. As a result, ACICS was unable to demonstrate compliance with the requirements of this section at this time. §602.21 Review of standards. (a) The agency must maintain a systematic program of review that demonstrates that its standards are adequate to evaluate the quality of the education or training provided by the institutions and programs it accredits and relevant to the educational or training needs of students. (b) The agency determines the specific procedures it follows in evaluating its standards, but the agency must ensure that its program of review-(1) Is comprehensive; (2) Occurs at regular, yet reasonable, intervals or on an ongoing basis; (3) Examines each of the agency's standards and the standards as a whole; and (4) Involves all of the agency's relevant constituencies in the review and affords them a meaningful opportunity to provide input into the review. The agency maintains an integrated approach to ensure that its program of review is regular and comprehensive. The review is conducted annually and focuses on different parts (approximately one-fifth) of the accreditation criteria. As a result, over the five-year period all of the standards have been reviewed with opportunities for comment made available to the agency’s constituencies. Each annual survey asks the agency’s constituencies, including member institutions, evaluators, commissioners and staff, state regulatory agencies and students to focus on the selected standards with regard to their relevancy and adequacy. That is, the relevancy of the standard to the educational/training needs of students, and the adequacy of the standard to evaluate the quality of education/training provided by the institution. The results of the annual survey are considered during the agency’s policy meetings where issues can result in proposed revisions of standards and a request for further comments from the constituents. In addition, to obtain further comments on its standards and practices, ACICS conducts regional meetings, employer surveys, and webinars for its constituents. A recent webinar was conducted on April 28, 2016 regarding the agency’s new approaches to verifying placement information. The webinar reported that ACICS is adding an additional on-site team member specifically for the purpose of review the placement data reported by the institution. However, the agency’s current system for comprehensively reviewing its standards has an apparent flaw. The agency is focused on the section of the standards scheduled for review each year. Yet the agency does not have an effective mechanism for reviewing standards out of the scheduled cycle that have allowed significant issues at problematic institutions to go unchecked. Although the agency may claim that its procedures already allow for that special focus to take place at any time, its implementation has been demonstrably inadequate. Therefore, the agency needs to revise its standards review program to encourage a special focus on any deficiencies in its standards that are identified during each decision-making meeting. In addition, the agency needs to provide documentation that it is expeditiously and consistently implementing that specialized standards review, as needed. Analyst Remarks to Response: The draft staff analysis found the agency out of compliance with this criterion. Department staff noted that the agency did not have a systematic process to review its standards outside of the scheduled review cycle to address issues at problematic institutions, and make changes as appropriate. The agency discusses in its response, the methods and practices it employs which would inform its decision regarding standards revisions or updates. As an example, the agency references that it tracks all accreditation findings contained in team reports and identifies those most frequently cited. The information is used to evaluate which standards are worthy of revision to strengthen student protections or to clarify the Council’s intent. However, in previous criteria it has been noted that verification of the information reviewed by the onsite visit team and used by the agency’s decision making body has been problematic and unreliable. The agency also discusses new requirements/policies to help it determine whether standards changes should be accomplished outside of its current standards review process. The agency references its Campus Effective Plan (CEP). However, the agency has not provided a completed CEP for review nor has it provided evidence of how this plan was helpful in determining whether a standards change is/was necessary. The CEP was a major discussion in the meeting of the Institutional effectiveness Committee (IEC); however, It was 25 noted in 602.16(a)(1) there is no documentation explaining the IEC’s role or training. The IEC is not recognized as a decision maker for this agency, so the IEC should not make agency policy. Therefore the role of this committee to establish or approve policy regarding the CEP is unclear. The agency informs Department staff that it has adopted tougher standards regarding the review of recruitment and admissions practices, because of fraud and misrepresentation by institutions. However, these policy changes appear to have been made long after indications of fraud and misrepresentation were publicly known at institutions cited in the draft analysis. §602.22 Substantive change. (3) The agency's substantive change policy must define when the changes made or proposed by an institution are or would be sufficiently extensive to require the agency to conduct a new comprehensive evaluation of that institution. This criterion requires ACICS to define when substantive changes and/or proposed substantive changes are, or would be, sufficiently extensive to require the agency to conduct a new comprehensive evaluation. ACICS written policy is to “conduct a comprehensive on-site evaluation of the institution if substantive changes that have been made or are proposed are sufficiently extensive that the institution’s capacity to maintain compliance with accreditation standards requires an immediate assessment.” To determine whether the proposed or adopted changes are sufficiently extensive, the agency notes when the types and/or number of changes are so substantial that the nature and scope of the accredited institution will no longer be the same since last evaluated and in its place a new institution has evolved (“Effect of Extensive Substantive Changes” 2-2-102). The petition narrative indicates that in order to monitor the proposed/adopted substantive changes at each institution, ACICS documents each institution’s substantive change request, and each substantive change approved by the decision-makers. Then, the agency’s practice is to give the list to the decision-makers every time it considers a subsequent substantive change request from that institution. However, Department staff could find no written policy supporting that practice to help ensure its consistent implementation. The agency also provided its “November 2015 Executive Committee Meeting Minutes with Exhibits and Council approval letter” (Exhibit 113). However, when Department staff examined that exhibit, there was no clear list of previously-proposed or approved substantive changes for the decision-makers to rely upon in their deliberations. Furthermore, there was no indication in the materials that consistent decisions are being made to require, or not require, a new institutional evaluation. Therefore, the agency needs to provide a written policy that clearly describes the mechanism by which ACICS documents and monitors the proposed and adopted substantive changes at each institution. In addition, the agency needs to provide documentation that its decision-makers are given a clear list of previously-proposed or approved substantive changes that could trigger a new institutional evaluation. Furthermore, the agency needs to provide documentation that its decision-makers are making consistent decisions regarding whether or not to require a new institutional evaluation. Analyst Remarks to Response: In the draft staff analysis the agency was found out of compliance. The agency needed to provide a written policy that clearly describes the mechanism by which ACICS documents and monitors the proposed and adopted substantive changes at each institution. In addition, the agency needs to provide documentation that its decision-makers are given a clear list of previously-proposed or approved substantive changes that could trigger a new institutional evaluation. In addition the agency must provide documentation that its decision-makers are making consistent decisions regarding whether or not to require a new institutional evaluation. After review of the agency’s response it is clear that the agency has not adequately addressed all of the issues identified by Department staff in the draft analysis. It is still not clear based on the provided documentation whether on a consistent bases decision makers are given a clear list of previously-proposed or approved substantive changes that would trigger a new comprehensive evaluation. It is also unclear as to whether decision-makers are making consistent decisions regarding whether or not to require a new institutional evaluation. It is apparent that the agency does not have a written policy that identifies the specific reasons/types of substantive changes that would require a new comprehensive evaluation or a written process to determine whether a new comprehensive evaluation is required on a consistent basis. In addition, the agency states in its narrative “that to ensure that the aggregation of substantive changes trigger a new comprehensive review if a written threshold is breached, each substantive change is awarded points in a rubric and weighted depending on the accumulated impact on the institution’s nature and character”. However, the agency did not provide the rationale or associated policy for this points system and rubric. The agency also did not discuss what this written threshold is or how it was established. The agency has not provided the information or documentation to demonstrate compliance with this criterion. 26 §602.24 Additional procedures certain institutional accreditors must have. (c) Teach-out plans and agreements. (1) The agency must require an institution it accredits or preaccredits to submit a teach-out plan to the agency for approval upon the occurrence of any of the following events: (i) The Secretary notifies the agency that the Secretary has initiated an emergency action against an institution, in accordance with section 487(c)(1)(G) of the HEA, or an action to limit, suspend, or terminate an institution participating in any title IV, HEA program, in accordance with section 487(c)(1)(F) of the HEA, and that a teach-out plan is required. (ii) The agency acts to withdraw, terminate, or suspend the accreditation or preaccreditation of the institution. (iii) The institution notifies the agency that it intends to cease operations entirely or close a location that provides one hundred percent of at least one program. (iv) A State licensing or authorizing agency notifies the agency that an institution's license or legal authorization to provide an educational program has been or will be revoked. The current petition notes that the incidence of ACICS institutions closing precipitously has prompted debarment actions against at least 10 owners and operators of five different institutions. In addition, the agency has issued a notice of intent to debar against former executives of Corinthian Colleges for the cessation of operations at four Everest Institute campuses in California coincidental with the corporation's bankruptcy filing. Furthermore, based on adverse information provided by the federal government, state agencies and other sources, ACICS has also required the submission of teach-out plans for Michigan Jewish Institute, ITT Tech Institutes and other institutions who may be at risk of ceasing operations. As well, the agency provided documentation of these recent actions related to teach-outs. In the past, the agency provided documentation that its written policies explicitly corresponded with the requirements of this section of the criteria regarding teach-out plans and agreements. Although the current petition narrative has referred to some written policies (particularly “Teach-Out” 2-2-303), they do not explicitly refer to these requirements. Therefore, the agency needs to document that it has written policies that correspond to the requirements of this section on teach-out plans and agreements, to include the agency's risk assessment framework and list of institutions considered 'at risk' by the agency. In addition, the agency needs to provide documentation that it is consistently implementing those policies in practice, and has a comprehensive process for reviewing the plans. Analyst Remarks to Response: In response to the Department staff draft analysis, ACICS provided in (Exhibit 180) Accreditation Criteria effective July 1, 2016 demonstrating written policies documenting its requirements regarding teach-out plans and agreements. While the agency has, in practice required its teach out plans to include the requirements of this section, its revised teach out policies do not included the terminology used in 602.24(C)(1). It should also be noted, that the requirements for teach-out plans and agreements that were in the agency’s Accreditation Criteria approved January 2016 do not include the language of this section. The agency needs to revise its teach out plan policies to include all the requirements in this section of the criteria. §602.27 Other information an agency must provide the Department. (a)(6) The name of any institution or program it accredits that the agency has reason to believe is failing to meet its Title IV, HEA program responsibilities or is engaged in fraud or abuse, along with the agency's reasons for concern about the institution or program; and (a)(7)If the Secretary requests, information that may bear upon an accredited or preaccredited institution's compliance with its Title IV, HEA program responsibilities, including the eligibility of the institution or program to participate in Title IV, HEA programs.(b) If an agency has a policy regarding notification to an institution or program of contact with the Department in accordance with paragraph (a)(6) or (a)(7) of this section, it must provide for a case by case review of the circumstances surrounding the contact, and the need for the confidentiality of that contact. Upon a specific request by the Department, the agency must consider that contact confidential. 27 The agency has attested that there have been no changes to its policies and/or practices in this area. However, Department staff is concerned because there have been instances of fraud and abuse discovered at several ACICS-accredited institutions within the last few years, including misrepresentations to students and falsification of placement rates. Therefore, the agency needs to provide documentation that it consistently notified the Department regarding suspected Title IV-related fraud and abuse during the last five years, or to attest that it had not found any occasion to do so. Analyst Remarks to Response: In response to the draft analysis, the agency has provided a copy of their Accreditation Criteria that will be effective on July 1, 2016 and attested that there have not been any findings of Title IV fraud or abuse during this recognition period. However, the Accreditation Criteria cited simply reference selected Title IV requirements, and for the most part do not concern 602.27 or compliance with it. The only portion relevant to 602.27 the agency cites is a “guideline” found in paragraph (4) of Appendix G, which contains a commitment to disclosure to the Department that is significantly weaker than that required by 602.27(a)(6), and which ignores the requirements of 602.27(a)(7) and (b) entirely. The Accreditation Criteria also do not become effective until July 1, so there could not be documentation of implementation of the Criteria even if they were compliant, which as discussed they are not. In addition, the agency has not demonstrated that it complies with 602.27 in practice, and there is ample evidence it does not. For example, the agency has not explained why it did not share with the Department the information it had about the abuse practiced by, for example, Michigan Jewish Institute and CSI, discussed under 602.16(a)(1)(i) and (x), nor can its claims of a lack of any abuse to report be squared with the documented abuse regarding placement rates discussed under 602.16(a)(1)(i). P ART III: THIRD P ARTY COMMENTS Staff Analysis of 3rd Party Written Comments Approximately 40 written third-party comments were received regarding this agency. The comments reflected negative views regarding the agency. Most of the comments are on behalf of individuals, but there were 4 comments on behalf of organizations or State entities. Most of the commenters did not tie their areas of alleged noncompliance to specific sections of the Secretary’s Criteria for Recognition. The vast majority of comments submitted to the Department were associated with a group called the Debt Collective. In these 33 comments submitted by students, the overwhelming information provided is that the institution in which they attended misrepresented various aspects of the program to the students. In a number of cases, students expressed discontent with the quality of the instruction they received in terms of both faculty qualifications and curriculum, with the large costs associated with the program, and in regards to the processes used and information provided as they were recruited and/or admitted to the program. As these comments appear to be complaints about specific institutions, the Department does not typically commence a review of an agency based on individual complaints unless and until a complainant exhausts the agency’s published complaint procedures. It is not clear that all of the issues raised in the comments would indicate noncompliance with the Secretary's Criteria for Recognition by the agency. However, there are areas within this analysis where the commenters’ concerns are parallel to those of the Department. The Department has noted in this analysis that it has questions related to the agency’s review of student achievement and recruiting and admissions practices in Section 602.16(a)(1). A group of Veterans’ Organizations raised concerns regarding the default rates of the institutions that are accredited by ACICS, about the student outcome and recruiting standards of ACICS being weaker than other accrediting agencies, and the recruitment practices at various institutions accredited by ACICS that involve misrepresentations about cost and job availability. The Department has noted in this analysis that it has questions related to the agency’s review of student achievement and recruiting and admissions practices, in Section 602.16(a)(1), and the agency’s monitoring of institutions in Section 602.19(b). A group of consumer organizations raised concerns regarding the poor track record of student achievement by ACICS institutions, that the agency relied too much upon institutional administrators in its work and lacked sufficient representation from other constituencies, that a large number ACICS-accredited institutions are under investigation, and that ACICS fails to acknowledge any necessary improvement, but rather it actively defends the success of its current work. The Department has noted in this analysis that it has questions related to the agency’s review of student achievement in Section 602.16(a)(1)(i), the agency’s wide acceptance in Section 602.13, the agency’s monitoring of institutions in Section 602.19(b), and the agency’s enforcement actions in Section 602.20. A group of state attorney generals raised concerns that ACICS has failed to take action when improper job placement statistics are reported by an institution, that ACICS has an inadequate job placement verification processes, that ACICS demonstrates a lack of transparency and cooperation with investigations into student outcomes, that more action was needed as a result of the Corinthian Colleges investigations, and that ACICS failed repeatedly to take action in response to public enforcement actions by state and federal law enforcement. The Department has noted in this analysis that it has questions related to the agency’s review of student achievement in Section 602.16(a)(1)(i), the agency’s wide acceptance in Section 602.13, the agency’s monitoring of institutions in Section 602.19(b), and the 28 agency’s enforcement actions in Section 602.20. A memo from the Center for American Progress (CAP) raises similar questions from other commenters about ACICS’s standards for student placement rates, default rates, student outcomes, and recruitment standards. The request for additional information of the agency is reflected in the agency’s review of student achievement and recruiting and admissions practices, in Section 602.16(a)(1), and the agency’s monitoring of institutions in Section 602.19(b). CAP also raises concerns about the number of the lawsuits brought against ACICS. This information cited and in need of additional information is included in this analysis in the student achievement in Section 602.16(a)(1)(i), the agency’s wide acceptance in Section 602.13, the agency’s monitoring of institutions in Section 602.19(b), and the agency’s enforcement actions in Section 602.20. Note: Due to space limitations, not every single lawsuit was able to be named individually in those sections. Agency Response to 3rd Party Comments ACICS requested members of the community, including students, graduates, employers, the work-force development community and policy makers to express their opinions regarding the agency's value to them personally and to their organizations. Staff Analysis of Agency Reponse to 3rd Party Comments In response to the third party comments, ACICS asked for their stakeholders (including students, graduates, employers, the workforce development community and policymakers) to provide background regarding the value of attending/working with an ACICS accredited institution. The agency subdivided the responses into three exhibits: letters of support (Exhibit 150), employer letters (Exhibit 150-2), and letters from students (Exhibit 150-3). The letters of support include 54 testimonies from multiple parties on the experiences they have had in/related to programs that are accredited by ACICS at 15 colleges, with 34 of the 54 comments pertaining to a single institution. A number of these letters contain identical language, suggesting that they may have been part of a letter campaign. The employer letters capture 84 signatures of individuals attesting to the quality of education that a 16th institution, ITT Technical, has provided and the benefit the employers have received as a result of their relationship with that institution. However, ACICS put ITT on show cause on April 20, 2016, with a letter demonstrating that ACICS itself had serious concerns about “the institutions’ administrative capacity, organizational integrity, financial viability, and ability to serve students in a manner that complies w/ ACICS standards.” The final exhibit includes 64 letters from ITT Technical students who attended that institution and had positive experiences. While the information provided demonstrates support by these individuals for ACICS-accredited institutions, it does not address the concerns that the draft staff report raised. The agency did not provide a narrative response to address the specific concerns relevant to the Secretary's Criteria including those previously mentioned about student outcomes and the falsification of that information (including verification of placement data), misleading recruitment and advertising practices, questions regarding Title IV responsibilities (including those about default rates), and those concerns related to lacking faculty qualifications and curriculum standards. The agency also did not give a clear indication if the issues that were brought forth through the 3rd party comment process have been considered and/or resolved through the agency's complaint standards and processes. The narrative did not specifically respond to address any of the comments, not those from individuals, nor the comments from the 19 attorneys general, nor the comments from the groups of consumer and veteran's organizations, nor from the Center for American Progress. While not included in this section of the petition, the agency did provide documentation of their written response to the Attorney General’s letter under section 602.19(b). 29