Embedded Change Report Substantive Change Recommendation Form After the team reaches consensus, the team chair completes this form to summarize and document the team’s view. Notes and evidence should be essential and concise. Submit the completed report as a single electronic document (in PDF format) on hlcommission.org/document_upload/. Institution: Scottsdale Community College City, State: Scottsdale, AZ Visit Date: 01/30-31/2017 Change Requested: Distance Delivery Names of Change Peer Reviewers: (List the names, titles and affiliations of each peer reviewer.) Dr. Kenneth E. Urban, President, West Shore Community College, Scottville, MI Dr. Charles Jacobs, Biology Faculty, Henry Ford Community College, Dearborn, MI Part A: Analysis (See Part 1: General Questions in change application) 1. Classification of Change(s) Complete Incomplete Notes or additions if marked incomplete: 2. Special Conditions – Institutional Context Complete Incomplete Notes or additions if marked incomplete: 3. Required Approvals Complete Incomplete Notes or additions if marked incomplete: Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 1 Substantive Change Recommendation Form: Embedded Change Report 4. For Contractual/Consortial Arrangements Contractual Consortial Check all that apply: on-ground delivery off-campus delivery distance education other: Not Applicable (Skip 4a-4c) correspondence education Complete Incomplete Notes or additions if marked incomplete: 4a. Key Services Provided by Partner Complete Incomplete Notes or additions if marked incomplete: 4b. Level of Programming and Enrollment Affected Complete Incomplete Notes or additions if marked incomplete: 4c. Overall Proportion of Affected Programs Provided by Partner Complete Incomplete Notes or additions if marked incomplete: 5. For applications regarding Competency-based Education (CBE) programs, complete 5a-5e. Otherwise continue to question 6 on Essential Elements. For CBE, including direct assessment, credit-hour based CBE, and hybrid programs, for which competencies alone are used to evaluate student achievement and progress toward a degree or certificate: 5a.The degree or certificate program is consistent with college-level work and rigor, establishing academic outcomes and competency statements comparable to similar programs offered by the institution: Acceptable Not acceptable Evidence: 5b.The institution has submitted with its application a current credit hour worksheet from the Federal Compliance Packet OR has on file a recent (within the past three years) credit hour worksheet, which it has used to determine credit-hour equivalency for any program involving direct assessment: Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 2 Substantive Change Recommendation Form: Embedded Change Report Complete Incomplete Notes or additions if marked incomplete: 5c. The institution has determined that “sufficient educational activity” takes place in the CBE program and is consistent with the federal definition of the credit hour or is applied to the credit-hour equivalency used by the program (i.e., educational activity that reasonably approximates not less than one hour of classroom instruction and two hours of out-of-class work each week during a typical academic semester): Complete Incomplete Notes or additions if marked incomplete: 5d. The program includes policies and procedures for meeting the federal requirement that “regular and substantive” interaction takes place between students and instructors: Acceptable Not Acceptable Not Applicable; note that if this program is a correspondence program, the institution is also required to complete a separate Distance Delivery substantive change application. Evidence: 5e. The institution has made a reasonable determination of what is expected of enrolled students regarding the normal time to complete the CBE program (typically expressed as “satisfactory academic progress”) and uses that determination to report student progress: Acceptable Not Acceptable Evidence: 6. Essential Elements. The categories below relate to the evidence expected across subsections of Part 2 of the change application. 6a. Planning and Design of the proposed change, including preparation for and fit of the proposed change to the institution. Acceptable Not Acceptable Evidence: In meetings with individuals and groups from SCC, the team noted a deep commitment to online delivery. The college provides extensive training built around the Quality Matters framework for faculty members offering online courses and has a standing committee made up of faculty and administrators that oversees and steers high-level strategy for online delivery. While SCC offers a variety of training opportunities, these training sessions are not mandated and instead are left to the decision of individual faculty members and the offering department. And while there is an institutional committee that drives high-level strategy for online delivery, many decisions Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 3 Substantive Change Recommendation Form: Embedded Change Report about online delivery are pushed down to the departmental level, which may pose a challenge for wide-scale deployment of online programs. 6b. Capacity for the proposed change, including resources and commitment of the institution. Provide an evaluation of the sufficiency, qualifications and experience of the faculty teaching the discipline and at the level of the proposed change. Acceptable Not Acceptable Evidence: SCC is strongly committed to online delivery; during Spring 2017, the college offered nearly 140 online courses across 47 curricular disciplines. The team noted that the college has wrap-around services such as advising and tutoring in place, as well as access to other support services such as the library. There is a robust set of training opportunities for online delivery, and SCC has built standards around the Quality Matters model. The team noted a wide range of experience in online delivery, ranging from typical lecture-based classes through lab-type classes. The faculty members with whom the team engaged were enthusiastic and appeared to be dedicated to the pedagogy of online delivery. 6c. Services and Support for the proposed change Acceptable Not Acceptable Evidence: SCC provided evidence that they have addressed wrap-around services in support of online students, including: advisement and registration either online or on phone; testing via proctor agreements or ProctorU (an online service); and tutoring via the phone, or through synchronous and asynchronous delivery (although there were no online options for math tutoring). SCC has an online Strategic Plan for the next five years, which includes wrap-around services; the current plan has four goals, three of which have been completed. The team was told that the remaining goal is at 50% complete. Each online class is populated with a "Start Here" module, which includes links to online wraparound services. However, the team noted that the module is not mandated (see discussion elsewhere in this evaluation about the Residential Faculty Policy, which SCC uses as a reason for being unable to mandate online standards); this means that individual faculty members are free to delete the module. The team noted that, just as with face-to-face courses and faculty, online faculty members must meet a requirement to provide five academic-support hours to students. Faculty members demonstrated to the team that they provide such support through a variety of means, including via Skype, phone, face-to-face office hours, email, and online. The team noted that, in the opinion of the eLearning Steering Committee, online faculty tended to be more accessible to students than other faculty. 6d. Evaluation, Assessment and Improvement Processes for the proposed change Acceptable Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 4 Substantive Change Recommendation Form: Embedded Change Report Not Acceptable Evidence: The team saw evidence that SCC has a propensity to engage in evaluation and assessment; for instance, as a means of addressing the lower success rate exhibited by online students the eLearning Steering Committee created and deployed a "Zero Week" pilot designed to help students by providing support and training for individual classes in the week before the start of the semester. The consensus of the Committee was that the Zero Week pilot was improving student success (although the committee did not provide metrics to support this assertion), and was in the process of expanding the Zero Week pilot to more classes. SCC uses the Quality Matters framework for its training and deployment of online classes. Because of its culture, Quality Matters certification is not required; only 10 of 174 faculty members are certified. The eLearning Steering Committee and online faculty members both asserted however that all training for online delivery was built upon the Quality Matters framework and that because of that the framework was inculcated into the culture of online delivery. It was clear to the team that SCC was making a concerted effort to improve student success in online courses. When asked how SCC assesses the efficacy of approaches and improvements, some examples included: • As expertise develops, we get better and better at retaining students • Placed high-stakes assessment early in the course: Don’t complete it, you might get dropped • Ensuring students have clear expectations of effort entailed in online classes • Constant monitoring by the faculty • Great training via the Center for Teaching and Learning 6e. Quality and Integrity of the proposed change, including potential positive or negative effects Acceptable Not Acceptable Evidence: After discussions with individuals and groups, it was clear to the team that SCC has many of the structures necessary for successful online program deployment in place. Where the institution has an opportunity to improve is in consistency across its online offerings. Close examination of the ten courses reviewed during this evaluation showed some with clear links to a course syllabus and some without; wide variability in the types of learning activities and they way they were labeled; a different "look and feel" between the courses; the use of the Canvas LMS in most cases with two instructional areas (Math and Spanish) using completely different LMS; and inconsistencies in the availability of tutoring and support resources (for instance, there was strong support for writing with online tutoring via the Writing Center, but no such tutoring for mathematics). In discussion with the eLearning Steering Committee, it was clear that while that group drove highlevel decision making about online delivery, much of the actual implementation was driven down into individual departments and faculty members. It appeared to the team that this was the source of the "inevitable inconsistencies" the institution appeared consigned to in online delivery, rather than the external factors (including students and technology) to which the Committee and individual online faculty attributed them. In the team's opinion, addressing this fundamental question of standardization could be instrumental in building upon the strong foundation SCC has already constructed. Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 5 Substantive Change Recommendation Form: Embedded Change Report Part B: Recommendation and Rationale Recommendation: Approve Request Approve Modified Request Deny Request Rationale for the team’s recommendation to approve: If the recommendation is a modification of the institution’s request, make clear how the team modified the original request. Rationale for the team’s recommendation to deny: If recommending denial of the request, explain what was inadequate. SCC proposed approval of 48 different certificates and degrees for online delivery. Proposed initiation dates ranged from Fall 2017 through Fall 2020. During the review the team noted strong foundational components critical to online delivery and a clear passion for such delivery. At the same time, there were clear areas for improvement, including a limited amount of standardization across individual courses. The team noted that although SCC offered a wide range of training for online instruction, such training was not required; in discussion with a variety of individuals and groups the Residential Faculty Policy (RFP), SCC's contract with the faculty, was cited as the reason training could not be mandated. Further, authority for reviewing and overseeing online delivery was pushed down to the department level. This decentralization in review and oversight authority led to variability between the ten online courses reviewed by the team that, in the opinion of the team, made the courses more faculty- than student-centered. The lack of consistency may be a contributing factor in the online student success rate, which was 10% lower than either face-to-face or hybrid delivery modalities. One recurring source of concern for the team was what SCC characterized as the "inevitable inconsistencies" in online delivery. When pressed on this, both the eLearning Steering Committee and online faculty often seemed to attribute these to the student or technology rather than the course or institutional approach; discussions with the groups included comments such as: • "We don't know what browser the students are using" • "The students lack the technical skills for online courses" • "Students don't read the material we provide" • "Canvas is constantly updating, with changes occurring constantly" • "It's the human factor: All the tools can be there, but if the instructor isn’t using the tools properly there can be problems" • "No standard for how we’re delivering the content" • "Many faculty are evolving into online delivery, so they’re at different places in the development of their skill set" • "Many classes used a variety of media; when you add other technologies you complicate the process and thus engender inconsistency" An on-site review of content in ten (10) different courses across multiple disciplines showed a wide range of variability between the courses, including: Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 6 Substantive Change Recommendation Form: Embedded Change Report 1. Multiple learning management systems (eight used the Canvas LMS, while Mathematics and Spanish each used a different LMS using Canvas only as a gateway); 2. An inconsistent deployment of student help facilities in the courses (according to faculty members, courses are all initially populated with a "Start Here" module, but faculty may choose to delete it from the course); 3. Inconsistencies in course navigation (only seven of the ten courses had a direct link to a syllabus in the lefthand navigation bar; some navigation areas had as few as two links while some had as many as eight, and; there were inconsistencies in how links were named); 4. The college provided some outstanding examples of online support systems (for instance, the Writing Center provided synchronous, asynchronous, telephone, and walk-in support). However, there was no methodology for providing support for online mathematics students. It was unclear what support was available for other courses (although faculty members did conduct online office hours and otherwise appeared to be highly available to students). The inconsistencies noted above seem to have been driven by faculty or departmental choices, giving the appearance of being faculty- rather than student-centered design. In the team's observation, this is due to the decentralized approached used by the institution for managing online delivery. The team observed that such faculty-centered design may at least partially explain the wide gulf in Fall 2016 student success between faceto-face (75.2%) and online (65.4%) courses. When the team sought clarification on the reasons for the inconsistencies faculty and administrators cited the inability of the institution to control the content of online courses, ceding all course-related decisions to faculty members. The institution uses Quality Matters as a model for designing training, but training is not mandatory (again, citing faculty control as a reason that training cannot be mandated). To date, the institution reported that only 10 of 174 faculty members have formally completed QM training, and only 20% of courses are "QM-certified". Using a recognized model such as QM is effective, but only to the extent that it is consistently applied and deployed. It is absolutely true that faculty must be in control of academic content. However, that does not preclude the institution from establishing complete standards for online courses and programs. If the institution chooses to reapply for authorization to deliver online programs, it must: 1. Develop and apply standards for training faculty members delivering online courses; 2. Develop student-centered standardization of courses to make it easier for students to navigate, and; 3. Develop an institutional program or system to ensure consistency between online courses and programs across departments and disciplines. Although the team is recommending a denial of the requested stipulation change, the institution has much of which it can be proud: It has an engaged faculty, it has a culture of evidence (e.g. the deployment of a “Zero Week” pilot to improve student success), it is evident a great deal of thought has gone into the provision of wrap-around services that are provided, and the institution has the institutional resources (both technology and staff) to support online delivery. With the suggested changes, the team would encourage the institution to reapply in the future. Stipulations or limitations on future accreditation relationships: If recommending a change in the institution's level for review of future changes (locations, programs, delivery, etc.), state both the old and new level and provide a brief rationale for the recommended change. Check the Institutional Status and Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 7 Substantive Change Recommendation Form: Embedded Change Report Requirement (ISR) report for the current wording. Monitoring: In limited circumstances, the team may call for a follow-up interim monitoring report. (Note that some types of substantive changes have built-in follow-up reviews; for example, the Campus Evaluation Visit.) If the team concurs that monitoring is necessary, indicate the topic, timeline and expectations for that monitoring. Interactions with Institutional Constituencies and Materials Reviewed: List the titles or positions, but not names of individuals with whom the team interacted during the review and the principal documents, materials and web pages reviewed. Individuals: Vice President of Academic Affairs eLearning Steering Committee Online faculty (15) Web pages and materials reviewed: Letter from Associate Vice Chancellor for Academic Affairs at MCCCD on college authority for online SCC web site online delivery pages (various) Courses Reviewed • • • • • • • • • • ACC 211: Financial Accounting ASM 104: Bones, Stones and Human Evolution BPC 110: Computer Usage and Applications CIS 235: e-Commerce ENG 101: First-Year Composition GLG 101: Intro to Geology GLG 111: Geologic Disasters and the Environment HUM 209: Women and Films IFS 110: Information Skills in the Digital Age MAT 220: Calculus with Analytic Geometry I Audience: Peer Reviewers Form Published: 2015 © Higher Learning Commission Process: Substantive Change Contact: changerequests@hlcommission.org Page 8 Internal Procedure Institutional Status and Requirements Worksheet INSTITUTION and STATE: Maricopa Community Colleges-Scottsdale Community College, AZ TYPE OF REVIEW: Open Pathway Comprehensive Evaluation DESCRIPTION OF REVIEW: Visit to include embedded change request to initiate distance delivery. Comprehensive evaluation to include Federal Compliance reviewer. DATES OF REVIEW: 1/30/2017 - 1/31/2017 No Change in Institutional Status and Requirements Accreditation Status Nature of Institution Control: Public Recommended Change: nc Degrees Awarded: Associates Recommended Change: nc Reaffirmation of Accreditation: Year of Last Reaffirmation of Accreditation: 2007 - 2008 Year of Next Reaffirmation of Accreditation: 2016 - 2017 Recommended Change: 2026-27 Accreditation Stipulations General: Prior Commission approval is required for substantive change as stated in Commission policy. Recommended Change: nc Additional Location: Prior HLC approval required. Recommended Change: nc Internal Procedure Institutional Status and Requirements Worksheet Distance and Correspondence Courses and Programs: Approval for distance education is limited to courses. The institution has not been approved for correspondence education. Recommended Change: nc Accreditation Events Accreditation Pathway Open Pathway Recommended Change: nc Upcoming Events Monitoring Upcoming Events None Recommended Change: Interim Report due June 30, 2019 on assessment of student learning. Institutional Data Extended Operations Branch Campuses None Recommended Change: nc Additional Locations None Recommended Change: nc Correspondence Education None Recommended Change: nc Internal Procedure Institutional Status and Requirements Worksheet Contractual Arrangements None Recommended Change: nc Consortial Arrangements None Recommended Change: nc