TMUS REPORT OF THE WSCUC TEAM For Reaffirmation of Accreditation To The Master’s University and Seminary March 21-23, 2018 Team Roster Team Chair: John Jackson, ​President, William Jessup University Assistant Chair: Anita Henck, ​Dean, School of Education, Azusa Pacific University Kevin Grant, ​Director of Student Affairs Assessment & Research, California Polytechnic University, San Luis Obispo Kimberly Renna, ​Vice President for Finance & Business Operations, Franciscan School of Theology Mark Riddle, ​Director of Educational Effectiveness, West Coast University Tamela Hawley, ​Vice President and Liaison, WSCUC ___________________________________________ The team evaluated the institution under the 2013 Standards of Accreditation and prepared this report containing its collective evaluation for consideration and action by the institution and by the WASC Senior College and University Commission (WSCUC). The formal action concerning the institution’s status is taken by the Commission and is described in a letter from the Commission to the institution. This report and the Commission letter are made available to the public by publication on the WSCUC website. ___________________________________________ TABLE OF CONTENTS SECTION I – OVERVIEW AND CONTEXT A. Description of the Institution and its Accreditation History, as Relevant B. Description of Team’s Review Process C. Institution’s Reaccreditation Report and Update: Quality and Rigor of the Report and Supporting Evidence SECTION II – EVALUATION OF INSTITUTIONAL ESSAYS A. Component 1: Response to Previous Commission Actions B. Component 2: Compliance With the Standards and Federal Requirements; Inventory of Educational Effectiveness Indicators C. Component 3: Degree Programs: Meaning, Quality and Integrity of Degrees D. Component 4: Educational Quality: Student Learning, Core Competencies, and Standards of Performance at Graduation E. Component 5: Student Success: Student Learning, Retention, and Graduation F. Component 6: Quality Assurance and Improvement: Program Review, Assessment, Use of Data and Evidence G. Component 7: Sustainability: Financial Viability, Preparing for the Changing Higher Education Environment H. Component 9: Reflection and Plans for Improvement SECTION III – FINDINGS, COMMENDATIONS, AND RECOMMENDATIONS FROM THE TEAM REVIEW APPENDICES A. Federal Compliance Forms 1. Credit Hour Review 2. Marketing and Recruitment Review 3. Student Complaints Review 4. Transfer Policy Review B. Distance Education 2 WSCUC Team Report for TMUS: March/2018 SECTION I – OVERVIEW AND CONTEXT A. Description of Institution and Relevant Accreditation History Background Information: ​The history of The Master’s University and Seminary (TMUS) dates to its original founding as a seminary in 1927. After several iterations of institutional scope and focus, the school moved to its current location in Placerita Canyon, north of Los Angeles, in 1961. Initial regional accreditation was received by the institution (then Los Angeles Baptist College) in 1975. In 1985, John F. MacArthur was appointed as College President and the name of the school was changed to The Master’s College. The seminary was rebirthed in 1986. Enrollment and financial growth occurred from 1985 through approximately 2012. In more recent years, traditional undergraduate enrollment has been stagnant or declining. Since 2007, the addition of graduate and international programs have provided the primary areas of growth for TMUS. While online courses were first offered in 2007, TMUS sought WSCUC permission to offer fully online programs in 2015. In 2016, the institutions’ names were formally changed by the board to The Master’s University and Seminary. TMUS has a clearly stated mission: “to empower students for a life of enduring commitment to Christ, biblical fidelity, moral integrity, intellectual growth, and lasting contribution to the Kingdom of God worldwide”. The university currently offers traditional undergraduate degree programs in 13 fields of study encompassing 57 emphases and 4 online degree programs. At the graduate level, the university offers master’s degrees in Business Administration (MBA), Biblical Counseling (MABC), Biblical Studies (MABS), and Education (MEd), as well as fifth-year California Single Subject and Multiple Subject credential programs. The seminary offers five degree programs: the Bachelor of Theology (BTh), Master of Divinity (MDiv), Master of Theology (ThM), Doctor of Philosophy (PhD), and Doctor of Ministry (DMin). The MDiv is offered at five distance locations (Springfield, VA; Kingsburg, CA; Dallas, TX; Spokane, WA; and Bozeman, MT) as well as on the Los Angeles campus. Beginning in 2016, TMS established the Center for Israeli Studies, a program charged with facilitating the training of 3 WSCUC Team Report for TMUS: March/2018 nationals on campus at TMS and working with churches in Israel to aid the development of expository ministries. The seminary began offering ThM courses in a modular format in summer 2017 (Att. 1.05). The university and seminary are in conversation to begin jointly offering a five-year Bachelor of Arts (BA) through MDiv program in the coming year (Att. 1.06). TMS also began offering a Spanish language Master of Arts in Biblical Ministry in the 2017-18 academic year. In Fall 2016, the university had 911 students in traditional undergraduate programs, 149 students in graduate programs, and 368 students in online courses and programs. The seminary enrollment included 22 Diploma in Theology (DipTh) students, 42 BTh students, 293 MDiv students, 28 ThM students, 4 PhD students, and 72 DMin students for a total of 461 students. The total Fall 2016 TMUS unduplicated student enrollment was 1,889 students. Institution’s Relevant Recent Accreditation History​: TMUS has experienced several accreditation-related activities since the Educational Effectiveness Review (EER) visit in Spring 2009. These actions included a Special Visit in February 2012, an Interim Report in March 2013, an Interim Report in February 2014, and a Progress Letter submitted in March 2015. The February 2012 Special Visit team was tasked with evaluating TMS’s progress relating to: ● faculty scholarship and doctoral program culture ● assessment and program review processes ● strategic planning ● engagement with women’s perspectives within the seminary curricula. The Special Visit Team’s report affirmed the progress made to date. The team was encouraged with the breadth of institutional participation in addressing these issues. They commented particularly on the maturity of the evidence-based assessment program. In March 2013, the university submitted a required Interim Report to WSCUC that addressed: ● scholarship of the master’s level faculty 4 WSCUC Team Report for TMUS: March/2018 ● assessment and program review activities ● strategic planning, including efforts to improve minority and female student retention. The WSCUC Review Panel Action Letter affirmed the university’s progress in each of these areas after the teleconference call with the review panel. As a follow-up to the 2012 Special Visit, TMS submitted a February 2014 Interim Report to WSCUC that addressed continued progress in regard to: ● encouraging faculty scholarship ● academic engagement with women’s experiences and spiritual perspectives ● systematic approaches to integration of diversity across the institution. In the Onsite Review, with regard to faculty scholarship, the WSCUC review panel encouraged TMS to “further spark faculty to engage with colleagues in the field outside of the institution” (Institutional Report, page 7). While the panel noted TMS's efforts to examine the presence of women’s experiences and perspectives within the curriculum, they encouraged the seminary to “use its findings in ways that can demonstrate to the next review team that the assessment process in this area has resulted in positive change, perhaps as broader dispersion of women’s perspectives through the curriculum.” In March 2015, TMU submitted a Progress Letter which outlined continued progress relating to: ● the analysis of retention data ● progress on promoting scholarly activity through founding of a university imprint, ● use of assessment data to develop the institution ’s programs. B. Description of Accreditation Visit Team’s Review Process The 2018 Accreditation Visit Team (AVT) thoroughly reviewed materials provided by the institution, third-party comments directed to WSCUC in years since the last accreditation report, the institutional report, all self-study supporting documents, additional documents submitted by the institution in response to the AVT’s Offsite Review Lines of Inquiry, materials provided onsite, and messages 5 WSCUC Team Report for TMUS: March/2018 received through the confidential email account. Team members accepted various areas of responsibilities as both primary and secondary readers of each area of review. They prepared worksheets reviewing each of the components, standards, federal required policies, the Inventory of Institutional Effectiveness Indicators, and any special materials related to distance education. As preparation for the Offsite Review (OSR), the AVT conducted multiple conference calls to review the completed worksheets. At the OSR, the AVT developed Lines of Inquiry that identified key issues for further exploration and helped to develop questions to be asked during the accreditation interviews. Additional materials were requested from TMUS prior to the onsite review and responses were uploaded to the team folder. Finally, as is routine, a confidential email account was established by WSCUC; the TMUS community was notified by the Academic Liaison Officer of this opportunity for feedback. The AVT received--before and during the visit--significant confidential email input from the community which became part of the data that were reviewed. During the accreditation visit (AV), the AVT conducted interviews with members of the board, a broad array of administrators (president, provost/ALO, vice presidents, deans, and other administrative directors), faculty members, students, staff members, and alumni. For the duration of the AV, the team collected information, examined the interview outcomes, and triangulated data to refine questions in subsequent interviews. Throughout the process, the AVT encouraged open communication and diligently reviewed the information provided by the AV interviews. As the AV drew to a close, team members collectively constructed the final set of commendations and recommendations based upon careful and comprehensive analysis of the evidence gleaned from institutional materials and AV interviews. C. Institution’s Reaccreditation Report and Update: Quality and Rigor of the Report and Supporting Evidence As part of a WSCUC institutional review process, a review team is tasked with reviewing institutional responsiveness and compliance with WSCUC standards, federal requirements, and an inventory of educational effectiveness indicators. By seeking regional accreditation, the institution is 6 WSCUC Team Report for TMUS: March/2018 commiting to align with accreditation standards and commonly accepted practices across U.S. higher education. This elective process of seeking external accreditation provides common ground across institutions ensure that “quality, integrity, accountability, and transparency” will be represented throughout a student-centered focus on learning (2013 WSCUC Handbook of Accreditation, p. 10). TMUS described their 2018 Institutional Review Report Process as follows: [P]reparation for the report began in Spring 2015 with the formation of a steering committee comprised of representatives from the Board of Directors, Administration, Faculty, and Staff. The steering committee developed an Institutional Research (IR) strategy in the Fall of 2015 to support the development of the report. Seven Component committees were established and given a clear description of responsibilities along with a timeline for completion of the writing portion of the report. From October 2015 to January 2017, component committees met at various times and developed templates to complete the assigned work. In February 2016, Dr. Oberg (ed: WASCUC representative) spent a day on TMU’s campus meeting with each of the component committees. During the meetings, committee chairs were responsible to present an overview of the planning they had completed to date, the intended approach and focus for their component, and to obtain feedback from Dr. Oberg in regard to their planned strategy. Drafts of the first seven IR Report components were written through the 2016-17 academic year, and then submitted to the Steering Committee by May 2017. From that time until submission in July 2017, the primary focus was to integrate the various sections of the report, eliminate redundancies, identify/locate missing information, and bring a single voice to the document. … During spring 2017, component committees developed and submitted all supporting report attachments. The AVT found that the Institutional Review document provided a clear identification of key challenges and opportunities the institution is facing and was hopeful that the six strategic issues identified at the end of the report would provide the framing narrative for evidence of strategic planning that aligned institutional realities with tactical decision making. At this juncture, following the onsite review and interviews and input from the TMUS community, the AVT remains unclear as to whether those six issues are indeed the framework of clarity that we believed them to be upon receipt and review of the written report. We strongly exhort the board, administration, faculty, and staff to address these six matters, and other items we will address in this report, in a diligent and timely fashion. Further, we believe the board must address a number of very specific items related to Accreditation Standards and Effectiveness indicators. 7 WSCUC Team Report for TMUS: March/2018 The institutional report as submitted provided a snapshot of the state of the institution, as well as the history of its mission, vision, and academic commitments. While it is likely that the report was current at the time of writing (Spring 2017), there were significant changes between that submission and the Off Site Review (OSR) and again between the OSR and the accreditation visit. These changes included a transition in the Accreditation Liaison Officer (ALO), substantial institutional reorganization, significant change in cabinet-level leadership at both the university and seminary, and the tuition reset which was announced in Fall 2017. Consequently, the requests for additional data were substantial as the AVT wanted to ensure, as best as possible, a real-time review of TMUS’s status and practices at the time of the March 2018 Onsite Visit. Often written reports didn’t align with current practices and documents hadn’t been updated. Thus, prior to and during the visit, the AVT sought to verify or extend the understanding of the report submitted by the institution, including use of data from onsite interviews and the updates provided by the ALO in response to team data requests as they became aware of the breadth of changes occurring. Following the OSR, additional documents were requested and studied to foster greater levels of understanding of the current status of TMUS personnel and operations. As the AVT became aware of changes, it sought to compare current operations to the most recent institutional report, but also attempted to map it to prior accreditation review comments and findings, with a goal to ensure responsiveness to ongoing reviews of TMUS accreditation standing. This was complicated by the departure of several long-term leaders who were familiar with the IIR documents, as well as the relatively short tenure of many key institutional leaders and their lack of familiarity of required accreditation components, standards, and federal policies. Therefore, it often appeared to be a limited continuity of process and personnel -- from past accreditation visits to the submitted institutional report to the AVT’s presence on campus. 8 WSCUC Team Report for TMUS: March/2018 SECTION II – EVALUATION OF INSTITUTIONAL ESSAYS Component 1: Response to previous Commission actions The AVT believes that TMUS has responded with some measure of substance to previous Commission actions in preparation for the 2017 Report and the 2018 Onsite Visit and is therefore in compliance with the foundational elements of this Component. We do, however, express concern later in this document that we believe that TMUS appeared to operate with little awareness of and attention to WSCUC standards and previous actions until the time of preparation for the current visit. As indicated previously, the 2012 Special Visit, the 2013 Interim Report, the 2014 Interim Report, and a Progress Report in 2015, expressed consistent concerns for TMUS: ● faculty scholarship and doctoral program culture ● assessment and program review processes ● strategic planning ● engagement with women’s perspectives within the seminary curricula. The AVT was pleased to see evidence of tangible efforts in each dimension referenced in prior reports and visits. It is clear that when WSCUC actions are taken, there is a specific and credible response from the institution. We honor and appreciate the progress that has been made to date. However, we express concern that with the volume of personnel transitions that have taken place in the last 12-24 months, that the institution appears to have little continuity or institutional knowledge regarding accreditation standards and requirements. Component 2: Compliance with the standards and federal requirements; Inventory of educational effectiveness indicators The WSCUC standards, federal requirements, and the inventory of educational effectiveness indicators provide common ground across institutions ensure that “quality, integrity, accountability, and 9 WSCUC Team Report for TMUS: March/2018 transparency” will be represented throughout a student-centered focus on learning (2013 WSCUC Handbook of Accreditation, p. 10). Standard 1: Defining institutional purposes and ensuring educational objectiv​es The AVT’s finding, which is subject to Commission review, is that the institution has not demonstrated sufficient evidence of compliance with Standard 1. Specific findings are outlined below. Institutional purposes: Formally approved statements of purpose that define values and character: ​The Master’s University and Seminary clearly defines its purposes and has designed educational objectives that are aligned with its mission to “empower students for a life of enduring commitment to Christ, biblical fidelity, moral integrity, intellectual growth, and lasting contribution to the Kingdom of God worldwide.” The fidelity of the religious mission of the university was pervasive across all sectors (CFR 1.1). The institution’s mission is clearly articulated and practiced, and is well known and practiced across all stakeholder sectors. Clear educational objectives: ​The AVT witnessed evidence of the university meeting its educational objectives and stated purposes throughout review and site visit processes. Institutional Review (IR) staff explained how syllabi are reviewed at the program level and by the university’s Curriculum Committee to assure that curricula and program learning outcomes are aligned with the university’s mission and its institutional learning outcomes (CFRs 1.2 and 2.2). Student achievement data is assembled by IR staff, disseminated internally, and discussed by faculty (CFRs 1.2 and 2.10). Faculty support the evaluation of learning outcomes, but the AVT found little evidence of the use of assessment data by administration or support staff (other than IR, which is only involved in gathering and reporting, not acting on the findings). In addition, while measures of retention and graduation are available on the university’s website, these data are difficult to find for those unfamiliar with the site (CFRs 1.2 and 2.10). 10 WSCUC Team Report for TMUS: March/2018 Integrity and Transparency: Academic freedom, policies and practices: ​Both the TMU and TMS Faculty Handbooks have statements affirming commitments to academic freedom, within the context of institutional values. Concerns emerged for the AVT about the lack of boundaries between dual roles of some key institutional leaders. Multiple reports were provided to the AVT that several persons were believed to have been dismissed from faculty roles for reasons of “lack of fit” with theological values that were not explicitly outside stated theological standards. While these could not be thoroughly investigated as part of this review, the volume of comments and specific details provided leave a concerning question of whether academic freedom exists (CFR 1.3) and whether there is institutional autonomy from external entities (CFR 1.5). Diversity, policies, programs, and practices: ​In prior WSCUC reports, TMUS’s lack of diversity in hiring and curriculum was noted. ​In 2018, records and interviews demonstrated the presence of female faculty in university faculty ranks and in junior ranks of administration. The seminary reported the presence of female scholarship being included in supplemental reading lists in the syllabi of required courses which is a beginning response but is not sufficiently robust to be considered complete. Commitments to diversity in academic content and hiring practices do not need to be seen as contrary to institutional values regarding the role of women in ministerial leadership but still require additional, timely, and specific attention. No specific plans with measurable outcomes and specific timetables have been developed despite prior WSCUC findings regarding concerns with fidelity to accreditor values of diversity (CFRs 1.4 and 3.1). Education as primary purpose; autonomy from external entities: ​The AVT recognizes the historic connections between the separate yet interrelated university, seminary, and Grace Community Church (GCC), and the ways each entity has historically benefitted from these ties. Yet, despite specific lines of inquiry in the On Site Review, those asked could identify no operational guidelines in place when 11 WSCUC Team Report for TMUS: March/2018 individuals with dual allegiance are involved in a matter that involves the realm of an accredited institution of higher education (CFR 1.5). Notably, significant proportions of TMUS faculty and staff have dual relationships with institutional leaders, as many worship at GCC where President MacArthur has been the senior pastor since 1969, in addition to working at the university or seminary where he is President. Further, a number of cabinet-level leaders of TMUS are also on staff at GCC. Consequently, employees’ higher education supervisors can also be their pastoral staff member. Some interviewees noted that lives were so intertwined that, should they be dismissed or leave TMU over a substantial difference of opinion, they lost their entire support community -- as a TMUS alumnus and employee as well as a congregant in Grace. They reported this was the reason for past silence on matters of concern. Further, in conversation with members of the board, the AVT developed concern that the board’s statements of extraordinary confidence in President MacArthur have resulted in less oversight of the administration than is expected, especially in a period of significant flux as has recently occurred. Board members stated unequivocally that they could say “no” to the president, but could not identify any occasions on which they actually had done so and self-identified as President MacArthur’s closest friends in the world (CFR 1.5). And, with at least one board leader also serving as the associate dean and adjunct faculty of a regional site and other board members serve as adjunct faculty, the presence of competing roles became evident. Truthful representation to students/public; fair and equitable policies; timely completion. ​The AVT found that, in many regards, the institution’s interactions with students demonstrates integrity and transparency (CFRs 1.6 and 3.6). One significant exception emerged relative to the late Fall 2017 announcement that tuition rates would be re-calibrated to more accurately reflect the cost of attendance, including the tuition discount rate. Thus, tuition rates were substantially reduced and the discount rate simultaneously reduced to a greater degree in the process, similar to other institutions moving to some version of this practice. Yet, TMU students noted that this change was announced in chapel as a tuition 12 WSCUC Team Report for TMUS: March/2018 cut, and they only later learned that approximately one-third of the student body would experience a reduction in tuition, while the remaining two-thirds would either see no change or find that their net tuition costs were higher than before. This lack of transparency resulted in significant parent and student concerns (CFRs 1.6 and 1.7). During the On Site Review, the AVT heard of a days-old plan to supplement the new tuition process for continuing students who would be adversely affected by the new tuition practice with supplemental scholarships. There was no ability for the AVT to either review the fiscal analysis that led to this decision, the fiscal impact on the operating budget, and/or the real-time impact on students. The AVT noted with concern the auditor findings that “multiple students who are family members of donors or related parties at TMUS received institutional aid … [which] appeared to be above what is typically offered to all students and is an inconsistency in awarding … according to the currently policy.” This is troubling, both for students without such connections, but also as a matter of integrity with donor, tax-exempt donations, and financial aid practices. The lack of timely responsiveness to auditor reports is concerning (CFRs 1.6, 1.7, 3.6, and 3.7). Operational integrity, sound business practices, timely and fair response to complaints, and evaluation of institutional performance. ​Student complaints are logged, but not in a way that supports analysis or decision-making. While it may be true that “TMUS is reevaluating and updating its methods of tracking student complaints over time to be able to better support the student experience” (IR, Attachment 2.07, p. 5), no formal practices have been implemented to date that address this important issue. Consequently, there are concerns about fidelity to all aspects of “operational integrity; sound business practices; timely and fair responses to complaints; evaluation of institutional performance” (CFR 1.7). In campus interviews and confidential emails, the AVT received clear and concerning reports of recent personnel actions (hirings, terminations, promotions, demotions, and lack of regular evaluations) 13 WSCUC Team Report for TMUS: March/2018 that have the appearance of seemingly arbitrary actions. While it is possible that the lack of clarity in some of these cases may be due to requirements related to confidentiality, the sheer number of these cases and the depth of issues provides cause for concern. This, coupled with the finding of a pervasive ​culture and climate of fear, intimidation, bullying, and uncertainty which appears to be created by the newly appointed management, provides significant areas of concern regarding sound and ethical business practices; these concerns were exacerbated for the AVT by significant community member reports of the fear of speaking up, despite protections of a whistleblower policy (CFRs 1.1, 1.7, 3.2, and 3.6). In summary, ​regarding sound business practices, TMUS is regularly audited financially, but important recent recommendations have not been addressed. In 2017, the auditor highlighted as a “significant deficiency” that there were several instances of management overriding or circumventing controls that were in place to process payments or contracts outside established policies. Further, the report noted that there was the appearance of conflicts of interest with the President’s son-in-law supervising a contract from which he benefits, as well as institutional aid being awarded to related parties exceeding typical award amounts, but there was no evidence at the time of the visit that these concerns had been addressed in more than a cursory manner (CFRs 1.1, 1.6, 1.7, 3.2, 3.4, 3.6, 3.8, and 3.9). Honest, open communication with WSCUC including notification of material matters; implementation of WSCUC policies. ​During the site visit, the AVT became aware of five distance sites where programs are offered (typically in a hybrid model) that have not been appropriately reported to the Commission. According to the WSCUC Substantive Change Manual, a reason for reporting to the Commission would be if an institution experienced “Rapid growth in campuses or off-campus locations for educational program delivery.” However, WSCUC was not aware of multiple of these hybrid programs that had been started in multiple states. Of additional concern is that lack of timely reports to WSCUC of significant leadership transitions at the Cabinet level. Even during the review process, an 14 WSCUC Team Report for TMUS: March/2018 Academic Liaison Officer transition occurred and was only learned through email “bounce backs” to the AVT several months after it occurred (CFR 1.8). Standard 2: Achieving Educational Objectives through Core Functions T​he ​AVT​’s finding, which is subject to the Commission’s review, is that the institution has not demonstrated sufficient evidence of compliance with several components of Standard 2. Teaching and Learning: Review of the academic programs, their content, standards, and degree levels found traditional standards were in place at the university at a basic level. Student responses were uneven relative to program delivery (regarding academic advising and faculty turn-around time on student inquiries), with some students enjoying their experiences while others noted substantial overlap between requirements of undergraduate and graduate programs in the same disciplines (CFRs 2.1, 2.2a, 2.2b, and 2.3). This, coupled with faculty turnover, led the AVT to question the relative stability of programs between periods of accreditation review (CFRs 2.1 and 3.1). Also, of note is the number of faculty who are alumni of TMUS. The lack of diversity of experience and thought, in conjunction with the limited engagement with professional development, professional organizations, and academic presentations in non-TMUS sectors is contrary to the expectation that faculty will have robust engagement with the profession and enhance their their ability to keep curriculum and practices current and comparable to other institutions students might consider (CFR 2.1, 2.8, and 3.3). Of greatest concern within this standard are findings of non-normative processes for awarding some degrees, including doctorates. In the review process, team members triangulated data about reports of some being doctorates being awarded on request of senior leadership, bypassing traditional autonomous faculty engagement. In one case, a student was awarded a PhD despite completing no coursework at the institution and bringing a dissertation that had not been approved by his prior European 15 WSCUC Team Report for TMUS: March/2018 institution, and the reports are that the degree was approved prior to the dissertation being translated into English. In another case, a student’s faculty-graded final exam earned a failing grade which resulted in a student appeal. Subsequently, another grader (other than the faculty member of record) submitted a passing grade for the student which was officially recorded. These are outside academic norms, raising significant questions about autonomous faculty roles being recognized (CFRs 1.1, 1.3, 1.5, 1.6, 1.7, 2.1, 2.2, 2.2b, 2.3, 2.4, and 2.6). Scholarship and Creative Activity: With previous WSCUC reviews expressing concerns regarding faculty scholarship, the importance of this standard was clearly noted. Yet, by 2018, faculty scholarship has shown limited growth, with most publications coming from seminary faculty whose work is accepted in the institution’s theology journal. While this is a good start, the lack of publication in independent, blind peer-reviewed venues (a higher education standard) and being published primarily by one’s institutional colleagues remains an area of concern (CFRs 2.8 and 2.9). Support for faculty professional development is cited in the Faculty Handbook and referenced in the Self-Study, yet there is limited evidence of regular support for faculty engagement in professional development (CFRs 2.8, 2.9, and 3.3) with some reports of “freezes” on faculty travel and book funds due to fiscal challenges at the institution. Further, widespread reports of decreased financial support to the library adds to the concerns for successful faculty and student scholarship resources (CFRs 3.4 and 3.5). Student Learning and Success: The ​AVT​ was able to meet with the executive leadership team of the Student Government who, by demographics, are more "traditional" college students. Many of these students affirmed their affinity for TMU and were in support of the values-based curriculum. They further expounded on their positive student life experiences. Specifically, the new marketing direction was highlighted as a positive movement in making the campus feel more "collegiate and professional". 16 WSCUC Team Report for TMUS: March/2018 Notably, the open session for students to engage with the AVT resulted in no student participants. The room in which the visit team members were to meet found students arriving early for another event and three of those students stepped up and offered to speak with the AVT. None were “traditional students” in terms of living in the residence halls, but they expressed strong support for the institution’s values and the quality of education they were receiving. ​In particular, one student shared her desire to have more of an opportunity and place to share and celebrate global cultures within the student experience. Meanwhile, in Fall 2017, there were significant changes to the traditional TMU Student Life operations and systems of support provided to students (CFRs 2.11 and 2.13), with lack of full clarity of the new organizational structure at the time of the site visit. A Vice President for Student Life/Dean of Students had previously served on the Academic Cabinet, but prior to the site visit, that position was abolished and the span of care divided into two sectors. Student Services (Student Activities, Campus Safety, Fall Welcome Week, and Health Center) is now a direct report to the Chief Operating Officer (in addition to his responsibilities for Human Resources, Plant Operations, Information Technology, TMU Enrollment, Church Relations, Creative & Marketing, and Athletics). Residential Life/Outreach (Residence Facilities, Summer Missions, International Students, and Multicultural Student Affairs (vacant)) now reports to the former VP of Student Life who is now a Dean of Students. In interviews with the COO and his direct reports, as well as some of the personnel remaining in the former Office of Student Life, it became clear that the full articulation of TMU’s student services philosophy had not emerged, nor was the COO familiar with mandated higher education policies and procedures -- FERPA, Clery Act, and VAWA (Violence Against Women Act), nor required reporting of related consumer data. Some confidential reports provided to the ​AVT​ indicated that the 22-year veteran leader of Student Life (who was now overseeing Residence Life/Outreach) had been notified he would leave the university soon after the On Site Visit was complete. If this personnel transition does indeed occur, the lack of 17 WSCUC Team Report for TMUS: March/2018 experienced senior personnel in Student Life (which is particularly crucial for a residential campus) will be notable. ​In the context of contemporary life, the absence of trained personnel to appropriately respond to sexual assault claims in a judicious and supportive environment is troubling ​(CFRs 1.6, 2.10, 2.11, 2.13, 3.1, 3.5, and 3.6).​ At the seminary, the Dean of Students serves as the designated Title IX administrator. The seminary website lacks the mandated Title IX and Clery reports. The Graduate Student Handbook contains a link to the university's HEOA (Higher Education Opportunity Act) page, which has links to Title IX reporting, but one must search the Education Department's website to obtain current reports. Standard 3: Developing and Applying Resources and Organizational Structures to Ensure Quality and Sustainability The ​AVT​’s finding, which is subject to the Commission’s review, is that the institution has not demonstrated sufficient evidence of compliance with several components of Standard 3. Faculty and Staff: The institution’s faculty and staff are deeply invested in the institution and its missional values. The institutional report indicates that students rate the faculty high on quality. Through the onsite process, the faculty demonstrated their devoted commitment to the institution. These factors evidence a match between institutional mission and employees (CFR 3.1) Meanwhile, the institutional report notes that faculty workloads often do not allow sufficient time for professional growth, research and scholarship and on-site reports were that resources for books and travel had been “frozen” in response to fiscal issues. Without time and resources for faculty growth, the institution’s ability to evolve with the changing educational landscape, advancements in technology, educational objectives, academic policies, integrity of its programs and quality of delivered education may falter in the future. ​Moreover, reports of insufficient time for professional development, research, and scholarship suggests the need for additional faculty in order to to meet current needs ​ (CFR 3.2). 18 WSCUC Team Report for TMUS: March/2018 During the ​AVT​’s onsite review, the accounting staff presented budget reports that demonstrated some financial resources allotted for higher educational advancement of employees through an application process. Individual departmental budgets allow for the department to allocate funds for the professional development of their departmental staff although limited evidence was provided of this being implemented. (CRF 3.3) The institution’s report noted that management’s leadership was highly rated by faculty and staff. This assertion appeared to be called into significant question during the On Site Visit. In the months leading up to the team visit, the institution sustained significantly high turnover in the executive management level as well as many staff positions. The ​AVT ​was informed and observed there is a culture and climate of fear, intimidation, bullying, and uncertainty which was ascribed to the newly appointed management team, (CFR 3.1). Further, reports from the confidential emails raised notable concerns, including the potential for lack of fidelity to a number of key policies, including the Whistleblower’s Act (included in the Employee Handbook). The hiring practices for several key management positions neglected responsible observance of the institution’s policies or best practice guidelines. Positions were simply appointed rather than advertised, without any form of search process conducted. Job descriptions and qualification standards for the positions were not established prior to the appointment of several key management positions. A number of those appointed to cabinet-level roles had no prior experience in the area represented by their new assignments. And, in interviews, it became evident that few of the senior leaders were actively engaged in the professional higher education associations relative to their particular areas of responsibility (CFR 3.2). In particular, hiring practices appear contrary to the expectation to diversify the ranks of faculty and staff populations. Specifically, during the accreditation review interviews, information emerged that employment opportunities are typically posted only within The Master’s University website, without 19 WSCUC Team Report for TMUS: March/2018 listing jobs with external professional entities. Through the onsite review, it became apparent that a significantly large portion of the faculty and staff are graduates of the university and/or the seminary (CFR 3.2). And, the “rigorous selection” process cited by recent hires as seminary faculty led to follow-up questions by AVT members; this selection process was found to be a 1-hour interview. While there is full support for hiring for “institutional and missional match,” unless future hiring practices intentionally focus beyond TMUS students and alumni and Grace Community Church parishioners, the expected diversification will likely not occur. The lack of diversity has been a repeated finding in WSCUC reviews and requires attention. The institutional report indicated concern that the ratio of pay to cost of living for employees has not reached acceptable levels. TMUS data evidences lower median salaries when compared to peer groups in Southern California faith-based schools for faculty and staff. This affects recruitment and could inhibit hiring to insure institutional purpose and educational objectives (CFR 3.1). During the onsite review, many faculty and staff reported concerns about their current salary and pay rates and the lack of regular raises. Notably, they commended the institution on the benefits that are offered to employees, particularly full institutional payment for health insurance. While there are established procedures for personnel reviews in the Employee Handbook, Onsite Review interviews received multiple reports that these procedures are not followed consistently throughout the organization nor on a regular basis which correlated with the institutional report of a low rating by faculty and staff on the process of regular performance reviews (CFR 3.2). Keeping institutional, department and individual goals at the forefront enhances the employees’ sense of team. Fiscal, Physical, and Information Resources: ​The institution has received an unqualified (“clean”) audit opinion over the last 5 fiscal years ending June 30, 2017 (CFR 3.4). However, the auditors have also presented management with a letter of recommendations for the 2016 and 2017 fiscal years. These reports noted a significant deficiency in internal controls with several instances where 20 WSCUC Team Report for TMUS: March/2018 established controls and policies were circumvented by the management to facilitate the processing of payments or contracts. The auditor’s management letter also noted internal control issues with several related party transactions. Multiple students who are family members of donors or related parties to the institution were awarded financial aid in excess of the general student population. The institution awarded significant marketing contracts to a related party without going through the proper approval processes (CFR 3.4). During the accreditation team’s onsite review, the newly appointed CFO demonstrated his knowledge, concern and immediate action being taken for the resolution of these key accounting issues identified in the 2017 audit management letter. This is positive, albeit in the context that the CFO holds what are apparently two or more substantive responsibilities -- as CFO at TMU, as a seminary faculty member, and as a significant member of the pastoral staff at Grace Community Church (CFR 3.8). In mid-Fall 2017, the institution initiated a tuition reset for University students that will be effective Fall 2018. The board approved the reset of tuition fees, significantly reducing the tuition costs to attend the undergraduate program at the university, but with a corresponding and deeper cut to the institutional scholarships provided by TMUS. After this plan was announced to the public, the institution determined a negative impact to one-third of the returning undergraduate students. The board stepped up to fund the difference for affected students, approving it the week before the Onsite Visit. During the onsite review, faculty, staff and students commented on the poor communication and implementation of the tuition reset project. The Finance Department demonstrated its understanding of the financial impact of the tuition reset and appeared to be addressing the concern for the initial negative impact on the institution’s operations due to the tuition reset. This negative impact on the institution’s budget will continue until enrollment increases to the desired goal. Efforts were in process during the visit to find areas to reduce costs to account for this reduction in revenue (CFR 3.4). 21 WSCUC Team Report for TMUS: March/2018 The IR noted that the institution has funded major facilities improvements and expansion over the last six years. Renovations of physical classroom space as well as technology enhancements are planned for the near future. During the onsite review, the ​AVT​ noted the beautiful university and seminary campuses, also learning that technology resources and access are available to support the academic offerings for the students and research/scholarship for its faculty (CFR 3.5). Resource planning appears to generally be integrated with all other institutional planning. Yet, faculty reported concerns on cuts to library services -- both in process and the lack of communication. Deep cuts to library resources can have significant impact on program quality, particularly for distance students whose sole access to resource are online subscriptions and services (CFRs 2.8 and 3.5). This concern requires further and timely attention. The institution has seen a decrease in enrollment of its undergraduate programs in recent years but has seen an increase in its graduate, seminary, and online programs. The institutional budget for 2018 shows a $1.2 M negative change in operational net assets and forecasts a negative operational net asset change for the next 2 years. This enrollment-based operating loss will be offset by annual donation contributions of $5M to cover operating expenditures. Historic annual unrestricted contributions have accounted for $2.6M to $4.5M per annual budget over the last 5 years, with 2017 unrestricted contributions at $3.7M. The institution is clearly and routinely dependent on donor-based contributions for its near-term and future operations. Thus, the IR indicates gifts and private grants are an area of strength for the institution. This was confirmed and demonstrated during the site visit by the professionalism, experience, and dedication of the development teams at the university and the seminary. Through multiple campaigns over the last seven years, the university and seminary have generated over $28M in donation support. The development efforts and donors’ commitment to this institution are remarkable. There is, however, significant concern about long-term institutional reliance upon significant gifts for institutional annual operating budgets, particularly if and when there is a presidential transition. 22 WSCUC Team Report for TMUS: March/2018 Organization Structures and Decision-Making Processes​: ​The institution’s report expressed high marks on the effectiveness of its leadership but noted some concerns of faculty and staff regarding a top-down approach to decision-making. The leadership team’s decisions were regularly described as bypassing established policies, procedures, and standards. Staff in some sectors were aware of irregularities, but were afraid to speak up to senior leaders (CFR 3.6). There were numerous reports in the confidential email process which provided documentation the authors believed demonstrated that those who “spoke up” to senior leaders were subsequently terminated. The organizational chart presented a crisp organizational structure yet was quickly dated, with several updates required between the submission of the Institutional Report and the Offsite Review, and again between the Off Site and Onsite Reviews. The organizational chart had many significant changes due to terminations, new hiring, reorganizations, and demotions during a very short period of time. The community described an atmosphere of confusion of leadership and decision-making processes which added to the disquieting spirit within the institution. With the volume and depth of major institutional and organizational changes, the community communicated a lack of understanding of roles, reporting hierarchy, responsibilities, and accountability which complicated conversations in the On Site Review, as changes were believed to still be occurring in real-time (CFR 3.7). TMUS has not adhered to the clearly-stated WSCUC standard that accredited institutions have a “full-time CEO and full-time CFO”. Specifically, the President/CEO and the CFO hold significant roles and responsibilities at other organizations. The standard requires these officers’ primary or full time responsibilities be to the institution (CFR 3.8). Further, the expectation of Standard 3.8 for “sufficient qualified administrators” is not fully realized in the new leadership ranks, with some alarming reports. The ​AVT​ was informed of situations where employees recently oversaw areas where they are enrolled in academic programs in their span of care (CFRs 2.1 and 3.8). With the recent turnover of cabinet-level leaders, most of the new team are recent alumni of the institution and lack higher education experience or 23 WSCUC Team Report for TMUS: March/2018 connection to professional associations or peers in similar professional roles. In most cases, recent senior leaders have dual pastoral/TMUS roles and lack higher education experience beyond student ranks (CFR 3.8). The institution’s governing board does not appear to demonstrate an independent governance role which is a WSCUC expectation. Virtually all board members have long-term relationships with the President and come from within his specific circle of influence, self-identifying as “his closest friends in the world.” At least one board member also serves as an Associate Dean, overseeing a out-of-state TMS site. With many board members being so integrally tied to the President, the ability to independently evaluate the Chief Executive Officer/President’s performance is hindered (CFR 3.9). Limited evidence was provided of faculty roles in academic leadership. The Academic Affairs Council of deans appears engaged in academic leadership but would not fully meet the expectations of faculty voice in academic decisions, as they carry a level of administrative rank. Further, reports of administrative intrusion in academic decisions (including bringing in an individual with a newly completed doctorate into a first-time faculty role at Associate Professor level; pressing for regrading of a failing exam and substituting a staff grade for that by the professor-of-record; and bypassing faculty input on awarding of several doctoral degrees) are outside the reasonable and ordinary practices of faculty academic leadership (CFR 3.10). Standard 4: Creating an Organization Committed to Quality Assurance, Institutional Learning, and Improvement The ​AVT​’s finding, which is subject to the Commission’s review, is that the institution demonstrated sufficient evidence of compliance with Standard 4. The Master’s University and Seminary have the basic structures for quality assurance at the institutional level. Departmental annual reports contribute to the institutional tracking of student success and learning outcomes. Periodic departmental reviews are scheduled and conducted to maintain a 24 WSCUC Team Report for TMUS: March/2018 satisfactory level of institutional commitment towards student learning and departmental standards of performance. Quality Assurance Processes: ​The Master’s University and Seminary have requisite systems in place to review and assess their academic programs, having utilized the Assessment Information Management System (AIMS) for over 10 years. However, as the institutional self-report alludes, there appears to be a rather large gap of time where program reviews were on hiatus, with many “delinquent” program reviews. Concurrently, the faculty at TMS provided a lower effectiveness rating for this standard. Ultimately, the systems are in place and appear to be satisfactory when they are properly utilized (CFR 4.1). The University’s Institutional Research (IR) department is composed of two individuals whose time is divided between IR and other responsibilities; their work is supported by IT staff. The stated goal is to combine these into a full time position, although the timing and institutional feasibility for this is unclear. While the IR department seems to be underdeveloped and underutilized, this is not uncommon, particularly in smaller institutional settings and their work was appropriate, given the resources. The underutilization and reputation of the IR department could be improved with better visibility and communication as to the research and data being tracked within the scope of their duties. Institutional Learning and Improvement: ​The University participates in the Ruffalo Noel Levitz Student Satisfaction Inventory (SSI) to address current student perceptions on the campus and their educational experience. In general, students report satisfaction with their TMU experience and education. The 2016 respondent demographics matched the overarching university demographic profile. Additionally, the 2016 version of the survey results shows a slightly higher student satisfaction level when compared to the 2014 data. In general, TMU students are more satisfied on virtually all scales when compared to other 4-year private institutions within the US, although there was no data to specifically compare the Master’s University with other faith-based comparison schools. In general, the 25 WSCUC Team Report for TMUS: March/2018 students report a high importance on virtually all scales with a corresponding higher than average satisfaction with their experience. The students that interacted with the ​AVT​ also echoed these sentiments and spoke very positively of their University experiences noting the excellence of instruction within the classroom and highlighting the campus experience (CFRs 2.3, 2.5, 2.10, 2.11, 2.13, 3.5, 4.3, and 4.5). With recognition of an appropriately-developed system of analysis, the gap was found in the lack of regular communication and utilization of assessment data, particularly as it relates to sharing it with faculty and staff. Simply conducting assessment scans and departmental reviews does not equate to a robust assessment culture. The results and findings must be disseminated to all appropriate parties and then action steps must be implemented to pursue change and improvement based on the findings. Here TMUS can give particular attention to “closing the loop” of the assessment cycle by improving the follow-through, communication and action plans as a result of their assessment processes. Based on the self-report, TMUS recognizes this shortcoming and is attempting to bring greater attention to the matter (CFR 4.3). In the ongoing inquiry of teaching and learning to improve curricula, pedagogy, and assessment, students of both TMU and TMS provide higher scores on this assessment than do the respective faculty in each unit. There appears to be adequate commitment on the part of the university focusing on teaching and pedagogy. Recently an educational technologist position has been created to assist and train faculty in new developments in this field (CFR 4.4). Current students score their role in assessing institutional effectiveness highly positively, while other constituents are less optimistic. Specifically, alumni desire to be more included in the assessment and feedback loop on how their input can make a difference. While this evidences alumni investment in their alma mater and its future, recently many alumni expressed concern over the directional shifts taking place on TMU’s campus and within the leadership team (CFR 4.5). Systems, structures and programs are 26 WSCUC Team Report for TMUS: March/2018 in place to facilitate the gathering and dissemination of key data. However, staff and faculty represented that while people may be invited to provide voice to the process, their voice isn’t utilized and there is strong belief that senior administration will operate independently of constituent input. While many decisions are ultimately in the purview of administrators, multiple voices and data is important to institutional health. In the context of reports of staff and faculty reports of not feeling heard or valued and significant reports of TMUS being an environment of intimidation and fear, the strategic planning and future success of the institution may be jeopardized if voices are silenced (CFR 4.6). Faculty and board members are less satisfied with the institution’s planning, anticipating, and responding to a changing higher education environment (CFR 4.7), with the perception that TMUS’s timing is “too little” or “too slow.” A highlight of the review was the professionalism, planning, and faculty support for the growing online programs. This is both market responsive and needed from a fiscal diversification perspective (CFRs 3.1, 3.4, 3.5, and 4.7). TMUS has wisely developed online and distributed learning courses to ensure the reach and accessibility of their educational mission. The online programs team demonstrated competence in their professional field and knowledgeable of distributed learning models as well as their target population. Federal Compliance with Required Policies In reviewing the university website for requisite reporting, the review team found some of the materials difficult to access; since the site visit, some related updates to the website appear to have been made. When asked about required data reporting, there was a concerning lack of familiarity by the COO (in whose domain the website rests) of responsibilities for the Clery Act, VAWA reporting, and ADA requirements of communication when asked for data availability. ​ ​At the seminary, the Dean of Students serves as the designated Title IX administrator, but the seminary website lacks the mandated Title IX and Clery reports and ADA communication requirements. The Graduate Student Handbook contains a link to 27 WSCUC Team Report for TMUS: March/2018 the university's HEOA (Higher Education Opportunity Act) page, which has links to Title IX reporting, but one must search the Education Department's website to obtain current reports. Credit Hour and Program Length Review:​ TMUS appears to be in alignment with federal credit hour and program length requirements. The M.Div. requires higher units than current elsewhere, with national trends showing this degree decreasing both in units and those opting for more flexible options such as the Th.M. program has implemented. The course syllabi are available online and include the credit hour calculations. Academic Affairs provides periodic and random credit hour audits. Seminary students expressed some concern over the amount of content being placed into each course noting particularly the amount of reading. This corresponded with courses that have been redesigned or reduced in units but retaining the same level of hourly work/reading expectations. Marketing and Recruitment Review:​ While not all of the needed information is easily accessible via the institutional website, TMUS appears to be well within compliance on this component. In recent years, the marketing department has made significant improvement to the branding and image of the TMUS, particularly focused on the undergraduate experience. The visitation team noticed the improved and polished look of many of the recruitment materials. Additionally, the students involved highlighted the more “collegiate” atmosphere these changes have brought to the campus, both in terms of advertising, marketing, and the look and feel of the campus itself. Student Complaints Review: ​TMUS has policies in place for handling complaints, the students who spoke with the team were aware of the processes they should follow, and the university seems to follow its own policies, so the university meets WSCUC’s complaint policy requirements. That said, there is no central system in place for tracking student complaints for the required six years​, so it is nearly impossible for administration to have a sense of the magnitude of problems,or of patterns that may be emerging. It is highly recommended that the university follow through on its stated intention (as stated in 28 WSCUC Team Report for TMUS: March/2018 the IR as a strategic emphasis) that a system for tracking complaints be put into place at the earliest opportunity. Transfer Policy Review: ​TMUS is in compliance with federal and WASC standards with regard to its transfer policies. Transfer student information, admissions requirements, and transfer policy are available in abbreviated form on the TMUS website and described in detail in the academic catalog. With the exception of Bible-focused courses, the requirements for nearly all General Education courses may be met through transfer courses. Administrative staff who recruit transfer students also provide advising on an individual basis so that students will know the courses which will articulate to TMUS and those that will not. Component 3: Degree Programs: Meaning, Quality, and Integrity of the Degrees The University’s stated mission is to “empower students for a life of enduring commitment to Christ, biblical fidelity, moral integrity, intellectual growth, and lasting contribution to the Kingdom of God worldwide.” The institution offers a variety of degree programs that are aligned with its mission, fit the institution’s values, objectives, and learning outcomes, and align with the university’s mission (CFRs 1.1, 1.2, 2.1, and 2.2). While the university’s motto and its Institutional Learning Outcomes both address concerns central to the meaning of a TMU degree, the university’s mission is foundational to both undergraduate and graduate degrees. The meaning of TMS degrees stems from its particular mission to prepare pastors; among these ideas is the seminary’s commitment to “the practice of pastoral ministry within the context of the local church” and “the pursuit of academic excellence” (TMUS 2018 Institutional Review Report). Quality is assured through the assessment process, which includes evaluating features of programs (curriculum and major subject exam results) in terms of external benchmarks (CFRs 2.1, 2.4, 2.7, and 4.5). Programs review their learning outcomes (PLOs) annually, and conduct a full program 29 WSCUC Team Report for TMUS: March/2018 review every five years. Students in some programs take nationally-normed field examinations, and some PLOs at the undergraduate level are associated with these exams, which gives faculty a sense of how their students perform in relation to those of other universities. Faculty with whom the team met described the workings of the assessment process capably, and were somewhat more enthusiastic about the process than is sometimes reported by faculty elsewhere. The Institutional Review Report notes that the seminary takes a similar approach to assessment, reviewing one program each year, in five year cycles. Integrity is assured by means of the scaffolding course content; curriculum maps identify required courses in which PLO-related content is introduced, taught, and measured. Prerequisites are reviewed by faculty as part of program review, and they are listed in the course catalog. TMU programs all contain senior-level seminars with capstone projects linked to the PLOs. Institutional Research staff provide additional support to the assessment process, providing data such as retention/attrition, graduation rates, and licensure exam pass rates (CFRs 2.6, 2.7, 4.1, 4.3, 4.4, and 4.6). Program reviews include self-studies, in which programs compare their requirements to similar programs at other institutions. These reviews also make use of external reviewers. As noted previously, grading and degree integrity emerged as an area of concern relative to incidents of a grade change in the seminary, outside usual and customary practices. The report was that a faculty member gave a student a failing grade. When the student appealed the grade, the TMS leadership had the assignment regraded by someone other than the professor of record; this second individual issued a passing grade that was recorded in lieu of the faculty member’s assigned grade. Multiple reports were received -- through the confidential email file and in conversations with TMS staff and leadership -- of doctorates being awarded outside defined program and degree requirements. Faculty processes and voices appear to have been silenced and/or overridden by competing concerns from other sectors of leadership. These are alarming reports as they speak to the meaning, quality, and integrity of a degree from TMS, the integrity of the institution and its leaders, and the fidelity to approved standards and 30 WSCUC Team Report for TMUS: March/2018 traditional academic practices as outlined in Standard 1, Defining Institutional Purposes and Ensuring Educational Objectives (CFRs 1.3, 1.5, 1.6, and 1.7); Standard 2, Achieving Educational Objectives Through Core Functions (CFRs 2.1, 2.2b, 2.4, 2.6, and 2.14); Standard 3, Developing and Applying Resources and Organizational Structures to Ensure Quality and Sustainability (CFRs 3.6, 3.7, and 3.10); and Standard 4, Creating an Organization Committed to Quality Assurance, Institutional Learning, and Improvement (CFRs 4.3 and 4.5). Component 4: Educational Quality: Student Learning, Core Competencies, and Standards of Performance at Graduation The institution demonstrates its commitment to the core competencies in Institutional Learning Outcome 3, “[The Master’s University bachelor’s degree graduate should be able to demonstrate:] Well developed intellectual skills in the areas of written and oral communication, quantitative reasoning, analytical thinking, and information literacy.” As described in the Institutional Review Report, these educational priorities fit well into the university’s mission. As noted in the review of Component 6, TMUS’s assessment process continues to evolve, with recent efforts aimed at identifying “key courses and assignments in each major program that support the development and formative assessment of each core competency” (TMUS 2017 IR Report, p. 27). Faculty are to be commended for their efforts along these lines; the list appears to be quite comprehensive, and samples from a diverse pool of assignments (IR Report, Attachment 4.04). As suggested above, the assessment process relies heavily on capstone assignments, which also provide opportunities to assess the core competencies (Attachment 4.02). Programs offer many opportunities to nurture students’ writing. Even highly quantitativelyfocused programs have assignments that require students to “show their work” in essays and other written projects. The AIMS report for 2015-2017 suggests that 86% of students write at an acceptable level; the 31 WSCUC Team Report for TMUS: March/2018 Academic Resource Center provides one-on-one tutoring for students who are in need of further assistance. The demonstration of oral communication skills begins in a general education course required for incoming students (C100 -- Spoken Communication). Across the curriculum, students have many opportunities to develop these skills. In many cases, the points of oral communication assessment are also points at which critical thinking may be assessed. Critical Thinking is associated with another required undergraduate course (MA240 -- Critical Thinking and Quantitative Analysis), which addresses essential elements of the scientific method (such as recognizing induction and deduction, and differentiating between theories and hypotheses) for students from less quantitative majors. The lists of assignments mentioned above demonstrate that students across the university have many opportunities to exercise these skills and receive feedback, and faculty have many opportunities to assess students’ progress. Both general education and major coursework support the development of information literacy. Courses throughout the curriculum require students to evaluate the arguments made in a variety of library resources. Examples include the English Department in which upper division courses require students to make use of peer-reviewed scholarly articles in their research, and Music History which requires students to weave together the insights of music historians in a cogent assignment. Students in less quantitative majors take the previously mentioned General Education Critical Thinking course which supports the development of quantitative reasoning. Students in majors that require more mathematics take statistics courses that are attuned to the needs of the program. Assignments and projects in majors such as Business and Biology provide many opportunities for the assessment of students’ acquisition of quantitative reasoning skills. There is a general University-wide commitment to educational quality and standards of performance in reviewing and improving upon the core curriculum as its own academic program (CFRs 32 WSCUC Team Report for TMUS: March/2018 2.2, 2.7, and 4.3). Processes are in place to evaluate student learning at entry and exit levels for undergraduates. As noted in the Institutional Review Report, TMS’s programs are graduate level, and learning goals have been developed with the specialized roles of graduates in mind. That said, a review of assignments across these programs suggest that most provide opportunities to observe the development of all core competencies with the exception of quantitative reasoning. Component 5: Student Success: Student Learning, Retention, and Graduation (CFRs 1.2, 2.7, 2.13) CFR 1.2 highlights student learning and requires an institution to provide “Clear educational objectives; indicators of student achievement at institution, program and course levels; retention/ graduation rates and evidence of student learning made public.” The Master’s University and Seminary graduation and retention rates are on par with other private, non-profit institutions. Interestingly, US News and World Report shows that while TMU’s scores are positive in the student-to-faculty ratio (top 10%), they are near the bottom in terms of the number of classes with more than 50 students enrolled (bottom 15%). Yet the numbers are comparatively low across all institutions, and thus, are not problematic. However, in terms of peer evaluation and student statistical success, TMU appears to be in the middle of the pack in the majority of metrics. This is a bit perplexing when they have one of the most open admission policies within their comparison group and the highest acceptance rate within their Christian Peer Group (more than 30% higher than average). It is not clear that the institution has a defined cohort group with whom they intend to compare themselves nor defined and specific standards of which quartiles they have set as goals, regarding admissions rates, student-to-faculty ratios, etc. CFR 2.7 emphasizes “Program review including SLOs, retention/graduation data, external evidence and evaluators”. Student learning in many courses is assessed through use of an appropriate competency/skills rubric. While the rubric is minimally sophisticated, TMUS has begun planning to this 33 WSCUC Team Report for TMUS: March/2018 end which is to be commended. There are systems, programs and processes in place to assess the classroom learning experiences and relate that learning to the department and university learning outcomes. Annual review reports are submitted by each respective dean. Departmental self-studies scheduled to be carried out every five years and are designed to provide an introspective analysis of the department while also offering comparisons to similar institutions. TMU is attempting to connect academic program reviews with co-curricular programs, and they have built a preliminary structure for identifying student learning outcomes within the co-curricular. This movement in terms of student life assessment data collection is a good start. However, significant work is needed in making these robust and effective, particularly in light of the redesign of the Student Life area of the university. CFR 2.13 focuses on whether there are “Appropriate student support services planned, implemented, and evaluated. The stated/provided “learning outcomes” for the co-curricular or Student Life Department are not defined as measurable learning outcomes. More attention is required for the development and refinement of Co-Curricular learning outcomes and the related assessment. While an alignment matrix attempts to demonstrate where each outcome is being represented throughout the co-curricular involvement, none of the data is connected back to programming or departmental responsibilities. TMU would strongly benefit from a unified curriculum map of the co-curricular efforts. Focused attention could be shown to where particularized outcomes are being introduced, developed, or mastered as this is currently only minimally represented. Component 6: Quality assurance and improvement: Program review, assessment, use of data and evidence While some programs at TMU have assessed learning outcomes since the late 1990s, ILOs were first adopted in 2011 and assessment began in earnest. In 2015, PLOs related to WSCUC’s Core Competencies were added to programs’ evaluation requirements, and programs were charged with 34 WSCUC Team Report for TMUS: March/2018 identifying courses in which the core competencies might be assessed during the university’s preparation for the current visit. The prior visiting team noted that the assessment process at the time was ambitious; these recent additions have made the process even more ambitious. The requirements for annual department review of learning outcomes are described in Attachment 6.06. These include the evaluation of student performance (and “loop-closing measures”) as they apply to outcomes in general education and the major. Given the breadth of content and the sheer number of items which must be assessed, it seems unlikely that even the largest of departments has adequate analytical capacity to benefit from the exercise. Evidence that this approach is overly ambitious may be found in the examples of annual reviews provided in the Institutional Report. The review provided for Mathematics dates back to AY 2011-12, while those for Music and DCP date back to 2012-13. While these may have been included in the report on the basis of their exemplary quality, it seems more likely that the annual reporting requirements may have been relaxed in service of other priorities. Further, a number of rubrics provided as samples (e.g., in Music) featured criteria and performance categories but provided no descriptions of how the two were related (the internal matrix was blank). Without those descriptors, the quality of the evaluation is problematic at best, and the rubric is not useful for providing feedback to students. In Visit Team members’ discussion with faculty about assessment, Erik Gilbert’s recent article about the value of assessment (in the ​Chronicle of Higher Education)​ provided insight into faculty’s regard for the current system. There seemed to be a fair amount of agreement with his description of assessment as a task force on faculty that rarely delivers on its promise of improved educational quality. While the current approach to annual reviews and program reviews probably provide a general sense of evaluation of programs, the institution would likely benefit from simplifying the process, which could potentially provide the level of engagement and insight that leads to substantive change. Institutional Review (IR) staff reported that the current AIMS system which is used for compiling and reporting 35 WSCUC Team Report for TMUS: March/2018 assessment outcomes is being replaced with another system, so specific recommendations may be affected by the new system. Nonetheless, a basic reminder is that providing too much data for consideration can be as problematic as having too little. If the process suffers as a consequence of its ambitions, an even larger obstacle to sustainability of program review may be the tenuous link between review results and administrative decision-making involving the allocation of university resources (CFRs 4.2, 4.5, and 4.6). It is difficult to make a case for additional resources when relying on data of dubious quality, and the volume of data which must be reviewed annually under the current requirements leaves little time or energy for sustained, thoughtful consideration of the intersecting factors that contribute to student learning. Finally, while there is an abundance of evidence that programmatic changes have been made on the basis of outcomes assessment and program review (e.g., in the annual reviews that appear in the Institutional Review Report), there is very little evidence that improvements resulted from these changes (CRF 4.1, 4.2). This, too, may be a sign that the annual reporting requirements are not leaving sufficient time for analysis of data over time. Component 7: Sustainability: Financial viability, preparing for the changing higher education environment The institution’s leadership team and the board acknowledge that sustainability, in today’s challenging higher educational environment, is one of the most significant concerns of the university, along with its ability to adapt to this changing landscape. As a faith-based institution, TMUS has a smaller constituency from which to recruit which adds to its unique enrollment challenges. The institution is projecting a negative change in net assets from operations for the next three years ranging from $1.3M to $385K annually. This negative change includes the release of funds to meet the operational needs of the university and seminary. The release of funds ranges from $4.1 M to $4.9M 36 WSCUC Team Report for TMUS: March/2018 annually each year for three years. These negative changes are a result of reduced undergraduate enrollment, a tuition reset, and the reduction in auxiliary revenue. The institution had a negative net change in its cash in 2017 of $729K. Current liabilities increased $3.4M from 2016 to 2017. Future negative change in net assets and scheduled debt repayments will put an additional strain on the unrestricted operational investment funds. This presents the institution with demands on financial resources and a burden on the alignment of unrestricted operating funds. The development efforts and the commitment of their donors to the future of the institution is a great strength for the university. The institution’s gifts and grants historically produce approximately $3M to $4M annually in operational general funds. The Development Office is projecting general donations of $5M annually over the next three years. Meanwhile, the institution’s investment balance presents a $4M increase from 2016 to 2017. This appears to be primarily due to $3M in endowed funds received in 2017. The institution’s strategic plan addresses the future and how it is prepared to face the fundamental changes in faith-based and private higher education. The institutional plan focuses on expanding student enrollment (tuition reset initiative), developing online programs, diversifying programs, a drive for enhanced quality programs, implementing a competitive salary structure, enhanced institutional technology, and student career planning and placement. While these strategies are in line with other successful initiatives, all require added resources -- for online program development, program improvement, professional development, enhanced salaries, improved technology, and additional student services. The Master’s University and Seminary appear to be addressing the sustainability issue and have articulated a robust strategic plan to address needed growth. Metrics have been created to measure the progress of these initiatives. While the development efforts of TMUS and, specifically, President MacArthur, are remarkable, there are significant concerns about the capacity to raise funds after a leadership succession. Further, the team notes that achieving operational results to insure the growth of 37 WSCUC Team Report for TMUS: March/2018 net assets should remain a high priority for the institution. Thus, there is a growing need to rely less on donations for annual operating funds and to grow the endowment to moderate year-to-year enrollment impact on long-term institutional sustainability. Component 9: Conclusion: Reflection and plans for improvement TMUS reflections on their plans for improvement included the following specific statements: The self -study process also identified a number of areas that will need significant attention going forward. Six important issues will be listed here that encompass many of the smaller specific items cited in the previous report components. First, given recent adjustments in the university and seminary administrative structures, it will be important to ensure that there is an effective transition in organizational reporting and communication channels and processes. With the significant number of personnel changes, there is a need for heightened attention to provide for the orderly transfer of institutional knowledge regarding internal history, policies, and procedures, as well as regulations from external governing authorities. Second, continued attention must be given to ensuring the fiscal sustainability of the institution. This a particularly challenging area given the necessity of remaining affordable to constituent students while addressing the long overdue priority of increasing the salary levels of faculty and staff. Third, and related to the above, is the importance of growing student enrollment levels. Effective marketing strategies will play a key role here. Expanding the availability of online courses and degree programs is another element that is likely to offer a TMUS education in a more accessible and affordable format. Fourth is the important challenge to find mechanisms to better reflect Southern California’s ethnically diverse environment through attracting a higher proportion of minority students, as well as a greater number of non-white faculty and staff. Fifth is the necessity to continue to effectively promote academic advancement and scholarship among faculty and among students in all TMUS graduate programs. A number of steps have already been taken in this area (e.g., the Centers of Excellence at TMU). However, providing support for scholarship and limiting teaching loads to provide time for research and student mentoring both require additional financial resources that relate to the challenges cited above. In addition, the positive transitions to recruit and hire younger faculty to replace retiring individuals often brings the need to support the new faculty in pursuit of a terminal degree. Finally, due to the national focus on student assessment processes as a key mechanism for demonstrating institutional educational effectiveness, the successful and efficient transition from TMUS’s AIMS system to AEFIS will be a priority for attention during the 2017-18 academic year. The team appreciates the serious reflection contained in these six strategic observations and will comment further about the substance of the six items in the Recommendations portion of the report. However, there are additional concerns noted previously that must be addressed as well as the six identified areas of future focus. 38 WSCUC Team Report for TMUS: March/2018 SECTION III – FINDINGS, COMMENDATIONS, AND RECOMMENDATIONS After concluding the Onsite Review, the Accreditation Visit Team more deeply understood the substantive nature of the concerns referenced in prior reports. What became even more clear to the team was that the need for systemic change across the institution referenced in prior reports has escalated significantly and now is a substantial issue for the institution to resolve in order to align with WSCUC Standards. Engagement by the Visit Team with board members, administration, faculty, staff, and students of The Master’s University and Seminary provided a consistent message of the commitment of the TMUS community to the identity and mission of the university. However, many other items came to the team’s attention through document and communication review, the onsite visit, and the extraordinary volume of emails received by the team. There is a high level of concern over the volume of personnel transitions since the last review, particularly in the year prior to the Institutional Report and WSCUC team visit. Many aspects of The Master’s University’s academic programs are strong and exemplify good practices. Faculty and staff are fully engaged, committed to student success, and invested long-term in the institution and its values. Academic programs, online program development, and assessment meet or exceed requisite standards and TMUS is to be commended for this. The Master’s University and Seminary’s greatest asset is the cadre of fully-committed faculty and staff, many of whom have been affiliated with the institution for decades. Thus, the cacophony of concerns about the current culture of the institution -- broad and significant reports of a pervasive culture of fear, intimidation, bullying, and uncertainty -- is even more notable and concerning. Specific reports of unethical behavior by key leaders and fear of reprisal were consistent and corroborated through multiple sources. Yet, with no place where faculty feel safe to report being required to award degrees outside usual boundaries; for staff to address being required to issue payments or financial aid outside usual and 39 WSCUC Team Report for TMUS: March/2018 customary procedures; and for a student worker to share having developed an exit strategy should their supervisor continue to bully and yell at them, the related reports of lack of leadership ethics and accountability that emerged was unmatched for members of this review team. This is escalated by the intertwined nature of relationships between senior leaders with each other (including several sets of family relationships), with the board, with businesses owned by key leaders and donors, with Grace Community Church, and with key donors that would not pass many institutions’ conflict of interest policies. It seems this has been part of the operation for so long that it is practiced without question. But, the role of accreditation is to provide an independent and impartial review. And this team’s findings include numerous practices that are outside what is reasonable and customary. The catalyst for the groundswell of reporting concerns points to a senior hire within the last twelve to eighteen months that has provided the tipping point for a culture that has long deferred to a president out of respect (and, for many, because he is also their pastor). The significant changes at the administrative cabinet level, coupled with the pervasive belief that long-time employees have been dismissed for speaking up, has resulted in a toxic environment that must be immediately addressed -- both as an accredited institution and as an institution rooted in Christian values of integrity and respect for people. Sadly, the institution may not be able to regain its balance without deep and significant changes. When one leader explained to the review team that if individuals are committed to an institution’s values, salary increases become less important, it was disturbing. If the institution loses the confidence of deeply committed employees, the institutional impact could be significant. Similarly, the lack of fidelity to WSCUC’s requirement for an accredited institution to have a full time president and CFO will likely require some further level of leadership transition. Simultaneously, the significant reliance on donors for annual operating funds is precarious, particularly with a president who has served for 33 years. As noted previously, the fundraising capacity of the President MacArthur is substantial and extraordinary and to be celebrated as a key part of the 40 WSCUC Team Report for TMUS: March/2018 sustainability equation for the institution. The institution itself noted that financial challenges due to flat or declining traditional enrollment are attenuated through fundraising activities. While there is great cause for celebration in the donor support of the institution, there are significant concerns about the sustainability of this model. Operational income and expense should be closely monitored by the board and administration in the years to come. The strong and lengthy tenure of the President is commendable but the likely pending transition in executive leadership of the President that may affect the capacity of the institution to continue to achieve the extraordinary fundraising results that have been experienced the last several years. Of primary concern to the team was the seeming lack of reference to and intentional framework for compliance with Standards and CFRs that are part of the accreditation review process. While the Institutional Report reference the Standards and CFRs, additional requested materials suggest that the accreditation standards are not woven into the operational life of the university. This concern was extended to a deeper level upon examination of the prior Commission reports and actions which raise similar and foundational concerns as those that this team observed at substantive levels. The seminary leadership appeared to take prior Commission visit recommendations and respond positively to them; we express the hope that TMUS and its board members and administrators will respond with clarity and focus to the recommendations the Commission will make through this present report. The self-study responded to each of the prior WSCUC Commission actions, acknowledging needed changes. Nonetheless, many responses to prior actions have not been operationalized and remain unresolved while others show steady progress but remain incomplete. At this stage in the institution’s history and pattern of engagement with WASC/WSCUC, the review team provides the following commendations and recommendations. Commendations The team commends the institution for the following: 41 WSCUC Team Report for TMUS: March/2018 1. The board appeared engaged and dedicated to the mission of the institution. 2. Faculty, staff, and students are deeply invested in the institution and its missional values. 3. Leadership and staff in both TMU and TMS Development Offices, as well as the fundraising influence of President MacArthur and his legacy of 33 years, are notable. 4. S​tudent ratings of their experience based on the Student Satisfaction Inventory and rating reported in the Wall Street Journal are high. Undergraduate student interactions affirmed this experience. 5. The university and seminary have increased their data analytic capacity since the last WASC/WSCUC visit. 6. The Finance team demonstrated knowledge, concern, and plans for action regarding resolution of key accounting issues identified in the institutions’ 2017 audit report. 7. The online program team evidences professional competence in the area of distributed education with knowledge of their population and appropriate pedagogical approaches. 8. Community feedback on marketing and branding materials is very positive. Recommendations The team recommends the institution address the following: 1. The team found a pervasive culture and climate of fear, intimidation, bullying, and uncertainty among significant numbers of faculty and staff. The team recommends the board immediately establish a special task force to address these substantial​ ​concerns, including related personnel actions (CFR 1.3, 1.6, 1.7, and 3.6). 2. The team found a pattern of seemingly arbitrary personnel actions (hirings, terminations, promotions, demotions, and lack of regular evaluations) with a lack of due process and in 42 WSCUC Team Report for TMUS: March/2018 violation of existing TMUS Employee and Faculty Handbooks. This included reports that some individuals had been notified they would be dismissed after the Onsite Review Team visit. The team recommends that the board hold the President accountable for ensuring that his leadership team and those in their span of care follow institutional policy in personnel decisions and require quarterly reporting on staff and faculty transitions and evaluations, with note of the importance of honoring the Whistleblower Act and related policy in the Employee Handbook (1.6, 1.7, 3.2, 3.6, 3.7, and 3.9). Specifically, the team recommends the board authorize an independent review of all faculty and staff departures over the last 2-3 years. There were multiple independent reports of concerning irregularities. 3. The institution fails to meet the WSCUC requirement for governing board independence. Virtually all board members have long-term relationships with the President and come from within his specific circle of influence. The team recommends that the board immediately review the WSCUC Independent Governing Board Policy and conform accordingly (CFRs 3.7 and 3.9, and Independent Governing Board Policy). 4. The team noted several instances of academic irregularity outside established procedures and best practices. This includes the awarding of some doctoral degrees with significant leadership pressure and outside usual academic standards and reviews, significant faculty hiring from alumni ranks as well as faculty hiring practices which lack substantial review, awarding faculty rank without traditional faculty and/or chair input, and individuals directly or indirectly overseeing academic programs in which they are enrolled. The team recommends that institutional leadership assiduously follow the policies of the Faculty Handbook and higher education traditional practices, with the Provost being 43 WSCUC Team Report for TMUS: March/2018 directly accountable to the board for reporting on faculty hirings, releases, and awarding of degrees (CFRs 1.3, 2.1, 2.2, 2.4, 3.10, and 4.6). 5. The team notes with concern a pattern of operational irregularities, including the 2017 auditor’s specific finding on appearance of conflicts of interest with the President’s son-in-law supervising a contract from which he benefits, as well as institutional aid being awarded to related parties exceeding typical award amounts. The lack of response to this finding six months after it was identified as an auditor “significant finding” adds to the concerns of leadership willingness to promptly align with accreditation and financial audit standards. The team recommends that the institution seek and receive legal and auditor guidance and implement practices to insure the resolution of all reported conflicts of interest (CFRs 3.6 and 3.7)​. 6. The institution fails the required standard to have a full time Chief Executive Officer and Chief Financial Officer. While the team recognizes that the individuals in the roles appear to be exceedingly capable, concerns remain about these and other key leaders holding two or more significant fulltime roles and note this is not in keeping with WASC expectations (particularly for the CEO and CFO) nor is it advisable in these times of enrollment and fiscal challenges. The team recommends the institution clearly articulate that the President (CEO) and CFO are full time roles and that the board responsibly monitor compliance with this standard (CFR 3.8). 7. The institution has entered into a period of substantial management and leadership changes with resultant institutional confusion due to the lack of timely personnel announcements. Further, some individuals have been hired for specific professional roles 44 WSCUC Team Report for TMUS: March/2018 despite a lack of higher education experience or training, and other have been hired without job descriptions being provided and/or searches being conducted. The lack of awareness of key higher education regulatory expectations and professional standards for institutions of higher education by some leaders without higher education experience or affiliation is concerning. The team recommends that all leaders at the director through executive level join and regularly engage in professional associations related to their work assignment, to ensure familiarity with professional standards and practices, as well as regulatory expectations. Further, institutional leadership is encouraged to establish regular mechanisms to update the community regarding staff assignments and program launches (CFRs 1.3, 1.6, 1.8, 3.6, and 3.7). 45 WSCUC Team Report for TMUS: March/2018 APPENDICES Federal Compliance Forms and Inventory of Educational Effectiveness Indicators Attachment 2.07 TMUS 2017 Federal Compliance Forms Attachment 2.08 Credit Hour Supporting Documents Attachment 2.09 The Master’s Seminary Credit Hour Policy & 2016 Audit Materials Attachment 2.10 Analysis of TMUS Recruitment Policies and Marketing Materials Attachment 2.11 TMUS Policies & Procedures on Student Complaints Attachment 2.12 TMUS Transfer Credit Policies Attachment 2:13 Inventory of Educational Effectiveness Indicators for The Master's University & Seminary 46 WSCUC Team Report for TMUS: March/2018 Attachment 2.07: TMUS 2017 FEDERAL COMPLIANCE FORMS OVERVIEW There are four forms that WSCUC uses to address institutional compliance with some of the federal requirements affecting institutions and accrediting agencies: 1 – Credit Hour and Program Length Review Form 2 – Marketing and Recruitment Review Form 3 – Student Complaints Review Form 4 – Transfer Credit Policy Review Form Teams complete these four forms and add them as appendices to the team report. They are included here in order for the institution to provide the necessary information for the team. Teams are not required to include a narrative about any of these matters in the team report but may include recommendations, as appropriate, in the Findings, Commendations, and Recommendations section of the team report. 1 - CREDIT HOUR AND PROGRAM LENGTH REVIEW FORM Under the federal requirements referenced below, WSCUC is required to demonstrate that it monitors the institution’s credit hour policy and processes as well as the lengths of its programs. Credit Hour - §602.24(f) The accrediting agency, as part of its review of an institution for renewal of accreditation, must conduct an effective review and evaluation of the reliability and accuracy of the institution's assignment of credit hours. (1) The accrediting agency meets this requirement if(i) It reviews the institution's(A) Policies and procedures for determining the credit hours, as defined in 34 CFR 600.2, that the institution awards for courses and programs; and (B) The application of the institution's policies and procedures to its programs and coursework; and (ii) Makes a reasonable determination of whether the institution's assignment of credit hours conforms to commonly accepted practice in higher education. (2) In reviewing and evaluating an institution's policies and procedures for determining credit hour assignments, an accrediting agency may use sampling or other methods in the evaluation. Credit hour is defined by the Department of Education as follows: A credit hour is an amount of work represented in intended learning outcomes and verified by evidence of student achievement that is an institutionally established equivalency that reasonably approximates not less than— (1) One hour of classroom or direct faculty instruction and a minimum of two hours of out of class student work each week for approximately fifteen weeks for one semester or trimester hour of credit, or ten to twelve weeks for one quarter hour of credit, or the equivalent amount of work over a different amount of time; or (2) At least an equivalent amount of work as required in paragraph (1) of this definition for other academic activities as established by the institution including laboratory work, internships, practica, studio work, and other academic work leading to the award of credit hours. See also WSCUC Senior College and University Commission’s Credit Hour Policy. Program Length - §602.16(a)(1)(viii) Program length may be seen as one of several measures of quality and as a proxy measure for scope of the objectives of degrees or credentials offered. Traditionally offered degree programs are generally approximately 120 semester credit hours for a bachelor’s degree, and 30 semester credit hours for a master's degree; there is greater variation at the doctoral level depending on the type of program. For programs offered in non-traditional formats, for which program length is not a relevant and/or reliable quality measure, reviewers should ensure that available information clearly defines desired program outcomes and graduation requirements, that institutions are ensuring that program outcomes are achieved, and that there is a reasonable correlation between the scope of these outcomes and requirements and those typically found in traditionally offered degrees or programs tied to program length. 1 - CREDIT HOUR AND PROGRAM LENGTH REVIEW FORM Under the federal requirements referenced below, WSCUC is required to demonstrate that it monitors the institution’s credit hour policy and processes as well as the lengths of its programs. Material Reviewed Questions/Comments (Enter findings and recommendations in the Comments sections as appropriate.) Policy on credit hour Is this policy easily accessible? If so, where is the policy located? Comments: Process(es)/ periodic review of credit hour Schedule of on-ground courses showing when they meet  YES  NO Faculty Policy located in Faculty Handbooks. Does the institution have a procedure for periodic review of credit hour assignments to ensure that they are accurate and reliable (for example, through program review, new course approval process, periodic audits)?  YES  NO If so, does the institution adhere to this procedure?  YES  NO Comments: Average number of hours of student engagement in each activity for each course are a required part of each course syllabus. So the review is a regular part of each semester’s instructional activities. Does this schedule show that on-ground courses meet for the prescribed number of hours?  YES  NO Comments: The published course schedules show the weeks, hours and days that courses meet. Sample syllabi or equivalent for online and hybrid courses Please review at least 1 - 2 from each degree level. Sample syllabi or equivalent for other kinds of courses that do not meet for the prescribed hours (e.g., internships, labs, clinical, independent study, accelerated) Please review at least 1 - 2 from each degree level. How many syllabi were reviewed? Type of courses reviewed:  online  hybrid What degree level(s)?  AA/AS  BA/BS  MA  Doctoral What discipline(s)? Are students doing the amount of work per the prescribed hours to warrant the credit awarded? Comments: How many syllabi were reviewed? What kinds of courses? What degree level(s)?  AA/AS  BA/BS  MA  Doctoral What discipline(s)? Are students doing the amount of work per the prescribed hours to warrant the credit awarded?  YES  NO  YES  NO Comments: Sample program information (catalog, website, or other program materials) How many programs were reviewed? What kinds of programs were reviewed? What degree level(s)?  AA/AS  BA/BS  MA  Doctoral What discipline(s)? Does this material show that the programs offered at the institution are of an acceptable length?  YES  NO 2 - MARKETING AND RECRUITMENT REVIEW FORM Under federal regulation §602.16(a)(1)(vii), WSCUC is required to demonstrate that it monitors the institution’s recruiting and admissions practices. Material Reviewed **Federal Requirements Questions and Comments: (Enter findings and recommendations in the Comments sections of this table as appropriate.) Does the institution follow federal requirements on recruiting students?  YES  NO Comments: The Director of Admissions is responsible to stay current on all federal requirements on recruiting students. All federal expectations are included in the Admissions Department Handbook. All admission counselors go through extensive training before going to work in the department so that they fully understand the laws and regulations that they are to obey. The Admissions Department also receives outside training that includes pertinent laws and regulations that are part of the recruiting process in a higher education institution. Degree completion and cost Does the institution provide information about the typical length of time to degree?  YES  NO Does the institution provide information about the overall cost of the degree?  YES  NO Comments: Information regarding the length of time needed to complete a degree and the cost of earning a degree is in the catalog, on the website (www.masters.edu), in print, and given in person by counselors who interact with every applicant to ensure that they understand both the time it will typically take to complete the degree and the estimated cost to earn the degree. Careers and employment The length of time to earn a degree is outlined in the University and Seminary Catalogs as well as on the websites. General education requirements, individual major requirements, and graduation requirements are all presented. Once students have begun their studies on campus, faculty advisors work with each student each semester to monitor progress, give advice on course selection, and prepare the student for graduation. Attachment 2.10 provides additional information on the Seminary’s efforts to provide students with cost information. Does the institution provide information about the kinds of jobs for which its graduates are qualified, as applicable?  YES  NO Does the institution provide information about the employment of its graduates, as applicable?  YES  NO Comments: Possible career information is available on TMU websites and in the academic catalogs. The Human Resources staff, Student Employment staff, and the Director of Alumni Relations work together to provide current students with career services. Two career preparation workshops are offered each semester. The HR Director works individually with seniors who are preparing resumes. Alumni come in for conferences to teach current students how to prepare for the workforce. TMU plans to purchase a software package, “Handshake,” that will connect to every student in a Facebook-like format. The Seminary conducts Placement Workshops each semester to help prepare graduates for job interviews after graduation. In addition, a placement system is in place that allows graduating students to connect with churches in need of pastoral personnel and a church finder feature on the www.tms.edu which includes TMS graduates. Attachment 2.10 provides additional information regarding the Seminary’s placement efforts. **Section 487 (a)(20) of the Higher Education Act (HEA) prohibits Title IV eligible institutions from providing incentive compensation to employees or third party entities for their success in securing student enrollments. Incentive compensation includes commissions, bonus payments, merit salary adjustments, and promotion decisions based solely on success in enrolling students. These requirements do not apply to the recruitment of international students residing in foreign countries who are not eligible to receive Federal financial aid. 3 - STUDENT COMPLAINTS REVIEW FORM Under federal regulation*§602-16(1)(1)(ix) WSCUC is required to demonstrate that it monitors the institution’s student complaints policies, procedures, and records. (See also WSCUC Senior College and University Commission’s Complaints and Third Party Comment Policy.) Material Reviewed Policy on student complaints Questions/Comments (Enter findings and recommendations in the Comments sections of this table as appropriate.) Process(es)/ procedure Does the institution have a procedure for addressing student complaints? YES  NO If so, please describe briefly: Students are directed to work with the leadership in the office that most closely relates to the area of their complaint (e.g., academic-related complaints by Provost’s office, inter-student conflicts by the Dean of Student Life, etc.) If so, does the institution adhere to this procedure? YES  NO Does the institution have a policy or formal procedure for student complaints? Is the policy or procedure easily accessible? If so, where? The policies and procedures are located in the faculty and student handbooks. Comments: See TMUS Institutional Review Report Attachment 2.11 for specific location of various policies. Comments: Records YES  NO YES  NO See TMUS Institutional Review Report Attachment 2.11 for details about how various types of student complaints are addressed. Does the institution maintain records of student complaints? YES  NO If so, where? TMS Student complaints kept in their permanent file. TMU student academic complaints kept in Registrar’s office. Student Life complaints kept in student file in Student Life Office. Does the institution have an effective way of tracking and monitoring student complaints over time? YES  NO If so, please describe briefly: TMUS is reevaluating and updating its methods of tracking student complaints over time to be able to better support the student experience. Comments: See TMUS Institutional Review Report Attachment 2.11 for specific location of various policies. 4 – TRANSFER CREDIT REVIEW FORM Under federal requirements*, WSCUC is required to demonstrate that it monitors the institution’s recruiting, transfer, and admissions practices accordingly. Material Reviewed Transfer Credit Policy(s) Questions/Comments (Enter findings and recommendations in the Comments sections of this table as appropriate.) Does the institution have a policy or formal procedure for reviewing and receiving transfer credit?  YES  NO If so, is the policy publicly available?  YES  NO If so, where? Transfer credit policies are published in the academic catalogs. Does the policy(s) include a statement of the criteria established by the institution regarding the transfer of credit earned at another institution of higher education?  YES  NO Comments: *§602.24(e): Transfer of credit policies. The accrediting agency must confirm, as part of its review for renewal of accreditation, that the institution has transfer of credit policies that-(1) Are publicly disclosed in accordance with 668.43(a)(11); and (2) Include a statement of the criteria established by the institution regarding the transfer of credit earned at another institution of higher education. See also WSCUC Senior College and University Commission’s Transfer of Credit Policy. Att. 2.13: Inventory of Educational Effectiveness Indicators for The Master's University & Seminary Category Outcomes Developed Where Published Types of Evidence that Graduates Achieve Outcomes Who & How is Evidence Evaluated? How are the Findings Used? Last Program Review University Programs Undergraduate Co-curricular Programs Yes TMU Catalog, TMU AIMS Survey of entry & end of college activity patterns; and surveys of students, graduates, alumni, & pastors. PLOs are incorporated them into the annual program reviews and assessments. A Bi-Annual All Student Survey with follow up focus groups is being implemented to enable cross-referencing with program assessments. Departmental faculty review student learning assessment data on annual basis, and as part of the Departmental Self-study (DSS). Needed changes are documented in the Annual Review (AAR) & DSS Reports. Needed changes to the PLOs, assessment methods, and/or program structure are documented & action is taken by appropriate person(s). Spring, 2008, Next DSS scheduled for 2017-18 B.A./B.M./B.S. General Education Yes TMU Website, TMU Catalog, TMU AIMS Institutional exams; student work portfolios; and surveys of students, graduate, alumni, employers, & graduate school advisors. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Comprehensive GE review & revision in 201314 Portfolio items from selected major courses, comprehensive senior exam in senior seminar; and surveys of students, graduate, alumni, employers, & graduate school advisors. Portfolio items from selected major courses, comprehensive senior exam in senior seminar; and surveys of students, graduate, alumni, employers, & graduate school advisors. Portfolio items from selected major courses, comprehensive senior exam in senior seminar; and surveys of students, graduate, alumni, employers, & graduate school advisors. Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Plan to begin departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Summer, 2016 Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2015 Plan to begin departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Fall, 2015 Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Fall, 2012 (program being sunset in Spring, 2018) Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Plan to begin departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Spring, 2015 Bachelor of Arts Majors: Biblical Counseling (online format) Yes TMU Website, TMU Catalog, TMU AIMS Biblical Studies (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS Biblical Studies (online format) Yes TMU Website, TMU Catalog, TMU AIMS Christian Ministries (degree completion format) Yes TMU Website, TMU Catalog, TMU AIMS Christian Ministries (on-line format) Yes TMU Website, TMU Catalog, TMU AIMS TMUS 2017 IEEI Worksheet pg. 1 Att. 2.13: Inventory of Educational Effectiveness Indicators for The Master's University & Seminary Category Outcomes Developed Where Published Types of Evidence that Graduates Achieve Outcomes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Communication (w/ emphases) Yes English (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS History (w/ emphases) Yes Kinesiology & Physical Education (w/ emphases) Yes Liberal Studies Who & How is Evidence Evaluated? DepartmentalPrograms faculty review student learning University How are the Findings Used? Last Program Review Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2015 assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2014 TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; ACAT Major Exam in History; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2013 TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Fall, 2012 Formalized in context of Teacher Education emphasis Liberal Studies Teacher Education Yes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis. Complete selfstudy activities as required by the California Commission on Teacher Credentialing (CCTC). Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. CCTC Required Biannual Review in Spring 2015 Mathematics (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Fall, 2016 Music (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; ETS Major Field Exam in Mathematics, and surveys of students, graduate, alumni, employers, & graduate school advisors. Portfolio items from selected major courses; ETS Major Field Exam in Music; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis. DSS completed as part of requirement for National Association of Schools of Music (NASM) accreditation. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Departmental Self-study for NASM Accreditation in Spring 2013 TMUS 2017 IEEI Worksheet pg. 2 Att. 2.13: Inventory of Educational Effectiveness Indicators for The Master's University & Seminary Category Outcomes Developed Where Published Types of Evidence that Graduates Achieve Outcomes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Organizational Management (degree completion format) Organizational Management (online format) Yes Yes TMU Website, TMU Catalog, TMU AIMS Political Studies (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS TMUS 2017 IEEI Worksheet Who & How is Evidence Evaluated? DepartmentalPrograms faculty review student learning University How are the Findings Used? Last Program Review assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Fall, 2012 (program being sunset in Spring, 2018) Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Plan to begin departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Spring, 2015 Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2013 pg. 3 Att. 2.13: Inventory of Educational Effectiveness Indicators for The Master's University & Seminary Category Outcomes Developed Where Published Types of Evidence that Graduates Achieve Outcomes Who & How is Evidence Evaluated? How are the Findings Used? Last Program Review University Programs Bachelor of Science Majors: Biological Sciences (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; ETS Major Field Exam in Biology; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Fall 2015 Business Administration (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; ETS Major Field Exam in Business; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2012 Computer & Information Sciences (with emphases) Yes TMU Website, TMU Catalog, TMU AIMS Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2007 (scheduled for Fall, 2014) Home Economics Family & Consumer Sciences (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; ETS Major Field Exam in Computer Science; and surveys of students, graduate, alumni, employers, & graduate school advisors. Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 1997 (program being sunset in 2017) Bachelor of Music Majors (w/ emphases) Yes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; ETS Major Field Exam in Music; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis. DSS completed as part of requirement for National Association of Schools of Music (NASM) accreditation. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Departmental Self-study for NASM Accreditation in Spring 2013 Master of Arts in Biblical Counseling Yes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2016 TMUS 2017 IEEI Worksheet pg. 4 Att. 2.13: Inventory of Educational Effectiveness Indicators for The Master's University & Seminary Category Outcomes Developed Where Published Types of Evidence that Graduates Achieve Outcomes TMU Website, TMU Catalog, TMU AIMS Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Portfolio items from selected degree courses (especially program capstone course); and surveys of students, graduate, alumni, employers, & graduate school advisors. Portfolio items from selected major courses; and surveys of students, graduate, alumni, employers, & graduate school advisors. Master of Arts in Biblical Studies (Summer Intensive format) Yes Master of Arts in Biblical Studies (online format) Yes TMU Website, TMU Catalog, TMU AIMS Master of Business Administration (online format) Yes TMU Website, TMU Catalog, TMU AIMS TMUS 2017 IEEI Worksheet Who & How is Evidence Evaluated? DepartmentalPrograms faculty review student learning University How are the Findings Used? Last Program Review Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2015 assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Plan to begin departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Fall, 2016 Plan to begin departmental faculty review student learning assessment data on annual basis, and as part of the DSS. Needed changes are documented in the AAR & DSS Reports. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Fall, 2015 pg. 5 Att. 2.13: Inventory of Educational Effectiveness Indicators for The Master's University & Seminary Category Outcomes Developed Where Published Types of Evidence that Graduates Achieve Outcomes Who & How is Evidence Evaluated? How are the Findings Used? Last Program Review University Programs Seminary Programs Diploma of Theology Yes TMS website; AIMS; TMS Catalog Entrance/exit exams; alumni survey; Corporate faculty review of assessment data; portfolio items from major courses individual faculty assessment annually with and culminating project. administration. Needed changes are documented & action taken by appropriate persons. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2016 Completed in context of M.Div. Review Bachelor of Theology Yes TMS website; AIMS; TMS Catalog Entrance/exit exams; alumni survey; Corporate faculty review of assessment data; portfolio items from major courses individual faculty assessment annually with and culminating project. administration. Needed changes are documented & action taken by appropriate persons. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2016 Completed in context of M.Div. Review Master of Divinity Yes TMS website; AIMS; TMS Catalog Entrance/exit exams; alumni survey; Corporate faculty review of assessment data; portfolio items from major courses individual faculty assessment annually with and culminating project. administration. Needed changes are documented & action taken by appropriate persons. Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2016 Master of Divinity-hybrid Yes TMS website; AIMS; TMS Catalog Entrance/exit exams; alumni survey; Corporate faculty review of assessment data; portfolio items from major courses individual faculty assessment annually with and culminating project. administration. Needed changes are documented & action taken by appropriate persons. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Fall, 2015 Master of Divinity --Distance Education Yes TMS website; AIMS; TMS Catalog Entrance/exit exams; alumni survey; Corporate faculty review of assessment data; portfolio items from major courses individual faculty assessment annually with and culminating project. administration. Needed changes are documented & action taken by appropriate persons. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. New program, implemented Fall, 2015 Master of Theology Yes TMS website; AIMS; TMS Catalog Portfolio items from selected courses; Thesis (which is made available via TREN); alumni survey, Needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2017 Doctor of Philosophy Yes Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Spring, 2009; Program revised 201415;Scheduled for Spring 2019 TMUS 2017 IEEI Worksheet Corporate faculty review of assessment data; individual faculty assessment annually with administration; Thesis assessment. Needed changes are documented & action taken by appropriate persons. TMS Website; Qualifying exams; Colloquiums; Colloquium presentations are evaluated by entire PhD. Handbook; Dissertation (which is made available faculty; Disssertation is evaluated by major faculty & TMS Catalog via TREN). outside reader; Dissertation defense before entire faculty. pg. 6 Att. 2.13: Inventory of Educational Effectiveness Indicators for The Master's University & Seminary Category Doctor of Ministry Outcomes Developed Yes Where Published TMS Website; TMS Catalog; D.Min. Handbook Master of Biblical Program in TMS Website; Leadership Development TMS Catalog; (Spanish Program) Types of Evidence that Graduates Achieve Outcomes Who & How is Evidence Evaluated? How are the Findings Used? Last Program Review Pre/post program sermon eval/ D.Min. facultyPrograms review & evaluate assessment daty; University comparisons; Preaching Project or Preaching Research Project is evaluated by D.Min. Research in Preaching Project (which faculty and the candidate defends project before is made available via TREN). advisor, D.Min program director and at least one additional TMS faculty member. Plan that needed changes to the PLOs, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action. Fall, 2010; Program revised 20142015;Scheduled for Spring 2018 Entrance/exit exams; portfolio items Corporate faculty review of assessment data; from courses and culminating individual faculty assessment annually with project in capstone course. administration. Needed changes are documented and action taken by appropriate persons Needed changes to the PLO’s, assessment methods, curricula, and/or course content are assigned to appropriate person(s) for follow-up action Program Launch Fall 2017, pending WSCUC approval TMU AIMS = Assessment Information Management System TMUS 2017 IEEI Worksheet pg. 7