EY RV T SU EN AL ATI PIT P RE OS ON H LTU er CU Us TY 012 ive FE 2 SA at rt ar o mp Rep Co se ba ata D Agency for Healthcare Research and Quality Advancing Excellence in Health Care www.ahrq.gov PATIENT SAFETY The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services. Hospital Survey on Patient Safety Culture: 2012 User Comparative Database Report Prepared for: Agency for Healthcare Research and Quality U.S. Department of Health and Human Services 540 Gaither Road Rockville, MD 20850 www.ahrq.gov Contract No. HHSA 290200710024C Managed and prepared by: Westat, Rockville, MD Joann Sorra, Ph.D. Theresa Famolaro, M.P.S. Naomi Dyer, Ph.D. Dawn Nelson Scott Alan Smith, Ph.D. AHRQ Publication No. 12-0017 February 2012 This document is in the public domain and may be used and reprinted without permission except those copyrighted materials noted for which further reproduction is prohibited without specific permission of copyright holders. Suggested Citation: Sorra J, Famolaro T, Dyer N, et al. Hospital Survey on Patient Safety Culture 2012 user comparative database report. (Prepared by Westat, Rockville, MD, under Contract No. HHSA 290200710024C). Rockville, MD: Agency for Healthcare Research and Quality; February 2012. AHRQ Publication No. 12-0017. No investigators have any affiliations or financial involvement (e.g., employment, consultancies, honoraria, stock options, expert testimony, grants or patents received or pending, or royalties) that conflict with material presented in this report. ii Table of Contents Executive Summary .........................................................................................................................1 Survey Content...........................................................................................................................1 2012 Database Hospitals ............................................................................................................2 Survey Administration Statistics................................................................................................2 Characteristics of Participating Hospitals ..................................................................................2 Characteristics of Respondents ..................................................................................................2 Areas of Strength for Most Hospitals ........................................................................................3 Areas With Potential for Improvement for Most Hospitals .......................................................3 Results by Hospital Characteristics ...........................................................................................4 Results by Respondent Characteristics ......................................................................................4 Trending: Comparing Results Over Time..................................................................................5 Trending Results by Hospital Characteristics ............................................................................6 Trending Results by Respondent Characteristics ......................................................................6 Action Planning for Improvement .............................................................................................7 Purpose and Use of This Report ......................................................................................................8 Chapter 1. Introduction ....................................................................................................................9 Survey Content...........................................................................................................................9 2012 User Comparative Database and Report .........................................................................10 Chapter 2. Survey Administration Statistics ..................................................................................12 Chapter 3. Characteristics of Participating Hospitals ....................................................................15 Bed Size ...................................................................................................................................15 Teaching Status ........................................................................................................................16 Ownership and Control ............................................................................................................16 Geographic Region ..................................................................................................................17 Chapter 4. Characteristics of Respondents ....................................................................................18 Work Area/Unit .......................................................................................................................18 Staff Position............................................................................................................................19 Interaction With Patients..........................................................................................................20 Chapter 5. Overall Results .............................................................................................................21 Chapter 6. Comparing Your Results ..............................................................................................30 Description of Comparative Statistics .....................................................................................30 Composite and Item-Level Comparative Tables .....................................................................33 Appendixes A and B: Overall Results by Hospital and Respondent Characteristics ..............40 Chapter 7. Trending: Comparing Results Over Time ....................................................................43 Description of Trending Statistics ...........................................................................................44 Composite and Item-Level Trending Results ..........................................................................45 Bar Charts of Trending Results................................................................................................52 Appendixes C and D: Trending Results by Hospital and Respondent Characteristics............56 Chapter 8. What's Next? Action Planning for Improvement ........................................................59 Seven Steps of Action Planning ...............................................................................................59 References ......................................................................................................................................64 Notes: Description of Data Cleaning and Calculations .................................................................65 Data Cleaning...........................................................................................................................65 Response Rates ........................................................................................................................65 Calculation of Percent Positive Scores ....................................................................................66 Item and Composite Percent Positive Scores...........................................................................67 Minimum Number of Responses .............................................................................................68 Percentiles ................................................................................................................................68 iii List of Tables Table 1-1. Patient Safety Culture Composites and Definitions ........................................................ 9 Table 2-1a. Overall Statistics for the 2012 Database Participating Hospitals ................................ 13 Table 2-1b. Statistics for Nontrending and Trending Hospitals in 2012 Database ........................ 13 Table 2-2. Summary Statistics for 2012 Database Participating Hospitals .................................... 13 Table 2-3. Survey Administration Statistics ................................................................................... 13 Table 2-4. Average Hospital Response Rate by Mode ................................................................... 14 Table 2-5. Types of Staff or Work Areas/Units Surveyed .............................................................. 14 Table 3-1. Distribution of Database Hospitals and Respondents by Bed Size Compared With AHA-Registered Hospitals .............................................................................................. 16 Table 3-2. Distribution of Database Hospitals and Respondents by Teaching Status Compared With AHA-Registered Hospitals ............................................................................. 16 Table 3-3. Distribution of Database Hospitals and Respondents by Ownership and Control Compared With AHA-Registered Hospitals ............................................................... 17 Table 3-4. Distribution of Database Hospitals and Respondents by Geographic Region Compared With AHA-Registered Hospitals ............................................................................. 17 Table 4-1. Distribution of Database Respondents by Work Area/Unit .......................................... 19 Table 4-2. Distribution of Database Respondents by Staff Position .............................................. 20 Table 4-3. Distribution of Database Respondents by Interaction With Patients ............................ 20 Table 6-1. Interpretation of Percentile Scores ................................................................................ 32 Table 6-2. Sample Percentile Statistics ........................................................................................... 33 Table 6-3. Composite-Level Comparative Results for the 2012 Database ..................................... 34 Table 6-4. Item-Level Comparative Results for the 2012 Database ................................................... Table 6-5. Average Distribution of Work Area/Unit Patient Safety Grades--2012 Database Comparative Results ................................................................................................. 39 Table 6-6. Average Distribution of Number of Events Reported in the Past 12 Months-- 2012 Database Comparative Results ........................................................................................ 39 Table 7-1. Summary Statistics for Most Recent and Previous Data Submissions From the 650 Trending Hospitals ............................................................................................................. 44 Table 7-2a. Example of Trending Statistics .................................................................................... 44 Table 7-2b. Example of Other Trending Statistics ......................................................................... 45 Table 7-3. Trending: Composite-Level Results .............................................................................. 46 Table 7-4. Trending: Item-Level Results ........................................................................................ 47 Table 7-5. Trending: Distribution of Work Area/Unit Patient Safety Grades ................................ 51 Table 7-6. Trending: Distribution of Number of Events Reported in the Past 12 Months ............. 51 Notes Table 1. Example of Computing Item and Composite Percent Positive Scores Table 2. Data Table for Example of How To Compute Percentiles iv List of Charts Chart 5-1. Composite-Level Average Percent Positive Response--Across All 2012 Database Hospitals .................................................................................................................... 24 Chart 5-2. Item-Level Average Percent Positive Response--Across All 2012 Database Hospitals ................................................................................................................................... 25 Chart 5-3. Average Percentage of Respondents Giving Their Work Area/Unit a Patient Safety Grade--Across All 2012 Database Hospitals................................................................ 29 Chart 5-4. Average Percentage of Respondents Reporting Events in the Past 12 Months-- Across All 2012 Database Hospitals......................................................................................... 29 Chart 7-1. Trending: Percentage of Hospitals That Increased, Decreased, or Did Not Change on Each Composite ...................................................................................................... 53 Chart 7-2. Trending: Percentage of Hospitals That Increased, Decreased, or Did Not Change on Work Area/Unit Patient Safety Grade .................................................................... 54 Chart 7-3. Trending: Percentage of Hospitals That Increased, Decreased, or Did Not Change on Number of Events Reported ................................................................................... 54 Chart 7-4. Trending: Distribution of Hospitals by Number of Composites That Increased, Decreased, or Did Not Change by 5 Percentage Points or More .............................................. 55 Chart 8-1. Plan-Do-Study Cycle Appendixes cited in this report are provided electronically at: www.ahrq.gov/qual/hospsurvey12/. v Executive Summary In response to requests from hospitals interested in comparing their results with those of other hospitals on the Hospital Survey on Patient Safety Culture, the Agency for Healthcare Research and Quality (AHRQ) established the Hospital Survey on Patient Safety Culture comparative database. Since the first annual user comparative database report, which was released in 2007 and included data from 382 U.S. hospitals, the number of hospitals and respondents included in the database has grown each year. The 2012 user comparative database report displays results from 1,128 hospitals and 567,703 hospital staff respondents. This report also includes a chapter on trending that presents results showing change over time for 650 hospitals that administered the survey and submitted data more than once. Hospitals do not necessarily administer the hospital patient safety culture survey every year. They may administer it on an 18-month, 24-month, or other cycle. Therefore, the comparative database is a "rolling" indicator. It retains data for up to 3.5 prior years when a hospital does not have new data to submit, replaces older data with more recent data when available, and adds data from hospitals submitting for the first time. This user comparative database report was developed as a tool for the following purposes: Comparison--To allow hospitals to compare their patient safety culture survey results with those of other hospitals. Assessment and Learning--To provide data to hospitals to facilitate internal assessment and learning in the patient safety improvement process. Supplemental Information--To provide supplemental information to help hospitals identify their strengths and areas with potential for improvement in patient safety culture. Trending--To provide data that describe changes in patient safety culture over time. Survey Content The hospital survey, released in November 2004, was designed to assess hospital staff opinions about patient safety issues, medical errors, and event reporting. The survey includes 42 items that measure 12 areas, or composites, of patient safety culture: 1. Communication openness. 2. Feedback and communication about error. 3. Frequency of events reported. 4. Handoffs and transitions. 5. Management support for patient safety. 6. Nonpunitive response to error. 7. Organizational learning--continuous improvement. 8. Overall perceptions of patient safety. 9. Staffing. 10. Supervisor/manager expectations and actions promoting safety. 1 11. Teamwork across units. 12. Teamwork within units. The survey also includes two questions that ask respondents to provide an overall grade on patient safety for their work area/unit and to indicate the number of events they reported over the past 12 months. 2012 Database Hospitals The 1,128 hospitals in the 2012 database fall into two categories: 508 hospitals from the previous database report that are still included in the 2012 report. 620 hospitals that submitted data for the 2012 report. Survey Administration Statistics The average hospital response rate was 53 percent, with an average of 503 completed surveys per hospital. Most hospitals (66 percent) administered Web surveys. Hospitals administering a Web survey had, on average, lower response rates (51 percent) compared with response rates from paper (61 percent), yet higher response rates compared with mixed-mode surveys (49 percent). Most hospitals (90 percent) administered the survey to all staff or a sample of all staff. Characteristics of Participating Hospitals Database hospitals represent a range of bed sizes and geographic regions. Most database hospitals are nonteaching (66 percent) and non-government owned (80 percent). Overall, the characteristics of the 1,128 database hospitals are fairly consistent with the distribution of U.S. hospitals registered with the American Hospital Association (AHA). Characteristics of Respondents There were 567,703 hospital staff respondents from 1,128 hospitals. The top three work areas of respondents were: o Other (30 percent).i o Medicine (12 percent). o Surgery (10 percent). i Many respondents chose "Other," which allowed them to note their specific work area or unit. However, this information was not collected from the hospitals. 2 The top three staff positions of respondents were: o Registered nurse or licensed vocational nurse/licensed practical nurse (35 percent). o Other (21 percent).ii o Technician (e.g., EKG, Lab, Radiology) (11 percent). Most respondents (76 percent) indicated that they had direct interaction with patients. Areas of Strength for Most Hospitals Four areas of strength emerged. Percent positive is the percentage of positive responses (e.g., Agree, Strongly agree) to positively worded items (e.g., "People support one another in this unit") or negative responses (e.g., Disagree, Strongly disagree) to negatively worded items (e.g., "We have safety problems in this unit"). The four areas with the highest percent positive responses were: 1. Teamwork Within Units (average 80 percent positive response)--the extent to which staff support each other, treat each other with respect, and work together as a team. 2. Supervisor/Manager Expectations and Actions Promoting Patient Safety (average 75 percent positive response)--the extent to which supervisors/managers consider staff suggestions for improving patient safety, praise staff for following patient safety procedures, and do not overlook patient safety problems. 3. Organizational Learning--Continuous Improvement (average 72 percent positive response)--the extent to which mistakes have led to positive changes and changes are evaluated for effectiveness. 4. Management Support for Patient Safety (average 72 percent positive response)--the extent to which hospital management provides a work climate that promotes patient safety and shows that patient safety is a top priority. Areas With Potential for Improvement for Most Hospitals Three areas showed potential for improvement: 1. Nonpunitive Response to Error (average 44 percent positive response)--the extent to which staff feel that their mistakes and event reports are not held against them and that mistakes are not kept in their personnel file. 2. Handoffs and Transitions (average 45 percent positive response)--the extent to which important patient care information is transferred across hospital units and during shift changes. 3. Staffing (average 56 percent positive response)--the extent to which there are enough staff to handle the workload and work hours are appropriate to provide the best care for patients. ii Many respondents chose "Other," which allowed them to specify their position. However, this information was not collected from the hospitals. 3 Results by Hospital Characteristics Bed Size The smallest hospitals (6-24 beds) had the highest percent positive average across all patient safety culture composites (68 percent); larger hospitals (400 beds or more) had the lowest (60 percent). Smaller hospitals (49 beds or less) had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (80 percent); larger hospitals (400 beds or more) had the lowest (71 percent). Teaching Status and Ownership and Control Nonteaching hospitals on average scored higher than teaching hospitals by 5 percentage points on Teamwork Across Units (60 percent positive compared with 55 percent positive) and Handoffs and Transitions (47 percent positive compared with 42 percent). Non-government-owned hospitals reported more events (47 percent) than governmentowned hospitals (41 percent). Geographic Region East South Central,iii West South Central, and South Atlantic/Associated Territories hospitals had the highest average percent positive response across all composites (65 percent positive); New England hospitals had the lowest (60 percent positive). West North Central hospitals had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (78 percent); New England hospitals had the lowest (69 percent). Pacific/Associated Territories hospitals had the highest percentage of respondents who reported one or more events in the past year (49 percent); the lowest percentage of respondents reporting events was in the West South Central region (41 percent). Results by Respondent Characteristics Work Area/Unit Respondents in Rehabilitation had the highest average percent positive response across the composites (69 percent positive); Emergency had the lowest (57 percent positive). iii States and territories are categorized into AHA-defined regions as follows: New England: CT, MA, ME, NH, RI, VT West North Central: IA, KS, MN, MO, ND, NE, SD Mid-Atlantic: NJ, NY, PA West South Central: AR, LA, OK, TX South Atlantic/Associated Territories: DC, DE, FL, GA, MD, NC, SC, VA, WV, Puerto Mountain: AZ, CO, ID, MT, NM, NV, UT, WY Rico, Virgin Islands Pacific/Associated Territories: AK, CA, HI, OR, East North Central: IL, IN, MI, OH, WI WA, American Samoa, Guam, Marshall Islands, Northern Mariana Islands East South Central: AL, KY, MS, TN 4 Rehabilitation had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (85 percent); Emergency had the lowest (64 percent). ICU (Any Type) had the highest percentage of respondents reporting one or more events in the past year (64 percent); Rehabilitation had the lowest (39 percent). Staff Position Respondents in Administration/Management had the highest average percent positive response across the composites (74 percent positive); Pharmacists had the lowest (60 percent positive). Administration/Management had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (86 percent); Pharmacists had the lowest (68 percent). Pharmacists had the highest percentage of respondents reporting one or more events in the past year (71 percent); Unit Assistants/Clerks/Secretaries had the lowest (17 percent). Interaction With Patients Respondents with direct patient interaction were more positive on Handoffs and Transitions compared with those without direct patient interaction (47 percent positive compared with 39 percent). Respondents without direct patient interaction were more positive than those with direct patient interaction on Management Support for Patient Safety (77 percent positive compared with 71 percent). Respondents without direct patient interaction had a higher percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (80 percent) than respondents with direct patient interaction (75 percent). More respondents with direct patient interaction reported one or more events in the past year (50 percent) than respondents without direct patient interaction (30 percent). Trending: Comparing Results Over Time Results regarding changes over time on the patient safety culture composites, patient safety grade, and number of events reported for the 650 hospitals (of the 1,128 total database hospitals) that administered the survey and submitted data more than once are highlighted. Trending Hospitals For the 650 hospitals with trending data, the average length of time between previous and most recent survey administrations was 20 months (range: 6 months to 66 months). The distribution of the 650 trending hospitals by bed size, teaching status, and ownership and control is similar to the distribution of the 1,128 database hospitals. Trending: Overall Summary Statistics The average percent positive scores on the patient safety culture composites increased by 1 percentage point (ranging from 0 to 2 percentage points). 5 The average percentage of respondents who gave their work area/unit a patient safety grade of "A-Excellent" or "B-Very Good" increased by 1 percentage point. The average number of respondents reporting one or more events decreased by 1 percentage point. Additional Trending Statistics The charts in Chapter 7 provide results for two additional ways of summarizing changes in patient safety composite scores over time. The first series of charts displays the number of hospitals that increased, decreased, or did not change by 5 percentage points or more for each composite, patient safety grade, and number of events reported. The second set of charts displays the distribution of trending hospitals by number of composites that increased, decreased, or changed less than 5 percentage points. Trending Results by Hospital Characteristics Trending: Bed Size Hospitals with 50-99 beds had the greatest increases in percent positive response over time on all 12 composites (an average increase of 2 percentage points). Hospitals with 50-99 beds had the greatest increase in the percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (a 3 percentage point increase, from 75 percent to 78 percent). Trending: Teaching Status and Ownership and Control Nonteaching hospitals showed increases up to 2 percentage points on all 12 patient safety composites; teaching hospitals showed increases up to 1 percentage point on half of the composites and decreases of 1 percentage point on Supervisor/Manager Expectations. Government-owned hospitals showed increases up to 2 percentage points across 11 composites; non-government-owned hospitals showed increases up to 2 percentage points on 9 composites. Trending: Region West North Central hospitals had the greatest increases in percent positive response over time on 6 of the 12 composites (average increase of 2 percentage points). West North Central hospitals had the greatest increase in the percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (a 3 percentage point increase, from 75 percent to 78 percent). Trending Results by Respondent Characteristics Trending: Work Area/Unit Rehabilitation had the greatest increase in percent positive response on all 12 patient safety culture composites (average increases of 3 percentage points). Emergency, Radiology, and Rehabilitation had the greatest increases over time in the average percentage of respondents giving their work area/unit a patient safety grade of 6 "Excellent" or "Very Good" (3 percentage point increases, from 62 percent to 65 percent, 79 percent to 82 percent, and 82 percent to 85 percent, respectively). Anesthesiology and Lab had the greatest increases in the average percentage of respondents reporting one or more events in the past year (3 percentage point increases). The largest decrease was in Psychiatry/Mental Health (a 4 percentage point decrease). Trending: Staff Position Patient Care Asst./Aide/Care Partner had the greatest increase in positive response over time on 4 of the 12 patient safety composites (average increase of 2 percentage points). Pharmacists had the greatest increase over time in the average percentage of respondents giving their work area/unit a patient safety grade of "Excellent" or "Very Good" (a 3 percentage point increase). Dietitians had the greatest decrease over time in the average percentage of respondents reporting one or more events in the past year (an 11 percentage point decrease). Trending: Interaction With Patients Respondents with direct interaction with patients showed an increase of 1 percentage point across 11 patient safety culture composites; respondents without direct interaction showed an increase of 1 percentage point across 10 composites. Action Planning for Improvement The delivery of survey results is not the end point in the survey process; it is just the beginning. Often, the perceived failure of surveys to create lasting change is actually due to faulty or nonexistent action planning or survey followup. Seven steps of action planning are provided to give hospitals guidance on next steps to take to turn their survey results into actual patient safety culture improvement: 1. 2. 3. 4. 5. 6. 7. Understand your survey results. Communicate and discuss the survey results. Develop focused action plans. Communicate action plans and deliverables. Implement action plans. Track progress and evaluate impact. Share what works. 7 Purpose and Use of This Report In response to requests from hospitals interested in comparing their results with those of other hospitals on the Hospital Survey on Patient Safety Culture, the Agency for Healthcare Research and Quality (AHRQ) established the Hospital Survey on Patient Safety Culture comparative database. Since the first annual comparative database report, which was released in 2007 and included data from 382 U.S. hospitals, the number of hospitals and respondents contributing to the database report has grown each year. The Hospital Survey on Patient Safety Culture 2012 User Comparative Database Report consists of data from 1,128 hospitals and 567,703 hospital staff respondents who completed the survey. The 1,128 hospitals in the 2012 report fall into two categories: 508 hospitals from the previous database report that are still included in the 2012 report. 620 hospitals that submitted data for the 2012 report. Hospitals do not necessarily administer the hospital patient safety culture survey every year. They may administer it on an 18-month, 24-month, or other cycle. Therefore, the comparative database is a "rolling" indicator. It retains data for up to 3.5 prior years when a hospital does not have new data to submit, replaces older data with more recent data when available, and adds data from hospitals submitting for the first time. This user comparative database report was developed as a tool for the following purposes: Comparison--To allow hospitals to compare their patient safety culture survey results with those of other hospitals. Assessment and Learning--To provide data to hospitals to facilitate internal assessment and learning in the patient safety improvement process. Supplemental Information--To provide supplemental information to help hospitals identify their strengths and areas with potential for improvement in patient safety culture. Trending--To provide data that describe changes in patient safety culture over time. The report presents statistics (averages, standard deviations, minimum and maximum scores, and percentiles) on the patient safety culture composites and items from the survey. This 2012 report also includes a trending chapter that describes patient safety culture change over time for 650 hospitals with data from two administrations of the survey. Appendixes A and B present overall results by hospital characteristics (bed size, teaching status, ownership and control, geographic region) and respondent characteristics (hospital work area/unit, staff position, interaction with patients). Appendixes C and D show trend results for the 650 trending hospitals, broken down by hospital characteristics (bed size, teaching status, ownership and control, and geographic region) in Appendix C and respondent characteristics (hospital work area/unit, staff position, interaction with patients) in Appendix D. 8 Chapter 1. Introduction Patient safety is a critical component of health care quality. As health care organizations continually strive to improve, there is growing recognition of the importance of establishing a culture of patient safety. Achieving a culture of patient safety requires an understanding of the values, beliefs, and norms about what is important in an organization and what attitudes and behaviors related to patient safety are supported, rewarded, and expected. Survey Content Recognizing the need for a measurement tool to assess the culture of patient safety in health care organizations, the Medical Errors Workgroup of the Quality Interagency Coordination Task Force (QuIC) sponsored the development of a hospital survey focusing on patient safety culture. The Agency for Healthcare Research and Quality (AHRQ) funded and supervised development of the Hospital Survey on Patient Safety Culture. Developers reviewed research pertaining to safety, patient safety, error and accidents, and error reporting. They also examined existing published and unpublished safety culture assessment tools. In addition, hospital employees and administrators were interviewed to identify key patient safety and error-reporting issues. The Hospital Survey on Patient Safety Culture, released by AHRQ in November 2004, was designed to assess hospital staff opinions about patient safety issues, medical errors, and event reporting. The survey includes 42 items that measure 12 areas, or composites, of patient safety culture. Each of the 12 patient safety culture composites is listed and defined in Table 1-1. Table 1-1. Patient Safety Culture Composites and Definitions Definition: The extent to which... Patient Safety Culture Composite 1. Communication openness 2. Feedback and communication about error 3. Frequency of events reported 4. Handoffs and transitions 5. Management support for patient safety 6. Nonpunitive response to error Staff freely speak up if they see something that may negatively affect a patient and feel free to question those with more authority Staff are informed about errors that happen, given feedback about changes implemented, and discuss ways to prevent errors Mistakes of the following types are reported: (1) mistakes caught and corrected before affecting the patient, (2) mistakes with no potential to harm the patient, and (3) mistakes that could harm the patient but do not Important patient care information is transferred across hospital units and during shift changes Hospital management provides a work climate that promotes patient safety and shows that patient safety is a top priority Staff feel that their mistakes and event reports are not held against them and that mistakes are not kept in their personnel file 9 Table 1-1. Patient Safety Culture Composites and Definitions (continued) Definition: The extent to which... Patient Safety Culture Composite 7. Organizational learning--Continuous improvement 8. Overall perceptions of patient safety 9. Staffing 10. Supervisor/manager expectations and actions promoting safety 11. Teamwork across units 12. Teamwork within units Mistakes have led to positive changes and changes are evaluated for effectiveness Procedures and systems are good at preventing errors and there is a lack of patient safety problems There are enough staff to handle the workload and work hours are appropriate to provide the best care for patients Supervisors/managers consider staff suggestions for improving patient safety, praise staff for following patient safety procedures, and do not overlook patient safety problems Hospital units cooperate and coordinate with one another to provide the best care for patients Staff support each other, treat each other with respect, and work together as a team The survey also includes two questions that ask respondents to provide an overall grade on patient safety for their work area/unit and to indicate the number of events they reported over the past 12 months. In addition, respondents are asked to provide limited background demographic information about themselves (their work area/unit, staff position, whether they have direct interaction with patients, etc.). The survey's toolkit materials are available at the AHRQ Web site (www.ahrq.gov/qual/patientsafetyculture/) and include the survey, survey items and dimensions, user's guide, feedback report template, information about the Microsoft Excel(TM) Data Entry and Analysis Tool, and the Hospital Patient Safety Improvement Resource List. The toolkit provides hospitals with the basic knowledge and tools needed to conduct a patient safety culture assessment and ideas regarding how to use the data. 2012 User Comparative Database and Report Since its release, the hospital survey has been widely implemented across the United States. Hospitals administering the survey have expressed interest in comparing their results with those of other hospitals as an additional source of information to help them identify areas of strength and areas for improvement. In response to these requests, AHRQ funded the Hospital Survey on Patient Safety Culture comparative database to enable hospitals to compare their survey results with those of other hospitals and to examine trends in patient safety culture over time. Hospitals interested in submitting to the database should go to the AHRQ Web site for more information (www.ahrq.gov/qual/hospsurveydb/y2dbsubmission.htm). Data Limitations The survey results presented in this report represent the largest compilation of hospital survey data currently available and therefore provide a useful reference for comparison. However, several limitations to these data should be kept in mind. 10 First, the hospitals that submitted data to the database are not a statistically selected sample of all U.S. hospitals, since only hospitals that administered the survey on their own and were willing to submit their data for inclusion in the database are represented. However, the characteristics of the database hospitals are fairly consistent with the distribution of hospitals registered with the American Hospital Association (AHA) and are described further in Chapter 3. Second, hospitals that administered the survey were not required to undergo any training and administered it in different ways. Some hospitals used a paper-only survey, others used Webonly surveys, and others used a combination of these two methods to collect the data. It is possible that these different modes could lead to differences in survey responses; further research is needed to determine whether and how different modes affect the results. In addition, some hospitals conducted a census, surveying all hospital staff, while others administered the survey to a sample of staff. When a sample was drawn, no data were obtained to determine the methodology used to draw the sample. Survey administration statistics that were obtained about the database hospitals, such as survey administration modes and response rates, are provided in Chapter 2. Finally, the data hospitals submitted have been cleaned for out-of-range values (e.g., invalid response values due to data entry errors) and blank records (where responses to all survey items were missing). In addition, some logic checks were made. Otherwise, data are presented as submitted. No additional attempts were made to verify or audit the accuracy of the data submitted. 11 Chapter 2. Survey Administration Statistics This chapter presents descriptive information regarding how the 2012 database hospitals conducted their survey administration. Highlights The 2012 database consists of data from 567,703 hospital staff respondents across 1,128 participating hospitals. The average hospital response rate was 53 percent, with an average of 503 completed surveys per hospital. Most hospitals (66 percent) administered Web surveys, which resulted in lower response rates (51 percent) compared with response rates from paper (61 percent) but higher response rates compared with mixed-mode surveys (49 percent). Most hospitals (90 percent) administered the survey to all staff or a sample of all staff from all hospital departments. The 2012 database consists of survey data from 1,128 hospitals with a total of 567,703 hospital staff respondents. Participating hospitals administered the hospital survey to their staff between January 2008 and June 2011 and voluntarily submitted their data for inclusion in the database. Hospitals do not necessarily administer the hospital patient safety culture survey every year. They may administer it on an 18-month, 24-month, or other cycle. Therefore, the comparative database is a "rolling" indicator. Data from prior years are retained in the database when a hospital does not have new data to submit; older data are replaced with more recent data when available; and data are added from hospitals submitting for the first time. In order to keep the database current, data more than 3.5 years old are removed. Thus, 129 hospitals that administered the survey prior to January 1, 2008, were dropped from the 2012 database. Overall statistics for the hospitals included in the 2012 database are shown in Table 2-1a according to when the data were submitted. The 2012 database includes 508 hospitals carried over from the 2011 report and new data submissions from 620 hospitals. As shown in Table 21b, the 2012 database includes 478 hospitals that submitted data to the database once and 650 trending hospitals that submitted data to the database more than once. 12 Table 2-1a. Overall Statistics for the 2012 Database Participating Hospitals Overall Statistic Retained From the 2011 Database Submitted for the 2012 Database Total 2012 Database 508 263,543 620 304,160 1,128 567,703 Number of hospitals Number of individual survey respondents Table 2-1b. Statistics for Nontrending and Trending Hospitals in 2012 Database Overall Statistic Nontrending (Submitted Once) Trending (Submitted More Than Once) Total 2012 Database 478 218,167 650 349,536 1,128 567,703 Number of hospitals Number of individual survey respondents Table 2-2 presents data on the number of surveys completed and administered, as well as response rate information. Table 2-2. Summary Statistics for 2012 Database Participating Hospitals Summary Statistic Average 12 8,725 1,182 53% Number of surveys administered per hospital Hospital response rate Maximum 503 Number of completed surveys per hospital Minimum 15 4% 13,000 100% Table 2-3 presents data on the type of survey administration mode (paper, Web, or mixed mode). Table 2-3. Survey Administration Statistics Survey Administration Mode Paper only Web only Both paper and Web TOTAL 2012 Database Hospitals Number Percent 2012 Database Respondents Number Percent 239 746 143 21% 66% 13% 55,194 424,366 88,143 10% 75% 16% 1,128 100% 567,703 101% Note: Percentages may not add to 100 due to rounding. 13 Table 2-4 shows average response rate by survey mode. Paper survey administration had a higher average response rate than Web or mixed mode. Table 2-4. Average Hospital Response Rate by Mode Survey Administration Mode Average Hospital Response Rate Paper only Web only 61% 51% Both Web and paper 49% Table 2-5 displays results for the types of staff and work areas/units surveyed within the hospitals. Table 2-5. Types of Staff or Work Areas/Units Surveyed Types of Staff or Work Areas/Units Surveyed All staff, or a sample of all staff, from all work areas/units Selected staff only Selected work areas/units only Selected staff and selected work areas/units TOTAL 2012 Database Hospitals 2012 Database Respondents Number Percent Number Percent 1,014 90% 533,915 94% 63 31 20 1,128 6% 3% 2% 101% 18,918 7,137 7,733 567,703 3% 1% 1% 99% Note: Percentages may not add to 100 due to rounding. 14 Chapter 3. Characteristics of Participating Hospitals This chapter presents information about the distribution of database hospitals by bed size, teaching status, ownership and control, and geographic region. Although the hospitals that voluntarily submitted data to the database do not constitute a statistically selected sample, the characteristics of these hospitals are fairly consistent with the distribution of hospitals registered with the American Hospital Association (AHA). The characteristics of database hospitals by bed size, teaching status, ownership and control, and geographic region are presented in the following tablesiv and are compared with the distribution of AHA-registered hospitals included in the 2010 AHA Annual Survey of Hospitals.v Highlights Database hospitals represent a range of bed sizes and geographic regions. Most database hospitals are nonteaching (66 percent) and non-government owned (voluntary/nonprofit or proprietary/investor owned) (80 percent). Overall, the characteristics of the 1,128 database hospitals are fairly consistent with the distribution of hospitals registered with the American Hospital Association. Bed Size Table 3-1 shows the distribution of database hospitals and respondents by hospital bed size. Overall, the distribution of database hospitals by bed size is similar to the distribution of AHAregistered U.S. hospitals. Most of the database hospitals (61 percent) have fewer than 200 beds, which is similar to but lower than the percentage of AHA-registered U.S. hospitals (74 percent). iv To ensure hospital confidentiality, at least 20 hospitals had to be in a particular breakout category before data would be displayed for that category. v Data for U.S. and U.S. territory AHA-registered hospitals were obtained from the 2006 or 2010 AHA Annual Survey of Hospitals Database, (C) 2010 Health Forum, LLC, an affiliate of the American Hospital Association. Hospitals not registered with AHA were asked to provide information on their hospital's characteristics such as bed size, teaching status, and ownership. 15 Table 3-1. Distribution of Database Hospitals and Respondents by Bed Size Compared With AHARegistered Hospitals Bed Size 6-24 beds 25-49 beds 50-99 beds 100-199 beds 200-299 beds 300-399 beds 400-499 beds 500 or more beds TOTAL AHA-Registered Hospitals Number Percent 657 10% 1,418 22% 1,347 21% 1,326 21% 709 11% 409 6% 218 3% 323 5% 6,407 99% 2012 Database Hospitals Number Percent 74 7% 165 15% 196 17% 250 22% 192 17% 94 8% 63 6% 94 8% 1,128 100% 2012 Database Respondents Number Percent 7,322 1% 22,687 4% 47,914 8% 94,361 17% 120,566 21% 72,147 13% 68,752 12% 133,954 24% 567,703 100% Note: Percentages may not add to 100 due to rounding. Teaching Status As shown in Table 3-2, similar to the distribution of AHA-registered hospitals, most database hospitals were nonteaching. However, there was a smaller percentage of nonteaching hospitals in the database (66 percent) compared with AHA-registered hospitals (76 percent). Table 3-2. Distribution of Database Hospitals and Respondents by Teaching Status Compared With AHA-Registered Hospitals Teaching Status Teaching Nonteaching TOTAL AHA-Registered Hospitals Number Percent 1,516 4,891 6,407 24% 76% 100% 2012 Database Hospitals Number Percent 386 742 1,128 34% 66% 100% 2012 Database Respondents Number Percent 322,030 245,673 567,703 57% 43% 100% Note: Percentages may not add to 100 due to rounding. Ownership and Control As shown in Table 3-3, most database hospitals were non-government owned (80 percent), which is similar to the distribution of AHA-registered U.S. hospitals. 16 Table 3-3. Distribution of Database Hospitals and Respondents by Ownership and Control Compared With AHA-Registered Hospitals Ownership and Control Government (Federal or nonFederal) Nongovernment (voluntary/nonprofit or proprietary/investor owned) TOTAL AHA-Registered Hospitals Number Percent 2012 Database Hospitals Number Percent 2012 Database Respondents Number Percent 1,645 26% 229 20% 109,656 19% 4,762 74% 899 80% 458,047 81% 6,407 100% 1,128 100% 567,703 100% Note: Percentages may not add to 100 due to rounding. Geographic Region Table 3-4 shows the distribution of database hospitals by AHA-defined geographic regions.vi The largest percentages of database hospitals are from the East North Central region (25 percent) and the South Atlantic/Associated Territories region (16 percent). Table 3-4. Distribution of Database Hospitals and Respondents by Geographic Region Compared With AHA-Registered Hospitals Region New England AHA-Registered Hospitals Number Percent 2012 Database Respondents Number Percent 9% 82 7% 70,275 12% 271 4% 30 3% 23,838 4% 1,016 16% 182 16% 101,597 18% East North Central 925 14% 279 25% 142,124 25% East South Central 533 8% 110 10% 36,841 6% West North Central 803 13% 142 13% 47,469 8% West South Central 1,089 17% 139 12% 56,740 10% 509 8% 73 6% 39,543 7% 659 6,407 10% 99% 91 1,128 8% 100% 49,276 567,703 9% 99% Mid-Atlantic South Atlantic/Associated Territories Mountain Pacific/Associated Territories TOTAL 602 2012 Database Hospitals Number Percent Note: Percentages may not add to 100 due to rounding. vi States and territories are categorized into AHA-defined regions as follows: New England: CT, MA, ME, NH, RI, VT West North Central: IA, KS, MN, MO, ND, NE, SD Mid-Atlantic: NJ, NY, PA West South Central: AR, LA, OK, TX South Atlantic/Associated Territories: DC, DE, FL, GA, MD, NC, SC, VA, WV, Puerto Rico, Mountain: AZ, CO, ID, MT, NM, NV, UT, WY Virgin Islands Pacific/Associated Territories: AK, CA, HI, East North Central: IL, IN, MI, OH, WI OR, WA, American Samoa, Guam, Marshall Islands, Northern Mariana Islands East South Central: AL, KY, MS, TN 17 Chapter 4. Characteristics of Respondents This chapter describes respondents within the participating hospitals. The data presented here are based on respondents' answers to survey questions about the hospital work area/unit where they spent most of their work time, their staff position, and their direct interaction with patients. Highlights There were 567,703 hospital staff respondents from 1,128 hospitals. The top three work areas of respondents were: o Other (30 percent). o Medicine (12 percent). o Surgery (10 percent). The top three staff positions of respondents were: o Registered nurse or licensed vocational nurse/licensed practical nurse (35 percent). o Other (21 percent). o Technician (e.g., EKG, Lab, Radiology) (11 percent). Most respondents (76 percent) indicated they had direct interaction with patients. Work Area/Unit Close to one-third of respondents (30 percent) selected "Other" as their work area, followed by "Medicine" (12 percent), and "Surgery" (10 percent) (Table 4-1). The Hospital Survey on Patient Safety Culture uses generic categories for hospital work areas and units. Therefore, a large percentage of respondents chose the "Other" response option, which allowed them to note their specific work area or unit. Participating hospitals were not asked to submit written or "other-specify" responses for any questions, so no data are available to further describe the respondents in the "Other" work area category. 18 Table 4-1. Distribution of Database Respondents by Work Area/Unit 2012 Database Respondents Number Percent Work Area/Unit Other 159,889 30% Medicine Surgery Many different hospital units/no specific unit Intensive care unit (any type) Radiology Emergency 62,688 52,072 44,024 36,402 30,215 30,111 12% 10% 8% 7% 6% 6% Laboratory Obstetrics Rehabilitation Pediatrics Pharmacy Psychiatry/mental health Anesthesiology 25,705 23,917 18,293 16,976 15,725 15,600 3,538 5% 4% 3% 3% 3% 3% 1% TOTAL Missing: Did not answer or were not asked the question Overall total 535,155 32,548 567,703 101% Note: Percentages may not add to 100 due to rounding. Staff Position More than one-third of respondents (35 percent) selected "Registered Nurse" or "Licensed Vocational Nurse/Licensed Practical Nurse (LVN/LPN)" as their staff position, followed by "Other" (21 percent) and "Technician (e.g., EKG, Lab, Radiology)" (11 percent), as shown in Table 4-2. As with the work area/unit question, many respondents chose the "Other" response option, which allowed them to note their specific staff position, but no data are available to further describe the respondents in the "Other" staff position category. 19 Table 4-2. Distribution of Database Respondents by Staff Position 2012 Database Respondents Number Percent Staff Position Registered nurse (RN) or licensed vocational nurse (LVN)/licensed practical nurse (LPN) Other Technician (EKG, Lab, Radiology) 191,402 35% 114,425 58,495 21% 11% Administration/management Unit assistant/clerk/secretary Attending/staff physician, resident physician/physician in training, or physician assistant (PA)/nurse practitioner (NP) Patient care assistant/hospital aide/care partner Therapist (respiratory, physical, occupational, or speech) Pharmacist Dietitian TOTAL 42,021 34,782 29,832 8% 6% 6% 29,726 25,160 10,339 3,332 539,514 6% 5% 2% 1% 101% Missing: Did not answer or were not asked the question Overall total 28,189 567,703 Note: Percentages may not add to 100 due to rounding. Interaction With Patients As shown in Table 4-3, most respondents (76 percent) indicated they had direct interaction with patients. Table 4-3. Distribution of Database Respondents by Interaction With Patients 2012 Database Respondents Number Percent Interaction With Patients YES, have direct patient interaction NO, do NOT have direct patient interaction TOTAL Missing: Did not answer or were not asked the question 407,884 130,135 538,019 29,684 Overall total 567,703 Note: Percentages may not add to 100 due to rounding. 20 76% 24% 100% Chapter 5. Overall Results This chapter presents the overall survey results for the database, showing the average percentage of positive responses across the database hospitals on each of the survey's items and composites. Reporting the average across hospitals ensures that each hospital receives an equal weight that contributes to the overall average. Reporting the data at the hospital level in this way is important because culture is considered to be a group characteristic and is not considered to be a solely individual characteristic. An alternative method would be to report a straight percentage of positive responses across all respondents, but this method would give greater weight to respondents from larger hospitals. (There are almost twice as many respondents from larger hospitals as from smaller hospitals.) Highlights Teamwork Within Units--This composite had the highest average percent positive response (80 percent), indicating it is a strength for most hospitals. Supervisor/Manager Expectations and Actions Promoting Patient Safety--This composite had the second highest average percent positive response (75 percent), indicating it is a strength for most hospitals. Nonpunitive Response to Error--This composite had the lowest average percent positive response (44 percent), indicating it is an area with potential for improvement for most hospitals. Handoffs and Transitions--This composite had the second lowest average percent positive response (45 percent), indicating it is an area with potential for improvement for most hospitals. On average, most respondents within hospitals (75 percent) gave their work area or unit a grade of "A-Excellent" (30 percent) or "B-Very Good" (45 percent) on patient safety; this was identified as an area of strength for most hospitals. On average, most respondents within hospitals (55 percent) reported no events in their hospital over the past 12 months. It is likely that this represents underreporting of events and was identified as an area for improvement for most hospitals. This section provides the overall item and composite-level results. The method for calculating the percent positive scores at the item and composite level is described in the Notes section of this document. 21 Composite-Level Resultsvii Chart 5-1 shows the average percent positive response for each of the 12 patient safety culture composites across hospitals in the database. The patient safety culture composites are shown in order from the highest average percent positive response to the lowest. Areas of Strength Teamwork Within Units--the extent to which staff support one another, treat each other with respect, and work together as a team. This patient safety culture composite had the highest average percent positive response (80 percent), indicating it is an area of strength across the database hospitals. Supervisor/Manager Expectations and Actions Promoting Patient Safety--the extent to which supervisors/managers consider staff suggestions for improving patient safety, praise staff for following patient safety procedures, and do not overlook patient safety problems. This patient safety culture composite had the second highest average percent positive response (75 percent). Organizational Learning--Continuous Improvement--the extent to which mistakes have led to positive changes and changes are evaluated for effectiveness. This patient safety culture composite tied for the third highest average percent positive response (72 percent). Management Support for Patient Safety--the extent to which hospital management provides a work climate that promotes patient safety and shows that patient safety is a top priority. This patient safety culture composite tied for the third highest average percent positive response (72 percent). Areas With Potential for Improvement Nonpunitive Response to Error--the extent to which staff feel that event reports and their own mistakes are not held against them and that mistakes are not kept in their personnel file. This patient safety culture composite had the lowest average percent positive response (44 percent), indicating it is an area with potential for improvement across the database hospitals. Handoffs and Transitions--the extent to which important patient care information is transferred across hospital units and during shift changes. This patient safety culture composite had the second lowest average percent positive response (45 percent). Staffing--the extent to which there are enough staff to handle the workload and work hours are appropriate to provide the best care for patients. This patient safety culture composite had the third lowest average percent positive response (56 percent). Item-Level Results Chart 5-2 shows the average percent positive response for each of the 42 survey items. The survey items are grouped by the patient safety culture composite they are intended to measure. Within each composite, the items are presented in the order in which they appear in the survey. vii Some hospitals excluded one or more survey items and are therefore excluded from composite-level calculations when the omitted items pertain to a particular composite. For the 2012 report, 44 hospitals were excluded from one or more composite-level calculations for this reason. 22 Areas of Strength The survey items with the highest average percent positive response (86 percent) were from the patient safety culture composite Teamwork Within Units: "People support one another in this unit" and "When a lot of work needs to be done quickly, we work together as a team to get the work done." Area With Potential for Improvement The survey item with the lowest average percent positive response (35 percent) was from the patient safety culture composite Nonpunitive Response to Error: "Staff worry that mistakes they make are kept in their personnel file." (In other words, an average of only 35 percent of respondents in each hospital Strongly disagreed or Disagreed with this negatively worded item.) Patient Safety Grade--Chart 5-3 shows the results from the item that asked respondents to give their hospital work area/unit an overall grade on patient safety. On average across hospitals, most respondents were positive, with 75 percent giving their work area or unit a patient safety grade of "A-Excellent" (30 percent) or "B-Very Good" (45 percent). Number of Events Reported--Chart 5-4 shows the results from the item that asked respondents to indicate the number of events they reported over the past 12 months. On average across hospitals, most respondents (55 percent) reported no events in their hospital over the past 12 months. Event reporting was identified as an area for improvement for most hospitals because underreporting of events means potential patient safety problems may not be recognized or identified and therefore may not be addressed. 23 Chart 5-1. Composite-Level Average Percent Positive Response, Across All 2012 Database Hospitals 24 Chart 5-2. Item-Level Average Percent Positive Response, Across All 2012 Database Hospitals (Page 1 of 4) Note: The item's survey location is shown to the right in parentheses. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). 25 Chart 5-2. Item-Level Average Percent Positive Response, Across All 2012 Database Hospitals (Page 2 of 4) Note: The item's survey location is shown to the right in parentheses. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). 26 Chart 5-2. Item-Level Average Percent Positive Response, Across All 2012 Database Hospitals (Page 3 of 4) Note: The item's survey location is shown to the right in parentheses. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item) 27 Chart 5-2. Item-Level Average Percent Positive Response, Across All 2012 Database Hospitals (Page 4 of 4) Note: The item's survey location is shown to the right in parentheses. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). 28 Chart 5-3. Average Percentage of Respondents Giving Their Work Area/Unit a Patient Safety Grade, Across All 2012 Database Hospitals 100% Percent 80% 60% 45% 40% 30% 20% 20% 4% 1% 0% A = Excellent B = Very Good C = Acceptable D = Poor E = Failing Overall Patient Safety Grade Note: Percentages may not add to 100 due to rounding. Chart 5-4. Average Percentage of Respondents Reporting Events in the Past 12 Months, Across All 2012 Database Hospitals 100% Percent 80% 60% 55% 40% 27% 20% 12% 4% 2% 1% 6 to 10 11 to 20 21 or more 0% None 1 to 2 3 to 5 Number of Events Reported Note: Percentages may not add to 100 due to rounding. 29 Chapter 6. Comparing Your Results To compare your hospital's survey results with the results from the database, you will need to calculate your hospital's percent positive response on the survey's 42 items and 12 composites (plus the two questions on patient safety grade and number of events reported). Refer to the Notes section at the end of this report for a description of how to calculate these percent positive scores. You will then be able to compare your hospital's results with the database averages and examine the percentile scores to place your hospital's results relative to the distribution of database hospitals. When comparing your hospital's results with results from the database, keep in mind that the database provides only relative comparisons. Even though your hospital's survey results may be better than the database statistics, you may still believe there is room for improvement in a particular area within your hospital in an absolute sense. As you will notice from the database results, there are some patient safety composites that even the highest scoring hospitals could improve on. Therefore, the comparative data provided in this report should be used to supplement your hospital's own efforts toward identifying areas of strength and areas on which to focus patient safety culture improvement efforts. Highlights There was considerable variability in the range of hospital scores (lowest to highest) across the 12 patient safety culture composites. Patient safety grades also had a wide range of response. In at least one hospital, only 4 percent of the respondents provided their unit with a patient safety grade of "A-Excellent," yet at another hospital 73 percent provided their unit with a patient safety grade of "A-Excellent". The number of events reported showed a wide range of response as well. In at least one hospital, 91 percent of respondents had not reported a single event over the past 12 months, while another hospital had 7 percent of respondents report 21 events or more. Description of Comparative Statistics In addition to the average percent positive scores presented in Chapter 5, a number of other statistics are provided to facilitate comparisons with the database hospitals. A description of each statistic shown in this chapter is provided next. 30 Average Percent Positive The average percent positive scores for each of the 12 patient safety culture composites and for the survey's 42 items (plus the two questions on patient safety grade and number of events reported) are provided in the comparative results tables in this chapter. These average percent positive scores were calculated by averaging composite-level percent positive scores across all hospitals in the database, as well as averaging item-level percent positive scores across hospitals. Since the percent positive is displayed as an overall average, scores from each hospital are weighted equally in their contribution to the calculation of the average.viii Standard Deviation The standard deviation (s.d.), a measure of the spread or variability of hospital scores around the average, is also displayed. The standard deviation tells you the extent to which hospitals' scores differ from the average: If scores from all hospitals were exactly the same, then the average would represent all their scores perfectly and the standard deviation would be zero. If scores from all hospitals were very close to the average, then the standard deviation would be small and close to zero. If scores from many hospitals were very different from the average, then the standard deviation would be a large number. When the distribution of hospital scores follows a normal bell-shaped curve (where most of the scores fall in the middle of the distribution, with fewer scores at the lower and higher ends of the distribution), the average, plus or minus the standard deviation, will include about 68 percent of all hospital scores. For example, if an average percent positive score across the database hospitals were 70 percent with a standard deviation of 10 percent and scores were normally distributed, then about 68 percent of all the database hospitals would have scores between 60 and 80 percent. Statistically "significant" differences between scores. You may be interested in determining the statistical significance of differences between your scores and the averages in the database, or between scores in various breakout categories (hospital bed size, teaching status, etc.). Statistical significance is greatly influenced by sample size, so as the number of observations in comparison groups gets larger, small differences in scores will be statistically significant. While a 1 percent difference between percent positive scores might be "statistically" significant (that is, not due to chance), the difference is not likely to be meaningful or "practically" significant. viii As described in the Notes section, an alternative method would be to report a straight percentage of positive response across all respondents, but this method would give greater weight to respondents from larger hospitals since they account for approximately twice as many responses as those from smaller hospitals. 31 Keep in mind that statistically significant differences are not always important, and nonsignificant differences are not always trivial. Therefore, we recommend the following guideline: Use a 5 percentage point difference as a rule of thumb when comparing your hospital's results with the database averages. Your hospital's percent positive score should be at least 5 percentage points greater than the database average to be considered "better" and should be at least 5 percentage points less to be considered "lower" than the database average. A 5 percentage point difference is likely to be statistically significant for most hospitals given the number of responses per hospital and is also a meaningful difference to consider. Minimum and Maximum Scores The minimum (lowest) and maximum (highest) percent positive scores are presented for each composite and item. These scores provide information about the range of percent positive scores obtained by hospitals in the database and are actual scores from the lowest and highest scoring hospitals. When comparing with the minimum and maximum scores, keep in mind that these scores may represent hospitals that are extreme outliers (indicated by large differences between the minimum score and the 10th percentile score, or between the 90th percentile score and the maximum score). Percentiles The 10th, 25th, 50th (or median), 75th, and 90th percentile scores are displayed for the survey composites and items. Percentiles provide information about the distribution of hospital scores. To calculate percentile scores, all hospital percent positive scores were ranked in order from low to high. A specific percentile score shows the percentage of hospitals that scored at or below a particular score. For example, the 50th percentile, or median, is the percent positive score where 50 percent of the hospitals scored the same or lower and 50 percent of the hospitals scored higher. When the distribution of hospital scores follows a normal bell-shaped curve (where most of the scores fall in the middle of the distribution, with fewer scores at the lower and higher ends of the distribution), the 50th percentile, or median, will be very similar to the average score. Interpret the percentile scores as shown in Table 6-1. Table 6-1. Interpretation of Percentile Scores Percentile Score Interpretation 10 percentile Represents the lowest scoring hospitals. th 25 percentile Represents lower scoring hospitals. th 50 percentile (or median) Represents the middle of the distribution of hospitals. th 75 percentile Represents higher scoring hospitals. th 90 percentile Represents the highest scoring hospitals. 10% of the hospitals scored the same or lower. 90% of the hospitals scored higher. 25% of the hospitals scored the same or lower. 75% of the hospitals scored higher. 50% of the hospitals scored the same or lower. 50% of the hospitals scored higher. 75% of the hospitals scored the same or lower. 25% of the hospitals scored higher. 90% of the hospitals scored the same or lower. 10% of the hospitals scored higher. th 32 To compare with the database percentiles, compare your hospital's percent positive scores with the percentile scores for each composite and item. Look for the highest percentile where your hospital's score is higher than that percentile. For example: On survey item 1 in Table 6-2, the 75th percentile score is 49 percent positive, and the 90th percentile score is 62 percent positive. Table 6-2. Sample Percentile Statistics Survey Item % Positive Response Survey Item Item 1 Min 10th %ile 25th %ile Median/50th %ile 75th %ile 90th %ile Max 8% 10% 25% 35% 49% 62% 96% If your hospital's score is 55%, your score falls here: If your hospital's score is 65%, your score falls here: If your hospital's score is 55 percent positive, it falls above the 75th percentile (but below the 90th), meaning that your hospital scored higher than at least 75 percent of the hospitals in the database. If your hospital's score is 65 percent positive, it falls above the 90th percentile, meaning your hospital scored higher than at least 90 percent of the hospitals in the database. Composite and Item-Level Comparative Tables Table 6-3 presents comparative statistics (average percent positive and standard deviation, minimum and maximum scores, and percentiles) for each of the 12 patient safety culture composites. The patient safety culture composites are shown in order from the highest average percent positive response to the lowest. Table 6-4 presents comparative statistics for each of the 42 survey items. The survey items are grouped by the patient safety culture composite they are intended to measure. Within each composite, the items are presented in the order in which they appear in the survey. The comparative results in Tables 6-3 and 6-4 show considerable variability in the range of hospital scores (lowest to highest) across the 12 patient safety culture composites. The standard deviation around the average percent positive scores ranged from 5.82 percent to 10.86 percent on the composites and ranged from 5.57 percent to 12.71 percent on the items. Patient safety grades, shown in Table 6-5, had a wide range of response, from at least one hospital where few of the respondents (4 percent) provided their unit with a patient safety grade of "A-Excellent" to a hospital where 73 percent did. Number of events reported also had a wide range of response, as shown in Table 6-6, from a hospital where 91 percent of respondents had not reported a single event over the past 12 months to a hospital where 89 percent of all respondents reported one event or more. 33 34 72% 72% 66% 64% 3. Organizational Learning--Continuous Improvement 4. Management Support for Patient Safety 5. Overall Perceptions of Patient Safety 6. Feedback & Communication About Error 63% 75% 2. Supervisor/Manager Expectations & Actions Promoting Patient Safety 62% 58% 56% 45% 44% 7. Frequency of Events Reported 80% 1. Teamwork Within Units Patient Safety Culture Composites Average % Positive 8. Communication Openness 9. Teamwork Across Units 10. Staffing 11. Handoffs & Transitions 12. Nonpunitive Response to Error 8.43% 10.86% 9.24% 9.82% 6.46% 7.51% 7.83% 8.26% 9.17% 7.00% 6.40% 5.82% s.d. Table 6-3. Composite-Level Comparative Results for the 2012 Database 20% 14% 27% 31% 36% 25% 30% 30% 37% 46% 49% 51% Min 34% 33% 45% 47% 54% 54% 55% 56% 61% 63% 67% 73% 10th %ile 38% 38% 50% 52% 58% 58% 59% 60% 67% 68% 71% 76% 25th %ile 43% 44% 56% 58% 62% 63% 64% 66% 72% 72% 75% 80% Median/ 50th %ile 48% 52% 62% 64% 66% 68% 70% 71% 78% 77% 79% 84% 75th %ile 54% 60% 68% 72% 69% 73% 74% 76% 84% 81% 83% 87% 90th %ile Composite % Positive Response 75% 88% 85% 93% 83% 92% 89% 89% 95% 94% 95% 98% Max 35 4. When one area in this unit gets really busy, others help out. Supervisor/Manager Expectations & Actions Promoting Patient Safety 1. My supv/mgr says a good word when he/she sees a job done according to established patient safety procedures. 2. My supv/mgr seriously considers staff suggestions for improving patient safety. 3. Whenever pressure builds up, my supv/mgr wants us to work faster, even if it means taking shortcuts. 4. My supv/mgr overlooks patient safety problems that happen over and over. Organizational Learning--Continuous Improvement 1. We are actively doing things to improve patient safety. 2. Mistakes have led to positive changes here. A11 69% 64% 84% 76% 74% 76% 73% 69% 78% 86% 86% 8.91% 8.07% 6.41% 6.53% 9.13% 7.15% 7.71% 7.49% 7.19% 5.57% 5.75% s.d. 25% 38% 50% 50% 5% 47% 24% 41% 30% 54% 52% Min 58% 54% 75% 68% 64% 67% 64% 60% 69% 79% 79% 64% 58% 80% 72% 69% 71% 68% 65% 74% 83% 82% 69% 64% 84% 76% 74% 76% 74% 69% 79% 87% 86% 75% 69% 88% 81% 79% 81% 78% 74% 83% 90% 89% 80% 74% 91% 85% 84% 85% 83% 79% 87% 93% 93% Survey Item % Positive Response Median/ 10th 25th 50th 75th 90th %ile %ile %ile %ile %ile 95% 89% 100% 95% 97% 100% 100% 100% 98% 100% 100% Max Note: The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). A13 A9 A6 3. B4R B3R B2 B1 2. 3. After we make changes to improve patient safety, we evaluate their effectiveness. 2. When a lot of work needs to be done quickly, we work together as a team to get the work done. 3. In this unit, people treat each other with respect. A3 A4 Teamwork Within Units 1. People support one another in this unit. Survey Items by Composite 1. A1 Item Average % Positive Table 6-4. Item-Level Comparative Results for the 2012 Database (Page 1 of 4) 36 Survey Items by Composite 4. Our procedures and systems are good at preventing errors from happening. Feedback & Communication About Error 1. We are given feedback about changes put into place based on event reports. 2. We are informed about errors that happen in this unit. 3. In this unit, we discuss ways to prevent errors from happening again. 8.67% 9.39% 9.41% 9.67% 61% 62% 64% 64% 72% 65% 56% 7.73% 8.66% 9.71% 8.41% 10.46% 75% 72% 9.65% 81% s.d. 43% 26% 6% 30% 23% 25% 27% 24% 38% 42% Min 62% 54% 45% 62% 52% 53% 51% 48% 62% 69% 66% 60% 50% 67% 58% 58% 56% 54% 69% 76% 72% 65% 56% 72% 64% 64% 62% 60% 75% 82% 77% 71% 63% 78% 70% 70% 68% 67% 81% 87% 81% 76% 69% 83% 76% 77% 74% 74% 87% 91% Survey Item % Positive Response Median/ 10th 25th 50th 75th 90th %ile %ile %ile %ile %ile 93% 93% 88% 97% 92% 94% 93% 92% 100% 100% Max Note: The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). C5 C3 6. C1 A18 Management Support for Patient Safety 1. Hospital mgmt provides a work climate that promotes patient safety. F8 2. The actions of hospital mgmt show that patient safety is a top priority. F9R 3. Hospital mgmt seems interested in patient safety only after an adverse event happens. 5. Overall Perceptions of Patient Safety A10R 1. It is just by chance that more serious mistakes don't happen around here. A15 2. Patient safety is never sacrificed to get more work done. A17R 3. We have patient safety problems in this unit. 4. F1 Item Average % Positive Table 6-4. Item-Level Comparative Results for the 2012 Database (Page 2 of 4) 37 Frequency of Events Reported 1. When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported? 2. When a mistake is made, but has no potential to harm the patient, how often is this reported? 3. When a mistake is made that could harm the patient, but does not, how often is this reported? Communication Openness 1. Staff will freely speak up if they see something that may negatively affect patient care. 2. Staff feel free to question the decisions or actions of those with more authority. 3. Staff are afraid to ask questions when something does not seem right. Teamwork Across Units 1. Hospital units do not coordinate well with each other. 2. There is good cooperation among hospital units that need to work together. 3. It is often unpleasant to work with staff from other hospital units. 4. Hospital units work well together to provide the best care for patients. Survey Items by Composite 68% 59% 60% 46% 63% 47% 75% 74% 59% 57% 10.17% 9.44% 10.51% 11.49% 7.33% 7.78% 6.61% 6.91% 8.42% 8.83% s.d. 19% 29% 21% 12% 35% 19% 47% 41% 15% 13% Min 56% 49% 47% 32% 54% 37% 67% 66% 49% 46% 62% 53% 53% 38% 58% 42% 71% 69% 53% 51% 68% 59% 59% 45% 63% 47% 75% 74% 59% 57% 75% 65% 66% 53% 67% 51% 80% 78% 65% 63% 82% 72% 74% 61% 71% 56% 84% 82% 70% 68% Survey Item % Positive Response Median/ 10th 25th 50th 75th 90th %ile %ile %ile %ile %ile 100% 93% 95% 93% 90% 75% 95% 100% 92% 87% Max Note: The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). F10 F6R F4 9. F2R C6R C4 8. C2 D3 D2 7. D1 Item Average % Positive Table 6-4. Item-Level Comparative Results for the 2012 Database (Page 3 of 4) 38 Staffing 1. We have enough staff to handle the workload. 2. Staff in this unit work longer hours than is best for patient care. 3. We use more agency/temporary staff than is best for patient care. 4. We work in -crisis mode? trying to do too much, too quickly. Handoffs & Transitions 1. Things -fall between the cracks? when transferring patients from one unit to another. 2. Important patient care information is often lost during shift changes. 3. Problems often occur in the exchange of information across hospital units. 4. Shift changes are problematic for patients in this hospital. Nonpunitive Response to Error 1. Staff feel like their mistakes are held against them. 2. When an event is reported, it feels like the person is being written up, not the problem. 3. Staff worry that mistakes they make are kept in their personnel file. Survey Items by Composite 35% 46% 50% 45% 44% 51% 41% 50% 68% 53% 56% 9.14% 8.84% 9.05% 11.79% 11.14% 10.34% 12.40% 11.02% 10.38% 9.73% 12.71% s.d. 9% 17% 15% 15% 6% 16% 8% 14% 0% 9% 13% Min 25% 36% 40% 32% 31% 39% 27% 37% 55% 40% 39% 29% 40% 44% 37% 36% 44% 32% 42% 62% 46% 47% 34% 46% 49% 44% 42% 50% 40% 49% 69% 53% 56% 40% 51% 55% 52% 50% 56% 48% 56% 75% 59% 64% 47% 58% 61% 61% 59% 64% 58% 65% 80% 65% 72% Survey Item % Positive Response Median/ 10th 25th 50th 75th 90th %ile %ile %ile %ile %ile 82% 80% 82% 92% 88% 89% 89% 88% 95% 85% 100% Max Note: The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). A16R A12R 12. A8R F11R F7R F5R 11. F3R A14R A7R A5R 10. A2 Item Average % Positive Table 6-4. Item-Level Comparative Results for the 2012 Database (Page 4 of 4) 39 Very Good Acceptable Poor Failing B C D E 1% 4% 20% 45% 30% Average % 1.02% 3.02% 7.00% 6.94% 9.52% s.d. 0% 0% 0% 12% 4% Min 0% 1% 12% 38% 19% 10th %ile 0% 2% 15% 42% 23% 0% 3% 20% 45% 29% 1% 5% 24% 49% 35% 2% 7% 29% 54% 42% 90th %ile 27% 12% 4% 2% 1% 1 to 2 events 3 to 5 events 6 to 10 events 11 to 20 events 21 events or more s.d. 1.04% 1.33% 2.44% 4.46% 6.21% 10.39% Note: Percentages for "Average %" may not add to 100 due to rounding. 55% Average % No events Number of Events Reported by Respondents 0% 0% 0% 0% 6% 11% Min 0% 0% 2% 6% 19% 42% 10th %ile 0% 1% 3% 9% 23% 48% 1% 1% 4% 11% 27% 55% 1% 2% 5% 14% 31% 62% Percentage of Responses Median/ 25th 50th 75th %ile %ile %ile Table 6-6. Average Distribution of Number of Events Reported in the Past 12 Months--2012 Database Comparative Results Note: Percentages for "Average %" may not add to 100 due to rounding. Excellent A Work Area/Unit Patient Safety Grade Percentage of Responses Median/ 50th 25th %ile %ile 75th %ile Table 6-5. Average Distribution of Work Area/Unit Patient Safety Grades--2012 Database Comparative Results 2% 3% 7% 17% 35% 68% 90th %ile 14% 35% 56% 80% 73% Max 7% 11% 19% 32% 51% 91% Max Appendixes A and B: Overall Results by Hospital and Respondent Characteristics In addition to the overall results on the database hospitals presented, Part II of the report presents data tables showing average percent positive scores on the survey composites and items across database hospitals, broken down by the following hospital and respondent characteristics: Appendix A: Results by Hospital Characteristics Bed size Teaching status Ownership and control Geographic region Appendix B: Results by Respondent Characteristics Work area/unit Staff position Interaction with patients The breakout tables are included as appendixes because there are a large number of them. Highlights of the findings from the breakout tables in these appendixes are provided on the following pages. The appendixes are available on the Web at: www.ahrq.gov/qual/hospsurvey12/. 40 Highlights From Appendix A: Overall Results by Hospital Characteristics Bed Size (Tables A-1, A-3) The smallest hospitals (6-24 beds) had the highest percent positive average across all patient safety culture composites (68 percent); larger hospitals (400 beds or more) had the lowest (60 percent). Smaller hospitals (49 beds or less) had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (80 percent); larger hospitals (400 beds or more) had the lowest (71 percent). Teaching Status and Ownership and Control (Tables A-5, A-8) Nonteaching hospitals on average scored higher than teaching hospitals by 5 percentage points on Teamwork Across Units (60 percent positive compared with 55 percent positive) and Handoffs and Transitions (47 percent positive compared with 42 percent). A higher percentage of respondents in non-government-owned hospitals (47 percent) than in government-owned hospitals (41 percent) reported events. Geographic Region (Tables A-9, A-11, A-12) East South Central, West South Central, and South Atlantic/Associated Territories hospitals had the highest average percent positive response across all composites (65 percent positive); New England hospitals had the lowest (60 percent positive). West North Central hospitals had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (78 percent); New England hospitals had the lowest (69 percent). Pacific/Associated Territories hospitals had the highest percentage of respondents who reported one or more events in the past year (49 percent); the lowest percentage of respondents reporting events was in the West South Central region (41 percent). Highlights From Appendix B: Overall Results by Respondent Characteristics Work Area/Unit (Tables B-1, B-3, B-4) Respondents in Rehabilitation had the highest average percent positive response across the composites (69 percent positive); Emergency had the lowest (57 percent positive). Rehabilitation had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (85 percent); Emergency had the lowest (64 percent). ICU (Any Type) had the highest percentage of respondents reporting one or more events in the past year (64 percent); Rehabilitation had the lowest (39 percent). 41 Staff Position (Tables B-5, B-7, B-8) Respondents in Administration/Management had the highest average percent positive response across the composites (74 percent positive); Pharmacists had the lowest (60 percent positive). Administration/Management had the highest percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (86 percent); Pharmacists had the lowest (68 percent). Pharmacists had the highest percentage of respondents reporting one or more events in the past year (71 percent); Unit Assistants/Clerks/Secretaries had the lowest (17 percent). Interaction With Patients (Tables B-9, B-11, B-12) Respondents with direct patient interaction were more positive on Handoffs and Transitions compared with those without direct patient interaction (47 percent positive compared with 39 percent). Respondents without direct patient interaction were more positive than those with direct patient interaction on Management Support for Patient Safety (77 percent positive compared with 71 percent). Respondents without direct patient interaction had a higher percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (80 percent) than respondents with direct patient interaction (75 percent). More respondents with direct patient interaction reported one or more events in the past year (50 percent) than respondents without direct patient interaction (30 percent). 42 Chapter 7. Trending: Comparing Results Over Time Many hospitals that administer the hospital survey have indicated that they intend to readminister the survey on a regular basis to track changes in patient safety culture over time. While the overall results presented earlier in this report reflect only the most recent survey data from all 1,128 participating hospitals, we have data from two administrations of the survey for 650 hospitals, allowing us to examine trends over time for these hospitals. This chapter presents trending results from these 650 hospitals. Highlights For the 650 hospitals with trending data, the average length of time between previous and most recent survey administrations was 20 months (range: 6 months to 66 months). The distribution of the 650 trending hospitals by bed size, teaching status, and ownership and control is similar to the distribution of the 1,128 database hospitals. The average percent positive scores on the patient safety culture composites increased by 1 percentage point (ranging from 0 to 2 percentage points). The average percentage of respondents who gave their work area/unit a patient safety grade of "A-Excellent" or "B-Very Good" increased by 1 percentage point. The average number of respondents reporting one or more events decreased by 1 percentage point. When reviewing the results in this chapter, keep in mind that survey scores might change, or not change, over time for a number of complex reasons. Important factors to consider are whether the hospital implemented patient safety initiatives or took actions between survey administrations and the length of time between administrations. Survey methodology issues can also play a big role in score changes. Low survey response rates for the previous or most recent administration, changes in the number of staff asked to complete the survey, or changes in the types of staff asked to complete the survey will make it difficult to interpret changes in scores over time. 43 Table 7-1 displays summary statistics from the previous and most recent survey administrations for the 650 trending hospitals. Table 7-1. Summary Statistics for Most Recent and Previous Data Submissions From the 650 Trending Hospitals Most Recent Survey Administration Summary Statistic Total number of respondents Number of completed surveys per hospital Hospital response rate Number of hospitals (out of 650) that administered the survey to all staff, or a sample of all staff, from all departments Previous Survey Administration 349,536 Average: 538 Range: 12-8,725 Average: 54% Range: 4-100% 595 (92%) 306,864 Average: 472 Range: 10-5,045 Average: 52% Range: 3-100% 543 (84%) Additional characteristics of the 650 trending hospitals follow: Most of the 650 trending hospitals (80 percent) administered the survey to the same types of staff in their previous and most recent administrations. The average change in response rate from the previous administration was a decrease of 1 percentage point (range: one hospital had a 90 percentage point decrease in response rate and one had an 85 percentage point increase). The average time between the previous and most recent survey administrations was 20 months (range: 6 months to 66 months). Note: Descriptive statistics on the 650 trending hospitals by bed size, teaching status, ownership and control, and region are provided in Appendix C (Tables C-1, C-2, C-3, and C-4). Description of Trending Statistics Table 7-2a shows examples of the types of statistics provided in this chapter. The tables show the average percentage of respondents who answered positively in the most recent survey administration (left column) and the previous administration (middle column) for the trending hospitals only. The change over time (Most Recent score minus Previous score) is shown in the right column. The change is a negative number if the most recent administration showed a decline and a positive number if the most recent administration showed an increase. Table 7-2a. Example of Trending Statistics Survey Item Most Recent Previous Change Item 1 Item 2 80% 80% 84% 78% -4% 2% Table 7-2b shows additional types of trending statistics that are provided. The maximum increase shows the score from the hospital or hospitals with the largest percent positive score increase on a particular composite or item. Similarly, the maximum decrease shows the score from the hospital or hospitals with the largest percent positive score decrease. 44 The average increase was calculated by including only hospitals that had an increase in their most recent score; hospitals that showed no change or decreased were not included when calculating the average increase. Similarly, the average decrease was calculated by including only hospitals that had a decrease in their most recent score; hospitals that showed no change or increased were not included when calculating the average decrease. Table 7-2b. Example of Other Trending Statistics Survey Item Maximum Increase Maximum Decrease Average Increase Average Decrease Item 1 Item 2 18% 21% -45% -19% 3% 5% -5% -6% Composite and Item-Level Trending Results Table 7-3 presents trending results on each of the 12 patient safety culture composites. The table shows average percent positive scores for the most recent and previous administrations, average change over time, maximum increase and decrease, and average increase and decrease over time. Table 7-4 presents similar trending results for the 42 survey items. Table 7-5 and Table 7-6 present the trending results for patient safety grade and number of events reported over the past 12 months, respectively. 45 46 Organizational Learning--Continuous Improvement Management Support for Patient Safety Overall Perceptions of Patient Safety Feedback & Communication About Error Frequency of Events Reported Communication Openness Teamwork Across Units 3. 4. 5. 6. 7. 8. 9. Staffing Supervisor/Manager Expectations & Actions Promoting Patient Safety 2. Nonpunitive Response to Error 11. 12. 44% 46% 57% 59% 62% 64% 65% 67% 73% 73% 75% 80% Most Recent 44% 45% 56% 58% 62% 63% 64% 65% 72% 72% 75% 79% Previous 0% 1% 1% 1% 0% 1% 1% 2% 1% 1% 0% 1% Change 21% 33% 38% 36% 21% 20% 34% 28% 27% 24% 23% 25% Maximum Increase -22% -24% -32% -25% -24% -21% -28% -20% -22% -27% -24% -21% Maximum Decrease Composite % Positive Response 5% 5% 6% 5% 4% 4% 5% 5% 5% 4% 4% 4% Average Increase -4% -5% -5% -4% -4% -3% -5% -4% -5% -4% -4% -3% Average Decrease Note: Based on data from 650 trending hospitals that had composite-level scores; the number of respondents was 349,536 for the most recent results and 306,864 for the previous results. Most recent, previous, and change columns display average percent positive scores across the trending hospitals. Handoffs & Transitions 10. Teamwork Within Units 1. Patient Safety Culture Composites Table 7-3. Trending: Composite-Level Results 47 Teamwork Within Units 1. People support one another in this unit. 2. When a lot of work needs to be done quickly, we work together as a team to get the work done. 3. In this unit, people treat each other with respect. 4. When one area in this unit gets really busy, others help out. Supervisor/Manager Expectations & Actions Promoting Patient Safety 1. My supv/mgr says a good word when he/she sees a job done according to established patient safety procedures. 2. My supv/mgr seriously considers staff suggestions for improving patient safety. 3. Whenever pressure builds up, my supv/mgr wants us to work faster, even if it means taking shortcuts. 4. My supv/mgr overlooks patient safety problems that happen over and over. Organizational Learning--Continuous Improvement 1. We are actively doing things to improve patient safety. 2. Mistakes have led to positive changes here. 3. After we make changes to improve patient safety, we evaluate their effectiveness. Survey Items by Composite 70% 65% 84% 77% 74% 76% 74% 70% 78% 86% 86% Most Recent 69% 64% 83% 76% 74% 76% 73% 69% 78% 85% 86% Previous 1% 1% 1% 1% 0% 0% 1% 1% 0% 1% 0% 32% 30% 19% 24% 22% 27% 38% 42% 26% 28% 28% -37% -41% -25% -25% -23% -30% -41% -28% -28% -19% -20% Item % Positive Response Maximum Maximum Change Increase Decrease 5% 6% 5% 4% 5% 5% 6% 5% 4% 4% 4% Average Increase -5% -5% -4% -4% -4% -5% -4% -4% -4% -4% -4% Average Decrease Note: Based on data from 650 trending hospitals. The number of respondents was 349,536 for the most recent results and 306,864 for the previous results, but the exact number of respondents will vary from item to item. Most recent, previous, and change columns display average percent positive scores across the trending hospitals. The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). A13 A9 A6 3. B4R B3R B2 B1 2. A11 A4 1. A1 A3 Item Table 7-4. Trending: Item-Level Results (Page 1 of 4) 48 Management Support for Patient Safety 1. Hospital mgmt provides a work climate that promotes patient safety. 2. The actions of hospital mgmt show that patient safety is a top priority. 3. Hospital mgmt seems interested in patient safety only after an adverse event happens. Overall Perceptions of Patient Safety 1. It is just by chance that more serious mistakes don't happen around here. 2. Patient safety is never sacrificed to get more work done. 3. We have patient safety problems in this unit. 4. Our procedures and systems are good at preventing errors from happening. Feedback and Communication About Error 1. We are given feedback about changes put into place based on event reports. 2. We are informed about errors that happen in this unit. 3. In this unit, we discuss ways to prevent errors from happening again. Survey Items by Composite 72% 66% 58% 73% 65% 65% 63% 61% 76% 81% Most Recent 71% 65% 56% 72% 64% 64% 62% 60% 74% 81% Previous 1% 1% 2% 1% 1% 1% 1% 1% 2% 0% 33% 41% 29% 29% 33% 37% 30% 28% 29% 27% -34% -23% -41% -23% -25% -27% -22% -28% -26% -27% Item % Positive Response Maximum Maximum Change Increase Decrease 6% 6% 6% 6% 6% 6% 5% 6% 6% 5% Average Increase -5% -5% -6% -4% -5% -5% -5% -5% -5% -5% Average Decrease Note: Based on data from 650 trending hospitals. The number of respondents was 349,536 for the most recent results and 306,864 for the previous results, but the exact number of respondents will vary from item to item. Most recent, previous, and change columns display average percent positive scores across the trending hospitals. The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). C5 C3 C1 6. A18 A17R A15 5. A10R F9R F8 4. F1 Item Table 7-4. Trending: Item-Level Results (Page 2 of 4) 49 Frequency of Events Reported 1. When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported? 2. When a mistake is made, but has no potential to harm the patient, how often is this reported? 3. When a mistake is made that could harm the patient, but does not, how often is this reported? Communication Openness 1. Staff will freely speak up if they see something that may negatively affect patient care. 2. Staff feel free to question the decisions or actions of those with more authority. 3. Staff are afraid to ask questions when something does not seem right. Teamwork Across Units 1. Hospital units do not coordinate well with each other. 2. There is good cooperation among hospital units that need to work together. 3. It is often unpleasant to work with staff from other hospital units. 4. Hospital units work well together to provide the best care for patients. Survey Items by Composite 69% 60% 60% 46% 63% 47% 76% 74% 60% 58% Most Recent 68% 59% 59% 45% 63% 47% 75% 74% 59% 56% Previous 1% 1% 1% 1% 0% 0% 1% 0% 1% 2% 39% 45% 38% 41% 49% 31% 20% 21% 27% 28% -24% -35% -33% -37% -25% -39% -28% -28% -34% -22% Item % Positive Response Maximum Maximum Change Increase Decrease 6% 6% 6% 6% 5% 6% 4% 4% 5% 5% Average Increase -5% -5% -5% -6% -5% -6% -4% -4% -4% -4% Average Decrease Note: Based on data from 650 trending hospitals. The number of respondents was 349,536 for the most recent results and 306,864 for the previous results, but the exact number of respondents will vary from item to item. Most recent, previous, and change columns display average percent positive scores across the trending hospitals. The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). F10 F6R F4 9. F2R C6R C4 8. C2 D3 D2 7. D1 Item Table 7-4. Trending: Item-Level Results (Page 3 of 4) 50 Staffing 1. We have enough staff to handle the workload. 2. Staff in this unit work longer hours than is best for patient care. 3. We use more agency/temporary staff than is best for patient care. 4. We work in -crisis mode? trying to do too much, too quickly. Handoffs & Transitions 1. Things -fall between the cracks? when transferring patients from one unit to another. 2. Important patient care information is often lost during shift changes. 3. Problems often occur in the exchange of information across hospital units. 4. Shift changes are problematic for patients in this hospital. Nonpunitive Response to Error 1. Staff feel like their mistakes are held against them. 2. When an event is reported, it feels like the person is being written up, not the problem. 3. Staff worry that mistakes they make are kept in their personnel file. Survey Items by Composite 36% 47% 50% 46% 44% 51% 42% 51% 68% 53% 56% 35% 46% 50% 45% 43% 50% 41% 49% 67% 53% 55% Previous 1% 1% 0% 1% 1% 1% 1% 2% 1% 0% 1% 27% 29% 42% 36% 58% 37% 37% 38% 78% 34% 32% -34% -25% -29% -28% -44% -32% -31% -36% -80% -33% -42% Item % Positive Response Maximum Maximum Change Increase Decrease 5% 6% 5% 6% 6% 6% 6% 7% 8% 6% 8% Average Increase -5% -5% -5% -5% -6% -5% -6% -6% -6% -6% -8% Average Decrease Note: Based on data from 650 trending hospitals. The number of respondents was 349,536 for the most recent results and 306,864 for the previous results, but the exact number of respondents will vary from item to item. Most recent, previous, and change columns display average percent positive scores across the trending hospitals. The item's survey location is shown to the left. An "R" indicates a negatively worded item, where the percent positive response is based on those who responded "Strongly disagree" or "Disagree," or "Never" or "Rarely" (depending on the response category used for the item). A16R A12R 12. A8R F11R F7R F5R 11. F3R A14R A7R A5R 10. A2 Item Most Recent Table 7-4. Trending: Item-Level Results (Page 4 of 4) 51 Very Good Acceptable Poor Failing B C D E 1% 4% 20% 45% 31% Most Recent 1% 4% 21% 46% 29% Previous 0% 0% -1% -1% 2% Change 13% 31% 27% 32% 35% Maximum Increase -7% -20% -29% -35% -44% Maximum Decrease 1% 2% 4% 4% 6% Average Increase -1% -2% -5% -5% -5% Average Decrease 27% 11% 4% 2% 1% 1 to 2 events 3 to 5 events 6 to 10 events 11 to 20 events 21 events or more 1% 2% 4% 12% 27% 54% Previous 0% 0% 0% -1% 0% 1% Change 4% 6% 10% 17% 22% 35% Maximum Increase -7% -8% -11% -34% -26% -33% Maximum Decrease 1% 1% 2% 3% 4% 6% Average Increase -1% -1% -2% -3% -4% -5% Average Decrease Note: Based on data from 650 trending hospitals that had data for this item. The number of respondents was 349,536 for the most recent results and 306,864 for the previous results. Most recent, previous, and change columns display average percent positive scores across the trending hospitals. Column totals in the table may not add to 100 percent due to rounding. 55% Most Recent No events Number of Events Reported by Respondents Percentage of Respondents Within Hospitals Table 7-6. Trending: Distribution of Number of Events Reported in the Past 12 Months Note: Based on data from 650 trending hospitals that had data for this item. The number of respondents was 349,536 for the most recent results and 306,864 for the previous results. Most recent, previous, and change columns display average percent positive scores across the trending hospitals. Column totals in the table may not add to 100 percent due to rounding. Excellent A Work Area/Unit Patient Safety Grade Percentage of Respondents Within Hospitals Table 7-5. Trending: Distribution of Work Area/Unit Patient Safety Grades Bar Charts of Trending Results Chart 7-1 shows the percentages of trending hospitals that increased, decreased, or did not change for each of the 12 patient safety culture composites. The chart shows that: Most hospitals changed less than 5 percentage points on the 12 composites. Staffing had the largest percentage of hospitals that increased 5 percentage points or more; 29 percent of hospitals increased by at least 5 percentage points. Communication Openness had the largest percentage of hospitals that decreased 5 percentage points or more; 19 percent of hospitals decreased by at least 5 percentage points. Chart 7-2 displays results for the percentages of trending hospitals that increased, decreased, or did not change on patient safety grades (percent providing grades of "A-Excellent" or "B-Very Good"; percentages may not add to 100 due to rounding) and shows that: 29 percent of hospitals increased by 5 percentage points or more. 52 percent of hospitals changed less than 5 percentage points. 18 percent of hospitals decreased by 5 percentage points or more. Chart 7-3 displays results for the percentages of trending hospitals that increased, decreased, or did not change in the proportion of respondents reporting one or more events and shows that: 19 percent of hospitals increased by 5 percentage points or more. 51 percent of hospitals changed less than 5 percentage points. 30 percent of hospitals decreased by 5 percentage points or more. Chart 7-4 displays the overall number of composites for which trending hospitals increased, decreased, or did not change: Most hospitals (64 percent) increased by 5 percentage points or more on at least one composite. 51 percent of hospitals decreased by 5 percentage points or more on at least one composite. More than half the hospitals (62 percent) changed less than 5 percentage points on seven or more composites. 52 53 Did Not Change (less than 5 percentage point change) (by 5 percentage points or more) Decreased Note: Based on data from 650 trending hospitals that responded to this item. Percentages may not add to 100 due to rounding. (by 5 percentage points or more) Increased Chart 7-1. Trending: Percentage of Hospitals That Increased, Decreased, or Did Not Change on Each Composite Chart 7-2. Trending: Percentage of Hospitals That Increased, Decreased, or Did Not Change on Work Area/Unit Patient Safety Grade Increased Did Not Change Decreased (by 5 percentage points or more) (increased or decreased by less than 5 percentage points) (by 5 percentage points or more) Note: Based on data from 650 trending hospitals that had data for this item. For each hospital, change over time was calculated for the percentage of respondents reporting a grade of "Excellent" or "Very Good." Percentages may not add to 100 due to rounding. Chart 7-3. Trending: Percentage of Hospitals That Increased, Decreased, or Did Not Change on Number of Events Reported Increased Did Not Change Decreased (by 5 percentage points or more) (increased or decreased by less than 5 percentage points) (by 5 percentage points or more) Note: Based on data from 650 trending hospitals that had data for this item. For each hospital, change over time was calculated for the percentage of respondents who reported one or more events over the past 12 months. Percentages may not add to 100 due to rounding. 54 Chart 7-4. Trending: Distribution of Hospitals by Number of Composites That Increased, Decreased, or Did Not Change by 5 Percentage Points or More Percentage of Trending Hospitals Distribution of Hospitals by Number of Composites That Did Not Change 100% 80% 60% 40% 25% 20% 31% 31% 7-9 10-12 11% 1% 0% None 1-3 4-6 Number of Composites Note: Based on data from 605 trending hospitals that measured all 12 survey dimensions. Twenty-four trending hospitals that did not measure all 12 survey dimensions are not included. Percentages may not add to 100 due to rounding. 55 Appendixes C and D: Trending Results by Hospital and Respondent Characteristics Part III of the report contains Appendixes C and D, which show trends over time for the 650 hospitals that administered the survey and submitted data more than once. Average percent positive scores from the most recent and previous administrations are shown on the survey composites and items, broken down by the following hospital and respondent characteristics: Appendix C: Trending Results by Hospital Characteristics Bed size Teaching status Ownership and control Geographic region Appendix D: Trending Results by Respondent Characteristics Work area/unit Staff position Interaction with patients Because there are many breakout tables, they are included in Appendixes C and D. Highlights of the findings from the breakout tables in these appendixes are provided on the following pages. The appendixes are available on the Web at: www.ahrq.gov/qual/hospsurvey12/. 56 Highlights From Appendix C: Trending Results by Hospital Characteristics Bed Size (Tables C-5, C-7) Hospitals with 50-99 beds had the greatest increases in percent positive response over time on all 12 composites (an average increase of 2 percentage points). Hospitals with 50-99 beds had the greatest increase in the percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (a 3 percentage point increase, from 75 percent to 78 percent). Teaching Status and Ownership and Control (Table C-9) Nonteaching hospitals showed increases up to 2 percentage points on all 12 patient safety composites; teaching hospitals showed increases up to 1 percentage point on half of the composites and decreased by 1 percentage point on Supervisor/Manager Expectations. Government-owned hospitals showed increases up to 2 percentage points across 11 composites; non-government-owned hospitals showed increases up to 2 percentage points on 9 composites. Geographic Region (Tables C-13, C-15) West North Central hospitals had the greatest increases in percent positive response over time on 6 of the 12 composites (average increase of 2 percentage points). West North Central hospitals had the greatest increase in the percentage of respondents who gave their work area/unit a patient safety grade of "Excellent" or "Very Good" (a 3 percentage point increase, from 75 percent to 78 percent). Highlights From Appendix D: Trending Results by Respondent Characteristics Work Area/Unit (Tables D-1, D-3, D-4) Rehabilitation had the greatest increase in percent positive response on all 12 patient safety culture composites (average increases of 3 percentage points). Emergency, Radiology, and Rehabilitation had the greatest increases over time in the average percentage of respondents giving their work area/unit a patient safety grade of "Excellent" or "Very Good" (3 percentage point increases, from 62 percent to 65 percent, 79 percent to 82 percent, and 82 percent to 85 percent, respectively). Anesthesiology and Lab had the greatest increases in the average percentage of respondents reporting one or more events in the past year (3 percentage point increases). The largest decrease was in Psychiatry/Mental Health (a 4 percentage point decrease). Staff Position (Tables D-5, D-7, D-8) Patient Care Asst./Aide/Care Partner had the greatest increase in positive response over time on 4 of the 12 patient safety composites (average increase of 2 percentage points). Pharmacists had the greatest increase over time in the average percentage of respondents giving their work area/unit a patient safety grade of "Excellent" or "Very Good" (a 3 percentage point increase). Dietitians had the greatest decrease over time in the average percentage of respondents reporting one or more events in the past year (an 11 percentage point decrease). 57 Interaction With Patients (Table D-9) Respondents with direct interaction with patients showed an increase of 1 percentage point across 11 patient safety culture composites; respondents without direct interaction showed an increase of 1 percentage point across 10 composites. 58 Chapter 8. What's Next? Action Planning for Improvement The seven steps of action planning outlined in this chapter are primarily based on the book Designing and Using Organizational Surveys: A Seven-Step Process (Church and Waclawski, 1998). Highlights The delivery of survey results is not the end point in the survey process; it is just the beginning. Often, the perceived failure of surveys to create lasting change is actually due to faulty or nonexistent action planning or survey followup. Seven steps of action planning are provided to give hospitals guidance on next steps to take to turn their survey results into actual patient safety culture improvement. Seven Steps of Action Planning Administering the hospital survey can be considered an "intervention," a means of educating hospital staff and building awareness about issues of concern related to patient safety. But it should not be the only goal of conducting the survey. Administering the survey is not enough. Keep in mind that the delivery of survey results is not the end point in the survey process; it is actually just the beginning. Often, the perceived failure of surveys as a means for creating lasting change is actually due to faulty or nonexistent action planning or survey followup. Seven steps of action planning are provided to help your hospital go beyond simply conducting a survey to realizing patient safety culture change. The progression is getting survey results, developing an action plan, and implementing the plan and tracking progress. The seven steps of action planning are: 1. 2. 3. 4. 5. 6. 7. Understand your survey results. Communicate and discuss survey results. Develop focused action plans. Communicate action plans and deliverables. Implement action plans. Track progress and evaluate impact. Share what works. 59 Step # 1: Understand Your Survey Results It is important to review the survey results and interpret them before you develop action plans. Develop an understanding of your hospital's key strengths and areas for improvement. Examine your hospital's overall percent positive scores on the patient safety culture composites and items. Which areas were most and least positive? How do your hospital's results compare with the results from the database hospitals? Next, consider examining your survey data broken down by work area/unit or staff position. Are there different areas for improvement for different hospital units? Are there different areas for improvement for different hospital staff? Do any patterns emerge? How do your hospital's results for these breakouts compare with the results from the database hospitals? Finally, if your hospital administered the survey more than once, compare your most recent results with your previous results to examine change over time. Did your hospital have an increase in its scores on any of the survey composites or items? Did your hospital have a decrease in its scores? When you consider the types of patient safety actions that your hospital implemented between each survey administration, do you notice improvements in those areas? After reviewing the survey results carefully, identify two or three areas for improvement to avoid focusing on too many issues at one time. Step # 2: Communicate and Discuss the Survey Results Common complaints among survey respondents are that they never get any feedback about survey results and have no idea whether anything ever happens as a result of a survey. It is therefore important to thank your staff for taking the time to complete the survey and let them know that you value their input. Sharing results from the survey throughout the hospital shows your commitment to the survey and improvement process. Use survey feedback as an impetus for change. Feedback can be provided at the hospital level and at the department or unit level. However, to ensure respondent anonymity and confidentiality, it is important to report data only if there are enough respondents in a particular category or group. Common rules of thumb recommend not reporting data if a category has fewer than 5 or 10 respondents. For example, if a department has only four respondents, that department's data should not be reported separately because there are too few respondents to provide complete assurance of anonymity and confidentiality. Summaries of the survey results should be distributed throughout the hospital in a top-down manner, beginning with senior management, administrators, medical and senior leaders, and committees, followed by department or unit managers and then staff. Managers at all levels should be expected to carefully review the findings. Summarize key findings, but also encourage 60 discussion about the results throughout the hospital. What do others see in the data and how do they interpret the results? In some cases, it may not be completely clear why an area of patient safety culture was particularly low. Keep in mind that surveys are only one way of examining culture, so strive for a deeper understanding when needed. Conduct followup activities, such as focus groups or interviews with staff to find out more about an issue, why it is problematic, and how it can be improved. Step # 3: Develop Focused Action Plans Once areas for patient safety culture improvement have been identified, formal written action plans need to be developed to ensure progress toward change. Hospitalwide, department-based, or unit-based action plans can be developed. Major goals can be established as hospitalwide action plans. Unit-specific goals can be fostered by encouraging and empowering staff to develop action plans at the unit level. Encourage action plans that are "SMART": Specific Measurable Achievable Relevant Time bound When deciding whether a particular action plan or initiative would be a good fit in your facility, you may find the guide Will It Work Here? A Decisionmaker's Guide to Adopting Innovations (Brach, Lenfestey, Roussel, et al., 2008) a useful resource (available at: www.innovations.ahrq.gov/content.aspx?id=2380). The guide helps users answer four overarching questions: Does this innovation fit? Should we do it here? Can we do it here? How can we do it here? Lack of resources is often a fundamental obstacle hindering implementation of action plans. Identify funding, staffing, or other resources needed to implement action plans and take steps to obtain these resources. It is also important to identify other obstacles you may encounter when trying to implement change and to anticipate and understand the rationale behind any potential resistance toward proposed action plans. In the planning stage, it is also important to identify quantitative and qualitative measures that can be used to evaluate progress and the impact of changes implemented. Evaluative measures will need to be assessed before, during, and after implementation of your action plan initiatives. 61 Step # 4: Communicate Action Plans and Deliverables Once action plans have been developed, the plans, deliverables, and expected outcomes of the plans need to be communicated. Those directly involved or affected will need to know their roles and responsibilities, as well as the timeframe for implementation. Action plans and goals should also be shared widely so that their transparency encourages further accountability and demonstrates the hospitalwide commitments being made in response to the survey results. At this step it is important for senior hospital managers and leaders to understand that they are the primary owners of the change process and that success depends on their full commitment and support. Senior-level commitment to taking action must be strong; without buy-in from the top, including medical leadership, improvement efforts are likely to fail. Step # 5: Implement Action Plans Implementing action plans is one of the hardest steps. Taking action requires the provision of necessary resources and support. It requires tracking quantitative and qualitative measures of progress and success that have already been identified. It requires publicly recognizing those individuals and units who take action to drive improvement. And it requires adjustments along the way. This step is critical to realizing patient safety culture improvement. While communicating the survey results is important, taking action makes the real difference. However, as the Institute for Healthcare Improvement (2006) suggests, actions do not have to be major permanent changes. In fact, it is worthwhile to strive to implement easier smaller changes that are likely to have a positive impact rather than big changes with unknown probability of success. The "Plan-Do-Study-Act" cycle (Langley, Nolan, Nolan, et al., 1996), shown in Chart 8-1, is a pilot-study approach to change. It involves first developing a small-scale plan to test a proposed change (Plan), carrying out the plan (Do), observing and learning from the consequences (Study), and determining what modifications should be made to the plan (Act). Implementation of action plans can occur on a small scale within a single unit to examine impact and refine plans before rolling out the changes on a larger scale to other units or hospitals. Chart 8-1. Plan-Do-Study-Act Cycle 62 Step # 6: Track Progress and Evaluate Impact Use quantitative and qualitative measures to review progress and evaluate whether a specific change actually leads to improvement. Ensure that there is timely communication of progress toward action plans on a regular basis. If you determine that a change has worked, communicate that success to staff by telling them what was changed and that it was done in response to the safety culture survey results. Be sure to make the connection to the survey so that the next time the survey is administered, staff will know that it will be worthwhile to participate again because actions were taken based on the prior survey's results. Alternatively, your evaluation may show that a change is not working as expected or has failed to reach its goals and will need to be modified or replaced by another approach. Before you drop the effort completely, try to determine why it failed and whether it might be worth it to make adjustments. Keep in mind that it is important not to reassess culture too frequently because lasting culture change will be slow and may take years. Frequent assessments of culture are likely to find temporary shifts or improvements that may come back down to baseline levels in the longer term if changes are not sustained. When planning to reassess culture, it is also very important to obtain high survey response rates. Otherwise, it will not be clear whether changes in survey results over time are due to true changes in attitudes or are caused by surveying different staff each time. Step # 7: Share What Works In step # 6, you tracked measures to identify which changes result in improvement. Once your hospital has found effective ways to address a particular area, the changes can be implemented on a broader scale to other departments within the hospital and to other hospitals. Be sure to share your successes with outside hospitals and health care systems as well. 63 References Agency for Healthcare Research and Quality. Hospital Survey on Patient Safety Culture. Available at: www.ahrq.gov/qual/patientsafetyculture. Accessed January 28, 2011. American Hospital Association (AHA) Annual Survey of Hospitals database. Chicago: Health Forum; multiple years (data used from 2006 and 2010). Brach C, Lenfestey N, Roussel A, et al. Will it work here? A decisionmaker's guide to adopting innovations. (Prepared by RTI International under Contract No. 233-02-0090). Rockville, MD: Agency for Healthcare Research and Quality; September 2008. AHRQ Publication No. 08-0051. Available at: www.innovations.ahrq.gov/content.aspx?id=2380. Church AH, Waclawski J. Designing and using organizational surveys: a seven-step process. San Francisco: Jossey-Bass; 1998. Improvement methods: the Plan-Do-Study-Act (PDSA) cycle. Washington, DC: Institute for Healthcare Improvement; 2006. Available at: www.ihi.org/IHI/Topics/Improvement/ImprovementMethods/HowToImprove. Accessed January 28, 2011. Langley C, Nolan K, Nolan T, et al. The improvement guide: a practical approach to improving organizational performance. San Francisco: Jossey-Bass; 1996. 64 Notes: Description of Data Cleaning and Calculations This notes section provides additional detail regarding how various statistics presented in this report were calculated. Data Cleaning Each participating hospital was asked to submit cleaned individual-level survey data. However, as an additional check, once the data were submitted, response frequencies were run on each hospital's data to look for out-of-range values, missing variables, or other data anomalies. When data problems were found, hospitals were contacted and asked to make corrections and resubmit their data. In addition, each participating hospital was sent a copy of its data frequencies to verify that the dataset received was correct. In order to keep the database current, data more than 3.5 years old are removed from the database. Thus, 129 hospitals that administered the survey prior to January 1, 2008, were dropped from the database. Response Rates As part of the data submission process, hospitals were asked to provide their response rate numerator and denominator. Response rates were calculated using the formula below. Response Rate Number ofcomplete,returned s urveys Number ofsurveys di stributed Ineligibl es Numerator = Number of complete, returned surveys. The numerator equals the number of individual survey records submitted to the database. It should exclude surveys that were returned blank on all nondemographic survey items but include surveys where at least one nondemographic survey item was answered. Denominator = The total number of surveys distributed minus ineligibles. Ineligibles include deceased individuals and those who were not employed at the hospital during data collection. As a data cleaning step, we examined whether any individual survey records submitted to the database were missing responses on all of the nondemographic survey items (indicating the respondent did not answer any of the main survey questions). Records where all nondemographic survey items were left blank by the respondent were found (even though these blank records should not have been submitted to the database). We therefore removed these blank records from the larger dataset and adjusted any affected hospital's response rate numerator and overall response rate accordingly. 65 Calculation of Percent Positive Scores Most of the survey's items ask respondents to answer using 5-point response categories in terms of agreement (Strongly agree, Agree, Neither, Disagree, Strongly disagree) or frequency (Always, Most of the time, Sometimes, Rarely, Never). Three of the 12 patient safety culture composites use the frequency response option (Feedback and Communication About Error, Communication Openness, and Frequency of Events Reported), while the other 9 composites use the agreement response option. Item-Level Percent Positive Response Both positively worded items (such as "People support one another in this unit") and negatively worded items (such as "We have patient safety problems in this unit") are included in the survey. Calculating the percent positive response on an item is different for positively and negatively worded items: For positively worded items, percent positive response is the combined percentage of respondents within a hospital who answered "Strongly agree" or "Agree," or "Always" or "Most of the time," depending on the response categories used for the item. For example, for the item "People support one another in this unit," if 50 percent of respondents within a hospital Strongly agree and 25 percent Agree, the item-level percent positive response for that hospital would be 50% + 25% = 75% positive. For negatively worded items, percent positive response is the combined percentage of respondents within a hospital who answered "Strongly disagree" or "Disagree," or "Never" or "Rarely," because a negative answer on a negatively worded item indicates a positive response. For example, for the item "We have patient safety problems in this unit," if 60 percent of respondents within a hospital Strongly disagree and 20 percent Disagree, the item-level percent positive response would be 80 percent positive (i.e., 80 percent of respondents do not believe they have patient safety problems in their work area). Composite-Level Percent Positive Response The survey's 42 items measure 12 areas, or composites, of patient safety culture. Each of the 12 patient safety culture composites includes 3 or 4 survey items. Composite scores were calculated for each hospital by averaging the percent positive response on the items within a composite. For example, for a three-item composite, if the item-level percent positive responses were 50 percent, 55 percent, and 60 percent, the hospital's composite-level percent positive response would be the average of these three percentages, or 55 percent positive.ix ix This method for calculating composite scores differs slightly from the method described in the September 2004 Survey User's Guide that is part of the original survey toolkit materials on the AHRQ Web site. The guide advises computing composites by calculating the overall percent positive across all the items within a composite. The updated recommendation included in this report is to compute item percent positive scores first, and then average the item percent positive scores to obtain the composite score, which gives equal weight to each item in a composite. The Survey User's Guide will eventually be updated to reflect this slight change in methodology. 66 Item and Composite Percent Positive Scores To calculate your hospital's composite score, simply average the percentage of positive response to each item in the composite. Here is an example of computing a composite score for Overall Perceptions of Patient Safety: 1. There are four items in this composite--two are positively worded (items A15 and A18) and two are negatively worded (items A10 and A17). Keep in mind that disagreeing with a negatively worded item indicates a positive response. 2. Calculate the percentage of positive responses at the item level. (See example in Table 1.) Table 1. Example of Computing Item and Composite Percent Positive Scores For Positively Worded Items, Number of "Strongly Agree" or "Agree" Responses For Negatively Worded Items, Number of "Strongly Disagree" or "Disagree" Responses Total Number of Responses to the Item Percent Positive Response on Item Item A15: positively worded -Patient safety is never sacrificed to get more work done? 120 NA* 260 120/260=46% Item A18: positively worded -Our procedures and systems are good at preventing errors from happening? 130 NA* 250 130/250=52% Item A10: negatively worded -It is just by chance that more serious mistakes don't happen around here? NA* 110 240 110/240=46% Item A17: negatively worded -We have patient safety problems in this unit? NA* 140 250 140/250= 56% Items Measuring Overall Perceptions of Patient Safety Composite Score % Positive = (46% + 52% + 46% + 56%) / 4 = 50% * NA = Not applicable. In this example, there were four items with percent positive response scores of 46 percent, 52 percent, 46 percent, and 56 percent. Averaging these item-level percent positive scores results in a composite score of .50, or 50 percent, on Overall Perceptions of Patient Safety. In this 67 example, an average of about 50 percent of the respondents responded positively to the survey items in this composite. Once you calculate your hospital's percent positive response for each of the 12 safety culture composites, you can compare your results with the composite-level results from the 1,128 database hospitals. Minimum Number of Responses Beginning with the 2010 database report, we enacted several new rules regarding a minimum number of responses for calculating the percent positive scores. First, we calculated percent positive scores only for hospitals that had at least 10 completed surveys. Second, item-level results were calculated only when there were at least three responses to the item. If a hospital had fewer than three responses to a survey item, the hospital's score for that item was set to missing. Third, if a hospital had fewer than five respondents in a breakout category (e.g., work area/unit, staff position, direct interaction with patients), then no statistics were calculated for that breakout category (i.e., all scores were set to missing). For example, if a hospital had five respondents indicating they worked in the Anesthesiology unit and four respondents indicating they worked in Pharmacy, that hospital would be included in the statistics displayed for Anesthesiology units but not in those displayed for Pharmacy units. These minimums also apply to the statistics displayed in Appendixes B and D (results by respondent characteristics). Percentiles Percentiles were computed using the SAS(R) software default method. The first step in this procedure is to rank order the percent positive scores from all the participating hospitals, from lowest to highest. The next step is to multiply the number of hospitals (n) by the percentile of interest (p), which in our case would be the 10th, 25th, 50th, 75th, or 90th percentile. For example, to calculate the 10th percentile, one would multiply 1,128 (the total number of hospitals) by .10 (10th percentile). The product of n x p is equal to j + g, where j is the integer and g is the number after the decimal. If g equals 0, the percentile is equal to the percent positive value of the hospital in the jth position plus the percent positive value of the hospital in the jth + 1 position, divided by 2 [(X(j) + X(j+1))/2]. If g is not equal to 0, the percentile is equal to the percent positive value of the hospital in the jth + 1 position. The following examples show how the 10th and 50th percentiles would be computed using a sample of percent positive scores from 12 hospitals (using fake data shown in Table 2). First, the percent positive scores are sorted from low to high on Composite "A." 68 Table 2. Data Table for Example of How To Compute Percentiles Hospital Composite "A" % Positive Score 1 2 33% 48% 3 4 5 6 7 8 52% 60% 63% 64% 66% 70% 9 10 11 12 72% 75% 75% 78% th ?10 percentile score = 48% th ?50 percentile score = 65% 10th percentile 1. For the 10th percentile, we would first multiply the number of hospitals by .10: (n x p = 12 x .10 = 1.2). 2. The product of n x p = 1.2, where j = 1 and g = 2. Since g is not equal to 0, the 10th percentile score is equal to the percent positive value of the hospital in the jth + 1 position: a. j equals 1. b. The 10th percentile equals the value for the hospital in the 2nd position = 48%. 50th percentile 1. For the 50th percentile, we would first multiply the number of hospitals by .50: (n x p = 12 x .50 = 6.0). 2. The product of n x p = 6.0, where j = 6 and g = 0. Since g = 0, the 50th percentile score is equal to the percent positive value of the hospital in the jth position plus the percent positive value of the hospital in the jth + 1 position, divided by 2: a. j equals 6. b. The 50th percentile equals the average of the hospitals in the 6th and 7th positions (64% + 66%)/2 = 65%. 69