DOI: 10.1111/1475-6773.13247 Health Services Research RESEARCH ARTICLE Assessment of nursing home reporting of major injury falls for quality measurement on nursing home compare Prachi Sanghavi PhD1   Shengyuan Pan BA1  Daryl Caudry MS2 1 Department of Public Health Sciences, The University of Chicago Biological Sciences, Chicago, Illinois 2 Abstract Objective: To assess the accuracy of nursing home self-report of major injury falls on Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts the Minimum Data Set (MDS). Correspondence Prachi Sanghavi, PhD, Department of Public Health Sciences, The University of Chicago Biological Sciences, 5841 S. Maryland Ave., MC2000, Chicago, IL 60637. Email: psanghavi@uchicago.edu Study Design/Methods: We linked inpatient claims for major injury falls with MDS Funding information Agency for Healthcare Research and Quality, Grant/Award Number: R01HS026957 Data Sources: MDS assessments and Medicare claims, 2011-2015. assessments. The proportion of claims-identified falls reported for each fall-related MDS item was calculated. Using multilevel modeling, we assessed patient and nursing home characteristics that may be predictive of poor reporting. We created a claimsbased major injury fall rate for each nursing home and estimated its correlation with Nursing Home Compare (NHC) measures. Principal Findings: We identified 150,828 major injury falls in claims that occurred during nursing home residency. For the MDS item used by NHC, only 57.5 percent were reported. Reporting was higher for long-stay (62.9 percent) than short-stay (47.2 percent), and for white (59.0 percent) than nonwhite residents (46.4 percent). Adjusting for facility-level race differences, reporting was lower for nonwhite people than white people; holding constant patient race, having larger proportions of nonwhite people in a nursing home was associated with lower reporting. The correlation between fall rates based on claims vs the MDS was 0.22. Conclusions: The nursing home-reported data used for the NHC falls measure may be highly inaccurate. KEYWORDS disparities, falls, long term care, nursing homes, public reporting 1   I NTRO D U C TI O N Compare (NHC). 2 Today, NHC reports nursing home performance on several patient safety indicators, among other measures for staff- In 1987, a landmark Institute of Medicine report concluded that the ing and inspections. It also creates summaries of these by assigning 1 quality of care in many nursing homes was seriously inadequate. stars to each home through the Five-Star Quality Rating System. Since the 1990s, a centerpiece of federal efforts to improve nursing However, as the underlying quality of resident care data, called the home quality has been a public reporting initiative by the Centers Minimum Data Set (MDS), is self-reported by nursing homes, it is for Medicare and Medicaid Services (CMS) called Nursing Home important to ask: are NHC patient safety measures accurate? This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes. © 2019 The Authors. Health Services Research published by Wiley Periodicals, Inc. on behalf of Health Research and Educational Trust. Health Serv Res. 2019;00:1–10.  wileyonlinelibrary.com/journal/hesr     1 2       SANGHAVI et al. Health Services Research Concerns about the accuracy of the MDS are long-standing. Investigations over the past decades by the Department of Health and Human Services Office of the Inspector General (HHS OIG),3,4 What This Study Adds the US Government Accountability Office (GAO), 5-8 CMS,9 and the 1. Section 1: The federal website Nursing Home Compare New York Times,10,11 as well as the limited academic research on the reports patient safety measures for nursing homes using have all found discrepancies between the MDS and other data that are self-reported by nursing homes. The accu- sources. Though state inspections have the potential to serve as in- racy of these data has long been a concern to academics dependent checks of quality of care, these too are partly guided by and policy makers, based on inconsistencies with meas- 12,13 topic, ures from other sources and small validation exercises. MDS-based quality measures.14 We assessed the quality of nursing home reporting of major 2. Section 2: We focused on the falls section of the nurs- injury falls by linking MDS assessments with Medicare hospital ad- ing home-reported data and found only 57.5 percent mission claims at the patient level. We focused on falls for two rea- of major injury falls, identified in claims data, were re- sons. First, they are a leading cause of death among older adults and ported, and that reporting was substantially lower for can lead to serious physical and psychological morbidity when not nonwhite people than white people. The data Nursing 15 fatal. Yet, because they are often preventable, they serve as an important measure of patient safety. Second, relative to other clin- Home Compare uses for reporting patient safety related to falls may be highly inaccurate. ical conditions measured by the MDS, we expected falls to be easy to identify and record. To our knowledge, this is the first rigorous national analysis of reporting on any section of the MDS. provider identification information, and publicly available data from NHC. We merged each MDS assessment with the most recent record from these datasets prior to the assessment. 2   M E TH O DS 2.1   Datasets 2.2   Identification of falls for MDS reporting We analyzed January 1, 2011, to September 30, 2015, hospital Medicare claims data are commonly used to identify fall-related in- admission claims of a 100 percent sample of Medicare beneficiar- juries, including to study the costs of medical care for falls,17,18 to ies from the Medicare Provider Analysis and Review (MedPAR) file assess fall events as outcomes,19,20 and to create measures of po- provided by CMS. We dropped the last three months of 2015 when tentially avoidable hospitalizations. 21 We started with the hospital diagnosis codes in claims switched from International Classification admission claims and determined a patient had experienced a fall of Diseases, Ninth Revision, Clinical Modification (ICD9-CM) to the if the admitting or primary diagnosis code fields, or primary exter- Tenth Revision (ICD10) in order to avoid any transition issues and nal cause code field reported an accidental fall (ICD9-CM external the use of separate methods for a short time frame. We used en- cause codes E880-E888, excluding E887), following an algorithm de- rollment and demographic information from the associated benefi- veloped by Kim et al for identifying fall-related injuries in Medicare ciary summary files (MBSF) to obtain age, sex, race/ethnicity, and inpatient claims data. 22 In the years of our data, reporting of external whether disability was the current reason for Medicare entitlement. cause codes is high, at about 90 percent for all injury cases (Appendix We used the monthly dual-status codes to flag individuals as being S1). We linked these claims to MDS assessments using beneficiary either full duals or restricted/nonduals in the month of their hospital identifiers and then applied criteria for each MDS fall-related item to admission. identify an appropriate denominator for reporting (Figure 1). In the same years as the Medicare claims, we used MDS 3.0 as- Based on the MDS assessment instrument user’s manual,16 we sessments, which collect information on the physical, clinical, and interpreted items J1700A-B as asking about falls that occurred prior 16 Nursing homes must to, and J1800-J1900C as asking about falls that occurred during, complete assessments at least every 92 days as part of a federal re- nursing home residency. As we describe below, in identifying the quirement to participate in the Medicare and Medicaid programs. claims-based falls for each MDS item, we applied the reporting rules We analyzed fall-related questions in Section J, Health Conditions from the manual to ensure only those falls for which the nursing of the MDS instrument (Table 1), focusing in particular on J1900C, homes had a clear reporting responsibility were included. psychological well-being of each patient. as the responses to this question for long-stay residents are used Items J1700A-B ask about falls in the six months prior to to create an NHC quality measure and are part of the star rating the current nursing home entry. To complete this information, algorithm. the steps for a nursing home are to ask family and review med- For facility-level characteristics, we used the Certification and ical records. To minimize any reporting discrepancies caused by Survey Provider Enhanced Reporting (CASPER) dataset, which are prior nursing home residencies during this look back period, we compilations of information collected for the Medicare and Medicaid required patients to have no evidence of nursing home residency certification process, LTCfocus data from Brown University for during the six months prior to hospital admission for the fall. We       3 SANGHAVI et al. Health Services Research TA B L E 1   Fall-related MDS 3.0 items a Section/item description Item Question Possible responses Binary variable created for reporting status Fall History on Admission/ Entry or Reentry J1700A Did the resident have a fall any time in the last month prior to admission/entry or reentry? Yes No Unable to determine 1 if Yes 0 if No or Unable to determine J1700B Did the resident have a fall any time in the last 2-6 months prior to admission/entry or reentry? Yes No Unable to determine 1 if Yes 0 if No or Unable to determine Any Falls Since Admission/ Entry or Reentry or Prior Assessment J1800 Has the resident had any falls since admission/entry or reentry or the prior assessment, whichever is more recent? Yes No 1 if Yes 0 if No Number of Falls Since Admission/Entry or Reentry or Prior Assessment J1900A No injury—no evidence of any injury is noted on physical assessment by the nurse or primary care clinician; no complaints of pain or injury by the resident; no change in the resident’s behavior is noted after the fall One Two or more None 1 if One or Two or more 0 if None J1900B Injury (except major)—skin tears, abrasions, lacerations, superficial bruises, hematomas, and sprains; or any fall-related injury that causes the resident to complain of pain. One Two or more None 1 if One or Two or more 0 if None J1900C a Major injury—bone fractures, joint dislocations, closed head injuries with altered consciousness, subdural hematoma One Two or more None 1 if One or Two or more 0 if None MDS item J1900C is used by Nursing Home Compare to create a patient safety measure and assign five-star ratings. dropped the first six months of claims to have this assessment his- during nursing home residency and is admitted to a hospital with tory for each patient. We then kept those cases with a nursing major injury, we expect the nursing home to report the fall under this home entry/admission assessment within one month and within item on a discharge assessment. two to six months after the hospital admission, respectively, for J1700A and J1700B. Items J1800-J1900C ask about falls during the current resi- 2.3   Constructed measures dency and should be completed on a discharge assessment when a resident is admitted to a hospital. Therefore, we required patients We defined short-stay residents as those whose stays would have to have a discharge assessment from the nursing home, indicating been covered by Medicare. Medicare covers up to 100 days of discharge to a hospital, within one day prior to the hospital admis- postacute nursing home care and requires the Prospective Payment sion. Though we only checked the discharge assessment for fall re- System (PPS) 5-day assessment to be completed within 8 days of ad- porting, we also required a readmission assessment from the same mission. 23 For patients who fell prior to nursing home residency, we nursing home within one day of the hospital discharge to remove looked for this assessment within 8 days of the entry/admission as- cases for which readmission to a different facility could arguably sessment. For patients who fell during residency, we looked for this have created confusion about reporting responsibility. Appendix assessment up to 101 days prior to the discharge to hospital. If we did S2 provides further detail, such as the specific MDS and claims not find the assessment, we determined the resident was long-stay. variables that were used. For finer control of variation in injury severity, we created New Though J1900C asks about major injury falls, all other fall-re- Injury Severity Scores (NISS) using ICDPIC software. 24-31 The NISS is lated items ask about any falls. Nonetheless, we restricted all neither normally distributed nor continuous, so in addition to the nu- our analyses to falls with a secondary ICD9-CM diagnosis code merical score, we created a categorical variable using a breakdown for conditions identified in the MDS definition for major injury, similar to other studies. 24,28,30 To adjust for health status, we used namely bone fractures, joint dislocations, closed head injuries with diagnosis codes on the hospital admission claim to construct com- altered consciousness, and subdural hematomas (specific ICD- bined Charlson/Elixhauser comorbidity scores.32 9CM diagnosis codes provided in Appendix S3). We refer to the At the nursing home level, we classified nursing homes into ter- final set of claims-identified falls for each MDS item as that item’s tiles of size by the total registered resident counts at the time of denominator. the CASPER report, with breaks at 65 and 105 residents. We also Though the MDS rules may appear complex, the context of the chief MDS fall item of interest, J1900C, is simple: If a person falls computed the proportions of residents who are duals, and fall within each race category. 4       SANGHAVI et al. Health Services Research 2.4   Outcome measures 3   R E S U LT S MDS item J1900C is used by CMS to create a quality measure as We identified 150,828 major injury falls in the hospital claims that part of its star rating algorithm. Our primary outcome measure occurred during nursing home residency, which we expect to have was a binary indicator of whether a fall in the J1900C denominator been reported under J1900C, the MDS item used by NHC (Figure 1). was reported or not. The binary indicators of whether claims-iden- Only 57.5 percent of these were reported (Table 2). Reporting on tified falls were reported in the other MDS items were second- this item was more complete for long-stay (62.9 percent) than for ary outcome measures. For J1700A-J1800, we counted the fall short-stay residents (47.2 percent), and for white (59.0 percent) than as reported if the nursing home marked “Yes”; for J1900A-C, we for nonwhite residents (46.4 percent). Long-stay white residents had counted the fall as reported if the nursing home marked “One” or the highest reporting rate (64.5 percent) and short-stay nonwhite “Two or more.” residents had the lowest reporting rate (37.4 percent). Including ad- We calculated a claims-based rate of major injury falls per 100 residents in each nursing home in 2014, our most recent complete ditional assessments beyond the discharge assessment did not improve reporting rates (see Appendix S6). year of data. For each nursing home, we totaled the number of major On the parent question for J1900C, J1800, which asks about injury falls during residency in the claims and divided by the total any falls regardless of injury severity, 62.6 and 82.8 percent of major registered resident counts snapshot variable from CASPER. injury falls identified in the claims were reported for short-stay nonwhite and long-stay white residents, respectively. Among the falls 2.5   Statistical analysis that were reported under J1800, it is possible some are misclassified under J1900A-B as having no or only minor injuries. However, due to the design of these particular survey questions, it is difficult to We computed national reporting rates for each MDS item, sep- ascertain this. arately for short- and long-stay, and white and nonwhite (black, Reporting completeness on J1700A, which asks about falls in Hispanic, Asian, and other race/ethnicity) nursing home residents. the month prior to the current nursing home residency, was high, For each item, we divided the total number of claims-identified ranging from 90.9 percent for long-stay nonwhite to 94.8 percent falls reported by the number of patients in the denominator, and for short-stay white patients. Item J1700B on falls between two to multiplied by 100. six months prior to the current stay had much lower reporting rates To assess patient- and nursing home-level characteristics predic- for all groups. tive of patient level reporting on J1900C, separately for short- and The final models for our main outcome, reporting on J1900C, long-stay patients, we estimated a linear multilevel model with nurs- included all the variables we assessed with linear hypothesis tests ing home random effects and year fixed effects. Candidate predic- (Table 3). Results were generally in the same direction in the short- tors at the nursing home level included our claims-based fall rate, and long-stay populations, so here we focus on the short-stay the Census region, ownership type, nursing home size, race mix, and patients. A higher NISS was associated with a higher reporting proportion dual status, and at the individual level included sex, age probability. After adjusting for nursing home-level differences in ra- (specified via linear splines with cutoff points at quantiles of the age cial composition, reporting was lower for Asians by 5.8 percentage distribution), race, comorbidity score, disability as a reason for cur- points (pp), for black people by 4.2 pp, and for Hispanics by 2.9 pp rent entitlement, dual status, and the NISS as both a categorical and than for white people. Holding constant patient race, having larger numerical variable. We disaggregated the within- and between-nurs- proportions of Asians, black people, or Hispanics (and correspond- ing home effects of race and dual status so that their coefficients ingly fewer white people) was also associated with lower reporting could be interpreted directly (see Appendix S4 for further detail). rates. The between- and within-nursing home coefficients of dual Though our main exhibits show linear models to allow interpretabil- status were positive, indicating that both residing in a home with ity, we included comparison tables with logistic regression models in many duals and being a dual were associated with higher reporting Appendix S5 that demonstrate the similarity of the results from the probability (16.9 pp and 5.8 pp, respectively). A 10 pp higher claims- two approaches. based fall rate in a nursing home was associated with 0.66 pp lower Finally, we defined quintiles of the claims-based fall rates and reporting probability. For-profit nursing homes were associated with within each, computed the percent of nursing homes with four- or lower reporting probability than all other ownership types, in par- five-star overall and quality-domain ratings. We also computed ticular a 4.8 pp lower probability of reporting a fall than a govern- means of these same ratings and the MDS-based major injury fall ment-owned nursing home. measure by quintile. Finally, we estimated the Pearson correlation The Pearson correlation between the 2014 claims-based major coefficients between the claims-based fall rates and these NHC injury fall rates and the MDS-based major injury fall rates reported measures. on NHC was 0.22 (Table 4). Correlations were also poor between The appendices provide further methodological detail, sensitiv- the claims-based fall rates and the NHC quality measure star ratings ity analyses (Appendices S6 and S7) in which we relax some of our (−0.05) and overall star ratings (0.05). Across the quintiles of claims- assumptions, and description of the code. based fall rates, about half of nursing homes had a four- or five-star       5 SANGHAVI et al. Health Services Research overall rating and at least 75 percent had a four- or five-star quality same nursing home within one day of the hospital admission and rating. discharge, respectively, and had discharge assessments from the nursing home. In a sensitivity analysis, allowing falls reported on additional assessments after readmission to the nursing home count 4   D I S CU S S I O N had little effect on the reporting rates. Comparisons between reporting rates of different MDS items This is the first national-level assessment of how nursing homes suggest some insights. The reporting rate for item J1700A, which self-report major injury falls data, which are used by CMS for quality asks about falls in the month prior to the current residency, was 94.3 measurement and public reporting. We found substantial underre- percent —much higher than the rates for J1800 and J1900C, which porting on the specific MDS item (J1900C) used by NHC. Reporting ask about falls during the current residency. This was contrary to ex- rates on the MDS of claims-identified falls by Asian, black, and pectations since in the former case nursing homes rely on second- Hispanic residents were substantially lower than those for white ary sources for fall information but in the latter have the resident people both within and across nursing homes, consistent with long- in their care. One explanation for this could be that administrative 33-35 processes at admission and discharge lend themselves to differences For questions about falls during nursing home residency, we con- in reporting accuracy; another could be that nursing homes underre- standing concerns about racial disparities in nursing home care. servatively identified a denominator population of major injury falls port when they may be considered responsible for a fall. in Medicare hospital admission claims that nursing homes should be Second, J1800, which asks about any falls during the current aware of both administratively and clinically. The individuals who stay, had a higher reporting rate than J1900C, which asks only about experienced these falls were discharged from and returned to the major injury falls during the current stay. It may be that nursing F I G U R E 1   Linkage of Medicare admission claims with MDS assessments to create denominators for fall-related MDS reporting outcomes. Notes: A, White boxes map out paths to the final denominators used to assess nursing home reporting of fall-related items on the MDS. Gray boxes identify observations that were not used in analysis. B, J1700A-J1900C refer to the specific MDS items under study and are described in Table 1. J1900C is the item that is used by CMS for quality reporting on NHC. C, CMS requires discharge assessments and items J1800J1900C in particular if a resident is admitted to a hospital. D, NH = nursing home 6       SANGHAVI et al. Health Services Research TA B L E 2   National reporting rates of major injury falls by race and short- vs. long-stay in 2011-2015 a Percent of major injury falls reported (25th, 75th percentile) d Short-stay Short-staya Long-stay White Nonwhite White Nonwhite 804 742 85 246 173 032 65 222 10 925 18 385 45 617 6 310 87 043 11 858 Long-stay Nonwhite White Nonwhiteb Fall item White 29 255 J1700A 94.8 (92.3, 100.0) 91.6 (91.7, 100.0) 94.0 (93.3, 100.0) 90.9 (97.6, 100.0) 4 013 J1700B 41.8 (8.3, 66.7) 33.2 (0.0, 66.7) 44.4 (0.0, 100.0) 33.2 (0.0, 100.0) J1800 67.8 (50.0, 100.0) 62.6 (0.0, 100.0) 82.8 (71.4, 100.0) 76.1 (60.0, 100.0) J1900A 17.4 (0.0, 33.3) 18.9 (0.0, 33.3) 23.5 (0.0, 33.3) 23.1 (0.0, 50.0) J1900B 18.0 (0.0, 30.0) 19.2 (0.0, 25.0) 21.0 (0.0, 33.3) 21.8 (0.0, 33.33) J1900Cc 48.6 (22.2, 80.0) 37.4 (0.0, 100.0) 64.5 (46.7, 87.5) 51.3 (0.0, 100) Patients who stayed in nursing homes for less than 101 days are classified as short-stay patients, otherwise long-stay patients. b c Number of major injury falls in item denominator Patients are categorized as either white or nonwhite, which includes black, Hispanic, Asian, and other race/ethnicity. MDS item J1900C is used by Nursing Home Compare to create a patient safety measure and assign five-star ratings. d The reporting rates are all statistically significant at an alpha level of 0.05. homes have trouble with injury severity classification prior to learn- of manipulation around surveyor visits. 38 Given this context and ing the hospital’s diagnosis. However, the patients in our J1900C our results, other MDS-based measures should also be assessed denominator returned to the same nursing home, nursing homes for accuracy. have 14 calendar days to submit discharge assessments, and detailed This study has limitations. First, our denominators only included policies and procedures are in place for submitting corrections to Medicare beneficiaries who had falls severe enough to lead to hos- the MDS. Nonetheless, it may be difficult in practice to follow these pital admission and be classified as major injury according to the rules; alternatively, nursing homes may be downgrading the severity MDS. For example, patients who received only outpatient service in to improve their quality ratings. an emergency department would not be included in our study. This To our knowledge, only state and federal offices with oversight does not affect reporting rates in our sample of serious falls, espe- responsibilities have compared patient-level MDS records with other cially for J1900C, the item used by NHC that focuses on major injury data sources, typically medical records or medical assessments.3,4,6-9 falls, which are more likely to result in hospital admission. However, For instance, in a 2014 audit, CMS compared MDS assessments with extrapolating from our estimates to reporting rates for less severe patients’ medical records in 25 volunteer nursing homes for up to falls is unlikely to produce accurate estimates. 10 patients per home.9 For falls, 26 percent of reviewed MDS as- Second, we relied on claims data, which are not medical records, sessments disagreed with the medical record as to whether the pa- for diagnosis information. If these contain errors, we may have incor- tient sustained a fall-related major injury. These studies, conducted rectly identified some cases or missed others, and our comorbidity in a handful of sites, based denominators on the MDS rather than a scores and injury severity scores may also not be accurate. At the validation source and therefore may have entirely missed cases un- same time, these data are used extensively in high-quality health ser- reported by the MDS. The HHS OIG and the GAO have repeatedly vices research, including to identify fall-related injuries. Furthermore, recommended that CMS check the reliability of MDS data, most re- it is unlikely that missingness in diagnosis coding for a fall-related cently in September 2018. 36 The poor correlation between our claims-based fall rates and the MDS-based NHC-reported fall rates indicates the MDS not hospital admission claim would be systematically correlated with nursing home reporting of a fall in the MDS, given the two institutions have separate administrative processes and incentives. only underreports but also may not be informative for comparing Third, it is possible that some claims-identified falls in our J1800- nursing homes. Correlations were also poor between the claims- J1900C denominators occurred in-hospital during a visit for an un- based measure and the quality-domain and overall five-star rat- related condition. However, this is unlikely because CMS considers ings, though this is less surprising given the five-star algorithm’s in-hospital falls to be “never events” and does not reimburse hospi- complexity and use of other information. Nonetheless, such incon- tals for the associated costs.39 If “never event” claims are submitted sistencies between the five-star ratings and other nursing home- for tracking purposes, they are expected to be submitted as “no-pay” level quality measures have been documented elsewhere.12,13,37 claims, which are not included in our MedPAR file. For example, in an analysis of fee-for-service Medicare beneficia- Finally, though we found weak correlations between our claims- ries, Neuman et al were unable to estimate consistent associations based major injury fall rates and other NHC measures, we were not between readmission or death risk and MDS-based measures, in- able to investigate reasons beyond poor MDS reporting. cluding those for pain and ulcers.12 Similarly, in a comparison of A few policy implications flow out of our study for measuring and facility-reported staffing data with potentially more objective monitoring falls in nursing homes. First, claims-based measures may payroll-based data, Geng et al found discrepancies and evidence be useful supplements or replacements for the MDS-based patient       7 SANGHAVI et al. Health Services Research TA B L E 3   Linear multilevel modelsb,d of MDS item J1900C a reporting outcome in 2011-2015   Short-stay   Long-stay   Patient-level characteristics Female 0.011*   0.018***   <78 0.003***   0.002***   78-85 0.005***   0.003**   85-90 0.000   0.003**   >90 0.003   0.001   Age linear spline Racec White (Ref)         Asian −0.058**   −0.041*   Black −0.042***   −0.037***   Hispanic −0.029*   −0.015   Other −0.001   0.006     0.034***   NISS 0.045*** NISS category 1-15 (Ref)         16-24 −0.441***   −0.329***   25-40 −0.934***   −0.728***   40-75 −1.546***   −1.290***   Disability status 0.009   Dual status 0.058*** Comorbidity score 0.001*   0.003     0.054***     −0.002***   Short-stay   Long-stay −0.066 −0.041   Nursing home-level characteristics Claims-based fall rate Between NH dual association 0.169*** 0.162*** 0.005 0.100*** −0.025 0.086*** Region South (Ref) Midwest   0.018**   0.014* Northeast −0.003 −0.010 West −0.032*** −0.044***   0.031*** 0.023*** −0.018*   0.028*** 0.020*** −0.024*** Ownership type For-profit (Ref)         Government 0.048*** 0.057*** 0.066*** 0.070*** Nonprofit 0.010 0.017** 0.031*** 0.037*** Other 0.006 0.012 0.023 0.022 Provider size Large (Ref)         Medium 0.026*** 0.028*** 0.018*** 0.017*** Small 0.036*** 0.040*** 0.028*** 0.028*** Between NH race associationc Asian −0.271*** −0.283*** −0.217*** −0.220*** Black −0.251*** −0.299*** −0.290*** −0.332*** Hispanic −0.278*** −0.318*** −0.267*** −0.302*** Other −0.022 −0.065 0.059 0.039 (Continues) 8       SANGHAVI et al. Health Services Research TA B L E 3   (Continued)   Short-stay   Long-stay   2011 (Ref)         2012 0.016* 0.020** 0.028*** 0.030*** Year 2013 0.028*** 0.036*** 0.028*** 0.032*** 2014 0.022*** 0.026*** 0.030*** 0.033*** 2015 Variance explained by random effects 0.021** 0.026*** 0.044*** 0.048*** 0.014 0.016 0.021 0.022 Variance explained by fixed effects 0.021 0.005 0.014 0.005 Residual variance 0.213 0.229 0.198 0.206 Within NH variance 0.017 0.000 0.008 0.000 Between NH variance 0.019 0.021 0.026 0.028 Abbreviations: NISS, New Injury Severity Score; NH, nursing home. *P < .05, **P < .01, ***P < .001. a MDS item J1900C is used by NHC to create a patient safety measure and assign five-star ratings. b Data are modeled at the patient level, and outcome is a binary indicator of whether the patient’s major injury fall was reported. c The patient-level race measure can be interpreted as follows in the case of black residents: On average, being black rather than white is associated with a 4.2 percentage point lower probability of a major injury fall being reported on J1900C, controlling for nursing home-level race mix. The nursing home-level race measure can be interpreted as follows in the case of more black residents: Holding constant patient race, increasing the proportion of black residents from 0 to 1 is associated with a 25.1 percentage point lower probability of a major injury fall being reported on J1900C. d Linear multilevel models are shown here for ease of interpretation. Appendix Tables S5 and S6 show multilevel logistic regression models and a comparison table to demonstrate the two approaches produce similar results. TA B L E 4   Correlations between claims-based fall rates and Nursing Home Compare measures in 2014 Percent of NHs with 4- or 5-star ratings NH average ratings Quintiles of claims-based fall rates, means, 10th, 90th percentiles Overall rating 6.0 (4.5, 8.1) 53.3 75.4 3.40 4.03 4.14 3.6 (3.1, 4.2) 51.5 78.2 3.36 4.11 3.55 Quality measure rating Overall rating Quality measure rating MDS 3.0 Major injury falls measure (N013.01) 2.6 (2.2, 2.9) 50.4 80.5 3.35 4.16 3.25 1.8 (1.5, 2.1) 48.8 77.1 3.29 4.08 3.13 1.1 (0.7, 1.4) 47.0 81.9 3.21 4.22 2.65 Correlation coefficients between claims-based fall rates and measure 0.046 −0.048 0.223 Abbreviation: NH, nursing home. a Claims-based fall rates are the number of major injury falls identified in Medicare Provider Analysis and Review (MedPAR) per 100 registered residents in each nursing home in the year 2014. b On NHC, the overall rating is based on a nursing home’s ratings for health inspections, quality measures (QMs), and staffing, while the quality rating is based on only the 16 physical and clinical QMs. The NHC MDS 3.0 measure (N013.01) is the percent of long-stay residents experiencing one or more falls with major injury. safety indicator. CMS has already introduced a few claims-based uti- Our study indicates an urgent need to assess the value and lim- lization measures, so the addition may not be overly burdensome. its of patient safety measurement that is based on the MDS. Given This would be consistent with CMS’s move toward payroll-based the amount of research that has been based on the MDS, it may be staffing data for NHC, which are likely more reliable than the previ- important to revisit some of our understanding of nursing home ous self-reported source.38 Second, it may be possible to make mod- quality of care. For example, given underreporting was worse for el-based corrections of MDS reporting rates based on nursing home underserved populations in our analysis, disparities in these settings characteristics and history. Finally, both claims data and models such may currently be poorly estimated. At the same time, alternative and as the one we developed may be used for automatic and targeted additional approaches to monitoring and measuring patient safety in auditing. nursing homes should be developed. Health Services Research AC K N OW L E D G M E N T S Joint Acknowledgment/Disclosure Statement: We thank Dr Alan Zaslavsky at Harvard Medical School and Dr Donald Hedeker at the University of Chicago for valuable conversations on statistics, and Dr David Grabowski at Harvard Medical School for resources that allowed early data analysis and helpful conversations about nursing home quality measures. C O N FL I C T O F I N T E R E S T We have no conflicts of interests pertaining to this article. I R B A P P R OVA L S The research protocol was approved by the relevant institutional review board at the University of Chicago. All data analyses were conducted by PS and SP. ORCID Prachi Sanghavi        9 SANGHAVI et al. https://orcid.org/0000-0002-0738-3193 REFERENCES 1. Committee on Nursing Home Regulation, Institute of Medicine. Improving the Quality of Care in Nursing Homes. Washington, DC: National Academies Press; 1986. 2. U S Government Accountability Office. GAO-12-390, Nursing Homes: CMS Needs Milestones and Timelines to Ensure Goals for the Five-Star Quality Rating System Are Met. 2012. 3. Department of Health and Human Services Inspector General. Nursing home patient assessment: Quality of care. 2001. http://oig. hhs.gov/oei/repor ​t s/oei-02-99-00040.pdf. 4. Department of Health and Human Services Inspector General. Inappropriate payments to skilled nursing facilities cost Medicare more than a billion dollars in 2009. 2012. http://oig.hhs.gov/oei/ repor​t s/oei-02-09-00200.pdf. Accessed December 18, 2019. 5. Kaiser Family Foundation. Implementation of Affordable Care Act Provisions To Improve Nursing Home Transparency, Care Quality, and Abuse. Prevention - Issue Paper. 2013. 6. U S Government Accountability Office. GAO-02-279, Nursing Homes: Federal Efforts to Monitor Resident Assessment Data Should Complement State Activities. 2002. 7. U S Government Accountability Office. GAO-03-187, Nursing Homes: Public Reporting of Quality Indicators Has Merit, but National Implementation Is Premature. 2002. 8. U S Government Accountability Office. GAO-06-117, Nursing Homes: despite increased oversight, challenges remain in ensuring high-quality care and resident safety. December 2005. 9. Survey and Certification Group, Centers for Medicare and Medicaid Services. MDS/Staffing focused surveys update. https​://www.cms. gov/Medic​are/Provi​der-Enrol​lment-and-Certi​f icat​ion/Surve​y Cert​ ifica​t ionG​e nInf​o/Downl​o ads/Survey-and-Cert-Letter-15-25.pdf. Published February 13, 2015. Accessed February 4, 2017. 10. Editorial Board. Is that really a five-star nursing home? New York Times. February 25, 2015:A22. 11. Thomas K. Medicare star ratings allow nursing homes to game the system. New York Times: 2014. 12. Neuman MD, Wirtalla C, Werner RM. Association between skilled nursing facility quality indicators and hospital readmissions. JAMA. 2014;312(15):1542-1551. 13. Han X, Yaraghi N, Gopal R. Winning at all costs: Analysis of Inflation in nursing homes’ rating system. Prod Oper Manag. 2017;27(2):215-233. 14. Rantz MJ, Flesner MK, Zwygart-Stauffacher M. Improving care in nursing homes using quality measures/indicators and complexity science. J Nurs Care Qual. 2010;25(1):5-12. 15. Burns E, Kakara R. Deaths from falls among persons aged ≥65 years — United States, 2007–2016. MMWR Morb Mortal Wkly Rep. 2018;67(18):509-514. 16. Centers for Medicare & Medicaid Services. Minimum Data Set (MDS) - Version 3.0, Resident Assessment and Care Screening, Nursing Home Comprehensive Item Set. https​: //www.cms.gov/ Medic​a re/Quali​t y-Initi​a tives-patie​n t-asses​s ment-instr​u ment​s / Nursi​n gHom​e Qual​i tyIn​i ts/MDS30​R AIMa​n ual.html. Published January 8, 2017. Accessed February 4, 2017. 17. Bohl AA, Phelan EA, Fishman PA, Harris JR. How are the costs of care for medical falls distributed? The costs of medical falls by component of cost, timing, and injury severity. The Gerontologist. 2012;52(5):664-675. 18. Hoffman GJ, Hays RD, Shapiro MF, Wallace SP, Ettner SL. The costs of fall-related injuries among older adults: Annual per-faller, service component, and patient out-of-pocket costs. Health Serv Res. 2016;52(5):1794-1816. 19. Pineles SL, Repka MX, Yu F, Lum F, Coleman AL. Risk of musculoskeletal injuries, fractures, and falls in medicare beneficiaries With disorders of binocular vision. JAMA Ophthalmol. 2015;133(1):60-66. 20. Ganz DA, Kim S-B, Zingmond DS, et al. Effect of a falls quality improvement program on serious fall-related injuries. J Am Geriatr Soc. 2015;63(1):63-70. 21. Segal M, Rollins E, Hodges K, Roozeboom M. Medicare-Medicaid Eligible Beneficiaries and Potentially Avoidable Hospitalizations. MMRR. 2014;4(1):E1-E13. 22. Kim S-B, Zingmond DS, Keeler EB, et al. Development of an algorithm to identify fall-related injuries and costs in Medicare data. Inj Epidemiol. 2016;3(1):1. 23. Centers for Medicare & Medicaid Services. MDS 3.0 Quality Measures User’s Manual - v9.0. October 2015. 24. MacKenzie EJ, Rivara FP, Jurkovich GJ, et al. A national evaluation of the effect of trauma-center care on mortality. N Engl J Med. 2006;354(4):366-378. 25. Osler T, Baker SP, Long W. A modification of the injury severity score that both improves accuracy and simplifies scoring. J Trauma. 1997;43(6):922-926. 26. Brenneman FD, Boulanger BR, McLellan BA, Redelmeier DA. Measuring injury severity: time for a change? J Trauma. 1998;44(4):580-582. 27. Sacco WJ, MacKenzie EJ, Champion HR, Davis EG, Buckman RF. Comparison of alternative methods for assessing injury severity based on anatomic descriptors. J Trauma. 1999;47(3):441-446. 28. Stevenson M, Segui-Gomez M, Lescohier I, Di Scala C, McDonaldSmith G. An overview of the injury severity score and the new injury severity score. Inj Prev. 2001;7(1):10-13. 29. Clark D, Osler T, Hahn D. ICDPIC: Stata module to provide methods for translating international classification of disease (Ninth Revision) diagnosis codes into standard injury categories and/or scores. 2010. http://ideas.repec.org/c/boc/bocod​e/s4570​28.html. Accessed December 18, 2019. 3 0. Sears JM, Blanar L, Bowman SM. Predicting work-related disability and medical cost outcomes: a comparison of injury severity scoring methods. Injury. 2014;45(1):16-22. 31. Meredith JW, Evans G, Kilgo PD, et al. A comparison of the abilities of nine scoring algorithms in predicting mortality. J Trauma. 2002;53(4):621–8. discussion628–9. 32. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749-759. 33. Mor V, Zinn J, Angelelli J, Teno JM, Miller SC. Driven to tiers: socioeconomic and racial disparities in the quality of nursing home care. Milbank Q. 2004;82(2):227-256. 10       SANGHAVI et al. Health Services Research 3 4. Konetzka RT, Werner RM. Review: Disparities in long-term care. Med Care Res Rev. 2009;66(5):491-521. 35. Smith DB, Feng Z, Fennell ML, Zinn JS, Mor V. Separate and unequal: Racial segregation and disparities in quality across U.S. nursing homes. Health Aff. 2007;26(5):1448-1458. 36. U S Government Accountability Office. GAO-18-694T, NURSING HOME QUALITY: Continued Improvements Needed in CMS’s Data and Oversight. September 2018:1-17. 37. U S Government Accountability Office. GAO-16-33, Nursing Home Quality: CMS Should Continue to Improve Data and Oversight. 2015. 38. Geng F, Stevenson DG, Grabowski DC. Daily nursing home staffing levels highly variable, often below CMS expectations. Health Aff. 2019;38(7):1095-1100. 39. Agency for Healthcare Research and Quality. Patient Safety Primer: Never Events. 2019. https​://psnet.ahrq.gov/prime​r/never-events. Accessed December 18, 2019. S U P P O R T I N G I N FO R M AT I O N Additional supporting information may be found online in the Supporting Information section.   How to cite this article: Sanghavi P, Pan S, Caudry D. Assessment of nursing home reporting of major injury falls for quality measurement on nursing home compare. Health Serv Res. 2019;00:1–10. https​://doi.org/10.1111/14756773.13247​