California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR The Data Standards and Quality Control Unit (DSQC) of the California Cancer Registry (CCR) performed a reabstracting audit remotely from the CCR offices in Sacramento in 2012. The audit utilized the Electronic Medical Record (EMR) remotely. One facility was audited; a community hospital in Region 6. The audit design evaluated the efficacy of performing an audit remotely while performing a routine reabstracting audit. The audit had two separate and equally important elements; audit performance and data quality. The audit performance component evaluated the efficiency of remote access, reliability and completeness of the EMR information accessed, and quality of the audit being performed in this manner. The data quality component evaluated the quality and completeness of the data reported. There were 58 data items reabstracted from each of the 60 source records, for a total of 3480 possible discrepancies. The list of audited data items are found on Table 1. This audit focused on diagnosis year 2010 cases. There were 60 recently abstracted and transmitted cases selected for this audit. The selected cases consisted of 15 colon cases (C18.0 – C18.9). There were 15 lung and bronchus cases (C34.0 – C34.9), 15 breast cases (C50.0 – C50.9) and 15 prostate cases (C61.9). This breakdown is demonstrated in Table 2. Performance Methodology Per audit protocol, the facility was sent an audit initiation letter which outlined the request for access to the EMR as well as the specific elements within the EMR that needed to be reviewed and reabstracted. The facility required a two week notice prior to commencing the audit in order to set up remote access. The auditor was required to sign and submit a Confidentiality Statement. A list of cases that were to be abstracted was provided to the facility. The auditor used CITRIX to access each facility’s electronic medical record (EMR). The EMR software that the hospital uses is named Horizon Patient Folder (HPF). Once approval was obtained and access granted, the auditor was given access to the EMR’s for a period of 60 days. It was requested that the facility provide the auditor with instructions on how to log into each system, how to navigate between the two systems, and how to find the documents that were needed to complete the audit. This was performed and the auditor was able to navigate the system with little effort. April 3, 2012 1 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Audit Performance Results This facility is a medium sized community hospital in a rural location in the northern central valley. Hospital demographics include: • • • Approximately 1200 cases abstracted and submitted to the CCR annually ACoS accreditation status: Approved as a Comprehensive Community Cancer Program Cancer registry staff is hospital based personnel The EMR at this hospital was easy to navigate and was relatively easy to locate information required to reabstract cases for the audit. There were advantages and disadvantages to performing this audit remotely. Some of the advantages to performing this reabstracting audit remotely for this facility include: • • Cost savings in the elimination of travel expenses Ability to perform other tasks and assignments while conducting the audit Disadvantages of performing reabstracting audits remotely on this facility include: • • Substantial amount of documents and information available resulted in a slower review of the medical record, impacting the audit completion timeline In some instances, documents needed to reabstract all data items were never located and resulted in a very large number of possible discrepancies sent back to facility for reconciliation Data Quality Overall There were a total of 104 discrepancies identified on this audit which resulted in an accuracy rate of 96.7%. A majority of the discrepancies occurred in the breast cases of which there were 38 (36.5%) discrepancies noted. Colon cases had the second highest number of discrepancies with 24 (23.1%); Lung was next with 21 (20.2%) discrepancies, and prostate had 21 (20.2%) discrepancies. This is demonstrated in Graph 1. April 3, 2012 2 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR The data items that make up the top discrepancies are: • • • • • • CS Extension Surgery Primary Site CS Site Specific Factor # 2 CS Site Specific Factor # 8 CS Site Specific Factor # 11 CS Site Specific Factor # 12 11 (10.6%) 10 (9.6%) 8 (7.7%) 8 (7.7%) 5 (4.8%) 5 (4.8%) The distribution of all discrepancies identified on this audit can be found on Table 3. The distribution of discrepancies by primary site are found in Tables 4 – 7. CS Extension Discrepancies The data item CS Extension had the highest number of discrepancies noted with 11 (10.6%). Eight (72.7%) discrepancies occurred in the lung cases, two (18.2%) were identified in the prostate cases, and one (9.1%) occurred in the colon cases. Of the eight discrepancies identified in the lung cases, five (62.5%) were recoded to a more specific CS Extension code documenting conditions such as vocal cord paralysis or erosion of an adjacent rib. Two discrepancies (25%) were recoded to a lower, more specific code. Both of these cases were coded to CS Extension code 115 which is defined as “Stated as T1a with no other information on extension.” Both of these cases had enough information to code these cases to CS Extension code 100 which is defined as “Tumor confined to one lung.” CS Extension codes that are defined as “stated AJCC stage,” such as code 115 listed above, are only to be used if there is no other information available on the chart. In these cases, there was enough information available to code the CS Extension to a specific code. One discrepancy was the result of a recode to a CS Extension code that documented a stage with no other information available. Surgery of the Primary Site Discrepancies Surgery of the Primary Site had the second highest number of discrepancies noted with 10 (9.6%) discrepancies. Five discrepancies (50%) were identified in the breast cases, four (40%) were identified in the colon cases, and one (10%) was found in the prostate cases. Of the five discrepancies among the breast cases, three (60%) were due to the confusion of the definition of tissue versus implant reconstruction. The remaining two cases (40%) were recoded to a more April 3, 2012 3 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR specific description of the surgical procedure performed. Of the four (40%) discrepancies in colon, three cases (75%) were recoded to a higher, more specific surgery code 40 (hemicolectomy) from the less specific surgery code 30 (partial colectomy/segmental resection). The data items CS Site Specific Factor #2 and CS Site Specific Factor #8 had eight discrepancies (7.7%) each. The details are as follows: CS Site Specific Factor #2 - Prostate The definition of this site specific factor in prostate is “the Prostatic Specific Antigen (PSA) Interpretation.” In all four discrepancies (100%) noted in this data item in the prostate cases, the codes were recoded from code 030 “Borderline; undetermined whether positive or negative” to 010 “elevated.” This is a significant finding in that the interpretation of the PSA value was coded to undetermined when the value of the PSA and interpretation was available. Furthermore, the value of the PSA was coded in CS Site Specific Factor #1 (Prostatic Specific Antigen (PSA) Lab Value). Note #2 for Site Specific Factor #2 states “Lab values for SSF’s 1 and 2 should be from the same laboratory test.” This note is repeated in the instructions for coding Site Specific Factor #1. None of these cases had discrepancies noted in Site Specific Factor #1. CS Site Specific Factor #2 - Lung There were two discrepancies (25%) in this data item in the lung cases. The definition for CS Site Specific Factor #2 in lung is “Visceral Pleural Invasion (VPI)/ Elastic Layer.” Both of these cases were recoded to 998 which is defined as “No histologic examination of the pleura.” CS Site Specific Factor #8 - Breast Of the eight discrepancies noted in CS Site Specific Factor #8, five (62.5%) occurred in the breast cases. For breast cases, CS Site Specific Factor 8 is defined as “HER2: IHC Test Lab Value.” All recodes were recoded from the code 999 (unknown or no information) to either code 002 (Score 2+) or to code 3 (Score 3+). This means that there was a value that was available for recode. CS Site Specific Factor #8 - Colon The three (37.5%) remaining CS Site Specific Factor #8 discrepancies were noted among the colon cases. The definition of CS Site Specific Factor #8 for colon is “Perineural Invasion.” All three (100%) of these recodes were identical. The original code of 999 (Unknown, Not documented in patient record) was recoded to 000 (None; no Perineural invasion present). The April 3, 2012 4 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR note listed in the instructions for coding this data item clearly state “Assign code 000 if histologic examination of primary site was performed, the pathology report is available for review, and perineural invasion is not mentioned.” Resource Allocation, Benefit, and Core Team • • Projected Hours to Complete Project: Estimated at 250 hours from project kick-off to Final Report submission. Actual Hours to Complete Project: Approximately 300 hours. The business benefit of this project is that a reporting facility data quality assessment was performed. Possible training issues specific to the reporting facility were identified. The core team included Kyle L. Ziegler, CTR, Scott Riddle, and Lois Inferrera, CTR Conclusion Performing audits remotely can be challenging depending on the type of EMR that is utilized. Nevertheless, auditing remotely is currently the most efficient and cost effective method of auditing. In many of the discrepancies identified in this audit, the information was available in the medical record but was not captured in the coding. The coding of the extension of disease at diagnosis, the identification of lab test values such as PSA or HER2, and the coding of surgical procedures performed are critical data elements to support cancer staging and treatment information. This indicates a training issue that may be unique to this facility. The scope of these issues may need to be further assessed as resources allow. If warranted, coding instructions, clarifications and reminders could be issued statewide through the newsletter, educational modules or Data Alerts. April 3, 2012 5 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Stage Data Item Social Security Number Date of Birth Gender Race 1 Race 2 Spanish/Hispanic Origin Marital Status Lymph Nodes Positive/Examined Date of Diagnosis Primary Site (including subsite) Laterality Histology Type (ICD-O-3) Behavior Grade Diagnostic Confirmation Grade Path System Grade Path Value Rx Summ - RX Status Date of Surgery Surgery of Primary Site Scope of Regional Lymph Nodes Regional Radiation Treatment Modality Regional Radiation Boost Chemotherapy Summary Hormone Summary CS Tumor Size CS Extension CS Lymph Nodes CS Mets at Diagnosis CS Mets Bone CS Mets Brain CS Mets Liver CS Mets Lung CS Site Specific Factor 1 CS Site Specific Factor 2 CS Site Specific Factor 3 CS Site Specific Factor 4 CS Site Specific Factor 5 CS Site Specific Factor 6 CS Site Specific Factor 7 CS Site Specific Factor 8 CS Site Specific Factor 9 CS Site Specific Factor 10 CS Site Specific Factor 11 CS Site Specific Factor 12 CS Site Specific Factor 13 CS Site Specific Factor 14 Treatment Data Item Tumor Data Demographic Data Items by Category Data Item CS Site Specific Factor 15 CS Site Specific Factor 16 CS Site Specific Factor 17 CS Site Specific Factor 18 CS Site Specific Factor 19 CS Site Specific Factor 20 CS Site Specific Factor 21 CS Site Specific Factor 22 CS Site Specific Factor 23 CS Site Specific Factor 24 CS Site Specific Factor 25 Table 1 April 3, 2012 6 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Number of Cases Reabstracted by Primary Site with Distribution of Discrepancies by Case and Primary Site Cases Discrepancies Total Number of Cases Sampled Number of Cases with Discrepancies Number of Discrepancies Colon (C18.0 - C18.9) 15 9 24 24% Lung (C34.0 - C34.9) 15 11 21 21% Breast (C50.1 - C50.9) 15 14 38 33% Prostate (C61.9) 15 10 21 22% Total 60 44 104 100% Primary Site Percent Table 2 April 3, 2012 7 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Distribution of Discrepancies by Data Item Data Item CS Extension Surgery Primary Site CS Site Specific Factor # 2 CS Site Specific Factor # 8 CS Site Specific Factor # 11 CS Site Specific Factor # 12 Date of Diagnosis Histology (ICD-O-3) Radiation Treatment Modality CS Site Specific Factor # 6 CS Site Specific Factor # 7 CS Site Specific Factor # 9 CS Site Specific Factor # 13 CS Site Specific Factor # 4 CS Site Specific Factor # 10 Diagnostic Confirmation CS Lymph Nodes CS Mets at Diagnosis RX Sum - Treatment Status Radiation Boost Hormone Summary Grade Lymph Nodes Examined/Positive CS Tumor Size Date Surgery Scope Regional Lymph Node Surgery Chemotherapy Summary CS Site Specific Factor # 1 CS Site Specific Factor # 3 CS Site Specific Factor # 5 CS Site Specific Factor # 22 CS Site Specific Factor # 23 Discrepancy 11 10 8 8 5 5 4 4 4 4 4 4 4 3 3 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 Percent 10.6% 9.6% 7.7% 7.7% 4.8% 4.8% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 3.8% 2.9% 2.9% 1.9% 1.9% 1.9% 1.9% 1.9% 1.9% 1.0% 1.0% 1.0% 1.0% 1.0% 1.0% 1.0% 1.0% 1.0% 1.0% 1.0% Table 3 April 3, 2012 8 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Distribution of Discrepancies by Primary Site 21 21 24 Breast Colon Lung Prostate 38 Graph 1 April 3, 2012 9 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Discrepancies by Primary Site – Breast (C50.0 – C50.9) Data Item Surgery Primary Site CS Site Specific Factor # 8 CS Site Specific Factor # 7 CS Site Specific Factor # 9 CS Site Specific Factor # 11 Histology (ICD-O-3) CS Site Specific Factor # 10 Hormone Summary Radiation Treatment Modality CS Site Specific Factor # 6 CS Lymph Nodes RX Sum - Treatment Status Radiation Boost Lymph Nodes Examined/Positive CS Tumor Size Chemotherapy Summary CS Site Specific Factor # 5 CS Site Specific Factor # 22 CS Site Specific Factor # 23 Discrepancy 5 5 4 4 3 2 2 2 1 1 1 1 1 1 1 1 1 1 1 Percent 13.2% 13.2% 10.5% 10.5% 7.9% 5.3% 5.3% 5.3% 2.6% 2.6% 2.6% 2.6% 2.6% 2.6% 2.6% 2.6% 2.6% 2.6% 2.6% Table 4 April 3, 2012 10 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Discrepancies by Primary Site – Colon (C18.0 – C18.9) Data Item Surgery Primary Site CS Site Specific Factor # 8 CS Site Specific Factor # 6 CS Site Specific Factor # 4 CS Site Specific Factor # 2 Date of Diagnosis CS Extension Histology (ICD-O-3) CS Lymph Nodes CS Mets at Diagnosis RX Sum - Treatment Status Grade CS Site Specific Factor # 1 Discrepancy 4 3 3 3 2 2 1 1 1 1 1 1 1 Percent 16.7% 12.5% 12.5% 12.5% 8.3% 8.3% 4.2% 4.2% 4.2% 4.2% 4.2% 4.2% 4.2% Table 5 April 3, 2012 11 California Cancer Registry 2012 Remote Reabstracting Audit CCR Project ID #300.70 Final Report Prepared by Kyle L. Ziegler, CTR Discrepancies by Primary Site – Lung (C34.0 – C34.9) Data Item Discrepancy Percent CS Extension 8 38.1% Radiation Treatment Modality 3 14.3% CS Site Specific Factor # 2 2 9.5% Date of Diagnosis 2 9.5% Diagnostic Confirmation 2 9.5% Histology (ICD-O-3) 1 4.8% CS Mets at Diagnosis 1 4.8% Radiation Boost 1 4.8% Date Surgery 1 4.8% Table 6 Discrepancies by Primary Site – Prostate (C61.9) Data Item CS Site Specific Factor # 12 CS Site Specific Factor # 2 CS Site Specific Factor # 13 CS Extension CS Site Specific Factor # 11 Surgery Primary Site CS Site Specific Factor # 10 Scope Regional Lymph Node Surgery CS Site Specific Factor # 3 Discrepancy 5 4 4 2 2 1 1 1 1 Percent 23.8% 19.0% 19.0% 9.5% 9.5% 4.8% 4.8% 4.8% 4.8% Table 7 April 3, 2012 12