Statewide Cancer Reporting System
(CalCARES Program, Institute for Population Health Improvement,
UC Davis Health System)
Mid-Year Progress Report
Grant Agreement Number 11-10828
Reporting Period: July 1, 2013 – December 31, 2013
Executive Summary
The CalCARES program, Institute of Population Health Improvement (IPHI), UC Davis
Health System (UCDHS) made significant progress toward meeting program objectives
during this reporting period. Notable accomplishments include: submissions of high
quality cancer data to federal funders, producing reports and manuscripts using
California Cancer Registry (CCR) data, successful deployment of Eureka versions 12.1
and 12.2, and working with California Department of Public Health (CDPH) staff to
resolve overarching administrative and program management issues.
Shortly after the first year of grant closed, it became apparent to IPHI that the hybrid
management model, (with day-to-day staff direction provided by both CPDH and
IPHI/UCD managers and split responsibility/authority for work activities) was not an
optimal arrangement. In addition, the scope of work (SOW), with 475+ pages of detailed
work activities, placed a considerable administrative burden on IPHI staff, often at the
expense of major work activities.
Divergent views of the management model and interpretation of the grant terms
necessitated meetings with both CDPH program level staff and directorate to discuss
and develop solutions to these issues. CDPH upper management indicated that it was
open to considering changes in the grant agreement. IPHI subsequently proposed
revisions to both the SOW and grant language to more accurately represent IPHI’s
collaboration with CDPH as an equal partnership. The approximately 40-page proposed
SOW revision collapses the existing activities at the Performance Objective/Required
Service Level, which does not change important objectives or required work. The intent
of the revision is to ease the administrative and reporting burden, while maintaining
sufficient levels of accountability. The proposal is currently under CDPH review. At the
program level, CalCARES and CDPH branch staff worked together to develop separate
operational plans that reflect the individual responsibilities and contributions of each
institution to overall CCR operations.
Delays in the submission and processing of Budget Amendment 3 (which was required
to align the originally submitted grant budget with actual program expenditures) were
unintended consequences of the management differences between the two institutions
during this reporting period. Although CDPH program staff granted IPHI the maximum
allowable $100,000 in budget flexibility, this was insufficient to cover all program
expenditures. Subsequently, IPHI made a tactical decision to terminate four contract
employees, as we could not obtain written assurance from CDPH that all expenditures
associated with retaining the contractors would be covered once the agreement was

 eventually executed. As these contractors were contributing to critical performance
objectives related to Eureka development and enhancement, their terminations resulted
in delays in various associated work activities
Shortly after this reporting period, CDPH fully executed Amendment 3 and is currently
reviewing Amendment 4 (IPHI’s proposed revisions to the SOW and contract language).
The full execution of Amendment 3 has provided IPHI with budget relief and associated
ability to once again move forward on critical work activities. CDPH approval of
Amendment 4 will provide both CDPH and CalCARES program staff with administrative
relief, and the CalCARES program increased operational flexibility to meet grant
deliverables.
Although the program faced some unique and significant management challenges
which impacted operations during this reporting period, IPHI staff continue to work
toward successful resolution of the issues in collaboration with CDPH staff. The
CalCARES program is optimistic that these challenges are largely behind us and we
look forward to working with all CCR stakeholders to contribute to extending the vision
and reach of the CCR.
Staffing and Hiring Updates
The CalCARES program filled several critical staff vacancies during the reporting
period, including the permanent appointments of existing staff as the Data Management
Systems (DMS) and Surveillance unit supervisors. We were successful in backfilling the
two vacant positions created by this move (Senior Systems Analyst and Research
Scientist III). Internal movement of staff and the loss of an additional staff person
resulted in unanticipated vacancies, which we are in the process of backfilling.
We have had difficulty recruiting the Programmer V and the Security Officer positions.
In an attempt to reach a larger audience, several meetings have been held with UCDHS
recruitment staff who in collaboration with the DMS unit manager identified and placed
advertisements on a variety of more specific sites. Even with the additional recruitment
activities, the DMS Unit has received very little interest in any of the programmer
positions. A barrier to the recruitment of programmers in general is the budgeted
salaries for the positions. Given that budgeted salaries are unlikely to change
significantly due both to current funding and internal equity issues, we have had to
come up with creative solutions to address these recruitment challenges. We redirected
funds from vacant programmer positions to fill these positions with temporary
contractors. While this is an interim solution, the DMS Unit manager will continue to
pursue other recruitment options with the goal of filling these vacancies with permanent
UCDHS staff.
Interviews with contractors are currently underway for the Security Officer position, with
the expectation to hire by February 15, 2014. In the meantime, we will redirect
resources from programming and database activities to make progress on the activities
associated with these positions.

 The inability to recruit programmers for these mission critical positions has resulted in a
delay in the publication of the Service Level Management Plans. These plans will be
completed and tested before they are ready for publication. The Disaster Recovery plan
is almost complete, and will go through a final review and testing prior to full
implementation by June 2014.
Significant Accomplishments by Unit
Production, Automation and Quality (PAQC) Unit
•

The PAQC unit filled the vacant Sr. Systems Analyst position. The successful
candidate was an internal Business Analyst from the PAQC Unit who had been
performing a knowledge transfer of some of those duties during the time of the
search for an appropriate candidate.

•

In October 2013, one of the cancer reporting standard setters, Surveillance
Epidemiology and End Results (SEER), requested a data submission. The
PAQC unit appropriately managed and monitored the data submission and all
related activities. The PAQC unit ran the latest SEER Edit Metafile against the
extract with resulting edit errors identified. CCR program staff assisted the SEER
regions by writing and implementing global fixes in order to correct edit errors
affecting a significant number of cases. Per SEER’s specifications, the PAQC
unit extracted the corresponding data file and successfully submitted data.

•

In November 2013, The PACQ unit received and prepared a request for data
submission per the North American Association of Central Cancer Registries
(NAACCR) and the National Program for Cancer Registries (NPCR)
specifications. The PAQC unit requested and received approval to extend the
deadline for data submission. The PAQC unit also created specifications and
prepared extract scripts for the Comparative Effectiveness Research (CER)
project. All relevant data submission due dates were met.

•

The PAQC unit identified and incorporated the 2013 annual data changes into
Volumes I, II, and III. The PAQC unit distributed the revised volumes and posted
them to the ccrcal.org website. The PAQC unit managed and presented the
associated education and training of statewide cancer registrars pertaining to the
2013 annual data changes. The development of education materials included a
PowerPoint presentation, live webinars, and in-person presentations to Regions
1/8 and Region 9. Staff distributed a copy of the PowerPoint presentation to the
Cancer Registry of Greater California at their request in order for regional registry
staff to educate their local registrars.

•

The PAQC Unit performed a Lung Recoding Audit. The PAQC Unit staff audited
each region in California, utilizing the PEER review method. This method
included both a primary and secondary auditor independently audited cases and

 then compared and reconciled their results. In December, the PAQC unit
distributed the final audit results to regional directors. A comprehensive
statewide report is currently in progress with an anticipated completion date of
January 2014.
•

The PAQC unit provides feedback to the North American Association of Central
Cancer Registries (NAACCR) Physician Reporting Workgroup as CCR’s
representative, with a particular emphasis on the standardization and
harmonization of cancer data. The PAQC unit is actively involved in providing
assistance to management and staff to increase their knowledge base with
regard to Meaningful Use and electronic physician reporting.

•

The PAQC unit addressed the automation goals and evaluated the correction
record processing with a goal of analyzing existing manual work efforts to
determine any potential for automation. This analysis identified multiple system
processing issues, which are in the process of being resolved. Following the
successful resolution of identified processing issues, the PAQC unit will draft
automation rules.

•

The PAQC unit evaluated the current manual tumor linkage process, to develop
recommendations for automation solutions. In December 2013 version release
of Eureka-DMS, the PAQC unit drafted, tested and implemented their
recommendation for a positive match automation rule. The PAQC unit drafted
consolidation automation rules. In September 2013, the PAQC unit implemented
the Class of Case 43 (Pathology Only) automation rules that were developed and
thoroughly tested during the last reporting period.

Data Management Systems (DMS) Unit
•

The DMU unit manager led the
contract re-negotiations. These
negotiations resulted an overall savings of over $100,000; extended the length of
the contract through June 2017 and included the following new features:
• 64GB of additional RAM for each ESX host
• 1TB additional hard drive (SAS) disk space for the production database
(which will help performance considerably)
• 1 Barracuda failover device for continuity management
• 1 RSA token failover device for continuity management
• 1 additional
for failover on the internal firewall for
continuity management
• 20 Mbps additional internet bandwidth

•

Successful deployments of the Eureka version 12.1 occurred on 9/20/2013. The
major developments/enhancements included:
o Fixed approximately 176 bugs and/or enhancements
o Updates to DB Inquiry for the purpose of Hospital User Direct Access

 o Implementation of Path Reports in Consolidation in support of the Patient
Centric design
o The migration of the PPAQ database to Eureka
o Ongoing Physician File work
o Implementation of the latest NAACCR edits
o Enhancements to Data Miner
o Re-coding Audit Module
o Enhancements to the Eureka software installer to automate additional
processes
•

Successful deployment of the Eureka 12.2 occurred on 12/20/2013. The major
developments/enhancements included:
o Fixing 142 bugs and/or enhancements
o Updates to File Upload UI for the purpose of Hospital User Direct Access
o Implementation of Tumor Linkage rules to auto-link incoming admissions
with existing Patient sets
o Highlighting SSF fields on a per schema basis to assist in Direct Entry as
well as VE and Consolidation
o Implementation of the latest NAACCR edits
o Ongoing Physician File work
o PPAQ enhancement to auto-assist Classification

Surveillance and Data Use (SDU) Unit
•

SDU unit published the report “Cancer in California, 1988-2009”, available
through the CCR website. The SDU unit produced the 1988-2010 annual report
and new and redesigned fact sheet on leukemia, non-Hodgkin lymphoma, breast,
prostate and thyroid cancers and are available through the CCR website. In an
effort to improve patient notification, the SDU Health Educator produced the new
“Public & Patient Information” tab on the CCR website.

•

SDU staff is collaborating with UCD Cancer Center researchers in a study on
characteristics of ovarian cancer long-term survivors. Another study to ascertain
use of the test Oncotype among breast cancer patients enrolled in Medi-Cal is
currently under way. As part of a CDC-led multi-institutional collaboration, the
senior epidemiologist co-authored the manuscript “Patterns of Loco-regional
Treatment for Non-Metastatic Breast Cancer by Patient and Health Systems
Factors”, which is now under CDC clearance for publication. The SDU unit
collaborated on the following abstracts presented at scientific conferences:
Morgan JW, Ji L, Cupino A, Saunders A, Soloman N, Dyke C, Morris CR, Lum
SS “Validity of Class of Case for identifying definitive surgery hospital for
stomach adenocarcinomas: limitation for use of consolidated cancer registry
data”, Chen Y, Morris CR, Leiserowitz GL, Cress R. “Characteristics of long term
ovarian cancer patients”, and Miller MD, Whitehead T, Giddings B, Cruz G,

 Metayer C, Zachek CM, Buffler P “Trends in Childhood Leukemia and Household
Chemical Exposures by Ethnicity in California”.
•

SDU continued to lead the data quality group, which now has representatives of
DMS and PAQC units as well as regional epidemiologists. Together, the group
uncovered problems with some race fields and resolved complex consolidation
rules for multi-racial patients. As a joint effort, SDU, DMS, and PAQC units
resumed work to develop consolidation logic to incorporate comorbidity fields in
the CCR database, a new requirement from CDC-NPCR. The SDU unit
participated in the SEER Rapid Response Surveillance Study “Evaluating areabased socioeconomic measures from American Community Survey data”. The
SDU unit incorporated the new SES index developed into the CCR research
data, which will now contain comparable SES measures from 1988 forward.

•

SDU staff revised the CCR cancer concern guidelines to reflect recent CDC
recommendations. The Cancer Concerns Workgroup met regularly and
implemented a new procedure to give CDPH reassurance on the assessment of
controversial cancer concerns. The SDU unit assessed eight-cancer concerns in
Palo Alto, East Oakland, Ventura, San Bernardino, Burney, Irwindale, Malibu,
and Hermosa Beach. The SDU unit prepared 20 research data sets, mostly
developed through data linkages between the CCR and other statewide
databases.

•

SDU staff is collaborating with the Cancer Center Shared Resource to implement
monthly educational meetings, which are open to CCR and regional staff. As a
result of this collaboration, the following presentations/ informal seminars were
conducted during this reporting period:
o Estimating Cancer Trends
o Cancer Stage at Diagnosis
o Measures in Cancer Surveillance and Research
o Sources of Population Data for Surveillance and Research
o Cancer Treatment Information in the CCR Database

•

The SDU unit was successful in filling the vacant Research Statistician and
Research Scientist III positions and are currently recruiting for remaining
Research Scientist I vacancy.

Program Support Unit (PSU)
During this reporting period, the Program Support Unit completed the following tasks
and ongoing activities:

 •
•
•
•
•

Provided program oversight to ensure the project had sufficient administrative
and clerical support, necessary supplies, equipment, and services.
PSU staff monitored monthly personnel, budgets and procurement activities to
ensure the fiscal integrity of the program.
On a regular basis, met with CDPH staff to review and discuss the monthly
expenditure reports and other administrative activities
Developed and submitted grant budget amendment A03
Developed and submitted both contract and budget amendment A04

Scope of Work Issues Requiring Discussion with CDPH
The IPHI-CCR program is requesting clarification on Goal 1: Manage and coordinate
statewide registry production activities.
Clarification requested: These are State activities as managing and coordinating the
statewide registry production activities are State responsibility, should this activity be
removed from the scope of work.
Scope of Work Evaluation Summary
The following objectives are at risk of falling behind schedule:
Goal 2: 1.4.1 – Automate internal manual Quality Assurance Testing.
Due to demands on for QA and help desk, documentation and engineering services
could not move forward. Work resumed with the hiring of the vacant QA position on
12/15/2013.
Goal 2: 1.6.1 – 1.6.5 – Shall perform software engineering to develop a Eureka solution
for “text-based drop-downs” in Eureka New Case Entry screen for cancer case
abstracting in support of 80% auto-Visual Editing.
Per discussions with State staff, the text drop-downs are software abstracting vendor
enhancement, which would warrant the removal or revision of this deliverable. There
are indicators that the ECC Cap protocols in electronic format may facilitate the process
of visual editing; however, this needs to be further analyzed.
Goal 2: 1.7 Shall perform software engineering to develop a Eureka solution for 80%
automated New Case Admission Linkage to Patient resolution.
Deadline dates do not correspond to activities; implemented initial rule with December
2013 release; next rule planned for June 2014.

 Goal 2: 1.18 (1.18.5 – 1.18.6, 1.18.11 – 1.18.12, 1.18.13 – 1.18.15 - Program
Probabilistic Linkage Rules to increase the number of single-patient tumor cases to auto
complete, Program Site Specific Visual Editing rules starting with Prostate and Program
Auto Complete Path Only Admission.
Some progress towards these objectives occurred; however, the original plan to link
name variants has become problematic and additional analysis and discussion is
required before implementation can begin.
Goals 2: 1.18.25 – 1.18.27 – Shall Develop Use Case for Site Specific Drop Down Text
tables starting with Prostate in order to verify codes with text in New Case Entry or
Visual Editing.
The delay is due to the decision by national standard setters to abandon Collaborative
Staging effective in 2016. Discussions are ongoing internally with both PAQC Unit staff
and State staff to determine the appropriate approach as well as the feasibility of
revising or removing this deliverable.
Goal 2: 1.20 – Perform software engineering to automate 55% of Eureka Quality
Assurance System testing.
Due to demands on for QA and help desk, work for this activity could not move. Work
resumed with the hiring of the vacant QA position on 12/15/2013.
Goal 2: 1.21 – Perform software engineering to automate 55% of Eureka Quality
Assurance Integration testing.
Due to demands on for QA and help desk, work for this activity could not move. Work
resumed with the hiring of the vacant QA position on 12/15/2013.
Goal 2: 1.22 – Perform software engineering to automate 55% of Eureka Quality
Assurance Regression testing.
Due to demands on for QA and help desk, work for this activity could not move. Work
resumed with the hiring of the vacant QA position on 12/15/2013.
Goal 2: 1:23 – 1.23.1 - Shall perform software engineering to improve the automated
Eureka E-path integration capabilities within Pre-Processing and Casefinding
subsystems. Shall analyze Business Process for continuous process improvement and
automation opportunities.
Due to other project demands, work for this activity could not move forward. The goal is
to resume work activity by June 2014.

 Goal 2: 1.25.1 – 1.25.5 - Shall perform software engineering to define and program
the complete CCR Data Standards Vol I-III into Eureka Automation Technologies
(i.e.) Business Rules Management System, Edits, Eureka business logic.
Further discussion and/or clarification from the State are required for this activity.
Volumes I and II cannot be automated as it is an instructional “how to” abstracting and
coding manual and provides layout instructions for software vendors. Volume III is the
edit documentation and corresponds with the edit automated metafile. There is no
automation opportunity for these instructional guidelines.
Recommend
removing/revising deliverable.
Goal 2: 1.30.3 – 6 - Shall develop plan, build and test solution for auto data collection of
cases into Eureka.
Despite hiring delays, E-path continued to provide case-finding information on a routine
basis.
The information comes directly from pathology labs to Eureka-DMS.
Additionally, we are in the process of developing programmatic solutions to receive
electronic submissions directly from physicians as part of meaningful use.
Goal 2: 4.1.4, 4.2.4, 4.3.4, 4.4.4, 4.5.4, 4.6.4, 4.7.4 – Shall maintain, support and test
disaster recovery plans for all database environments.
With the re-direction of the programming and database staff (until such time that the
Security Officer position is filled), work on this activity will proceed.
Goal 2: 7 – Perform software engineering to automate 55% of Eureka Quality
Assurance Regression testing.
Due to demands on for QA and help desk, work for this activity could not move. Work
resumed with the hiring of the vacant QA position on 12/15/2013.
Goal 2: 9.4 - Shall perform Technical Writing services to provide a desktop reference
manual that consists of the complete series of document deliverables that comprise; the
IT Service Level Management Plan, Problem Management Plan, the Change
Management Process Plan, the Software Release and Deployment Plan, the Software
Development Life Cycle Plan, the Capacity Management Plan, the Disaster Recovery
Plan, and the IT Incident/Event Management Plan.
With the re-direction of the programming and database staff work on this activity will
proceed as outlined below:
•
•
•
•
•

February 2014 - Schedule regular monthly meetings with key players
Gather all information from key players for each ITIL document required
Publish Problem Management Plan by 4/15/2014
Publish Change Management Process Plan by 4/30/2014
Publish Incident/Event Management Plan by 5/15/2014

 •
•

Publish IT Service Level Management Plan by 5/30/2014
Publish all other plans by 6/30/2014

Goal 2: 13.2 – 13.4 – Shall perform system engineering services to provide a Service
Level Management Plan to ensure that all operational services and their performance
are measured in a consistent, professional manner.
With the re-direction of the programming and database staff (until such time that the
Security Officer position is filled), work on this activity will proceed.
Goal 2: 14 - Ensure Data Center services provide 99.3% Eureka systems availability
between 6am to 6pm M-F (excluding holidays).
With the re-direction of the programming and database, staff work on this activity will
proceed in January 2014.
Goal 3: 1.1.1 - Produce annual incidence and mortality reports that exceed minimal
NPCR Program Standards for Data Use.
Although statistical tables were prepared and posted on the CCR website, the delay in
the annual reports was due to conflicting views regarding the appropriate content and
formatting of reports. These issues are resolved and we do not anticipate future release
delays.
Goal 3: 1.1.5 - Prepare maps outlining geographic distribution of late stage disease for
screenable cancers.
The produced maps did not include the most recent CCR data. The research scientist I,
who was in charge of preparing maps, took another position within CDPH. IPHI is
currently recruiting to fill this position and resume mapping activities.
Goal 3: 1.1.9 – Use expert knowledge to set priorities for the production of reports and
data products.
SDU research staff continued to set priorities for the production of reports. However,
these decisions were not guided by a surveillance committee, which has not been
formed yet (please see explanation for items 2.1.1 and 2.2.1 below).
Goal 3: 1.3.1 - Develop a plan for reducing the interval from close of diagnosis year to
the dissemination of cancer surveillance reports
This activity is contingent on Data Production reaching 95% completeness of reporting.
Once CDPH approves the contents and format of the 1988-2010 annual report, SDU
will be able to streamline the production of subsequent reports.

 Goal 3: 2.1.1 & 2.2.1 - Identify key issues, knowledge gaps and program needs, plan
prepared that describes focus of epidemiologic studies to be performed and rationale,
design and conduct studies. Establish epidemiology and research steering committee to
maximize efficiencies in using statewide data.
SDU continued to work closely with regional epidemiologists and, under IPHI
leadership, continues to define priorities for surveillance and research. During
discussions to revise the proposed CCR operational plan, our understanding is that the
development of a cancer surveillance committee is under the State’s helm, so SDU is
no longer pursing this development. If our understanding is incorrect, SDU will work
diligently to implement the committee.
Goal 3: 4.4.1 - Plan and implement a scientific conference to be held at minimum once
every two years.
This continues to be an unfunded activity; completion of this objective continues to
depend on SDU identifying and securing external funding.
Goal 3: 6.3.1 - Develop and describe proposal for new and expanded uses of CCR
surveillance data beyond the traditional scientific manuscript and cancer reports
Although SDU continues to strengthen relationships with cancer control collaborates,
based on the explanation provided in activity 2.11 and 2.2.1 above, work is unable to
progress without the establishment of a cancer surveillance committee.
Goal 3: 7.1.2 - Serve as principal investigator for funded projects.
SDU analysts are not eligible to submit grant proposals as principal investigators but will
be available to collaborate in proposals brought forth by other researchers.
Goal 4: 1.1.13 - Utilize project management processes and tools.
This activity requires further discussion and/or clarification with CDPH to evaluate
moving or removing this activity.
Goal 4: 6 – Ensure program efficiencies using current business strategy and CQI
methods e.g. Six Sigma, Kaizen, etc.)
This activity requires further discussion and/or clarification with CDPH to evaluate
moving or removing this activity.