President: Da Ma kelky Douglas County Vice President: laura Walker La rimer County Treasurer: Cheryl ernes Arapahoe County Secretary: linda Warsh Saguache County Past President: Chris Kline Ada ms County Largest County: Don Mares City and County of IDenyer Northwest Region: Janeen McGee I ake County Northeast Region: Jerri Spear I Ibert County Metro Region: l'rank Alexander Boulder County Southeast Region: Stacie Kwitek Fremont COunty Southwest Region: Tracey Garchar Mesa County San Luis Valley Region: Catherine Salazar Alamosa County . Gatorade Human Services Directors Association October 19, 2018 Reggie Bicha, Executive Director Colorado Department of Human Services 1575 Sherman St. Denver, CO 80203 ?fir ulti?illigf? Suma Nallapati, Secretary of Technology State Chief Information Of?cer Governor?s Of?ce of Information Technology 601 E. 18th Ave, Suite 250 Denver, CO 80203 Re: Trails Modernization Dear Director Bicha and Secretary Nallapati, cur?vul? CHSDA has compiled the attached document regarding the Trails modernization project in an effort to outline the challenges we face, safety concerns that exist, and our desire to better understand the plan to address these important concerns. The problems outlined in the paper are more than an ?inconvenience? to our operations. They are crippling our work and putting the safety of children and families at risk. The examples we provide below highlight the presence of bugs, corrupt data, missing referrals, and inaccurate scores related to safety and risk, which all affect the practitioner?s ability to use historical information to make decisions regarding current safety issues with children and families. Our workers cannot trust the data that is in Trails Legacy or Trails Modernization. We are requesting that CDHS and OIT formally respond to each of the concerns that have been identi?ed by providing a plan and timeline for resolution. We would also like to discuss how we can create a better governance structure so these issues do not arise in future projects. Thank you in advance for your consideration and timely response to these items. Sin__,ere1y, aw Dan Makelky President, CHSDA Director, Douglas County Human Services Trails Modernization Trails originally rolled out to counties in 2001. It was built using 1980s technology and as such, modernization was long past due when the Trails Modernization project began in 2015. Modernized Trails new Trails or Trails Mod) is being built as a web-based application with modern web features like scrolling windows, left hand navigation, and application tabs that allow users to be able to switch between windows without losing the work they had open in prior windows (something that has frustrated users in old Trails since its release). The roll-out of the project has been met with frustration from users as well as supervisors, managers, and directors and has compounded the challenges and shortcomings of a nearly 20- year-old system. While there have been meetings to discuss ?lessons learned,? this project telegraphed its shortcomings; the problems experienced upon release were foreshadowed by the learning curve of proj ect participants. The Colorado Trails Users Group (CTUG) is comprised of county and state subject matter experts (SME). CTUG members advised the Request for Proposal (RFP) process, and its members have guided the development of new Trails since the beginning of requirements gathering in August 2016. CTUG was not invited to participate in certain high-level meetings. Vendor push-back on was a constant frustration and the weekend prior to the rollout of release 4, CTUG identified bugs that were knowingly included in the modernization?s release; however, the state deployed this release during the last weekend in July. The challenges faced afterward were primarily ones of communication between all involved parties. As time has progressed, the delay between bug discovery and resolution/workaround has been reduced, but consistency and timeliness of message is still a struggle. The workarounds listed on are dated the end of August and those listed on the portal are from mid-September. The 55 bugs reported in Informational Memo 2018-0015 differ from the 171 in the IRACCF R1 Release Notes. In this Informational Memo, eight weeks after the release of Modernized Trails, the state acknowledged problems but listed no plan of action to address the challenges. What counties desire now is an urgency of action mirroring their own and a high?level awareness of the very real consequences caused by this latest release of Modernized Trails. Most importantly, counties are concerned about the risk to child safety when the new Modernized Trails system cannot be trusted as presently released. Communication CTUG is the counties? ?rst and often only timely point of contact for learning about modernization efforts. Beyond this, however, CTUG members are also the counties? representative SMEs. Many of the CTUG members, along with their regular job duties, typically participate in two day-long conference calls each week testing new Trails before new versions are released to the public, do post-release bug testing and reporting to the State, and come up with workarounds to communicate to their reSpective counties. CTUG representatives are responsible, when able, for communicating their respective counties? desires for Modernized Trails and the shortcomings of the current system. While CTUG has been able to meet with the vendor, CGI, to discuss system requirements and enhanced functionality for modernized Trails on multiple occasions, early on in the project, members were excluded from some key initial high-level meetings wherein scope and functionality were not only discussed, but decided. For example, CGI presented the user interface for Modernized Trails to CTUG, but the group was only allowed to make minor revisions, thus many of our requested user interface modifications were not implemented in the initial rollout of modernized Trails. Although exclusionary policies are certainly the prerogative of any leadership, when it comes at the cost of clear communication and produces ill-conceived, unsolicited products, those decisions must be examined. More recently, thanks to push back and complaints from CTUG, members have been able to participate in these high-level meetings and offer guidance on which ?xes to prioritize for the various Hot?xes and Service Packs. What remains worrisome, however, is that efforts and the state?s responses to demands have been the actions of individuals, not agencies. While both CTUG and individuals from the Department of Child Welfare and the Of?ce of Information Technology (OIT) have been able to work toward the success and improvement of this process, these are occurring outside traditional governmental modes of Operation. Graig Crawford, a Jefferson County employee, has dedicated a considerable amount of his time to a program (Trails Mod) benefitting all counties and has been instrumental in advocating greater county representation. That the successes we can count as a state have been achieved by individuals is fortunate. It also underscores the fragility of the process and its victories. In a county-driven governmental agency like CDHS, this underscores high-level communication failures and, in the case of IM-OCYF-ZOI 8-001 5, the appearance of downplaying the severity of Modernized Trails? challenges when only 55 bugs were listed. In mid-September, OCYF told counties they were aware of problems but had no strategy for addressing them. Prior to that: 0 Hotline staff were overwhelmed when many of them lost access due to the profiles issue. 0 Timely Assessment Closure, a frequently?referenced C-Stat measure has had its credibility damaged; safety assessments completion was hindered by bugs and this, among other issues prevented their closure. August?s statewide score dropped Some counties were impacted much more because of their models of practice. 0 Tum?around on resolution of bugs via ticket to state OIT for both Legacy and Modemized Trails has been severely impacted as Trails OIT staff have been re-assigned to work on other projects and systems. The remaining OIT staff assigned to Trails (understandably) cannot manage this volume of work and the number of service level agreement (SLA) violations for Trails incidents is growing daily. Bugs - There were/are some major pro?le issues in Trails Mod, some of which still haven?t been resolved, that required intervention by the County Trails Security Admin. The profile issues affected both hotline and intake staff and supervisors. The security admins had to give the hotline supervisor?s pro?le to all caseworker supervisors before they could approve their workers? approval requests. Another bug necessitated giving the ?generalist? pro?le to all users. This consolidates six other pro?les including provider, contracts/rates, and eligibility workers. Also, upon initial rollout, staff certi?cations weren?t entered timely in Trails Mod, which led to many hotline and intake staff being unable to do their work until resolved by the state. Unfortunately for many counties, despite a script being executed to ?x this in late July or early August, counties found that they had to reach out to Caitlin Smith and Kathy Clark with the state requesting that they correct staff certi?cations, since that is a change that counties cannot view or complete on their own in Trails Mod. As described above, some initial decisions on the overall look and feel were made without CTUG input. When the design was ultimately presented to CTUG, the recommended changes were met with resistance by the vendor since leadership at the state level had directed them to make the changes. Making the recommended changes was cited as not being in scope and resulted in a con?ict about a potential cost incurred without reason. The vendor repeatedly cited that CTUG recommendations to add new functionality to Trails were out of scope due to guidance in the RFP, where new Trails can only have ?20% new functionality? compared to old Trails. Although 20% is a very speci?c number, it has proven dif?cult to quantify in terms of actual functionality and therefore decisions on scope have often become subjective. CTUG members are concerned that this ?20% rule? has been routinely used by the vendor as justi?cation to deny vetted requests for new functionality. The Informational Memo sent to counties on September 18th stated ?approximately 55 bugs? had been identi?ed following rollout. The IRACCF R1 Hot Fix and Service Pack List released one week prior listed 17] that had been or were going to be ?xed. Since then, additional bugs have been identi?ed and/or created as a consequence of new releases. This list also does not include the non-critical bugs that were identi?ed prior to the July rollout that have not been resolved. Many of the bugs in the functionality released in July impacted caseworkers? ability to complete tools and forms required to complete assessments or close cases such as risk and safety assessments. These bugs led to decreased performance by counties on the C- Stat measure, Timeliness of Assessment Closure as workers were forced to keep assessments for extended periods of time awaiting ticket resolution or issues to be resolved in hot ?x or service pack releases. The risk assessment tool in modernized Trails is not scoring risk levels accurately. There are two questions in which the answers entered by the caseworker are reversed when the tool is saved. Additionally, there are errors in the scoring algorithm that lead to an incorrect overall risk score. This issue is not slated for resolution until the November service pack. The IV-E determination functionality has several critical bugs including the inability to print determination forms, inability to complete redeterminations (they all show as ineligible), and the inability to complete Adoption and RGAP determinations. 0 There are several outstanding issues with the Referral functionality including the inability to edit key information in referral a?er disposition but while the assessment is still open, inability to disposition a referral without a client being added, inability to generate referrals for prevention services (CCR and SafeCare), and numerous less critical issues that are impacting the ability of county intake and screening staff from entering and dispositioning referrals. . The databases that Counties use for reporting and data extracts have been down since September 26. This prevents counties from being able to run reports regarding performance, workload, placement rates, and more. This is more than an inconvenience to our operations as it potentially puts the safety of our children and families at risk. Fixes Several ?Hot?x? releases have been deployed to ?x major bugs associated with the July release, yet there are still some major issues that persist: The vendor has expressed concerns over the cost and resources required for more frequent ?Hot?x? builds and has pushed back when CTUG asked for additional Hotfix releases. Consequently, we had to wait until the September ?Service Pack? release for more major issues to get resolved. 0 Not all bugs were resolved in the September service pack release and again the vendor has pushed back against request for additional Hot?x releases. In addition, the September service pack caused new bugs that have yet to be resolved. Now we are being told we have to wait for the November service pack release to get a CTUG-prioritized list of bug ?xes resolved. Not only are counties unable to easily access workarounds for issues since they are so many and varied, but timelines for bug resolution have been uncertain. As of this writing (October 10th), the Known Issues, Tips, and Workarounds listed on for Trails Modernization is dated August 24th. Similarly, on the ?news? page they note that since ?Monday, August 29, all learners attending a Fundamentals class will NOT be required to complete a Trails Scavenger Hunt.? This aspect of training has been suspended because of frustration it causes learners (and, presumably, certi?ed caseworkers), but in a ?eld known for its struggles with worker retention this is doubly unfortunate. Worker stress (although broad in scope) has been cited as the most impactful predictor for employee turnover data-entry and performance standards that are complicated by bugs and lags in communication exacerbate existing frustrations. When addressing the problems experienced during this module?s release, impact must be anticipated and managed. While counties continue to wait on timely communication and resolution of bugs, work quality suffers. Lack of functionality has left workers utilizing case notes like notes jotted in the margins of textbooks. Important case information is being relegated to drafted asides and this is preventing opportunities to meaningfully utilize data. Counties are developing their own workarounds and ad-hoc solutions to mandatory ?elds that do not function properly. Some of these ?work-arounds? include: Providing incorrect/nonexistent addresses. Marking ?unknown primary caregiver? on assessments when the opposite is true, to get past a known bug. Entering fake birthdays for mothers under the age of 18 (only when their Date of Birth is unknown) as the system forces a Date of Birth. Noting that safety assessments are not required (when they are, but cannot be entered) and then writing them in case notes in order to comply with the statutory requirement to close assessments within 60 days (some counties). Relying on counties to have and use their own email (and knowledge of its use) to securely email their own developmental screenings to providers. 0 This bug has been fixed. Trails Mod is now capable of sending the attachment Moving Forward and Managing Impact With respect to the state?s response, data from this period is now suspect and must be managed carefully. Eight weeks passed before the problems were acknowledged. Counties continue to take action while the state is silent in the face of substantial criticism and pushes ahead in the rollout of modernized CBMS and CHATS, overtaxing OIT resources which are already stretched thin. The Colorado Human Services Directors Association therefore request the Colorado Department of Human Services formally respond to each of the following: Develop and implement statewide communication regarding known bugs and timelines for their ?xes. Formally announce how and whether scores will be adjusted for impacted period(s), presumably August and September of 201 8 for the Timeliness of Assessment Closure Stat measure. Provide a plan for how OIT will be supported moving forward as welt as a plan for how OIT will support and communicate with counties. Increased opportunities for worker training and updates about pending certi?cations and worker requirements from CWTS. Work to ?x the Results Oriented Management system (ROM), which seemed to break perhaps due to unrelated issues, but further exacerbates the chaos related to Modernized Trails as counties cannot rely on ROM reports to manage performance or identify challenges in the system.