The Denver Public Health department serves the residents of the City and County of Denver and provides many services to the entire Denver metro area. Denver is the largest city within Colorado and has more than 725,000 residents.  The population of Denver is 55% White, 29% Hispanic, 9% Black, 4% Asian, and 3% multi-racial.  Almost 12% of the Denver population has an income below poverty level.

One responsibility performed by Denver Public Health during the COVID-19 pandemic is case investigations.  These investigations require contacting people diagnosed with COVID-19 and collecting information about their symptoms, onset date, hospitalization status, and housing situation. On a daily basis, information about new cases are downloaded from the Colorado State database called the Colorado Electronic Disease Reporting System (CEDRS) so that epidemiologists and disease investigators can contact recently diagnosed individuals. With the initial surge in cases, many within Denver Public Health were called upon to support case investigations efforts.  One staff member from the Data Science and Informatics team with expertise in informatics and disease investigation was trained on the initially developed process to download the cases from CEDRS and manually transfer information into a case investigation spreadsheet.  The process was very labor intensive and time consuming. 

After being trained on the process and performing one shift, she recognized that process improvements could be made using informatics techniques.  At the conclusion of each of her shifts, she documented areas where improvements could potentially be made.  She then met with the epidemiologists overseeing case investigations, explained what improvements could be made, and asked them to prioritize items that would have the greatest impact.  Another member of the Data Science and Informatics team, who had expert programming skills, volunteered to develop informatics components to streamline the process, largely consisting of programs to replace manual steps and to improve data quality. As the collaboration continued, he also offered additional suggestions for improvement.

The new informatics component provided many enhancements.  It matched the individuals’ names, birth dates, and gender with hospital data to determine if they were hospitalized.   It standardized addresses and electronically matched them with lists to determine if the address was a long-term care facility, jail, homeless shelter, or a college dorm.  It also automatically flagged those who had passed away, minors, City workers, and first responders.  Another enhancement identified individuals from the same household so that only one outreach would be needed for that household.

After the informatics process was implemented, significant improvements were realized.   The initial process, which took an average of 6.8 cases per hour, now averaged 28.3 cases per hour. The enhancements also improved the actual case investigation process.  And these improvements, implemented in May 2020, have already saved over 4,900 hours and $253,000 dollars in personnel costs.  This equates to about 2.5 full-time equivalents (FTE).

Since March 2020, the City and County of Denver has had more than 46,000 cases of COVID-19.  The race and ethnicity of these cases consisted of 52% Hispanic, 35% White, 7% Black, 2% Asian, and 4% other or multiple races.  The largest age group with COVID-19, 26% of all cases, was for ages 20-29.  The second largest age group, 30-39, made up 21% of the cases.

One of the goals for case investigation was to contact all new cases within the first 24 hours of being identified.  In the earlier months of the pandemic, there were occasions where Denver averaged more than 100 cases per day.  In the months of November and December, the average number of new cases exceeded 700 per day.  With that many cases needing investigation, it became very important to provide the case information as quickly as possible to those performing case investigations.  Extracting the data from the State data downloads and converting it into the case investigation spreadsheets in a timely fashion was a critical step in the process.

The informatics components that were developed were resourceful and innovative in their approach.  The initial process that was set up required copying and pasting data elements one-by-one from the State CEDRS download file into the case investigation spreadsheet.  It was a slow, tedious process that required concentrated effort to make sure that the correct data were copied and pasted into the designated cells of the spreadsheet.  There was opportunity for errors, skipping cases, or duplicating cases.

One of the first changes to the initial process was to convert and format the downloaded data so that they could be imported directly into the case investigation spreadsheet.  This eliminated a lot of the manual effort of copying each individual data element and pasting it into a cell of the spreadsheet.  It also eliminated the possibility of duplicating or omitting cases during that process.

The next innovative step was combining current hospital data with the COVID case data.  Denver Public Health is a department within a larger Denver Health organization. Denver Health, the parent organization, provides an integrated safety-net health care system that serves about one-third of all residents within the City and County of Denver.  The program extracted the current inpatient population of the hospital and then was matched with name, birth date, and gender information from the COVID case file to identify the new cases who were currently hospitalized.  This enhancement eliminated the need for someone to manually search the electronic health record system for each person on the new COVID cases list.  It also provided additional information to the case investigator of the hospitalized status of the individual, along with more specific contact information and primary language spoken.

Another innovative time-saving step was linking the addresses of the new cases to various types of housing.  Lists were developed and updated to identify the addresses of long-term care facilities, group or foster homes, jails, public housing units subsidized by HUD, homeless shelters, and university dorms.  This information provided the case investigator with key information on steps which may need to be taken in order to contact the individual newly diagnosed with COVID-19.

The addresses from the State CEDRS database frequently had misspelled street names or missing apartment numbers.  By electronically standardizing and formatting the different elements of the addresses, the program more accurately matched the addresses of other patients and flagged them as same households or matched them to reference files that contained the addresses of Long term Care Facilities, Jails, Homeless Shelters and Denver Housing Authority housing. Technology called SPEDIS (Spelling Distance) was incorporated to determine possible address matches for incomplete addresses or similar addresses that were entered slightly differently. Depending on the ‘spelling distance’ value, it would then flag them as a possible ‘Same household’ for the case investigators to follow up on. This enabled addresses to be corrected, which also facilitated the matching to the various housing types mentioned earlier.

Other automatic flags were also established to help case investigators better understand the individuals to whom they were to contact.  The program flagged healthcare workers, first responders, minors, Denver City and County workers, individuals who had passed away, Denver Public school students, and professional athletes.  It also matched cases with the same phone number or the same address.  Each of these flags improved the case investigation process.

The final innovative step was to automatically send an email out to the case investigators when the completed file had been finalized and posted in the folder for them to being reviewing.

As these processes and functions were implemented, the case investigators provided valuable feedback to the program developer so that further enhancements and improvements could be made.

The practice that was implemented was an internal process improvement utilized primarily by those performing case investigations within the Disease Investigation team and team members from the Data Science and Informatics team who developed and ran the programs which generated the case spreadsheets for investigators.  As was mentioned earlier, the initial collaboration occurred when a member from the Data Science and Informatics team began supporting the conversion of the State CEDRS data files into the case investigation spreadsheets.  This initial step of being trained and gaining an in-depth understanding the existing process was critical in order to identify steps and opportunities for improvement.

Another essential element was the willingness of the leaders of the case investigation area to listen and be receptive to suggestions for improvements.  Rather than insist that the initial process be followed without exception, they were very responsive and excited about how the process could be improved and enhanced.  They provided valuable input on which features would be the most beneficial to them and identified the highest priorities for implementation.  Each of the case investigators involved in this process were also very helpful throughout the pandemic response.  If a case investigator discovered an address for one of the specified housing types (long-term care facility, homeless shelter, etc.), they would communicate that back to the person who generated the file and that information would be added to the reference lists used each day.  Also, throughout the many months of the pandemic response, other suggestions for improvement were encouraged and implemented.  There was a strong bond of teamwork and collaboration between the case investigation team and the Data Science and Informatics team throughout the entire pandemic time period.

The entire effort on this practice was initiated as a quality improvement activity. Quality improvement has grown within the Denver Public Health department during the past ten years such that it is now engrained in the culture of all employees.  The team member who was trained in the initial data conversion process and performed it for several weeks is part of the department’s quality committee and consistently looks for ways to eliminate waste and improve productivity and performance.  When faced with a process that required a lot of time to complete, her immediate thoughts were about possible opportunities for improvement.  Rather than immediately jump to solutions, she took the time to perform the initial process before proposing improvements. One of the lessons learned through various quality improvement projects is to have a good understanding of the processes before making significant, widespread changes.

The quality improvements made through this process saved valuable time, created higher quality data files, and implemented processes and data integration that enhanced the data being provided to the case investigators.  Not only was the time needed for the data transformation process reduced significantly, the improvements made cases investigations easier.  For example, prior to the improvement process changes, often case investigators would end up calling the same household multiple times when more than one household member was infected.  These types of situations were sometimes not well received since some of the same information may need to be conveyed to two different case investigators working through the case list.  With the improved process, members of the same household, identified by either address or phone number, could be more efficiently addressed in one single outreach.

One of the lessons learned was that not all suggestions for improvement were possible. Ideally it would have been nice to automate the manual step of downloading the new cases from the State system.  Unfortunately, that was not viable because the State did not have the resources to develop customized programs for various counties. Another suggestion was to upload a file into the State system that would change case record statuses.  Given the significant demands on State resources for the pandemic, that option was also not something which could be incorporated.

Another lesson learned was to make the process adaptable to change.  There were instances where the State CEDRS reports were changed, such as additional columns added, which impacted the programs that extracted the data from the downloads.  Developing a flexible system that could be easily adapted to these changes was important so that public health functions would not be impacted.  Another flexible structure was to set up the lists of addresses for various housing types so that they could be amended and revised easily.  Having a flexible system that could be adjusted based upon changing circumstances was important.

Perhaps one of the more important lessons was the team approach in improving quality.  By having interdisciplinary staff on the team, each with their own perspectives and ideas, more opportunities and suggestions for change were identified and implemented.  The case investigators were able to identify their needs and desires for the system, while other team members assessed how feasible the various suggestions were and what efforts were needed for incorporating those changes.  The team members were very supportive of each other throughout the entire process.

Finally, this entire quality improvement process has demonstrated that involving the right people during the initial development of processes is extremely valuable.  If a data collection system is being developed, it is important to involve in the early stages individuals who are subject matter experts with data collection, manipulation, and reporting.  Fortunately, in this situation many of the initial data collection and transformation issues were identified early within the pandemic response so that critical time savings were implemented and utilized throughout the many months of the pandemic.