The overall goal of PSL is to reduce overdoses, recidivism, and deaths in Duval County that are related to opioid-related use disorder (OUD), other substance use disorder (SUD), or co-occurring substance use disorder (CSUD) and mental health disorder (MHD).
Data collection and evaluation approach
The approach developed to evaluate OD2A followed the guidelines, steps, and standards of the CDC Evaluation Framework. Process measures were collected to measure and track grant activities, while data for outcome measures were collected to assess whether the program had its desired impact. DOH-Duval worked collaboratively with internal and external stakeholders to evaluate PSL. Prior to finalizing the evaluation plan, DOH-Duval met with stakeholders to discuss the evaluation plan, address any barriers or concerns, and receive feedback on the evaluation plan and preferred methods of disseminating findings. The evaluation plan was regularly assessed via feedback from OD2A partners and improvements were implemented based on recommendations. Data was reviewed with stakeholders to determine how it can be used for continuous quality improvement purposes.
To monitor process measures, a Microsoft Office Excel data collection tool was created with input from the PSL director. Monthly process indicators reported via the data collection tool included the following:
1. Total number of overdose survivors eligible to enroll in PSL
2. Number of overdose survivors with no program contact (e.g., expired, left against medical advice AMA, CRPS not available)
3. Number of overdose survivors that consented to PSL
4. Number of PSL enrollees that selected peer support services plus treatment for substance use disorder and the number of PSL enrollees that selected peer support services only
5. Number of PSL enrollees actively engaged in PSL
6. Number of PSL enrollees that completed PSL
7. Number of PSL enrollees that died from an overdose
8. Number of PSL enrollees placed in a treatment slot in the eTransX OCC system and that were linked to care
9. Number of PSL enrollees serviced by a CRPS at each participating ED and the number of PSL enrollees served by an OD2A funded CRPS
10. DOH-Duval also collects data related to the number of pregnant women enrolled in PSL that have a POSC and that are linked to care.
In addition to the process measures reported to DOH-Duval, data was also collected through surveys and interviews. This data is used to evaluate the effectiveness of PSL and for program improvement purposes. To date, the following surveys and interviews have been conducted:
1. Annual CRPS survey. CRPS staff are asked to provide their feedback regarding the following topics:
a. Needs of program enrollees (e.g., housing, employment, food, access to care, transportation)
b. Organizations PSL enrollees are most frequently linked to (e.g., housing support, food banks, mental health treatment facilities, clothing, employment services, additional services)
c. Perceived barriers that PSL enrollees experience to seeking treatment for OUD/SUD
d. Instruments and strategies utilized during interactions with PSL enrollees (e.g., SBIRT, URICA, link to NARCAN resources, motivational interviewing, sharing personal experiences)
f. Modes of communication that are most effective in reaching PSL enrollees
g. Perception of PSL’s effectiveness and the care coordination provided
i. Beliefs about the effectiveness of the eTransX OCC system, as well information on barriers, opportunities for improvement, and the most utilized features
j. Barriers to implementing linkage to care activities
k. Challenges faced due to COVID-19
2. Quarterly Professional Quality of Life Scale Survey (ProQOL) of CRPS
a. Measure levels of compassion satisfaction and compassion fatigue, which consists of burnout and secondary traumatic stress
3. Annual interview with Director of PSL
a. Interview includes questions concerning the functionality of PSL, successes, barriers, lessons learned, suggestions for improvement, program sustainability, and specific challenges faced due to the COVID-19 pandemic
4. Annual Stakeholder Meeting Survey
a. Frequency that stakeholders attend various community meetings (i.e., North Florida Poly-Drug Task Force quarterly meeting, OD2A Monthly Grant Partner meeting with DOH-Duval, Drug Epidemiology Network (DENS) meeting, etc.)
b. Usefulness of data presented at each meeting in informing overdose prevention efforts
c. Data sources most frequently utilized (e.g., JFRD Transport/EMS data, ED visits (ESSENCE-FL), PSL data, Medical Examiner, Vital Statistics Death data, Premier BioTech laboratory reports)
d. COVID-19’s impact on ability to conduct prevention and response work in the community
Program Reach and Outcomes
University of Florida’s Center for Health Equity and Engagement Research completed an analysis of death records from the Florida Department of Health’s Vital Statistics Office from November 2017 through November 2020 for the five-county area of Northeast Florida (Baker, Clay, Duval, Nassau, and St. Johns). Overdose death records during this time were compared between overdose survivors that consented to participate in PSL and overdose survivors that were approached by a CRPS but declined all PSL services. Primary and secondary causes of death were examined, the number of days the death occurred after the initial overdose visit to the ED, and whether the deceased participated in substance use treatment. The results of the analysis showed that since inception of PSL in 2017, 8,212 ED patients have been eligible to enroll in PSL. Of those, 837 (10%) did not have any program contact and 3,340 (41%) declined all PSL services. Of the 4,035 patients that consented to PSL, 3,443 (85%) elected to receive peer support services only and 583 (14%) elected to receive peer support services and treatment. During this time, 9 (0.2%) patients enrolled in PSL died due to an overdose. Analysis showed that the overdose rate of those who participated in PSL was statistically lower than those who did not participate in PSL (p = 0.02).
Data also suggests that participation in PSL is associated with a reduction in future JFRD response calls for program participants. Analysis conducted in 2018, reported a 52% decrease in JFRD overdose response calls to PSL enrollees. This analysis was performed by comparing the number of JFRD overdose response calls to each participant for the 12-months prior to enrolling in PSL and the 12-months post PSL program completion.
Premier BioTech Laboratory Testing
As part of this work, PBL tests urine samples of overdose survivors who consent to the PSL program. PBL tests for Opiates, Oxycodone, Cocaine, Amphetamine/Methamphetamine, Marijuana, Buprenorphine, Benzodiazepines, Tramadol, Heroin, Gabapentin, Dextromethorphan, Alcohol, and 13 different variations of Fentanyl. Between November 1, 2020, and June 30, 2021, a total of 222 samples from six different EDs were accepted for testing. The top five drugs with the highest overall positivity percent were alcohol (53%), fentanyl and fentanyl Analogs (49%), marijuana (43%), cocaine (35%), amphetamine (34%), and benzodiazepines (28%). PBL also conducts a detailed breakdown of fentanyl and analogs and opiates. In Duval County, fentanyl is found in 98% of the specimens, followed by the analog norfentanyl (97%), acetyl norfentanyl (19%), acetyl fentanyl (19%). Furanyl fentanyl was not found in any sample within this period. Morphine (67%) was the most common opiate followed by norhydrocodone (33%), hydrocodone (29%), hydromorphone (24%), dihydrocodeine (20%), codeine (7%), heroin (7%). Finally, an analysis of drug combinations shows that synthetic fentanyl was found with two or more substances in a total of 80 specimens, and with three or more substances in 54 specimens. When synthetic fentanyl was found with only one other substance, the most common substance was amphetamine (N = 56) or cocaine (N = 52).
The toxicology reports of PSL enrollees provided by PBL have increased our understanding of the illicit drug supply in Duval County and the geographic differences that exist based on hospital location. All test results are aggregated, analyzed, and presented at stakeholder meetings to inform future overdose prevention efforts.
Linkage to Care
PSL enrollees are linked to a variety of services and programs, both internal and external to Gateway Community Services Inc. The data below represent the percent of PSL enrollees that were linked to each service during the time frame of September 1, 2019, through August 31, 2020. The linkages are tracked in the eTransX OCC.
1. Substance use treatment – 68%
2. Mental health services – 77%
3. Support systems – 34%
4. Housing – 90%
5. Hygiene/food – 15%
6. Clothing – 34%
7. ID/legal – 8%
8. Medical needs – 34%
9. Medical insurance – 3%
10. Employment – 16%
11. Education – 3%
12. Life Skills – 92%
Sharing Results with Stakeholders
DOH-Duval disseminates surveillance data through a variety of mechanisms including data requests, brief reports, presentations, stakeholder meetings, and other publications and has built relationships with community partners that can immediately use surveillance data to prevent overdoses. The intended target audience for dissemination included the public, stakeholders, and policy makers working to prevent overdoses. DOH-Duval worked with stakeholders to identify additional target audiences for evaluation data and ensure that data is disseminated in a way that is clear, concise, useful, and practical. In addition to sharing evaluation results, communication materials included information that stakeholders may need to implement evaluation findings. Stakeholder meetings, community meetings, and briefs will continue be used as a mechanism to share timely data, communicate what is working well, what needs to be adjusted, and to share recommendations for using evaluation findings to improve overdose interventions. Success stories and reports are utilized to share accomplishments, demonstrate value and impact, and communicate how data can be used to improve overdose interventions. As part of this work, an evaluation reporting tool has been developed to track data dissemination and how evaluation data was used to inform overdose prevention efforts. A marketing specialist at DOH-Duval supports OD2A dissemination efforts, including the development of a DOH-Duval OD2A website and other marketing materials to share evaluation findings. PSL data is also presented by Gateway during the City of Jacksonville City Council Special Committee on Opioid Epidemic, Vaping & Mental Health committee meetings, which is free and open to the public.