rdunkelb
Thu, 05/04/2023 - 16:01
Edited Text
BLOOMSBURG UNNERSITY OF PENNSYLVANIA
As members of the DNP (Doctor of Nursing Practice) Project Committee, we certify that we
have read the DNP project prepared by Emmanuel Ahipue, titled "Telemental health: Increasing
access to mental health services and decreasing hospitalization" and recommend that it be
accepted as fulfilling the DNP project requirement for the Degree of Doctor of Nursing Practice.
[Faculty of record]
Peggy Z. Shipley PhD, RN
[Faculty Mentor]
�� j);,:Jrf, F/JF· G
�ember]
Date: 5'//S / Zt.fZ--z.
5/13/2022
Date: ____
_
Date:
Final approval and acceptance of this DNP project is contingent upon the candidate's submission
of the final copies of the DNP project to the Graduate College.
I hereby certify that I have read this DNP project prepared under my direction and recommend
that it be accepted as fulfilling the DNP project requirement.
DNP · oordinator, Chair, or Dean
[Graduate Nursing Department]
Date:
�/:w.
Telemental Health Services
1
TELEMENTAL HEALTH SERVICES: INCREASING ACCESS TO MENTAL HEALTH
SERVICES AND DECREASING HOSPITALIZATION
by
Emmanuel Ahipue
________________________
Copyright © Emmanuel Ahipue 2022
A DNP Project Submitted to the Faculty of the
COLLEGE OF NURSING
In Partial Fulfillment of the Requirements
For the Degree of
DOCTOR OF NURSING PRACTICE
BLOOMSBURG UNIVERSITY
2022
Telemental Health Services
2
BLOOMSBURG UNIVERSITY OF PENNSYLVANIA
As members of the DNP Project Committee, we certify that we have read the DNP project
prepared by Emmanuel Ahipue, titled Decreasing hospitalization readmissions and increasing
access to mental health services through increased assessment via telemental health in a
Wellness and Recovery clinic in Staten Island, Richmond County of New York City: A Quality
Improvement Project and recommend that it be accepted as fulfilling the DNP project
requirement for the Degree of Doctor of Nursing Practice.
_________________________________________________________________
Date: ____________
_________________________________________________________________
Date: ____________
_________________________________________________________________
Date: ____________
[Committee Chair Name]
[Committee Member Name]
[Committee Member Name]
Final approval and acceptance of this DNP project is contingent upon the candidate’s submission
of the final copies of the DNP project to the Graduate College.
I hereby certify that I have read this DNP project prepared under my direction and recommend
that it be accepted as fulfilling the DNP project requirement.
_________________________________________________________________
[Committee Chair Name]
DNP Project Committee Chair
[Academic Department]
Date: ____________
Telemental Health Services
3
ACKNOWLEDGMENTS
I wish first to acknowledge the grace of God for allowing me to complete this quality
improvement project. Despite the challenges in time management, travelling time to my clinical
site, and long working hours, you have continued to inspire and strengthen me to complete this
project. I am forever grateful and indebted to you.
I wish to acknowledge Dr. Shipley, my Clinical mentor. I have nothing but respect and
appreciation for your encouragement and prompt response to my questions. I also admire your
wealth of knowledge and constructive criticism during the phases of this QI project. You edited
my proposal for this project. I wish to acknowledge Dr. Jackson and Dr. Parke for their patience
and encouragement throughout the DNP courses. I would not have asked for better professors. I
wish to acknowledge G. Craig Wood, the biostatistical consultant, for processing the data.
I also wish to acknowledge Mr. Bello Muftau, my colleague at Project hospitality. As a
fellow psychiatric NP, I admire your insight and the experience in implementing the telemental
health at Project Hospitality. I would also like to acknowledge Mr. Gary Seigel for your
extraordinary leadership that you showed in bringing the team of mental health workers together
at Project Hospitality at the onset of the Corona virus pandemic. Your leadership set the stage for
the implementation of the telemental health in the facility. I can’ t help but acknowledge the
amazing staff of Project Hospitality. The implementation of this telemental health in our facility
has bonded us together and has brought out the best in all of us. I appreciate your dedication and
kindness to our patients. You are all amazing and have formed a formidable team in the Staten
Island District 49. I am immensely proud of the work we have done increasing access to mental
health in our community.
Telemental Health Services
4
DEDICATION
This project is dedicated to my wife, Ulonna, my sons, Justin and Joshua, and my daughter,
Alexandra. Thank you for your support, understanding and enduring love. You were all there at
every bump in the road and I appreciate you all endlessly. This project is also dedicated to my
mother, Justina Ahipue, and my father Paul Ahipue. Thank you, mom, for your prayers and
encouragement to keep moving beyond your fourth-grade education. Also, for babysitting the
grandchildren without complaining whenever I needed you. Dad, I know your spirit is with me
and happy with me in this accomplishment. Mom and Dad, you are both my rock.
Telemental Health Services
5
TABLE OF CONTENTS
LIST OF FIGURES .........................................................................................................................7
LIST OF TABLES ...........................................................................................................................8
ABSTRACT.....................................................................................................................................9
INTRODUCTION........................................................................................................................11
Background Knowledge/Significance.........................................................................................12
Local Problem ..............................................................................................................................14
Intended Improvement ................................................................................................................19
Project Purpose ................................................................................................................21
Project Question ...............................................................................................................22
Project Objectives ............................................................................................................22
Theoretical Framework ...............................................................................................................23
Literature Synthesis .....................................................................................................................26
Evidence Search ...............................................................................................................28
Comprehensive Appraisal of Evidence ..........................................................................29
Strengths of Evidence ......................................................................................................35
Weaknesses of Evidence ..................................................................................................36
Gaps and Limitations ......................................................................................................36
METHODS ...................................................................................................................................37
Project Design...............................................................................................................................37
Model for Implementation ..........................................................................................................39
Setting and Stakeholders .............................................................................................................41
Planning the Intervention ...........................................................................................................42
Participants and Recruitment.....................................................................................................45
Consent and Ethical Considerations ..........................................................................................45
Data Collection .............................................................................................................................46
Data Analysis ................................................................................................................................46
RESULTS .....................................................................................................................................48
Outcomes ......................................................................................................................................52
Telemental Health Services
6
TABLE OF CONTENTS - Continued
DISCUSSION ..............................................................................................................................59
Summary.......................................................................................................................................59
Interpretation ...............................................................................................................................61
Implications (Practice, Education, Research and Policy) ........................................................62
Limitations ....................................................................................................................................66
DNP Essentials Addressed .........................................................................................................69
Conclusions ...................................................................................................................................70
Plan for Sustainability .....................................................................................................70
Plan for Dissemination ....................................................................................................71
APPENDIX A:
SITE APPROVAL/AUTHORIZATION LETTER .........................................73
APPENDIX B:
CONSENT DOCUMENT (DISCLOSURE FORM, CONSENT FORM,
ETC.)................................................................................................................75
APPENDIX C:
EVALUATION INSTRUMENTS (STUDENT CREATED DATA
COLLECTION TOOLS – SURVEYS, QUESTIONNAIRES, INTERVIEW
QUESTIONS, ETC.) .......................................................................................78
APPENDIX D:
PROJECT TIMELINE .....................................................................................81
APPENDIX E:
OTHER DOCUMENTS AS APPLICABLE TO THE PROJECT (SUCH AS
BUDGET) ........................................................................................................82
REFERENCE…………………………………………………………………………………….84
Telemental Health Services
7
LIST OF FIGURES
Figure 1. Telemental health fishbone diagram. (Adapted from Chang (2015). No permission
needed to use and adapt……………………………………….……………………………………41
Figure 2. Screening tool implementation process………………………………………………49
Figure 3: Number of contacts by service type during the pre-TMH (in person office visits only,
September 15, 2019 – December 13, 2019) and post-TMH (Telemental Health, September 15,
2021 – December 13, 2021) ……………………………………………………………………54
Figure 4: Number in census report and number of substance abuse and psychiatric related
hospitalizations during the pre-period (in person visits only, September 15, 2019 – December 13,
2019) and post-period (TMH, September 15, 2021 – December 13, 2021) …………………….56
Figure 5. GAD-7 anxiety level before and after introduction of TMH services…………………57
Figure 6. PHQ-9 depression before and after the introduction of TMH services………………58
Telemental Health Services
8
LIST OF TABLES
Table 1. Definition of terms……………………………………………………………. 18
Table 2. The benefit of the Effectiveness-Efficiency-Equity conceptual framework….26
Table 3. High Risk Criteria Protocol……………………………………………………50
Table 4. Service types/ Contact logs…………………………………………………….54
Table 5. C-SSRS suicidality and after the introduction of TMH services………………58
Table 6. Hospitalizations greater than 24 hours……………………………………….65
Telemental Health Services
9
ABSTRACT
Purpose: Assess the ability of increased assessment via telemental health (TMH) to increase
access to care and decrease hospitalizations.
Background: There is mental health crisis in the U.S. and mental health services are insufficient
despite more than half of Americans (56%) seeking help. Patients seeking mental health
treatment have had to deal with the limited options in treatment and long waiting periods with
46% of Americans who have or know someone who has had to drive more than an hour
roundtrip to seek treatment, and 38% of Americans reporting that they have waited longer than
one week for mental health treatments.
Methods: This quality improvement (QI) project utilized retrospective chart audit at Project
Hospitality (PH) outpatient clinic to compare patients’ hospitalizations, readmissions, and access
to care between in-person pre-pandemic metrics with those occurring after institution of the New
York emergency TMH guideline initiated during the COVID-19 pandemic. Data was obtained
from PH’s electronic health record (EHR), Awards, evaluating these screening tools (Patient
Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder 7-item scale (GAD) and the
Columbia-Suicidal Severity Rating Scale (C-SSRS)). Data was obtained from interoperative
community EHR indicating where and when a patient obtained mental health treatment within
and/ or outside the program within the last 90 days.
Results: Availability of TMH through the emergency guideline has resulted in improved access
to mental health providers leading to improvement of symptoms, reduction in in-patient
hospitalization, and improved patient outcomes.
Telemental Health Services
10
Conclusion: The implementation of TMH has helped to improve patient access to mental health,
reduced loss to patient follow-up, and reduced in-patient hospitalization. Additional work is
needed to adopt this guideline into policy ensuring that patients have flexible access to care when
it is needed. Telecommunication aid, broadband, and internet access required and must be
adopted as medical necessity to fully adopt this innovation.
Telemental Health Services
11
INTRODUCTION
As the emergency TMH guidelines are entering into the second year, evaluation of the
effectiveness of TMH to increase patient access to their providers and decrease hospitalization at
PH is important. TMH use focuses on the second overarching goals of the Healthy People
2020 which aimed to achieve health equity, eliminate disparities, and improve the health of all
groups (Center for Disease Control and Prevention, 2020). There are several problems facing
the use of TMH/ telepsychiatry in non-emergent times. There is lack of nationwide policy
governing the provision of TMH care. Secondly, there is reimbursement disparity between
traditional in-person office visits and TMH services. Finally, there are common restrictions
placed on locations where providers and patients must be located to engage in TMH. This
project is currently operating under an emergency executive order, with month-to-month
extensions and revisions due to the COVID-19 pandemic. These revisions create uncertainty
among different organizations and the population being served.
At the onset of the COVID-19 pandemic, the request for TMH services, by both
providers and patients, increased significantly. A regulatory waiver was thereby issued to
maintain mental health services to vulnerable citizens throughout the duration of the declared
emergency. As a result, TMH was temporarily expanded for Medicaid-reimbursable services
to include telephonic and/ or video technology, including tools commonly available on smart
phones and other devices (Cuomo et al., 2020). Additional roles were included in this
emergency guidance as the definition of TMH practitioners were expanded to include any
professional, paraprofessional, or unlicensed behavioral health staff who deliver a qualified
service via TMH (Cuomo et al., 2020). TMH refers to “the use of technologies such as phone
or video calls between mental health professionals and other mental health professional,
Telemental Health Services
12
patients, service users, family members, or care givers to deliver mental health care” (Barnett
et al., 2021). Services includes delivering remote psychiatric evaluations, psychotherapy, and
medication management. The goal of this study is to use the findings to support the need for a
more permanent and effective TMH policy.
Background Knowledge/Significance
There is mental health crisis in America. Mental health services in the U.S. are
insufficient despite 56% of Americans seeking help (Wood et al., 2018). The attitudes of
Americans have improved towards receiving mental health services and 76 % of Americans are
now seeing mental health as important as physical health (Wood et al., 2018). This has further
increased the demand for mental health services which had already been shown to be
insufficient. Lack of access and long waiting periods have become the norm. Despite the high
demand for mental health services and societal awareness for mental health in the U.S., 74% of
Americans do not believe that these services are available for everyone and 47 % believe that the
options are limited (Wood et al., 2018). Medicaid-covered patients in publicly funded
community health centers experience the most extended delays in timely access (National
Council for Behavioral Health, 2017). The disparity in access to mental health care is
pronounced with patients who have low level income (annual household salary less than
$49,999), reside in under-resourced neighborhoods, and with insufficient insurance coverage
(Cohen Veterans Network & National Council for Behavioral Health, 2018). About 53% of
Americans who have sought mental health treatment were in low-income households (National
Council for Mental Wellbeing, 2018).
The largest proportions of those seeking mental health treatment are millennials (37%),
those in lower income households (53%), and those involved with the military either directly or
Telemental Health Services
13
through a secondary relationship (66%) (CVN & NCBH, 2018). Younger Americans, despite
having some of the more critical needs for mental health treatment, are unsure of their resources
and are more impacted by the stigma of mental health care (CVN & NCBH, 2018). The average
American is seeking mental health treatment and the need to receive care has grown (National
Council for Behavioral Health, 2017).
There are factors that prevent timely access to treatment for those who have established
mental health treatment. These factors include cost of mental health care for those with
insufficient insurance, social stigmas attached to mental health treatment, lack of access to
mental health facility or knowledge of where to find one, concern relating to quality of care, and
travelling associated with follow up appointments (Wood et al., 2018). Cost or poor insurance
coverage constitutes 42% of the barriers for Americans seeking mental health, 15% lack
information on where to start, 17% related to social stigma around seeking treatment, and 7%
ascribes to the quality of care as a barrier (CVN & NCBH, 2018). Patients are losing time while
waiting for treatment and traveling to their appointments. Patients seeking mental health
treatment have had to deal with the limited options in treatment and long waiting periods, with
46% of Americans who have or know someone who has had to drive more than an hour
roundtrip to seek treatment, and 38% of Americans reporting that they have waited longer than
one week for mental health treatments (Wood et al., 2018).
With expanded access under the Affordable Health Care reform, demand for mental
health care is high, adding to the problem of timely access to mental health treatment. The
National Council of Behavioral Health (2017) report indicates that the demand for psychiatry
will outstrip supply by 15,600 psychiatrists in 2025, based on estimates of retirement and new
entries into the workforce. Access is further diminished due to increasing numbers of private
Telemental Health Services
14
practice psychiatrists accepting cash or private insurance only (National Council for Behavioral
Health, 2017). There are factors affecting the shortage of psychiatrists in both the rural and urban
areas across in the country. The National Council for Behavioral Health Report (2017)
highlighted these reasons to include aging of the current workforce, low rates of reimbursement,
burnout, burdensome documentation requirements, and restrictive regulations around sharing
clinical information necessary to coordinate care.
Innovative techniques are needed to solve the problem of timely access to care (National
Council for Behavioral Health, 2017). This is necessary to capture the considerable number of
people suffering from a range of psychiatric disorders like schizophrenia, bipolar disorder, major
depression, anxiety, and other mood disorders, who needs mental health services. High quality
care can be maintained while patients are reached, regardless of the location of the patients. The
solutions to the shortage of psychiatry and timely access can only be achieved by a combination
of interrelated solutions and not exclusively on recruiting more psychiatrists or increasing the
payment or reimbursement rate (National Council for Behavioral Health, 2017). Telepsychiatry
is the most developed, innovative model in expanding access to psychiatric provider (National
Council for Behavioral Health, 2017).
Local Problem
PH has multiple programs that provide services to the community of the 49th district of
Staten Island, New York also known as the North shore of Staten Island. The served population
includes individuals with mental illness and some with co-occurring substance use disorder. The
challenges faced in providing services to this population includes keeping the patients engaged in
treatment long enough for recovery, reducing patients lost to follow up, and providing treatment
Telemental Health Services
15
for those who are uninsured or underinsured. Most of the patient population at PH have Medicaid
insurance.
This population often face health inequity due to mental illness and addiction. These
disparities can be related to unemployment, homelessness, and multiple health issues, including
addiction and mental illness. There are scenarios where there is complete disregard for an
established care or resources by the patients. Other instances, there is an issue of abandonment of
needed resources like shelter due to impulsive judgment and isolation from familial support.
These situations create ethical dilemmas for mental health workers when patients are not
engaging with treatment. Ethical dilemmas arise in areas related to continuing treatment when a
provider knows that the patient is still actively using drugs or whether to discharge a patient who
has disengaged from treatment by missing appointments yet knowing that he/ she needs the
medications for mental stability and functioning. Based on the nursing code of ethics, the
components of beneficence, justice, and nonmaleficence are challenged.
The South shore is the more affluent section of Staten Island and more economically
disadvantaged population with a more diverse population and larger percentage of people of
color is on the Island’s Manhattan-facing North shore (Milstein & Madden, 2017). Staten Island,
despite its relative affluence, had the highest all-cause mortality rate in New York City (Milstein
& Madden, 2017). Mortality here is driven by rates of cancer (lung cancer and breast cancer
among women), colorectal cancer, heart disease, and substance abuse respectively particularly in
the North shore area (Hinterland et al., 2018).
Staten Island has the highest rate of drug overdose deaths compared to the other New
York City boroughs (Serenity at Summit, 2022). Staten island was once the hot bed for the U.S.
opioid epidemic. Most hospitalizations for drug overdose in Staten Island are related to opioid
Telemental Health Services
16
use (Serenity at Summit, 2022). Based on Medicaid hospital admission data from 2016 to 2018,
there were 2,619 emergency department visits during this time attributed to opioid abuse and
1,851 of them where in people 18 to 44 years old (Serenity at Summit, 2022). Statistics indicate
that synthetic opioids accounted for 1,641 opioid-related overdose deaths in New York in 2016,
heroin caused 1,307 deaths, and 1,100 deaths were caused by prescription opioids (Serenity at
Summit, 2022). Out of the 306 overdose deaths in Staten Island from 2016 to 2018, 92 deaths
were in 2016, 108 deaths in 2017, and 105 deaths in 2018 (Serenity at Summit, 2022). Over 80%
of the fatal overdose in 2018 were seen in the Caucasian population and White males were more
likely to experience a mortal overdose (77%) (Serenity at Summit, 2022).
The rate of psychiatric hospitalizations in 2016 among adults in the North Shore is higher
than Staten Island and overall New York City (Richmond University Medical Center, 2019). The
North Shore of Staten Island in 2016, according to the Department of Health and Mental
Hygiene (DOHMH), had a rate of 963 psychiatric hospitalizations per 100, 000 compared to 707
Staten Island average, and the New York City average of 676 (The New York City Department
of health and Mental Hygiene, 2018). According to the County Health Rankings and Roadmaps
Richmond County of Staten Island’s premature death toll in 2020 was 5,600 compared to 5,500
with the top U.S. performers. Also, the ratio of mental health patients to mental health provider is
440:1, compared to 350:1 in New York City, and 290:1 with the top U.S. performers.
According to the DOHMH (2018), The New York State Department of Health, Statewide
Planning and Research Cooperative system, the rate of avoidable hospitalizations among adults
in the North Shore of Staten Island is higher than New York City rate in 2014. High psychiatric
hospitalization rates reflect the challenges residents in under-resourced neighborhoods face.
Telemental Health Services
17
These include difficulty accessing preventive services and early care, greater exposure to
stressors, and interruptions in health insurance coverage (DOHMH, 2018)
In New York City, the general wait times for psychiatry appointments were significantly
longer in the spring than in the summer (49.9 days versus 36.7 days) and the wait times of
therapy appointments were shorter in the community than the hospital clinics (19.1 days versus
35.3 days) (Olin et al., 2016). At the onset of the COVID-19 pandemic, when the clinic
adopted TMH services, there was a surge in mental health needs. Ways to access mental
health treatment in the North Shore community was initiated to meet the needs of the
community. As an Office of Mental Health (OMH) designated funding entity, there is annual
requirement to show effectiveness for increasing patient capacity within the clinic.
According to the New York State Office of Mental Health (OMH), there are clinic
services that must be available and offered as needed at any mental health clinic licensed by
OMH. These required services, as noted in Table 1, include assessments, therapies, injections,
and enhanced services (New York Office of Mental health, 2021a). These requisite services
serve as the template for allowable billing. Assessments include initial assessment and
psychiatric evaluation, therapies include individual psychotherapy, family/ collateral
psychotherapy, group psychotherapy, and psychotropic medication treatment. In-person clinic
visit serving adults covers injectable psychotropic medication administration and injectable
medication administration with education and monitoring. The enhanced services include crisis
intervention and monitoring.
Telemental Health Services
18
Table 1
Definition of term
Billable Services
1. Assessment
2. Therapies
3. Injections
4. Enhanced
services
Optional clinic services
1. Testing
2. Physical health
3. Injections
4. Enhanced
Services
Other Terms
1. Other Service
type
2. Outreach
Definition of terms
Initial assessment and psychiatric assessment
Individual psychotherapy, family/ collateral psychotherapy, group
psychotherapy, and psychotropic medication treatment.
Injectable psychotropic medication administration and injectable
psychotropic medication administration with education and
monitoring (both for clinics serving adults)
Crisis intervention and complex care management. A crisis
intervention refers to activities, including medication and verbal
therapy, which are designed to address acute distress and
associated behaviors when the individual’s condition requires
immediate attention. Complex care management is a clinical level
service which may be necessary as a follow up to psychotherapy,
psychotropic medication treatment or crisis service for the purpose
of preventing a change in community status or as a response to
complex conditions.
Description of services
Developmental testing, psychological testing, and psychiatric
consultation
Health physicals and health monitoring
Injectable psychotropic medication administration for clinics
serving children and injectable psychotropic administration with
education and monitoring for clinics serving children.
smoking cessation treatment for individual and group, and
Screening, Brief intervention, and referral to treatment (SBIRT).
Documentation of pertinent patient information necessary for
treatment that is not billable
Different attempts to contact a patient in connection with his
appointment or related matter.
3. Other definition
of terms
4. High-Risk
It involves weekly clinical team meetings to discuss any patient
Criteria protocol put on the HRCP pathway, calling patients who are missing
(HRSCP)
appointments and could potentially be loss to follow up, reporting
patients who are in Assisted Outpatient Treatment (AOT) who are
court ordered or on probation and not following up with treatment,
patients who have reported suicidal or homicidal thoughts within
the past six months, aggressive or violent patients, sexual
predators, and frequently hospitalized patients within the last six
months.
5. Contact log
A flowchart showing the service types in a period.
Telemental Health Services
6. Census
7. Number of the
census report
per contact
8. The number of
hospitalizations
per contact
19
The number of active patients in the wellness and recovery
program in a period.
This is the percentage of billable services received by each patient
in a period
The percentage of services received by hospitalized patients in a
period
To achieve the standard of the New York OMH, the program must find ways to
communicate with providers and patients to maintain quality care. A High-Risk Criteria protocol
(HRCP) developed by the PH became useful in monitoring patients. This HRCP involves weekly
clinical team meetings to discuss any patient put on the HRCP pathway, calling patients who are
missing appointments and could potentially be loss to follow up, reporting patients who are in
Assisted Outpatient Treatment (AOT) who are court ordered or on probation and not following
up with treatment, and patients who have reported suicidal or homicidal thoughts within the past
six months. Additionally, those patients who are frequently hospitalized within the past six
months, patients who are aggressive, violent, or with sexually inappropriate behaviors, patients
and anyone involved in domestic violence where the perpetrator continues to have access to the
victim are also included in the HRCP.
Intended Improvement
Project Hospitality (PH) is a non-for-profit outpatient Wellness and Recovery program
in Staten Island, New York. Telehealth/ TMH did not exist at the clinic before the Corona Virus
pandemic and was initiated in March 2020. The implementation was ongoing and there was lots
of learning and teaching as a team.
The PH program, an OMH funded entity, needed to show the effectiveness of TMH
services by increasing patients’ assessments. A quality improvement initiative was
Telemental Health Services
20
implemented which monitored patient outcomes by showing lack of deterioration to the point
of hospitalization for patients with co-occurring mental health and substance use disorder.
This was done using the self-assessment tools PHQ-9, the GAD-7, and C-SSRS. While CSSRS screens the suicide risks of the clients, the PHQ-9 assessment determined level of
depression of clients, the GAD-7 assessment was for anxiety levels. These three-assessment
metric are done during initial intake and every 6 months and based on patient symptom
changes. These assessments are completed in the scope of practice of licensed social workers
(LSWs) and mental health counselors (MHCs). These assessments are performed frequently on
patients based on their needs. The patient risk assessment tool was created to monitor patients
either at high risk of decompensation and/or suicide or those lost to follow up and are identified
by running the contact lapse report and then efforts are made to re-engage them in treatment.
Interventions to increase patient visits were initiated when the PHQ-9 score level was
ten and above or the GAD score is above 10 which indicates moderate to high anxiety. Also,
this is brought to the attention of the NPPs and other providers for a case conference and
modify medication management. The GAD-7 is associated with acceptable specificity and
sensitivity for detecting clinically significant anxiety symptoms (Mossman et al, 2018) like
the PHQ-9 which is sensitive and more specific for all ages from a cut off score of 10 or
above (Levis et al, 2019). The Columbia-Suicide Severity Rating Scale (C-SSRS) is a scale
designed to measure suicidal ideation severity and suicidal behaviors by capturing the pastmonth severity of suicidal ideation and measuring the past 3-month presence of actual and
aborted suicide attempts (Bjureberg et al., 2021). These screening tools PHQ-9, GAD-7, and
C-SSRS have been found to be valid and reliable in predicting level of depression, anxiety,
and adequate ability to detect suicidality in risk and behavior (Wood & Gupta, 2017).
Telemental Health Services
21
The hospitalization rate was checked monthly using the community based electronic
health records such as the PSYCKES and Healthix by the clinical supervisor. The current
emergency TMH guideline by the New York State Office of Mental Health (NYSOMH) utilized
verbal patient consent to provide services. Other verbal consents obtained included consent for
community based interoperable EHR, Psyckes, and Healthix. Psyckes is a HIPAA-compliant
web-based application designed to support clinical decision making, care coordination, and
quality improvement in New York State (NYSOMH, 2021b). While the Psyckes monitors
where and when a Medicaid patient receives mental health treatment, Healthix monitors all the
patients regardless of insurances and verbal consents are obtained from the patients to access
their information in these databases, allowing tracking of those patients who often use the
Emergency Department (ED) for care. The area of struggle for this organization is how to
monitor patients’ compliance to treatment regimen.
Project Purpose
The purpose for this DNP project was to evaluate the impact of TMH services in
increasing access to mental health services and reducing inpatient hospitalization because of
more frequent assessment usage. It was achieved by comparing records of in-person visits period
to the records of the emergency TMH guideline period.
Increasing access and preventing loss to patient follow up through TMH in outpatient
mental health wellness and recovery clinic was a new phenomenon in the PH program. The need
for access to mental health treatment grew throughout New York causing a shift to cater for the
mental health needs of the community at large. There were multiple systems needed to make the
system functional. Health care providers, insurers, patients, and others must work together to
ensure high quality, accessible treatment for all who need it (CVN & NCBH, 2018).
Telemental Health Services
22
Project Question
In an outpatient mental health wellness and recovery clinic in Staten Island New York,
does increased patient assessment via the implementation of TMH services lead to increased
access to mental health care and reduced hospitalization?
Project Objectives
The goal of this quality improvement project is to use findings to support the impact of
the New York emergency TMH guideline in increasing access to care and reducing inpatient
hospitalization.
The HRCP developed by the program was useful in evaluating patients’ level of risk
and assigning specialized treatment. Screening tools PHQ-9, GAD-7, and C-SSRS were used
to ascertain the severity of mental health symptoms. The use of these assessment tools would
be the primary source of data once a month or as needed based on patients’ symptom
presentation. The secondary source of data was obtained through the interoperable community
EHR like Psyckes and Healthix. These secondary sources of data indicated where and when a
client obtained care within and/ or outside the program within the last 90 days. These two
interoperable EHRs, Psyckes and Healthix, are community web-based applications in New York
state designed to obtain real-time patient data throughout the patients’ care used for care
coordination and quality improvement (Healthix, 2021; NYSOMH, 2021b).
It is important to note that the clinic initiated TMH under the emergency guideline in
April 2020 due to the extenuating circumstances posed by the COVID-19. There were
adjustments in the proposed implementation due to changes in New York State’s guidance,
which included loosening restrictions on duration of session for billing, allowing telephonic
services as audio only services, and protection of dual reimbursement where telehealth
Telemental Health Services
23
services not covered by Medicare but fully covered by Medicaid were allowed for
beneficiaries with both Medicare and Medicaid (Dorin, 2021). These guidelines allowed for
the provision of mental health services to continue despite health and safety concerns posed
by the COVID-19 pandemic.
Data obtained was compared between the office visit/ in-person office visit prepandemic metrics to those of the TMH. The outcome of TMH was evaluated by using the
assessment tools, organizational EHR (Awards) and interoperable community EHR.
Theoretical Framework
The project began with the goal of evaluating the efficiency, effectiveness, and experiences of
both the patients and the providers. An effective framework is required in knowing the impact of
the emergency TMH services in the clinic and how it impacts the patients, and their providers.
The evaluation approach utilized for this topic was the Effectiveness-Efficiency-Equity
conceptual framework. The 3 Es were important in how the plan came about. A framework
that reduces or eliminates disparity in mental health access as well as evaluate the usefulness of
the screening tools would be required explain the impact of the emergency TMH guideline. Each
component of this framework was examined and applied to the multiple facets implicit in
mental health services to determine the impact of TMH guideline on patients and how it
affected their access and outlook on mental health services.
Each of the three components in the Efficiency- Effectiveness- Equity conceptual
framework seeks to improve the provision of services. Sorato et al. (2020) define efficiency as “a
measure of the quality and/or quantity of output for a given level of input.” As applied to
healthcare and more specifically PH, it would refer to the reduction of wasted time between
seeing patients and providing services, particularly in the case of non-compliant patients who do
Telemental Health Services
24
not present for their appointments or those who are often rescheduled for their follow up
appointments. It also can refer to the length of time required between requesting and receiving an
appointment or travel time.
The emergency TMH guidelines allow for the provider to reach the patient over the
phone or through secure video services at the patients’ preferred location whereas the previous
guidelines dictated that patients had to be present at the physical clinic location to complete a
telehealth service. This perpetuated the challenges inherent with in-person care due to the need
for patients travel to a physical clinic setting where telehealth would take place. Due to the
shortage of psychiatrists and other mental health care providers, the emergency telehealth
guidelines eased the burden on providers allowing them to reach more patients with little
additional overhead cost. This eliminated the need for full-time clinic patient visits in the
traditional clinical setting.
Equity is defined by Sorato et al. (2020) as “the absence of avoidable or remediable
differences among groups of people whether those groups are defined socially, economically,
demographically, or geographically”. In the population involved in this study, there is a high
incidence of poverty and reliance on public assistance programs. Resources are limited and
negatively impact a person’s ability to attend healthcare appointments or to present to a clinic to
engage in telehealth as governed by the previous policy. Limitations include the lack of access to
a vehicle, travel costs, or access to affordable childcare.
On Staten Island, many providers in the South shore accept primarily private insurances
or cash payments, while the providers on the North shore more readily accept Medicare and
Medicaid. Access to telehealth can be afforded by out-of-pocket payment whereas those with
Medicare and Medicaid insurances must conform to the policy defined for telehealth for a
Telemental Health Services
25
session to be billable. Limitations in billable mental health services has created disparity in
access to telehealth based on insurance and affordability.
Effectiveness is the ability to produce desired results. The effectiveness of TMH can be
measured through indicators such as “waiting time to receive mental health treatment, the
availability of resources, the cost of access, availability of medical personnel, infrastructure and
equipment, and the cost of travel” (Sorato et al., 2020). Effectiveness also relies on the delivery
of care, improvement of standard screening scores such as PHQ-9, GAD and preventing hospital
admissions. The effectiveness of TMH services can be evident when patients are readily
connected with mental health providers instead of using the emergency department (ED) to
receive mental health services.
When effectiveness is paired with efficiency and equity, a health care system with little
waste and improved access can be established and meet patient care goals and achieve better
outcomes (Sylvia & Terhaar, 2018).
Telemental Health Services
26
Table 2
The benefit of the Effectiveness-Efficiency-Equity conceptual framework
Efficiency
Equity
A measure of quality and/
Absence of disparity
or quantity of output
1. Increase access to
1. Reduction of wasted
mental health services
time between seeing
by eliminating direct
patients and
patient cost.
provided services.
2. Reduces financial
2. Help manage
burden to patients as
patients’ time.
cost of access.
3. Ease the burden on
3. Eliminate the socioproviders by
economical,
reducing the impact
geographical, and
of shortage of mental
demographical impact
health workers.
of seeking mental
4. Eliminate overhead
health treatment
cost incurred in
clinic visits
Effectiveness
Ability to produce desired
results
1. Reduced waiting time
to receive mental
health treatment
2. Reduced cost of
access
3. Availability of mental
health workers,
infrastructure, and
equipment.
4. Eliminate cost of
travelling for
appointments.
Literature Synthesis
Mental health services in the U.S. are insufficient despite more than half of Americans
(56%) seeking help (National Council for Mental wellbeing, 2018). Physical limitations,
distance, and time are major obstacles to access mental health services (Haghnia et al., 2019).
Other limitations to receiving mental health services include inadequate specialty services,
maldistribution issues, and inadequate funding (Mace et al., 2018). These challenges have
plagued the behavioral health services and created the demand and need for an innovative
approach of providing care to individuals (Mace et al., 2018). Community-based agencies are
uniquely positioned to fill this gap in care; providing individualized care across settings,
coordinating services across agencies, and assisting individuals to transition at various stages of
Telemental Health Services
27
treatment (Williams, 2020). TMH can be used to further reduce the treatment gap and overcome
the challenges in accessing mental health services.
Reimbursement parity is one of the biggest barriers to TMH expansion (Warren &
Smalley, 2020). Lack of parity results from different rates for in-person office visits compared to
telehealth. In-person office visits are reimbursed fully for mental health services compared to
telehealth services. Currently, among the U.S. states and territories, there are twenty-three with
telehealth wavers, twenty-seven without wavers or that have closed wavers, and nineteen with
long-term or permanent interstate telemedicine including Guam, Commonwealth of the Northern
Mariana Islands, Puerto Rico, and the U.S. Virgin Islands (Federation of State Medical Boards,
2022). Due to the regulatory waiver, Executive Order 202 section 596 of Title 14, issued by the
New York state governor, the term TMH was temporarily expanded for Medicaid-reimbursable
services to include telephonic and/ or video, technology commonly available on smart phones
and other devices (Cuomo et al., 2020).
Less than 10% of the U.S. population used telehealth for a clinical encounter and only
18% of physicians provided such services prior to COVID-19 (Warren & Smalley, 2020).
Limited diagnosis, usually requiring physical comorbidities, would cover home-based telehealth
services (Warren & Smalley, 2020). For instance, radiological and critical care services were
reimbursable by telehealth when the patient or provider was at home. PH, a community-based
organization, did not apply for the TMH hub site license pre-pandemic despite eligibility due to
the difference in reimbursement fees. The hub site (provider location) also known as the distant
site is the consulting site where the provider is seeing the patient at a distance or consulting with
the patient’s provider, while the spoke site (patient location) also known as the originating site is
Telemental Health Services
28
where the patient or patient’s physician is located during the telehealth encounter (Medicare
Learning Network, 2021).
Smartphones are helping to bridge the disparity in access to treatment. While Black
Americans and Hispanics own smartphones at similar shares to whites, roughly 82% Whites
report owning a desktop or laptop computer compared to 58% of Black people and 57% of
Hispanics (Perrin & Turner, 2019). There are still concerns regarding disparities in access to
mental health services for those in rural, suburban, and urban communities. Technological
infrastructures like owning a desktop or laptop computer in these communities would ensure
improved access to TMH services.
Evidence Search
The evidence obtained for this review was from a computerized search of the library
databases. Health science databases reviewed include Cumulative Index to Nursing and Allied
Health literature (CINAHL), Centers for Disease Control and Prevention (CDC), PubMed,
Academic Search Premier (EBSCO), New York State Office of Mental Health (NYSOMH), and
the American Psychological Association (APA PsychInfo). The keywords used to retrieve
documents were “telepsychiatry,” “telemedicine,” “teleconferencing,” “telepsychology,”
“psychiatric clinics,” “access to mental health,” and “hospital readmission.” Alternate terms
include “telehealth,” “telemental health,” “community mental health centers,” “mental health
programs,” “walk in clinics,” “increased access to mental health,” “and decreasing
hospitalization readmission.” Limiters included last 10 years, English, primary research (clinical
study, clinical trial, clinical controlled trial, meta-analysis, randomized controlled trial). To
determine appropriateness, titles and abstracts of the results were screened. Out of the 240
articles resulted, ten studies were deemed appropriate via literature review and appraisal.
Telemental Health Services
29
Comprehensive Appraisal of Evidence
Knowledge of Telemental Health
Telemental health is the use of two-way interactive audio and video equipment used in
real time, to support clinical psychiatric care (New York Office of Mental Health, 2020). On July
3, 2019, the New York State Office of Mental Health (NYSOMH), changed the term
“telepsychiatry” to “telemental health” to demonstrate their broader use of technology and
expansion of “telemental health” services delivery (NYSOMH, 2020). This expansion enabled
additional providers: psychologists, social workers, mental health counselors, marriage and
family therapists, creative arts therapists, and psychoanalysts (NYSOMH, 2020) to deliver
psychiatric care remotely. To fully understand the need for TMH services, it is necessary to
understand the nature of mental disorders, costs, available resources, and treatment (Eisman et al,
2020)
Most mental disorders are a result of a combination of several “biopsychosocial” factors
rather than a single factor (Bashshur et al., 2016). According to the National Institute of Mental
Health (2021), mental illnesses include those that are diagnosable currently or within the past
year lasting a sufficient duration to meet diagnostic criteria within the Diagnostic and Statistical
manual of Mental Disorders (DSM-V) and excludes a developmental and or substance use
disorder. Age 14 is the beginning of 50 % of most lifetime mental illness and 75% have
manifested by age 24 (National Alliance on mental Illness (NAMI), 2021). Mental disorders
cause substantial distress or disruption in social, occupational, and/or family relationships, and
people with such disorders fail to meet important obligations in everyday life, work, or school
(Bashshur et al., 2016). Mental disorders are common, and it is difficult to determine the
prevalence of mental disorders in any population (Bashshur et al., 2016). Among U.S. adults,
Telemental Health Services
30
21% (52.9 million people) experienced mental illness in 2020, 5.6% (14.2 million people)
experienced serious mental illness, and 6.7% (17 million people) experienced a co-occurring
substance use disorder and mental illness in 2020 (NAMI, 2021). In 2016, there were 16.5% (7.7
million people) of U.S. youth aged 6 to 17 that experienced a mental health disorder (NAMI,
2021). In 2020, more than 12 million U.S adults had serious thoughts of suicide, 1 in 5
experienced mental illness, 1 in 20 experienced serious mental illness, and 1 in 15 experienced
substance use disorder and mental illness.
There are an estimated 12 million emergency department visits (1 out of every 8) by U.S.
adults related to mental illness and substance use disorders (NAMI, 2021). In 2020, 17 million
U.S. adults (32.1%) with mental illness experienced a concurrent substance use disorder (NAMI,
2021). Students aged 6 to 17 with mental, emotional, or behavioral concerns are three times more
likely to repeat a grade and high school students with significant symptoms of depression are
more than twice as likely to drop out of compared to their peers (NAMI 2021). The
unemployment rate among U.S. adults with mental illness is 6.4% compared to 5.1% of those
without (NAMI, 2021). There are at least 8.4 million people in the U.S. providing an average of
32 hours per week of unpaid care to adults with mental or emotional health issues (NAMI, 2021).
Among the people experiencing homelessness in the U.S., 20.8% have serious mental
illness and 37% of the adults incarcerated in the state and federal prison systems have a
diagnosed mental illness (NAMI, 2021). About 2 in 5 incarcerated people have mental illness
and an estimated 4,000 people with serious mental illness are held in solitary confinement in
U.S. prisons (NAMI, 2021). One in four people shot and killed in the U.S. by police officers
between 2015 and 2020 had mental health conditions (NAMI, 2021). Suicide is the second
leading cause of death among people aged 10 to 34 and the 10th leading cause of death in the
Telemental Health Services
31
U.S. (NAMI, 2021). Transgender adults are twelve times more likely to attempt suicide than the
general population (NAMI, 2021). Lesbian, gay, and bisexual youth are four times more likely to
attempt suicide than straight youth, and 78 % of people who die by suicide are male (NAMI,
2021).
Serious mental illness causes $193.2 billion in lost earnings in the U.S. economy each
year and mood disorders are the most common cause of hospitalization for all people in the U.S.
under the age 45 (NAMI, 2021). The global cost of depression and anxiety is $1 trillion in lost
productivity each year, and depression is the leading cause of disability worldwide (NAMI,
2022). The global cost of mental health disorders was estimated at $2.5 trillion per year in poor
health and reduced productivity in 2010 with a projected cost increase to $6 trillion by 2030 (The
Lancet Global health, 2020).
In 2020, 46.2% of U.S. adults with mental illness and 64.5% of those with serious mental
illness received treatment (NAMI, 2021). Among U.S. adults with mental illness, the annual rate
of treatment indicates that females (51.2%) received mental health services more than males
(37.4%); lesbian, gay and bisexual (54.3%) more than non-Hispanic White (51.8%), more than
non-Hispanic Black or African American (37.1%), and more than Hispanic or Latino (35.1%)
(NAMI, 2021).
The year 2020 was typified by uncertainty and loss brought about by the Coronavirus
(COVID-19) pandemic causing an impact on mental health of individuals around the world.
Manifest as Pronounced fear and anxiety about COVID-19, emotional distress resulting from
illness, bereavement, unemployment, income loss, and loneliness due to social isolation (The
Lancet Global health, 2020). Among adults in the U.S. in 2020, 1 in 5 report that the pandemic
had a significant impact on their mental health, 45% of those adults have mental health illness
Telemental Health Services
32
and 55% serious mental illness (NAMI, 2021). In 2020, among U.S. adults who received mental
health services, 17.7 million experienced delays or cancellation in appointments, 7.3 million
experienced delays in getting prescriptions and 4.9 million were unable to access needed care
(NAMI, 2021). Approximately 11% of adults with mental illness are without insurance coverage,
134 million people were living in a designated mental health professional shortage area, and 55%
of U.S. counties did not have a single practicing psychiatrist in 2020 (NAMI, 2021).
With the current shortage in the number of psychiatrists, and the predicted future shortage
of 15,600 to 16,060 by 2025, other mental health professionals are increasingly assuming
primary treatment roles (Harrar, 2021). This includes psychologists, nurse practitioners, mental
health counselors, and psychiatric social workers. As the number of inpatient psychiatric beds
decreases, the shortage of psychiatrists along with health insurance limitations of coverage,
determining which provider can be seen or the amount of help that can be received, has led to
long wait times for appointment, shorter doctor visits for psychiatric issues, less monitoring of
patients on prescriptions, and long wait times in the hospital (Harrar, 2021). The trends in mental
health treatment along with shortage of mental health personnel support the need for TMH
services.
Effects of telehealth health in increasing access to mental health services.
A case study report within a community mental health center in suburban Illinois
describes the planning, implementation, and evaluation of a telepsychiatry program (Mahmoud
et al., 2020). Identified telemental access barriers included clinician shortage, patient and staff
resistance, reimbursement, funding, technological challenges, infrastructure, patient privacy,
concerns about cyber security, and misinformation and misconception of telepsychiatry
(Mahmoud et al., 2020). A variety of access challenges were mitigated by defining clinician
Telemental Health Services
33
roles and developing clear workflow, increasing workers’ buy-in on telehealth by providing
training, assessing patients’ preferences as well as providing on-site alternatives, and enhancing
technology preparedness by adding a hotspot in case of internet failure (Mahmoud et al., 2020).
A double-blinded randomized control trial (RCT) examined the use of multimedia
applications (IMO voice calls, text messaging, Telegram, and Skype) for follow up treatment of
Iranian war veterans with PTSD (Haghnia et al., 2019). A significant positive relationship exists
between use and patient satisfaction. Additionally, there were benefits in reduced treatment costs
and completion rates.
Mace et al. (2018) conducted a mixed methods review of data collected from 329
behavioral health organizations to investigate the utilization patterns and barriers of telehealth
among behavioral health providers. The responding organizations represented all fifty states, the
District of Columbia and Puerto Rico. Half of the respondents (n=153; 47%) used telehealth
specifically for behavioral health services and the telehealth services most used by behavioral
health providers is videoconferencing (40%) followed by telephone (11%) (Mace et al., 2018).
Among the behavioral services respondents, psychiatrists are the largest group of users (78%) of
telehealth followed by mental health counselors (33%) (Mace et al., 2018).
The study also revealed the three most cited barriers to telehealth as; financing, including
the lack of reimbursement by insurers (59%), cost of implementing the infrastructure (56%), and
the cost of maintaining telehealth (56%) (Mace et al., 2018). The cost of teleconferencing
technology continues to decline because of innovations in the telecommunication media making
TMH services more cost effective (Guaiana et al, 2021). Other barriers highlighted by
respondents include the lack of leadership regarding telehealth, staffing challenges, client
challenges, and technical assistance barriers. The study also explored the primary reasons for
Telemental Health Services
34
implementing telehealth services. Study respondents agreed that telehealth is important to an
organization for reaching more patients, as it allows for the organizations to reach patients over a
geographical distance, and permits a more efficient workflow (Mace et al., 2018).
However, there are limitations to the generalizability of this study because of the sample
size. Even though there was representation from all the fifty states, there was unequal
representation in the responses to the electronic survey (i.e., NY 29.9%, PA 16.5% and six states/
area with only one response) (Mace et al., 208). Applicability of individual state-based findings
to other states may be challenging. Also, the sample for qualitative interviews was limited to
only five states compared to the RCT. (Mace et al., 2018).
An observational cross-sectional study by Cantor et al., 2021, reported that among the
8860-outpatient mental health treatment facilities in the United States, 3848 (43.4%) offered
telehealth services at the onset of the COVID-19 pandemic on April 16, 2020. This data was
from the National Mental health Services Survey (N-MHSS) of 2019 with response rate of 91%
(Cantor et al., 2021). The lack of telehealth access indicates that more needs to be done to
enhance telehealth policies and ensure continuous access to treatment even through another
pandemic. This study reveals limited telehealth infrastructure to meet the expected increase in
demand for mental health care during the pandemic. limitations of this study include
approximately 9.3% of the eligible facilities did not respond to the 2019 N-MHSS survey, the
behavioral treatment services locator did not contain treatment capacity or quality, and the data
collected did not reflect changes in telehealth services after the March 13, 2020, declared
national emergency (Cantor et al., 2021).
Effect of Telehealth in Reducing Hospitalization
Telemental Health Services
35
Brearly et al. (2020), investigated the implementation and outcome of a clinical quality
improvement project, Allied Transitional Telehealth Encounters post-inpatient Discharge
(ATTEND), which targeted veterans transitioning from inpatient to outpatient mental healthcare.
Here, Veterans, who met criteria, were provided 4G Android tablets with which they were
contacted via Clinical Video Telehealth (CVT) by a pharmacist to discuss their medications from
any location and at their convenience. Noted factors for a decrease in standard outpatient followup care were housing instability, transportation problems, distance from the clinic, forgetting to
attend appointments, driving related anxiety due to posttraumatic stress disorder (PTSD), poor
referral communication, and difficulty arranging time off work (Brearly et al., 2020).
Post inpatient mental health follow-up provided by the ATTEND clinic, helped veterans
to overcome barriers to care, increased patient satisfaction through prompt access, reduced the
likelihood of inpatient readmission, and lowered recidivism rate (Brearly et al., 2020). The
limitation of this implementation study model is the sample size of twenty patients. Although the
authors denied conflict of interest, this study was supported by the DVA Office of Academic
Affiliations Advanced Fellowship Program in Mental illness, Research, and Treatment and the
Mid-Atlantic Mental Illness Research, Education, and Clinical Center (Brearly et al., 2020).
Strengths of Evidence
The adoption of TMH services allows providers to increase continuity of care, extend
access beyond normal clinic hours, reduce patient travel burden, and help overcome clinician
shortages in rural and underserved populations which helps practices to enhance patient
wellness, improve efficiency, provide higher quality of care, and increase patient satisfaction
(American Medical Association [AMA], 2020). TMH has ensured that patients with mental
health problems, like depression, anxiety, attention-deficit/hyperactivity disorder (ADHD),
Telemental Health Services
36
bipolar disorder, schizophrenia, and other serious mental health conditions, who are adjusting to
new medications maintain contact with their mental health providers for their routine follow-ups.
It has enabled patients to participate in their routine virtual psychotherapy appointments.
Transportation barriers can be overcome when patients are able to access care
from their convenient locations.
Weakness of Evidence
Although there are benefits to TMH implementation, there are also challenges to
adopting this innovative technology. Some of the barriers include inconsistent reimbursement,
concerns over security, privacy, and confidentiality, concerns of providing safe and effective
care, and lack of evidence about impact on health care costs, utilization, or outcomes (AMA,
2020).
Gaps and Limitations
The resounding theme in these published studies discussed above is the correlation of
TMH with increased access to mental health services. There is increase in provider and patient
satisfaction while using TMH services in an industry that is overburdened by shortage of staff.
Though there is increased access of patients to their mental health providers for alleviation of
symptoms and preventing mental health crisis, there is little evidence discussed in the studies
about decreasing hospitalization. There is need for more current academic research studies to
evaluate the impact of TMH on reducing patients’ hospitalization readmission. There are not
enough studies that sufficiently examine the effect of telehealth on decreasing hospitalization.
There is the limitation to duplicability of studies using some of these studies because of the small
sample size.
Telemental Health Services
37
There are gaps in the use of TMH services. These studies have highlighted the
differences between states in their adoption of specific TMH policies. This makes the
duplicability of study difficult. A standardized national TMH policy would expand TMH access
at the end of the COVID-19 pandemic for regular use and maintain the infrastructures in place
should there be another pandemic. There is need to incentivize the training and recruitment of
mental health providers to help reduce the personnel shortage.
METHOD
This QI project aims to assess the ability of increased assessment via TMH to increase
access to care and decrease hospitalizations. This will be assessed by evaluating the impact of
increased monitoring and evaluation of anxiety, depression, and suicidality on hospitalization
rates and. The methodology section focused on project design, the model for implementation,
setting and stakeholders, planning the intervention, ethical consideration, data collection and
analysis of data collected.
Project design
Quality improvement in health care involves the analysis of performance in a system
and the development of a system designed to systematically improve care. It is a process of
systematic and continuous actions, which cause measurable improvements in the health status
of patient populations and health care services (Hickey & Brosnan, 2017). The goal here is to
utilize instruments to assess well-being and mental health status and their impact of health
care delivery and ensuring patient safety.
This QI project aims to assess the impact of increased assessment while using the current
New York State emergency TMH guidance. The current effects of TMH are being experienced
in real time while using the emergency guidance as a policy guide. However, the implementation
Telemental Health Services
38
of this QI project was designed via gap analysis to make course corrections by organizational
measures by looking at what was missing and determining ways to improve those faults. Areas
for correction include monitoring of patients’ remission of symptoms by frequently utilizing the
assessment tools, reducing patient loss to follow up by increasing contact with patients,
monitoring the effect of safety plan implementations by using the HRCP, and reducing inpatient
hospitalization.
This vulnerable population includes patients over the age of eighteen with mental illness,
some of whom have co-occurring substance use disorder. Ninety days retrospective data, for
three years prior to the roll out, was used as a comparison. The data in the retrospective timeline
is compared to the period of TMH implementation after the Institutional Review Board (IRB)
approval from September 15, 2021, until December 13, 2021.
This QI project utilizes a retrospective chart review of assessment tools. The degree of
change in mental health symptoms were measured by the pre and post interventions for both new
and established patients. Patients’ mental health interventions varied from patient to patient due
to the degree of their symptoms. In this QI project, interventions included easy access of patients
to their providers via telehealth, frequent therapy sessions based on assessed needs and/ or
medication adjustments determined by the HRCP.
The team reached a consensus to implement screening tools upon intake and at least once
a month during the implementation phase of data collection. The team includes the one clerk,
one intake specialist, four licensed Social Workers (LMSW) and two fee-for-service LMSW, two
Mental Health Counselors (MHC), two Nurse Practitioners in psychiatry (NPPs), a per diem
internal medicine physician, two substance abuse counselors, two billing personnel, the
Recovery Program director, and the Associate Area Director also filling in the role of Wellness
Telemental Health Services
39
Program director. The implementation timeline is from September 15, 2021, until December 15,
2021. The team discussed variables which could potentially impact the TMH services include
varying degree of serious mental illnesses, frequency of therapy already available, buy in of the
staff to assess patients by using the assessment tools, appointment kept by patients for their
therapy sessions and medication management sessions, and access to good working electronic
devices. We agreed that more frequent use of screening tools than once a month could potentially
skew the result of data collected.
Model for Implementation
The Plan-DO-Study-Act (PDSA) is a suitable implementation model to assess the stages
of change that is being implemented (AHRQ, 2020). Using the four-cycle model of testing
change/ improvement, the PDSA cycle develops a plan to evaluate the change (Plan), carrying
out the test (Do), observing and learning from the consequences (Study), and determining what
modifications should be made to the test (Act) (IHI, 2021).
We planned to use the mental health assessment tools PHQ-9, GAD, and C-SSRS to
assess for rate of symptoms of depression, anxiety, and suicidality, respectively. The rate of
patient hospitalization was assessed by checking the community based PSYCKES and Healthix
EHRs. To execute the plan, the PHQ-9, GAD, and C-SSRS were administered upon intake and at
least every 4 weeks for up to 90 days. After 90 days, gather data from the PSYCKES and
Healthix EHRs both pre and post intervention.
The assessment tools data were collected by the MHC and LMSW in the Wellness clinic
upon new patient intake and at least once every 90 days. The 90 days was the program policy for
completing a treatment plan for patients to accommodate non-Medicaid patients with
commercial insurances. Treatment planning is an ongoing process of assessing mental health
Telemental Health Services
40
status and needs of the patient, establishing his/ her treatment and rehabilitative goals, and
determining what services may be provided by the clinic to assist the patient in accomplishing
those goals (NYSOMH, 2021a). According the NYSOMH (2021a), the initial treatment plan
must be completed thirty calendar days from the admission date and reviewed annually after
admission or from the most recently completed treatment plan review unless the services are
covered by a third-party payer with a different requirement. Treatment plan is reviewed when
there are significant clinical changes like behavioral or medical diagnosis, changes in risk level,
increased or new symptoms, alterations in functioning, stressors, needs, circumstances, and
reviews may be required more frequently as determined by the treating clinician, patient, or
family member (NYSOMH, 2021a).
The implementation phase of this model is the “Do” phase. The plan was executed on
September 15, 2021. In this QI project, a majority of the TMH services provided were through
the telephone. This was the most readily available resource for the population under study. There
were cases in which ZOOM videoconferencing was utilized. There were other instances in which
patients received hybrid services of TMH and in-person visits.
Patients with worsening symptoms or high risk, as determined through use of the
assessment tools, received increased contact to their mental health providers. This increased
access continued until there was improvement as evidenced by low scores of the screening tools
or resolution of symptoms. There is increased number of new patient intakes who would be
screened using the assessment tools, both in-person visits and TMH.
The study model of the PDSA in the implementation of TMH can better be understood
using a fishbone diagram to capture the Effectiveness-Efficiency-Equity conceptual framework
discussed above.
Telemental Health Services
41
Figure 1
Telemental health fishbone diagram. (Adapted from Chang (2015). No permission needed to use
and adapt. Writer modified to focus on topic).
Human
Environment
System
Service Provider
Organization
Comfort w/ work flow
Leadership
Expertise on ICT
3rd party payers
Clinical information
systems
Education & Training
Work flow
re-engineering
Location / Travel time
Patient awareness
Disease characteristics
ICT skills
& knowledge
Reliability
of technology
Patient/ client
Social norms and values
(Stigma)
Training & support
OUTCOME
• Decreased
hospitalization
• Cost effective
• Increase access TO
___ health
Data quality
Medical liability
Transmission
Storage
ICT equipment
Interoperability
System Speed
Medical cost
(out-of-pocket)
User habit
Insurance fee schedule
ICT infrastructure
Budget
Resistance to change
Routine psychotherapy
opportunity, routine follow
ups with patient’s who are
adjusting to new medications
Reimbursement
Change
management
Comfort with patient
communication
Comfort with
Provider Interaction
Society
Organizational Culture
Privacy & security rules
• Enhance patient
wellness
• Provide higher
quality of care
Practice Certification
& license
Interface standards
Information Security
User Interface
Technology
Government authority
Rules /policies
To completely evaluate the “Act” phase of this implementation model, the
implementation data is compared to data obtained from the pre-pandemic, in-person visits period
of 2019. The total patient census at the given period would be compared. The frequency of
patient contact and assessment would be compared using service contact logs. Hospitalization
rate would be compared at both periods and length of hospitalization stays would be analyzed.
Settings and Stakeholders
The QI project was implemented in Project Hospitality (PH), an outpatient mental health
Wellness and Recovery Clinic in Staten Island, Richmond County of New York City. The larger
program consists of inpatient respite and long-term substance abuse program, housing programs,
shelters, case management and youth services. This outpatient Wellness and Recovery clinic is in
Telemental Health Services
42
a county with a population of 476,143 in 2019 (United States Census Bureau, 2019). About
59.6% of the county population were listed as White non-Hispanic, 11.6% as Black or African
American, 18.6% as Hispanic or Latino, and 10.9% as Asian alone (United States Census
Bureau, 2019).
The clinic was staffed with a clerk, an intake specialist, two billing personnel each for
recovery and wellness, two Licensed Mental Health Counselors (LMHC), four Licensed
Master Social workers (LMSW), two of them are fee-for-service, two Credentialed
Alcoholism and Substance Abuse counselors (CASAC), two Nurse practitioners in psychiatry
(NPPs), a per diem internal medicine physician, Program Director in wellness, and Program
Director in Recovery. This QI project was led by a nurse practitioner employed at the site as
partial fulfillment of a Doctor of Nursing Practice school project. The other NPP is the
clinical expert who helped in rolling out the TMH services within the organization. Common
stakeholders in healthcare improvement include, but not limited to patients, clinicians,
managers, executives, clinical assistants, and payers.
Planning the Intervention
The assessment tools used as intervention for the implementation of this QI project
include the PHQ-9, GAD, C-SSRS, and the community-based EHRs (Psyckes and Healthix). All
of these were used to assess patients’ status and frequency of hospitalization readmissions. After
a careful internet search, it was identified that the PHQ-9, GAD, and C-SSRS are all publicly
available and no written consent required (Wood & Gupta, 2017). The Psyckes and Healthix
required a written consent until the pandemic when a waiver was issued for obtaining verbal
consent, allowing for verbal consents to be entered into the PH’s EHR, Awards. Unlike the
Telemental Health Services
43
Psyckes, Healthix consent is traditionally obtained via verbal consent and entered electronically
by the intake specialist or the mental health workers.
The primary areas of knowing the effectiveness of the emergency TMH guideline is by
initiating a QI initiative to demonstrate effectiveness to support continued adoption of TMH
services during the pandemic and to provide support for a permanent adoption of the guideline
by showing effectiveness through reduction in recidivism, decreased inpatient hospitalization,
and improvement in patient outcomes. Recidivism rate is monitored using the High-Risk
Criteria Protocol (HRCP) adopted by the clinic and expanded from the New York State Office
of Mental Health Clinic Standards of care anchor (NYSOMH, 2021a). The “High Risk”
criteria include anyone that has had suicide attempt within the past six months, anyone
hospitalized for psychiatric reasons for the past six months, anyone who has been to the
emergency room due to psychiatric reasons within the past three months, anyone on AOT,
those experiencing suicidal and/ or homicidal ideation with a plan and deviating from their
normal baseline within the past three months, anyone that experienced an overdose within the
past six months, anyone reported to be engaged in violent behaviors or committed a violent
crime and has been released from prison within the past six months or is currently on parole
or probation for violent crime, anyone actively using drugs and/ or alcohol and exhibiting
behaviors perceived to be a potential danger to self or others, and anyone that is the victim of
a domestic violence situation where the perpetrator continues to have access to the victim
(Project Hospitality, n.d.).
The HRCP indicates that the clinical team should discuss and assess any patient put on
the high-risk list at the team meetings once a week. Patients on the high-risk list who missed
an appointment must be contacted via phone to ascertain the reason and rescheduled. Any
Telemental Health Services
44
patient reporting suicidal or homicidal ideation must go into the HRCP and CSSRS
administered at every session. If an Assisted Outpatient Treatment (AOT) patient, who is
court ordered to take medications, disappears, the legal system must be notified immediately,
and lack of attendance must be reported.
Improvement of symptoms was monitored using the assessment tools PHQ-9, GAD-7,
and C-SSRS. While the PHQ-9 assessment determines severity of depression of patients, the
GAD-7 tool is used to measure the level of general anxiety disorder, and the C-SSRS is a
systematic suicidal screening tool. This three-assessment metric was completed during initial
patient intake and every 30 days. Interventions to increase patient visits were initiated when
the PHQ-9 score level reached ten and above, the GAD score of ten or above, or moderate to
high suicidal ideation and behavior on the C-SSRS. The PHQ-9, GAD-7 and the C-SSRS are
valid and reliable in detecting depression, anxiety, and suicidal risks, respectively (Wood &
Gupta, 2017). The NPPs and other providers were notified by any mental health worker who
assessed the patient and identified these scores. Alerting the team to call for a case conference
of treating providers allows for modification of treatment plans in the form of medication
management adjustment and/ or increase in frequency of therapy.
Another way of tracking the effectiveness of increased access is by tracking
hospitalization through Psyckes and Healthix community based electronic health records
(EHRs). Psyckes is a HIPAA-compliant web-based application designed to support clinical
decision making, care coordination, and quality improvement in New York State (NYSOMH,
2021b). It is a community database that tracks patients’ care and shows the different programs
in which the patient receives care. Consents are sought from the patients to access their
Telemental Health Services
45
information in this database. Those patients who often use the Emergency Department (ER)
for care can successfully be tracked.
Participants and Recruitment
A convenience sample aged 18 years old and older, English speaking, with mental health
illness and/ or SUD attending PH were utilized. The exclusion criteria include patients who are
younger than 18 years old, those adult patients with serious mental health illness who are not of
sound mind, those unable to make decisions, and non-English speaking.
Consents and Ethical Considerations
The investigator requested and received permission from the Director of Policy and
Compliance of the Project Hospitality organization. There is no IRB approval requirement at the
organization. Also, the investigator obtained university IRB approval.
This QI project involves the use of the current emergency TMH guideline to improve
the mental health outcome of this population. Hence, there is no experimentation involved and
there is no increased potential for direct patient risk. The data needed for this QI project was
accessed or obtained through retrospective data collection from patients’ EHR and the
electronic databases (Awards). The ethical concerns involved in this project includes patient
safety, confidentiality, respect for persons, autonomy, and justice to the participants in this
project (Liem et al, 2020). These concerns are mitigated by using a flash drive and storage
space with a lock for locking up the devise in the office. A request for a password secured
laptop with Virtual Private Network (VPN) set up by the System Administrator was granted by
PH and provided for this QI project. Consents obtained in the physical chart or the PH’s EHR,
Awards, are confirmed by the investigator before accessing patient information in the Psyckes
and Healthix databases.
Telemental Health Services
46
Data Collection
Once the IRB was approved, the Psyckes and Healthix EHRs were reviewed at the onset
of data collection and at the end of the 90 days period. Team meetings were held weekly to
review the high-risk list, discuss any added information, review the treatment plan, and update
the list. The assessment tools were obtained from the organization’s EHR to assess depression,
anxiety, and suicidal scales using the PHQ-9, GAD-7, and C-SSRS, respectively. Data for PHQ9, GAD, and the C-SSRS were obtained every 4 weeks for a period of 90 days from PH’s EHR,
Awards.
Information obtained when accessing the EHR included diagnosis, medical diagnoses,
reasons for hospitalization, medical hospitalizations, gender, age, and the assessment tool scale.
Patients’ names were deidentified. Other data obtained from the high-risk protocol spreadsheet
included risk levels, proposed plans, and case review dates. The data obtained was reviewed by
the investigator to assess inclusion criteria, saved in flash drive, and locked at the program site in
the office where only the investigator could access it. Additionally, a password secured laptop
with the Virtual Private Network (VPN) was set up by the System Administrator of PH.
Data Analysis
The outcome of this QI project was analyzed by comparing the metrics of patients in
the control 90-day period in 2019 and those from the implementation period of 2021. To
measure access to mental health services, the census of active patients during in-person visits
pre-implementation, was compared to the number of active TMH visits during implementation
of 2021. Resolution of symptoms was determined by patients’ reports and measured by the
screening tools (PHQ-9, GAD-7, and C-SSRS) upon intake and every six months, patients
referred for the high-risk protocol. There was a change in the implementation of the screening
Telemental Health Services
47
tools because of the change in guidance proposed by the NYSOMH to implement treatment
plan upon intake and annually unless clinically necessary due to changes in mental health
symptoms, hospitalization and/ or deterioration of health. The change in guidance was
compensated by implementing the assessment tools upon intake, every six months and as
needed depending on patients’ presentations. Access was also measured in the overall number
of mental health service types provided within each 30-day period to indicate the form of
patient-provider interactions noted in the contact log report.
According to the NYSOMH (2021a), there are ten mental health clinic services that
must be available and offered at any mental health clinic licensed by the NYSOMH. The
required services fall under the following categories: assessments, therapies, injections, and
enhanced services (Table 1). While assessments involve interaction between a patient and a
mental health provider including initial assessment and psychiatric assessments, therapies
include individual psychotherapy, family/ collateral psychotherapy, group psychotherapy, and
psychotropic medication treatment (NYSOMH, 2021a). Injection services involve
administration of injectable psychotropic medication administration with education and
monitoring. Enhanced services include crisis intervention and complex care management
(NYSOMH, 2021a). There are eight optional services which include developmental testing,
psychological testing, psychiatric consultation, health physical, health monitoring, injectable
psychotropic medication administration for clinics serving children, injectable psychotropic
administration with education and monitoring for clinics serving children, and enhanced
services for smoking cessation treatment for individuals and group (NYSOMH, 2021a).
Patients placed on the HRCP were monitored frequently, at every patient contact with
their provider, weekly or biweekly, using the screening tools (PHQ-9, GAD-7, and C-SSRS)
Telemental Health Services
48
to monitor severity of symptoms, improvements, and viability of individualized treatment
plan. While the patients were under the HRCP, the average length of stay in the HRCP
pathway was measured along with revisions in treatment plan. Individualized treatment plans
were revised in the form of frequency for therapy/ medication management and/ or changes to
type of medication management.
The hospitalization rate was determined by the total number of psychiatric and
substance abuse hospitalizations within the 90 days in the control period of 2019 and that of
the study period of 2021. The length of hospital stay was evaluated in the control group and
study group and differentiated between inpatient hospitalization (> 24 hours stay) and ED
visits (< 24 hours stay). The reason for hospitalization was also evaluated to affirm
psychiatric and substance abuse related conditions for the purpose of this QI project.
RESULTS
The intervention was the ongoing implementation of telemental health and patient
assessment using the screening tools. This QI project used the convenience sample of patients
who are both in the Recovery and Wellness programs of the clinic and who are 18 years and
older. The PICOT questions whether increased patient access to mental health services and
increased assessment using the screening tools, PHQ-9, GAD-7, and C-SSRS via TMH enabled
decreased frequency of hospitalization for patients within the clinic. The data for the screening
tools were to be accessed monthly. The intake process in these programs involved the
assignment of risk levels based on screening in pre and post implementation as well as
implementation of resources which are specific for individualized patient care and based on these
assessments.
Telemental Health Services
49
During planning, there was a consensus by the team of providers to implement the
screening tools every four weeks for each patient as part of the NYSOMH requirement to update
treatment plans every six months. The program took it further to assess the treatment plans every
three months. Updates by the NYSOMH then changed requirement to perform screening using
assessment tools at intake and every 12 months, and with change in status. The team determined
to use tools at intake, every six months, and with change in status, and to continue with the
HRCP which was initiated before TMH.
Figure 2
Screening tool implementation process
Intake
Specialist
High Risk
Criteria
Protocol
OMH Guideline
(PHQ-9, GAD-7,
C-SSRS)
Within 30
days of
intake
Every 6
months
As needed
upon contact
(PHQ-9,
GAD-7, CSSRS)
1st Intake
2nd Intake
LMSW
LMSW
(LMSW,
(LMSW,
LMHC
LMHC
LMHC) Patients placed
on the high-risk criteria protocol have a GAD-7 score greater
than 10, or
LMHC)
PHQ-9 score greater than 10 and/ or have had suicidal behavior within previous three months,
Telemental Health Services
50
and/ or recent psychiatric hospitalization. Other criteria of suicidality, aggression, sexual
misconduct, victims of domestic violence, and assisted patient treatment for court mandated
patients are some examples of diagnostic categories of patients placed on the high-risk protocol.
The risk levels instituted included level 1 through level 4. While the low risk (level 1)
involved patients who wished to die without plan, intent, or suicidal behavior, Level 2 Moderate
risk patients are those with suicidal ideation more than a month ago without a plan, intent, or
suicidal behavior. For the low risk level 1 patients, the CSSRS was completed every 30 days and
the moderate risk level 2 patients had a CSSRS done biweekly.
Table 3
High Risk Criteria Protocol
Level 1
Low Risk
Patients who wished
to die without plan,
intent, or suicidal
behavior
Level 2
Moderate Risk
Patients with
suicidal ideation
more than a
month ago
without a plan,
intent, or
suicidal
behavior.
Level 3
Moderate Risk
Patients with
suicidal
behavior more
than 3 months
ago
Frequency of
CSSRS
Every 30 days
Biweekly
Disposition/
plan
Address
individualized patient
needs,
Case conferencing
among providers to
address symptoms,
weekly psychotherapy
Address
Individualized
patient needs,
Case
conferencing
among
providers to
address
Close
monitoring
with the CSSRS done
with every
patient contact.
Case
conferencing
among
providers.
Referral to
Health Homes
for home
checks, therapy
twice a week
Criteria
Level 4
High Risk
Patients with
suicidal ideation
with intent or
intent with a
plan in the past
month, and
recent
psychiatric
hospitalization.
CSSRS done on
every contact
Referred to the
NPP for
biweekly
assessment and
the patients’
family members
are involved as
collateral
protective factor.
Telemental Health Services
51
symptoms,
weekly therapy
The level 3 moderate risk involved those patients with suicidal behavior more than 3
months ago and they required close monitoring with the C-SSRS done with every patient
contact. Level 4 High risk involved patients with suicidal ideation with intent or intent with a
plan in the past month, and recent psychiatric hospitalization. The level 4 high risk patients had
their C-SSRS done on every contact, was referred to the NPP for biweekly assessment, and the
patients’ family members are involved as collateral protective factor. Systematic C-SSRS
screening while using the HRCP helped identify patients who were at risk for suicide in the
general clinic population and focused services on patients with the most need (Pumariega et al,
2020).
The initial plan was to obtain assessment data every 30 days, from the population sample,
during the intervention period. Data collected in the pre and post intervention period but did not
depict the overall sample but a sub-group of the sample. There were challenges encountered in
obtaining the necessary data for this QI project. The physical charts and Awards EHR were
reviewed for Psyckes and Healthix consents to enable the researcher entry into the interoperative
EHR to verify hospitalizations. While entering the hospitalizations from the Psyckes and
Healthix databases into Awards EMR, there was omission of pertinent mental health data such as
the reason for hospitalization or visiting the hospital emergency department by the personnel
entering the patients’ data. There were various categories of the C-SSRS assessment tool in
Awards and there are four separate places to obtain reports. These categories were listed as C-
Telemental Health Services
52
SSRS, C-SSRS, C-SSRS-Ongoing Weekly, and C-SSRS Intake/ Initial. There were no data in
the first two categories and minimal data in the C-SSRS- Ongoing Weekly. C-SSRS Intake/
Initial category contained more data than the other categories and sorting through the results was
confusing.
Outcome
TMH enabled PH to increased patient contact, mental health assessment, and decrease
hospital length of stay. The number on the overall patient census report increased by 21% after
the implementation of the TMH Services. However, when accounting for the number of patient
contacts with mental health workers, there was a 25% decrease in census per individual contacts
with providers after the implementation of the TMH Services. The number of patient contacts
with their providers increased by 61% after the implementation of the TMH Services. This
indicates that increased access to mental health services which allowed mental health providers
to be easily accessible by their patients as indicated by the service types provided at the TMH
period. The distribution of service types was different between (pre and post) time periods with
the largest increase within “Individual psychotherapy” (increased by >1000 contacts, an 8%
increase).
The number of psychiatric and substance abuse hospitalizations increased by 83% after the
implementation of the TMH Services. However, when accounting for the number of contacts, the
number of substance abuse hospitalizations per contact were not different between time periods.
The percent with >1 day length of stay was lower after the implementation of the TMH Services.
Although there were more hospital services needed at the implementation of the TMH, only 20%
of the total hospitalization were inpatient compared 46% during in-person visits.
Telemental Health Services
53
After statistical adjustment for gender identity, those in the post-TMH Services period
tended to have lower anxiety levels. Those in the post-TNH Services period tended to have lower
depression levels. It should be noted that while the number of patient contacts with wellness
(service) for mental health recovery increased after the implementation of TMH services, the
number of patient contacts with recovery (service) for patients with chemical dependency
decreased by 17% in the last month of data collection. The decreased contact by patients enrolled
in recovery could indicate that they may have dropped out of the program. There were twice the
number of suicidal responses following C-SSRS assessment in-person compared to the TMH
period. This increased number of responses in the in-person period could have several causes:
there were either increased patient intake (admission) into PH within the control period, or there
were patients placed on the HRCP within the same period, or there were just increased assessments
to update the treatment plan.
Contextual Elements
The independent variables in this QI project included the TMH services provided and the
frequency of assessments which were used to determine increased mental health access and
reduced inpatient hospitalization. Appointments kept by patients for their therapy sessions and
medication management sessions are other independent variables. Hospitalization is dependent on
TMH and assessment tools. Other dependent variables include buy-in of staff to assess patients
using assessment tools and access to good working electronic devices.
Contact Log
Contact logs were reviewed to describe the number of contacts by service type during the
period before and at the implementation of the TMH services. The table below includes a
summary of overall counts and percentages of total visits for the period under investigation. The
Telemental Health Services
54
number of contacts increased by 66% after implementation of the TMH services (calculated as
(4086-2532)/2532). The distribution of service types was different between time periods (chisquare p-value <0.0001), with the largest increase within “Therapies” (See Table 2, increased by
>1000 contacts, an 80% increase).
Table 4
Service types/ Contact logs
Required Services (Billable services)
Assessment
Therapies
Injections
Enhanced services
Optional Services (Non-billable)
Outreach (Non-billable)
In person OV
N=2466
37
1.5%
1707
69.2%
41
1.7%
189
7.7%
10
0.4%
482
19.5%
Telemental Health
N=4094
14
0.3%
3072
75.0%
31
0.8%
421
10.3%
13
0.3%
543
13.3%
Figure 3
Number of contacts by service type during the pre-period (in person office visits only, September
15, 2019 – December 13, 2019) and post-period (Telemental Health, September 15, 2021 –
December 13, 2021)
Telemental Health Services
55
3500
3072
In person OV
Telemental Health
Number of contacts
3000
2500
2000
1707
1500
1000
0
482 543
421
500
41 31
37 14
Billable
services:
Assessment
Billable
services:
Therapies
Billable
services:
Injections
187
Billable
services:
Enhanced
services
10 13
Optional non- Non-billable
billable
Outreach
services
Census
There were total of 280 on the census report in the period prior to introduction of TMH
services. This increased by 21% (up to 340 patients) in the period after introduction of TMH
services (See Figure 4).
However, the number of the census report per contact was 25% lower after introduction of
TMH services periods (Pre: 318 of 2532, 12.6%, Post: 374 of 4086, 9.2%; chi-square p-value
<0.0001). It implies the interaction of patient with their mental health providers while accessing
mental health treatment via TMH services is replicable and highly statistically significant.
Substance abuse and psychiatric related hospitalizations
There were a total of 48 substance abuse and psychiatric hospitalizations in the period prior
to introduction of TMH services. This increased by 83% (up to 88 cases) in the period after
introduction of TMH Services (See figure 4). Although this was a large increase, the number of
Telemental Health Services
56
substance abuse and psychiatric hospitalizations per contact was not statistically significant
between time periods (Pre: 48 of 2532, 1.9%, Post: 88 of 4086, 2.2%; chi-square p-value = 0.472).
Finally, the percent of psychiatric and substance abuse hospitalizations that required >1
day length of stay was lower after introduction of TMH Services (Pre: 45.8%, Post: 26.1%; chisquare p-value = 0.020). This implies that increased assessment in TMH period is effective because
of the decrease in prolonged inpatient hospitalization. It could also imply that the increased contact
of SUD patients who are receiving MAT kept them current with treatment and reduced
hospitalization.
Figure 4
Number in census report and number of substance abuse and psychiatric related hospitalizations
during the pre-period (in person visits only, September 15, 2019 – December 13, 2019) and post-
Number of patients
period (TMH, September 15, 2021 – December 13, 2021)
400
In person OV
Telemental Health
300
200
100
0
Census report
Substance abuse and psychiatric
hospitalizations
GAD7
Scores of GAD-7 range from zero to 21. The cut off points of five, ten, and 15 represent
mild, moderate, and severe levels of anxiety respectively, on the GAD-7 scoring scale (Spitzer et
al, 2006). The above categories were used to compare the percentage within each anxiety level
Telemental Health Services
57
between the pre-TMH period and TMH period using Ordinal Logistic Regression. Although there
was a slight trend towards lower anxiety in the post-period, the differences were not significant
(p=0.191).
However, there were differences in characteristics between the pre and TMH periods,
including gender identity (35% female in pre-period and 73% female in the post period). When
adjusting for gender identity within ordinal logistic regression, those female and male patients in
the post-TMH Services period tended to have lower anxiety levels (p=0.030) compared to those in
the in-person visits.
Figure 5
GAD-7 anxiety level before and after introduction of TMH services
GAD-7 anxiety level before and after introduction of
Telemental Health Services
100%
Severe, 13%
80%
Severe, 35%
60%
Moderate, 10%
40%
Mild, 35%
20%
0%
Moderate, 17%
Mild, 39%
None, 20%
Female
In person OV
Severe, 16%
Severe, 12%
Moderate, 6%
Moderate, 19%
Mild, 18%
Mild, 22%
None, 65%
None, 30%
Female
Telemental Health
None, 43%
Male
In person OV
Male
Telemental Health
Overall p-value comparing In person OV versus Telemental Health = 0.030
PHQ9
The analysis for differences in PHQ-9 used the established cutoff of <5 (no depression),
5-9 mild, 10-14 moderate, 15-19 moderately severe, and 20-27 severe. These categories were
used to compare the percentage within each depression level between the pre-TMH period and
Telemental Health Services
58
TMH period using Ordinal Logistic Regression. There was a significant trend towards lower
depression level in the post-period (p=.033).
Figure 6
PHQ-9 depression before and after the introduction of TMH services
100%
PHQ-9 depressionlevel before and after
introduction of Telemental Health Services
Any Severe, 13%
80%
Any Severe, 35%
60%
Moderate, 15%
40%
Mild, 24%
20%
0%
Moderate, 23%
Mild, 30%
None, 34%
None, 27%
In person OV
p-value = 0.033
Telemental Health
Note: Ordinal logistic regression adjusted for other factors (e.g., gender identity) did not change the conclusion
C-SSRS
The responses of individual items from the C-SSRS were compared between in-person
OV and TMH using Chi-square tests or Fisher’s exact test (when expected counts were <5). In
general, the rates of suicidal responses were higher during in-person visits, but these differences
were not significant. However, the small number of responses may lead to unstable or unreliable
results. Caution should be used when interpreting these results.
Table 5
C-SSRS suicidality before and after the introduction of TMH services
C-SSRS
In-person OV
Telemental health
p-value
Telemental Health Services
Wish to be dead
Suicidal thoughts (General
non-specific thoughts)
Suicidal thoughts with
method (w/o Specific plan or
intent)
Suicidal intent (without
specific plan)
Suicidal intent (with specific
plan)
Suicidal Behavior
Suicidal attempt during
lifetime
59
23%
14%
4
2
N=44
9%
5%
6
7%
0
0%
0.1771
6
7%
1
2%
0.4251
4
4%
0
0%
0.3021
4
30
4%
33%
1
13
2%
30%
0.9991
0.6592
21
13
N= 90
0.0591
0.1431
Note: ¹Fisher’s exact test, ²Chi-square test
DISCUSSION
Summary
During utilization of the emergency TMH services the number of contacts increased by
61%. The distribution of service types was different between the in-person visits and TMH with
the largest increase being within “Therapies” (increased by >1000 contacts, an 80% increase as
displayed in Table 4). The number on the census report increased by 21% after the implementation
of the TMH Services. However, when accounting for the number of contacts, there was a 25%
decrease in census per contact after the implementation of the TMH Services. Those patients who
needed increased contacts with their provider received it.
The number of psychiatric and substance abuse hospitalizations increased by 83% after the
implementation of the TMH Services. However, when accounting for the number of contacts, the
number of psychiatric and substance abuse hospitalizations per contact were not different between
time periods. The percent with >1 day length of stay was lower after the implementation of the
TMH Services. After statistical adjustment for gender identity, those in the post-TMH Services
period tended to have lower anxiety levels. Those in the post-TMH Services period tended to have
Telemental Health Services
60
lower depression levels as seen on Figure 6 and evidenced by the Ordinal Logistic Regression
(p=.033) when the percentages of each depression level are compared between the pre and TMH
period. Those patients assessed for suicidality were 51% less in the post-TMH period and 57%
less likely to attempt suicide compared to the pre-period.
The strengths of the project
The benefit of this QI project lies in the increased communication among team members
that resulted in the implementation of TMH services. It was difficult at first to get the team
working cohesively together. It was imperative for the project leader to remain vulnerable to
assemble a formidable team. The first stage was developing a functional team which included
building rapport among the team members and understanding each person’s strengths and their
experiences with regards to TMH. There was agreement by the team to attend weekly
interprofessional team meetings via Zoom. Team members then volunteered to take other roles.
A constructivist approach of learning and teaching enabled the collaborative effort of the team of
mental health workers to facilitate development of this project. Everyone, both trainer/ trainees,
relied on their knowledge in providing mental health services. Weekly interprofessional zoom
meetings, a PowerPoint presentation, role playing, and Inservice-type lectures were utilized to
provide organized information for the team.
The team of clinicians, both fee-for-service and direct employees contributed data and
their various experiences in the implementation of the TMH service in this organization were
another strength. These professionals influenced collaborative team building by forming the
changes that directly affected them, implementing the changes, and identifying the impact of
the changes they contributed, both positive and negative. This design creates a sense of
ownership, professional empowerment, and commitment, impacting the buy-in and
Telemental Health Services
61
implementation of TMH. This collaborative effort by the stakeholders increased job
satisfaction, mutual respect for the value of each discipline, and patient outcomes.
Interpretation
According to the AMA, the adoption of TMH services allowed mental health providers to
increase continuity of care, extend access beyond normal clinic hours, reduce patient travel
burden, and help overcome clinician shortages in rural and underserved populations helping
practices to enhance patient wellness, improve efficiency, provide higher quality of care, and
increase patient satisfaction (AMA, 2020). This was evident in PH in the following ways: TMH
ensured that patients with mental health problems, like depression, anxiety, ADHD, bipolar
disorder, schizophrenia, and other serious mental health conditions, who were adjusting to new
medications, maintained contact with their mental health providers for their routine follow-ups. It
enabled patients to participate in their routine virtual psychotherapy appointments.
Transportation barriers were overcome when patients accessed care from the convenience of
their current location.
After these screening tools were applied for the implementation period of three months,
there was decreased inpatient hospitalization rates for mental health and SUD admissions by
stratifying patients’ level of mental health decompensation risks using the HRCP and increasing
access to mental health treatment. These high-risk protocol levels included low, moderate, and
high risks and treatment was implemented according to patients’ needs. The type of treatment
changes ranged from increased frequency of therapy, type of therapy, and medication
adjustments. Access to mental health treatment was measured by frequency of contact with
mental health providers and number of follow up appointments kept.
Telemental Health Services
62
There were decreases in the metrics of the assessment tools (GAD-7 scores and the PHQ9) in the TMH period compared to the in-person visits indicating improved mental health status.
The number of patients screened for suicidality decreased in the TMH period despite increase in
the overall patient census of TMH period indicating fewer changes in mental health status or
mental health stability. There were increases in the census of the active wellness and recovery
patients, the contacts of patients with their providers, and an increase in the number of
psychiatric and substance abuse hospitalizations (Emergency Department utilization) with
decreased inpatient hospitalization. When considering the billable therapies, there was 135%
increase in individual psychotherapy, a 37% increase in group therapy and medication assisted
treatments (MAT), indicated as the chemical dependency groups, and a 33% increase in
psychotropic medications treatment.
Implications: Impact of the project on people and systems
This project impacts both the patients and the providers, due to increased access of
patients to providers and substance abuse counselors. The flow of communication between the
two parties generated individualized treatment plans and aided in improvement of symptoms.
The implementation of this QI project has affected system-wide workflow within the
Wellness and Recovery program. There is better flow of communication between the employees.
At the weekly interprofessional team meetings, team members provide informed contributions on
specific subject areas regarding patients’ treatments, engagement in therapy, and potential
patients’ risks. In these meetings each of the mental health workers are prepared to discuss and
resolve pertinent patient issues regarding patients’ welfare. The office served as central hub for
patient scheduling and intake. Patients’ calls, complaints, and referrals were overseen by these
two personnel in the office.
Telemental Health Services
63
Initially, just the clerk and intake specialist were present in the office to maintain social
distancing mandates. Systematically, each of the NPPs followed by coming into the clinic once a
week to administer injections, since there was no Registered Nurse to administer injections.
Patients were scheduled such that there were only two to three patients in the waiting area at a
given time, per CDC mandates.
The paper scheduling was replaced by the Award EHR patient scheduling. Notes were
written at the times of scheduled appointments, or an outreach note was written for those patients
who could not be reached at their scheduled time. The clerk reviewed notes and rescheduled the
patients as indicated by the mental health workers. Mental health providers like the LMSW,
LMHC, CASAC, and NPPs were expected to work remotely using private work designated
phone numbers to contact patients.
Reasons for any differences between observed and anticipated outcomes, including the
influence of context
The outcome of this QI project anticipated a decrease in mental health and substance
abuse related hospitalization, which did not occur. There was an increase in number of patient
hospitalizations, with a decrease in length of stay from an average of 42 days to < 1 day. These
findings could be due to the psychological effects of the sudden surge of the COVID-19 Delta
and Omicron variants and more people using the ER for COVID testing. It could also be because
of the declining participation in the number of patients in the recovery participating in the
chemical dependency groups and/ or MAT during the last month of the data collection in the
TMH period despite 37% overall increase.
There are people who know relatives, friends, and neighbors impacted by the COVID-19
pandemic. The COVID-19 pandemic brought many changes like social isolation, financial
Telemental Health Services
64
pressures, life uncertainty, and changes in routines. These changes led to increased life stress,
anxiety, and depression which can worsen. Others may have chosen to self-medicate by
increasing alcohol and/ or other drugs to cope with their mental health symptoms exacerbated by
the pandemic.
While patients were actively participating in their Wellness individual therapy and
medication management sessions, the number was declining in the Recovery program. Although
there was 37% growth in group therapy and MAT participation in the TMH period, there was
increased hospitalization. There were systemic changes which may have affected the results of
the study period. The only clerk who schedules clients’ appointment with their mental health
providers was out on extended sick leave and a replacement was in training. There was a period
between the new clerk replacement and the job training where we may have lost some patients to
follow up. There were also new Recovery and Wellness program directors who were adjusting to
their positions. Two Licensed Social workers resigned about a month apart before and at the
onset of the TMH study period. It was noted during data analysis that some of the patients were
not reassigned during the data collection phase and may have been lost to follow up after they
separated from their therapist.
According to the substance abuse counselors at Project Hospitality, patients struggling
with substance abuse recovery need the physical and emotional support of their peers and
counselors. There was a 31% increase of TMH services within the same monitoring period
(November 14 into December 13) compared to pre TMH. There was also 9% decline compared
to the previous 30-day period. As a result, we may have lost some patients within the study
period, which resulted in increased hospitalization. However, among the hospitalizations
recorded in the TMH period, only 20% were in the hospital for more than 24-hours compared to
Telemental Health Services
65
46% in the in-person visits. There was 22% decline in inpatient hospitalizations (greater than 24
hours) in TMH period.
Table 6
Hospitalizations greater than 24 hours
Hospitalizations
September
October
November
Total
In-Person visits (pre-TMH)
10
7
5
22/Total # 48 46%
TMH
3
9
6
18/total #88
20%
Costs and strategic trade-offs, including opportunity costs
Telepsychiatry is the most developed, innovative model in expanding access to
psychiatric providers. According to the National Council for Behavioral Health, the expansion of
telepsychiatry has been disadvantaged by burdensome regulations and limits of reimbursement.
Deliberately increasing the number of mental health providers that are well trained in TMH
services may impact the number of long-term hospital readmissions. Utilization of TMH may
impact substance use disorder patients with co-occurring mental illness who use the Emergency
Departments for stabilization.
The lack of timely access to mental health services affects private clinics, group
practices, individual offices, emergency departments in hospitals and community behavioral
health centers. Insufficient symptom management and non-compliance with medication can
occur with extended times between appointments, leading to frequent Emergency Departments
visits and more hospitalizations. This gap in mental health and substance use disorder services
significantly delays treatment and reduces the quality of treatment, resulting in unacceptable
patient experiences in care, poor outcomes, and higher costs (National Council for Behavioral
Health, 2017). Limited timely access to mental health care further impacts the providers’ ability
Telemental Health Services
66
to address other social determinants of health such as housing, employment, education, and
family engagement.
The direct cost of mental health treatment is the cost of mental health hospitalizations and
outpatient treatments, including medication and counseling, for the captured population. Indirect
cost includes, income loss, disability, the cost of incarceration for prisoners with serious mental
health problems and frequent emergency room visits. The incremental cost would be the cost of
training new providers to telepsychiatry and providing TMH communication models to patients.
Opportunity cost in telepsychiatry/ TMH implementation is the number of people lost to follow
up due to lack of continuous access to mental health treatment, the reimbursement cost per
treatment for in-person visits, and increased workload for providers. These results in high cost
for mental health treatment compared to low cost to train physicians in TMH services.
Limitations to the generalizability of the work
There were limitations experienced during this QI project. TMH was initiated via an
emergency order, prior to the beginning of this project, and at present remains in place. After
project initiation, there was a change from the New York Office of Mental Health, as to how to
evaluate the treatment plans of patients. The project initially proposed patient assessment with
screening tools (GAD-7, PHQ-9, and C-SSRS) by the team of providers within the program
(mental health workers), every 30 days within the 90 days study period. This study period was
from September 15 through December 13, 2021. However, the guidelines later extended
evaluations for all Medicaid patients to be completed at intake, then every 6 months or as needed
depending on presentation of symptoms. A measure was in place at PH to monitor severity of
symptoms using the high-risk protocol criteria. This consequently affected the implementation of
Telemental Health Services
67
the screening tools utilized to assess patients’ mental health such that the entire patient
population in the census was not assessed every 30 days using the screening tools as planned.
Secondly, there was significant turnover of staffing that affected patient contact and
implementation of the assessment tools. For instance, the clinic has gone through three wellness
program directors and one recovery program director since 2020. There were two licensed social
workers who resigned, one clerk who was out on leave, and one credentialed Alcoholism and
Substance Abuse Counselor (CASAC) who resigned during the study phase of this QI project in
2021.
Another limitation is the study design of using the convenience sample of patients from
both the wellness and recovery programs. The results may not be an accurate representation of
all the patients in both programs. The results of the screening tools do not truly reflect the total
number of patients assessed. Given the fluid nature of psychiatry, the frequency of screening
may have varied. For example, some patients may have more screening than normal based on
acute changes in their clinical course.
There were variations in the service types seen in the contact logs between the in-person
office visits of 2019 and the TMH study period of 2021. Some services were either not provided
in 2019 or were documented in the wrong service type. An argument could be made that the
same may have occurred in of the TMH period.
Psyckes EHR serves only Medicaid insurance, and it has delayed data update. Hence, it
may not have captured recent hospital visits/ admissions at the end of the 90 days. The Healthix
EHR is not limited to Medicaid and captures hospitalization of all patients with various
insurances.
Telemental Health Services
68
Although some providers may struggle due to the inadequate quality of reception of some
patients’ phone lines, overall communication has been improved and made easier. Patients who
may have been non-compliant were reengaged and follow up more readily available from their
preferred location.
A difficult area for this organization was how to monitor patients’ compliance to their
treatment regimen. This would necessitate running the patients contact lapse report at least
monthly to enquire about patients who have missed more than two appointments with their
providers, therapy, and medication management sessions. There should be consideration to
discharge those patients who may not comply with medications and TMH visits. Also, there
should be recommendations for easy admissions and access for the discharged clients to return to
the program.
The study was not fully able to meet the exclusion criteria by including those with serious
mental illness who are not of sound mind and unable to make decision, and non-English
speaking. This is because the data pulled from the Awards EHR was clustered and was unable to
differentiate individual patients who received services and how many times the patient received
the services. However, there were no patients younger than 18 years old included in the study.
Within the period of TMH implementation, the (PH) program secured a grant for interpretive
services for Spanish speaking patients and for those patients (documented and undocumented)
without insurance. Hence, Spanish speaking patients were not excluded from this project.
Although there are benefits to TMH implementation, there are also challenges to
adopting this innovative technology. These barriers include inconsistent reimbursement,
concerns over security, privacy, and confidentiality, concerns of clinical duty to provide safe and
Telemental Health Services
69
effective care, and lack of evidence about impact on health care costs, utilization, or outcomes
(AMA, 2020).
The data obtained from the study design of a convenience sample of patients from the
Wellness and Recovery programs may not fully depict the characteristics of the entire
population. The psychiatric and substance abuse hospitalization report obtained from the Award
EHR clustered medical, psychiatric, and substance abuse hospitalizations. The investigator
removed all medically indicated hospitalizations. However, there are cases where neither
psychiatric nor substance abuse hospitalizations could be determined. Therefore, the data is
clustered and cannot be further separated for analysis. The investigator accessed the Psyckes and
Healthix to verify the reasons for these hospitalizations.
The data obtained from the assessment tools are not the cumulative number of patients
assessed, but the results (scores) of assessments and indicates the number of times the
assessments were administered. A patient could be screened more than once in each period,
depending on needs and risks of the patients. The same for contact logs where a given patient
could be provided different service types and/ or same service in each period of data collection.
Efforts made to minimize and adjust for limitations
A cohort of the high-risk patients would have been more appropriate than using the
convenient sample of patients. Investing in the educational training of personnel that document
patients’ hospitalizations in Award her, as indicated in the Psyckes and Healthix her, would
increase the accuracy of the hospitalization report. The billing team participated in the planning
and implementation phase of TMH to increase understanding of federal, state, and private payer
requirements and regulations. The billing team was also provided education regarding the
documentation that captures the billable and other service types.
Telemental Health Services
70
DNP Essentials Addressed
The implementation of this QI project utilized all the essentials of doctoral of advanced
nursing practice (DNP). DNP essential I was utilized when the investigator incorporated
innovative ideas to improve care delivery for patients with mental health and co-occurring
substance abuse disorder. Essential II was incorporated when the investigator evaluated effective
strategies to enhance patient care and enhance the workflow within the PH organization. The
DNP Essential III incorporates nursing theories and frameworks that allowed methodical data
gathering which were utilized to implement the QI project. Essential IV was applied to data
extraction from practice information systems (Awards) and databases (Psyckes and Healthix)
and Essential V is by analyzing TMH guidelines and advocating for equitable policies to enhance
patient outcomes.
There has been interprofessional collaboration throughout the implementation of this QI
project that satisfies the DNP Essential VI. There have been collaborations with LCSWs, MHCs,
substance abuse counselors, the leadership in PH, and consultation with a biostatistician.
Essential VII was incorporated with the implementation of this QI project which aimed to
increase patient access to mental health care and reduce hospitalization by utilizing the TMH
emergency guideline to increase patient assessment. Essential VIII was employed when the
investigator utilized analytical skills in evaluating the current TMH emergency guideline and
providing education to the team of mental health professionals at PH.
Conclusions
Plan for Sustainability
This QI project would be useful for monitoring patients in high-risk situations and track
patients who could be lost to follow up. The sustainability of this QI project is dependent on the
continuous insurance reimbursement related to New York emergency TMH guidelines.
Telemental Health Services
71
Additionally, mental health workers spent time in addition to their scheduled work hours to
address patient needs. The mental health workers spent a vast amount of time overseeing billable
and unbillable services. This project may be beneficial to patients, but it could increase the
workload and stress of mental health providers.
While assessing the utilization of emergency TMH services to increase patient access to
mental health assessments and counseling, and decreasing hospitalization, our findings suggests
that there is need to adopt this guideline into policy and a hybrid approach would better serve the
patients in the recovery program. The hybrid approach entails a combination of in-person office
visits (for group therapy) for peer support and TMH (individual counseling) therapy. Future
studies could investigate a control sample of patients who are seen in in-person visits compared
to those patients who are solely receiving TMH services in real time.
Plan for Dissemination
The results of this QI project were presented to the doctoral committee and the team of
mental health workers at Project Hospitality using a poster and PowerPoints. There is need for
more in-service type education for the staff of Project hospitality on proper documentation of
service types to capture billable services.
Furthermore, there is a need to maintain TMH services as an option for patients in New
York City at the end of the COVID-19 pandemic. A robust TMH legislation which offers equal
reimbursement for TMH, and in-person visits is necessary to capture the increasing number of
patients seeking mental health services in an industry plagued with personnel shortage. This
legislation should increase and standardize the reimbursement of TMH services by Medicaid and
managed care organizations. Payment should be as much as the in-person visits to entice those
cash-only providers to start accepting Medicaid and Managed Care insurances. There is need for
Telemental Health Services
72
support for the legislation and provision of funding to recruit and train more mental health
providers in the North Shore. This would help to ensure the sustainability of the TMH services.
A next step to obtaining legislation of TMH services will be accomplished by contacting the
Council member for the 49th District of the New York City Council. Finally, Telecommunication
aid needs to be provided and adopted as a medical necessity to fully adopt this innovation.
Telemental Health Services
APPENDIX A:
SITE APPROVAL/AUTHORIZATION LETTER
73
Telemental Health Services
74
Telemental Health Services
APPENDIX B:
CONSENT DOCUMENT (DISCLOSURE FORM, CONSENT FORM,
ETC.)
75
Telemental Health Services
76
Telemental Health Services
77
Telemental Health Services
APPENDIX C:
78
EVALUATION INSTRUMENTS (STUDENT CREATED DATA
COLLECTION TOOLS – SURVEYS, QUESTIONNAIRES, INTERVIEW
QUESTIONS, ETC.)
Telemental Health Services
79
Telemental Health Services
80
Telemental Health Services
APPENDIX D:
PROJECT TIMELINE
81
Telemental Health Services
APPENDIX E:
82
OTHER DOCUMENTS AS APPLICABLE TO THE PROJECT (SUCH AS
BUDGET)
Telemental Health Services
83
Telemental Health Services
84
References
Agency for Healthcare Research and Quality. (2020). Plan-Do-Study-Act (PDSA) directions and
examples.
https://www.ahrq.gov/health-literacy/improve/precaustions/tool2b.html
American Medical Association (2020). Telehealth Implementation Playbook.
https://www.ama-assn.org/system/files/2020-04/ama-telehealth-implementationplaybook.pdf
Bashshur, R. L., Shannon, G. W., Bashshur, N., & Yellowlees, P. M. (2016). The empirical
evidence for telemedicine interventions in mental disorders. Telemedicine Journal and EHealth, 22(2), 87-113.
https://doi.org/10.1089/tmj.2015.0206
Brearly, T., Goodman, C., Haynes, C., McDermott, K., & Rowland, J. (2020). Improvement of
post inpatient psychiatric follow-up for veterans using telehealth. American Journal of
Health-System Pharmacy, 77(4), 288-294.
https://doi.org/10.1093/ajhp/zxz314
Bjureberg, J., Dahlin, M., Carlborg, A., Edberg, H., Haglund, A., & Runeson, B. (2021).
Columbia-Suicide Severity Rating Scale Screen Version: Initial screening for suicide risk in
a psychiatric emergency department. Psychological Medicine, 1-9.
doi:10.1017/S0033291721000751
Cantor, J. H., McBain, R. K., Kofner, A., Ma, Stein, B. D., & Yu, H. (2021). Availability of
outpatient telemental health services in the United States at the outset of the COVID-19
pandemic. Medical care, 59(4), 319-323.
https://doi.org/10.1097/MLR.0000000000001512
Telemental Health Services
85
Center for Disease Control and Prevention. (2020). National Center for health statistics: Healthy
people 2020. U.S. Department of Health & Human Services.
https://www.cdc.gov/nchs/healthy_people/hp2020.htm
Chang, Hyejung. (2015). Evaluation Framework for Telemedicine Using the Logical Framework
Approach and a Fishbone Diagram. Healthcare Informatics Research. 21. 230. 10.4258/hi
Cohen Veterans Network & National Council for Behavioral Health. (2018). America’s mental
health 2018.
https://www.cohenveteransnetwork.org/wp-content/uploads/2018/10/Research-Summary10-10-2018.pdf
County Health Rankings & Roadmaps. (2020). Health Outcomes: Overall rank of Richmond
County, New York. A Robert Wood Johnson Foundation Program.
https://www.countyhealthrankings.org/app/newyork/2020/rankings/richmond/county/outco
mes/overall/snapshot
Cuomo, A. M., Sullivan, A. M. T., & Tavella, C., & Revised March 17, 2. (2020). Supplemental
guidance regarding use of telehealth for people served by OMH licensed or designated
programs affected by the disaster emergency. Albany, New York.
https://omh.ny.gov/omhweb/guidance/supplemental-guidance-use-of-telemental-healthdisaster-emergnecy.pdf
Cuomo, A. M., Sullivan, A. M. T., & Tavella, C., & Revised March 17, 2. (2020). Use of
Telephone and Two-way Video Technology by OMH-Licensed, Funded or Designated
Providers and Clients Affected by the COVID-19 Pandemic. Albany, New York.
https://ahihealth.org/wp-content/uploads/2020/04/NYS-OMH-Consolidated-COVID-19Telemental-Health-Guidance.pdf
Telemental Health Services
86
Dorin, A. (2021). Behavioral health advocates support Governor Cuomo’s legislation to
expand access to telehealth in mental health and substance use services for all New
Yorkers. The Coalition for Behavioral Health. New York, NY.
Eisman, A. B., Kilbourne, A., M., Dopp, A. R., Saldana, L., Eisenberg, D. (2020). Economic
evaluation in implementation science: Making the business case for implementation
strategies. Psychiatry Research, 283.
https://doi.org/10.1016/j.psychres.2019.06.008.
Federation of State Medical Boards. (2022). U.S. States and Territories Modifying Requirements
for Telehealth in Response to COVID-19: Out-of-states physicians; preexisting providerpatient relationships; audio-only requirements, etc.
fsmb.org/siteassets/advocacy/pdf/states-waving-licensure-requirements-for-telehealth-inresponse-to-covid-19.pdf
Guaiana, G., Mastrangelo, J., Hendrikx, S., & Barbui, C. (2021). A systematic review of the use
of telepsychiatry in depression. Community Mental Health Journal, 57(1), 93-100.
https://doi.org/10.1007/s10597-020-00724-2
Healthix. (2021). Ixchange Blog: Healthix VIPs deliver healthcare interoperability solutions.
https://healthix.org/ixchange/healthix-vips-deliver-healthcare-interoperability-solutions/
Haghnia, Y., Samad-Soltani, T., Yousefi, M., Sadr, H., & Rezaei-Hachesu, P. (2019).
Telepsychiatry- based care for the treatment follow-up of Iranian war veterans with posttraumatic stress disorder: A randomized controlled trial. Iranian Journal of Medical
Sciences, 44(4), 291-298.
https://doi.org/10.30476/IJMS.2019.44944
Telemental Health Services
87
Harrar, S. (2021). Inside America’s Psychiatrist Shortage: An in-depth look at why it’s
happening, how bad it is, where it’s worst, and how to find the mental health help you need,
despite it. PSYCOM.
https://www.psycom.net/inside-americas-psychiatrist-shortage
Hickey, J. V. & Brosnan, C. A. (2017). Evaluation of health care quality for DNPs (2nd ed.).
Springer Publishing Company.
Hinterland, K., Naidoo, M., King, L., Lewin, V., Myerson, G., Noumbissi, B., Woodward, M.,
Gould, L. H., Gwynn, R. C., Barbot, O., Bassett, MT. (2018). Community Health Profiles
2018, Staten Island Community District 1: St. George and Stapleton. NYC.gov k57(59):1-20.
https://www1.nyc.gov/assets/doh/downloads/pdf/data/2018chp-si1.pdf
Institute for Healthcare Improvement. (2021). Tools: Plan-Do-Study-Act (PDSA) worksheet.
http://www.ihi.org/resources/Pages/Tools/PlanDoStudyActWorksheet.asp
Levis, B., Benedetti, A., Thombs, B. D. (2019). Accuracy of patient health questionnaire-9
(PHQ-9) for screening to detect major depression: Individual participant data metaanalysis. Quebec, Canada. BMJ 2019; 365:1476
https://doi.org/10.1136/bmj.l1781
Liem, A., Sit, H. F., Arjadi, R., Patel, A. R., Elhai, J. D., & Hall, B. J. (2020). Ethical
standards for telemental health must be maintained during the COVID-19
pandemic. Asian journal of psychiatry, 53, 102218.
https://doi.org/10.1016/j.ajp.2020.102218
Mace, S., Boccanelli, A., & Dormond, M. (2018). The use of telehealth within behavioral health
settings: Utilization, opportunities, and challenges. Behavioral Health Workforce Research
Center, University of Michigan.
Telemental Health Services
88
https://behavioralhealthworkforce.org/wp-content/uploads/2018/05/Telehealth-FullPaper_5.17.18-clean.pdf
Mahmoud, H., Naal, H., & Cerda, S. (2021). Planning and implementing telepsychiatry in a
community mental health setting: A case study report. Springer Science and Business Media
LLC.
https://doi.org/10.1007/s10597-020-00709-1
Medicare Learning Network. (2021). Telehealth services. Centers for Medicare & Medicaid
Services.
https://www.cms.gov/Outreach-and-Education/Medicare-Learning-NetworkMLN/MLNProducts/Downloads/TelehealthSrvcsfctsht.pdf
Milstein, D. & Madden, S. L. (2017). Tackling Youth Substance Abuse on Staten Island: A
collective impact project. Harvard T. H. Chan School of Public Health.
http://sipcw.org/wp-content/uploads/2019/09/Harvard-University-Case-Study-TacklingYouth-Substance-Abuse-on-Staten-Island.pdf
Mossman, S. A., Luft, M. J., Schroeder, H. K., Varney, S. T., Fleck, D. E., Barzman, D. H.,
Gilman, R., DelBello, M. P., & Strawn, J. R. (2017). The Generalized Anxiety Disorder
7-item scale in adolescents with generalized anxiety disorder: Signal detection and
validation. Annals of clinical psychiatry: official journal of the American Academy of
Clinical Psychiatrists, 29(4), 227–234A.
National Council for Behavioral Health. (2017). The Psychiatric Shortage: Causes and solutions.
Washington, DC.
https://www.thenationalcouncil.org/wp-content/uploads/2017/03/PsychiatricShortage_National-Council-.pdf?daf=375ateTbd56
Telemental Health Services
89
New York Office of Mental Health. (2020). Telemental health services: Interim guidance for
telemental health during the COVID-19 outbreak.
https://omh.ny.gov/omhweb/clinic_restructuring/telepsychiatry.html
New York State Office of Mental Health (2021a, January 1). New York State Office of Mental
health 14 NYCRR Part 599: “Clinic Treatment Programs” interpretive/ implementation
guidance. https://omh.ny.gov/omhweb/clinic_restructuring/part599/part-599.pdf
New York State Office of Mental Health. (2021b, February 1). PSYCKES. New York.
https://omh.ny.gov/omhweb/psyckes_medicaid/psyckes_medicaid_release_notes_710_fe
b_2021.pdf
Olin, S. C., O'Connor, B. C., Storfer-Isser, A., Clark, L. J., Perkins, M., Hudson Scholle, S.,
Whitmyre, E. D., Hoagwood, K., & Horwitz, S. M. (2016). Access to Care for Youth in a
State Mental Health System: A Simulated Patient Approach. Journal of the American
Academy of Child and Adolescent Psychiatry, 55(5), 392–399.
https://doi.org/10.1016/j.jaac.2016.02.014
Perrin, A. & Turner, E. (2019). Smartphones help blacks, Hispanics bridge some-but not alldigital gaps with whites. Pew Research Center.
https://www.pewresearch.org/fact-tank/2019/08/20/smartphones-help-blacks-hispanicsbridge-some-but-not-all-digital-gaps-with-whites/
Project Hospitality. (n.d.). High Risk Criteria Protocol
Pumariega, A. J., Posner, K., Millsaps, U., Romig, B., Starvarski, D., Gehret, M. J., Riley, K.,
Wasser, T. E., Walsh, G., Yarger, H. (2020). Systematic Suicide Screening in a General
Hospital Setting: Process and initial Results. World Soc. Psychiatry 2020; 2:31-42.
https://www.worldsocpsychiatry.org/text.asp?2020/2/1/31/281135
Telemental Health Services
90
Richmond University Medical Center. (2019). Community health needs assessment &
community service plan for Richmond County (Staten Island).
https://www.rumcsi.org/wp-content/uploads/2020/09/2019-CHNA-Web-site-FinalReport-2019-10-30.pdf
Serenity at Summit (2022). Addiction statistics and treatment in Staten Island. Retrieved from
https://serenityatsummit.com/new-york/staten-island/
Spitzer, R. L., Kroenke, K., Williams, J. B. W., Lowe, B. (2006). A brief measure for assessing
generalized anxiety disorder: The GAD-7. Arch Intern Med. 166(10):1092–1097.
doi:10.1001/archinte.166.10.1092
Sylvia, M. L., & Terhaar, M. F. (2018). Clinical Analytics and Data Management for the
DNP (2nd ed.). Springer Publishing Company.
The Lancet Global Health. (2020). Mental health matters. Elsevier Ltd 8(11).
https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(20)304320/fulltext?rss=yes
United States Census Bureau. (2019). QuickFacts: Richmond County (Staten Island Borough),
New York.
https://www.census.gov/quickfacts/richmondcountystatenislandboroughnewyork
Warren, J. C. & Smalley, K. B. (2020). Using telehealth to meet mental health needs during the
COVID-19 crisis. The Commonwealth Fund.
https://www.commonwealthfund.org/blog/2020/using-telehealth-meet-mental-health-needsduring-covid-19-crisis
Telemental Health Services
91
Williams, C. (2021). Using the hub and spoke model of telemental health to expand the reach of
community-based care in the United States. Community Mental Health Journal, 57(1), 4956.
https://doi.org/10.1007/s10597-020-00675-8
Wood, J. M. & Gupta, S. (2017). Using rating scale in a clinical setting. Clinical Reviews.
https://cdn.mdedge.com/files/s3fs-public/Document/August-2017/CR02709028.PDF
Wood, P., Burwell, J., Rawlett, K. (2018). New Study Reveals Lack of Access as Root Cause
for mental Health Crisis in America. National Council for Behavioral Health.
https://www.thenationalcouncil.org/press-releases/new-study-reveals-lack-of-access-asroot-cause-for-mental-health-crisis-in-america/
As members of the DNP (Doctor of Nursing Practice) Project Committee, we certify that we
have read the DNP project prepared by Emmanuel Ahipue, titled "Telemental health: Increasing
access to mental health services and decreasing hospitalization" and recommend that it be
accepted as fulfilling the DNP project requirement for the Degree of Doctor of Nursing Practice.
[Faculty of record]
Peggy Z. Shipley PhD, RN
[Faculty Mentor]
�� j);,:Jrf, F/JF· G
�ember]
Date: 5'//S / Zt.fZ--z.
5/13/2022
Date: ____
_
Date:
Final approval and acceptance of this DNP project is contingent upon the candidate's submission
of the final copies of the DNP project to the Graduate College.
I hereby certify that I have read this DNP project prepared under my direction and recommend
that it be accepted as fulfilling the DNP project requirement.
DNP · oordinator, Chair, or Dean
[Graduate Nursing Department]
Date:
�/:w.
Telemental Health Services
1
TELEMENTAL HEALTH SERVICES: INCREASING ACCESS TO MENTAL HEALTH
SERVICES AND DECREASING HOSPITALIZATION
by
Emmanuel Ahipue
________________________
Copyright © Emmanuel Ahipue 2022
A DNP Project Submitted to the Faculty of the
COLLEGE OF NURSING
In Partial Fulfillment of the Requirements
For the Degree of
DOCTOR OF NURSING PRACTICE
BLOOMSBURG UNIVERSITY
2022
Telemental Health Services
2
BLOOMSBURG UNIVERSITY OF PENNSYLVANIA
As members of the DNP Project Committee, we certify that we have read the DNP project
prepared by Emmanuel Ahipue, titled Decreasing hospitalization readmissions and increasing
access to mental health services through increased assessment via telemental health in a
Wellness and Recovery clinic in Staten Island, Richmond County of New York City: A Quality
Improvement Project and recommend that it be accepted as fulfilling the DNP project
requirement for the Degree of Doctor of Nursing Practice.
_________________________________________________________________
Date: ____________
_________________________________________________________________
Date: ____________
_________________________________________________________________
Date: ____________
[Committee Chair Name]
[Committee Member Name]
[Committee Member Name]
Final approval and acceptance of this DNP project is contingent upon the candidate’s submission
of the final copies of the DNP project to the Graduate College.
I hereby certify that I have read this DNP project prepared under my direction and recommend
that it be accepted as fulfilling the DNP project requirement.
_________________________________________________________________
[Committee Chair Name]
DNP Project Committee Chair
[Academic Department]
Date: ____________
Telemental Health Services
3
ACKNOWLEDGMENTS
I wish first to acknowledge the grace of God for allowing me to complete this quality
improvement project. Despite the challenges in time management, travelling time to my clinical
site, and long working hours, you have continued to inspire and strengthen me to complete this
project. I am forever grateful and indebted to you.
I wish to acknowledge Dr. Shipley, my Clinical mentor. I have nothing but respect and
appreciation for your encouragement and prompt response to my questions. I also admire your
wealth of knowledge and constructive criticism during the phases of this QI project. You edited
my proposal for this project. I wish to acknowledge Dr. Jackson and Dr. Parke for their patience
and encouragement throughout the DNP courses. I would not have asked for better professors. I
wish to acknowledge G. Craig Wood, the biostatistical consultant, for processing the data.
I also wish to acknowledge Mr. Bello Muftau, my colleague at Project hospitality. As a
fellow psychiatric NP, I admire your insight and the experience in implementing the telemental
health at Project Hospitality. I would also like to acknowledge Mr. Gary Seigel for your
extraordinary leadership that you showed in bringing the team of mental health workers together
at Project Hospitality at the onset of the Corona virus pandemic. Your leadership set the stage for
the implementation of the telemental health in the facility. I can’ t help but acknowledge the
amazing staff of Project Hospitality. The implementation of this telemental health in our facility
has bonded us together and has brought out the best in all of us. I appreciate your dedication and
kindness to our patients. You are all amazing and have formed a formidable team in the Staten
Island District 49. I am immensely proud of the work we have done increasing access to mental
health in our community.
Telemental Health Services
4
DEDICATION
This project is dedicated to my wife, Ulonna, my sons, Justin and Joshua, and my daughter,
Alexandra. Thank you for your support, understanding and enduring love. You were all there at
every bump in the road and I appreciate you all endlessly. This project is also dedicated to my
mother, Justina Ahipue, and my father Paul Ahipue. Thank you, mom, for your prayers and
encouragement to keep moving beyond your fourth-grade education. Also, for babysitting the
grandchildren without complaining whenever I needed you. Dad, I know your spirit is with me
and happy with me in this accomplishment. Mom and Dad, you are both my rock.
Telemental Health Services
5
TABLE OF CONTENTS
LIST OF FIGURES .........................................................................................................................7
LIST OF TABLES ...........................................................................................................................8
ABSTRACT.....................................................................................................................................9
INTRODUCTION........................................................................................................................11
Background Knowledge/Significance.........................................................................................12
Local Problem ..............................................................................................................................14
Intended Improvement ................................................................................................................19
Project Purpose ................................................................................................................21
Project Question ...............................................................................................................22
Project Objectives ............................................................................................................22
Theoretical Framework ...............................................................................................................23
Literature Synthesis .....................................................................................................................26
Evidence Search ...............................................................................................................28
Comprehensive Appraisal of Evidence ..........................................................................29
Strengths of Evidence ......................................................................................................35
Weaknesses of Evidence ..................................................................................................36
Gaps and Limitations ......................................................................................................36
METHODS ...................................................................................................................................37
Project Design...............................................................................................................................37
Model for Implementation ..........................................................................................................39
Setting and Stakeholders .............................................................................................................41
Planning the Intervention ...........................................................................................................42
Participants and Recruitment.....................................................................................................45
Consent and Ethical Considerations ..........................................................................................45
Data Collection .............................................................................................................................46
Data Analysis ................................................................................................................................46
RESULTS .....................................................................................................................................48
Outcomes ......................................................................................................................................52
Telemental Health Services
6
TABLE OF CONTENTS - Continued
DISCUSSION ..............................................................................................................................59
Summary.......................................................................................................................................59
Interpretation ...............................................................................................................................61
Implications (Practice, Education, Research and Policy) ........................................................62
Limitations ....................................................................................................................................66
DNP Essentials Addressed .........................................................................................................69
Conclusions ...................................................................................................................................70
Plan for Sustainability .....................................................................................................70
Plan for Dissemination ....................................................................................................71
APPENDIX A:
SITE APPROVAL/AUTHORIZATION LETTER .........................................73
APPENDIX B:
CONSENT DOCUMENT (DISCLOSURE FORM, CONSENT FORM,
ETC.)................................................................................................................75
APPENDIX C:
EVALUATION INSTRUMENTS (STUDENT CREATED DATA
COLLECTION TOOLS – SURVEYS, QUESTIONNAIRES, INTERVIEW
QUESTIONS, ETC.) .......................................................................................78
APPENDIX D:
PROJECT TIMELINE .....................................................................................81
APPENDIX E:
OTHER DOCUMENTS AS APPLICABLE TO THE PROJECT (SUCH AS
BUDGET) ........................................................................................................82
REFERENCE…………………………………………………………………………………….84
Telemental Health Services
7
LIST OF FIGURES
Figure 1. Telemental health fishbone diagram. (Adapted from Chang (2015). No permission
needed to use and adapt……………………………………….……………………………………41
Figure 2. Screening tool implementation process………………………………………………49
Figure 3: Number of contacts by service type during the pre-TMH (in person office visits only,
September 15, 2019 – December 13, 2019) and post-TMH (Telemental Health, September 15,
2021 – December 13, 2021) ……………………………………………………………………54
Figure 4: Number in census report and number of substance abuse and psychiatric related
hospitalizations during the pre-period (in person visits only, September 15, 2019 – December 13,
2019) and post-period (TMH, September 15, 2021 – December 13, 2021) …………………….56
Figure 5. GAD-7 anxiety level before and after introduction of TMH services…………………57
Figure 6. PHQ-9 depression before and after the introduction of TMH services………………58
Telemental Health Services
8
LIST OF TABLES
Table 1. Definition of terms……………………………………………………………. 18
Table 2. The benefit of the Effectiveness-Efficiency-Equity conceptual framework….26
Table 3. High Risk Criteria Protocol……………………………………………………50
Table 4. Service types/ Contact logs…………………………………………………….54
Table 5. C-SSRS suicidality and after the introduction of TMH services………………58
Table 6. Hospitalizations greater than 24 hours……………………………………….65
Telemental Health Services
9
ABSTRACT
Purpose: Assess the ability of increased assessment via telemental health (TMH) to increase
access to care and decrease hospitalizations.
Background: There is mental health crisis in the U.S. and mental health services are insufficient
despite more than half of Americans (56%) seeking help. Patients seeking mental health
treatment have had to deal with the limited options in treatment and long waiting periods with
46% of Americans who have or know someone who has had to drive more than an hour
roundtrip to seek treatment, and 38% of Americans reporting that they have waited longer than
one week for mental health treatments.
Methods: This quality improvement (QI) project utilized retrospective chart audit at Project
Hospitality (PH) outpatient clinic to compare patients’ hospitalizations, readmissions, and access
to care between in-person pre-pandemic metrics with those occurring after institution of the New
York emergency TMH guideline initiated during the COVID-19 pandemic. Data was obtained
from PH’s electronic health record (EHR), Awards, evaluating these screening tools (Patient
Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder 7-item scale (GAD) and the
Columbia-Suicidal Severity Rating Scale (C-SSRS)). Data was obtained from interoperative
community EHR indicating where and when a patient obtained mental health treatment within
and/ or outside the program within the last 90 days.
Results: Availability of TMH through the emergency guideline has resulted in improved access
to mental health providers leading to improvement of symptoms, reduction in in-patient
hospitalization, and improved patient outcomes.
Telemental Health Services
10
Conclusion: The implementation of TMH has helped to improve patient access to mental health,
reduced loss to patient follow-up, and reduced in-patient hospitalization. Additional work is
needed to adopt this guideline into policy ensuring that patients have flexible access to care when
it is needed. Telecommunication aid, broadband, and internet access required and must be
adopted as medical necessity to fully adopt this innovation.
Telemental Health Services
11
INTRODUCTION
As the emergency TMH guidelines are entering into the second year, evaluation of the
effectiveness of TMH to increase patient access to their providers and decrease hospitalization at
PH is important. TMH use focuses on the second overarching goals of the Healthy People
2020 which aimed to achieve health equity, eliminate disparities, and improve the health of all
groups (Center for Disease Control and Prevention, 2020). There are several problems facing
the use of TMH/ telepsychiatry in non-emergent times. There is lack of nationwide policy
governing the provision of TMH care. Secondly, there is reimbursement disparity between
traditional in-person office visits and TMH services. Finally, there are common restrictions
placed on locations where providers and patients must be located to engage in TMH. This
project is currently operating under an emergency executive order, with month-to-month
extensions and revisions due to the COVID-19 pandemic. These revisions create uncertainty
among different organizations and the population being served.
At the onset of the COVID-19 pandemic, the request for TMH services, by both
providers and patients, increased significantly. A regulatory waiver was thereby issued to
maintain mental health services to vulnerable citizens throughout the duration of the declared
emergency. As a result, TMH was temporarily expanded for Medicaid-reimbursable services
to include telephonic and/ or video technology, including tools commonly available on smart
phones and other devices (Cuomo et al., 2020). Additional roles were included in this
emergency guidance as the definition of TMH practitioners were expanded to include any
professional, paraprofessional, or unlicensed behavioral health staff who deliver a qualified
service via TMH (Cuomo et al., 2020). TMH refers to “the use of technologies such as phone
or video calls between mental health professionals and other mental health professional,
Telemental Health Services
12
patients, service users, family members, or care givers to deliver mental health care” (Barnett
et al., 2021). Services includes delivering remote psychiatric evaluations, psychotherapy, and
medication management. The goal of this study is to use the findings to support the need for a
more permanent and effective TMH policy.
Background Knowledge/Significance
There is mental health crisis in America. Mental health services in the U.S. are
insufficient despite 56% of Americans seeking help (Wood et al., 2018). The attitudes of
Americans have improved towards receiving mental health services and 76 % of Americans are
now seeing mental health as important as physical health (Wood et al., 2018). This has further
increased the demand for mental health services which had already been shown to be
insufficient. Lack of access and long waiting periods have become the norm. Despite the high
demand for mental health services and societal awareness for mental health in the U.S., 74% of
Americans do not believe that these services are available for everyone and 47 % believe that the
options are limited (Wood et al., 2018). Medicaid-covered patients in publicly funded
community health centers experience the most extended delays in timely access (National
Council for Behavioral Health, 2017). The disparity in access to mental health care is
pronounced with patients who have low level income (annual household salary less than
$49,999), reside in under-resourced neighborhoods, and with insufficient insurance coverage
(Cohen Veterans Network & National Council for Behavioral Health, 2018). About 53% of
Americans who have sought mental health treatment were in low-income households (National
Council for Mental Wellbeing, 2018).
The largest proportions of those seeking mental health treatment are millennials (37%),
those in lower income households (53%), and those involved with the military either directly or
Telemental Health Services
13
through a secondary relationship (66%) (CVN & NCBH, 2018). Younger Americans, despite
having some of the more critical needs for mental health treatment, are unsure of their resources
and are more impacted by the stigma of mental health care (CVN & NCBH, 2018). The average
American is seeking mental health treatment and the need to receive care has grown (National
Council for Behavioral Health, 2017).
There are factors that prevent timely access to treatment for those who have established
mental health treatment. These factors include cost of mental health care for those with
insufficient insurance, social stigmas attached to mental health treatment, lack of access to
mental health facility or knowledge of where to find one, concern relating to quality of care, and
travelling associated with follow up appointments (Wood et al., 2018). Cost or poor insurance
coverage constitutes 42% of the barriers for Americans seeking mental health, 15% lack
information on where to start, 17% related to social stigma around seeking treatment, and 7%
ascribes to the quality of care as a barrier (CVN & NCBH, 2018). Patients are losing time while
waiting for treatment and traveling to their appointments. Patients seeking mental health
treatment have had to deal with the limited options in treatment and long waiting periods, with
46% of Americans who have or know someone who has had to drive more than an hour
roundtrip to seek treatment, and 38% of Americans reporting that they have waited longer than
one week for mental health treatments (Wood et al., 2018).
With expanded access under the Affordable Health Care reform, demand for mental
health care is high, adding to the problem of timely access to mental health treatment. The
National Council of Behavioral Health (2017) report indicates that the demand for psychiatry
will outstrip supply by 15,600 psychiatrists in 2025, based on estimates of retirement and new
entries into the workforce. Access is further diminished due to increasing numbers of private
Telemental Health Services
14
practice psychiatrists accepting cash or private insurance only (National Council for Behavioral
Health, 2017). There are factors affecting the shortage of psychiatrists in both the rural and urban
areas across in the country. The National Council for Behavioral Health Report (2017)
highlighted these reasons to include aging of the current workforce, low rates of reimbursement,
burnout, burdensome documentation requirements, and restrictive regulations around sharing
clinical information necessary to coordinate care.
Innovative techniques are needed to solve the problem of timely access to care (National
Council for Behavioral Health, 2017). This is necessary to capture the considerable number of
people suffering from a range of psychiatric disorders like schizophrenia, bipolar disorder, major
depression, anxiety, and other mood disorders, who needs mental health services. High quality
care can be maintained while patients are reached, regardless of the location of the patients. The
solutions to the shortage of psychiatry and timely access can only be achieved by a combination
of interrelated solutions and not exclusively on recruiting more psychiatrists or increasing the
payment or reimbursement rate (National Council for Behavioral Health, 2017). Telepsychiatry
is the most developed, innovative model in expanding access to psychiatric provider (National
Council for Behavioral Health, 2017).
Local Problem
PH has multiple programs that provide services to the community of the 49th district of
Staten Island, New York also known as the North shore of Staten Island. The served population
includes individuals with mental illness and some with co-occurring substance use disorder. The
challenges faced in providing services to this population includes keeping the patients engaged in
treatment long enough for recovery, reducing patients lost to follow up, and providing treatment
Telemental Health Services
15
for those who are uninsured or underinsured. Most of the patient population at PH have Medicaid
insurance.
This population often face health inequity due to mental illness and addiction. These
disparities can be related to unemployment, homelessness, and multiple health issues, including
addiction and mental illness. There are scenarios where there is complete disregard for an
established care or resources by the patients. Other instances, there is an issue of abandonment of
needed resources like shelter due to impulsive judgment and isolation from familial support.
These situations create ethical dilemmas for mental health workers when patients are not
engaging with treatment. Ethical dilemmas arise in areas related to continuing treatment when a
provider knows that the patient is still actively using drugs or whether to discharge a patient who
has disengaged from treatment by missing appointments yet knowing that he/ she needs the
medications for mental stability and functioning. Based on the nursing code of ethics, the
components of beneficence, justice, and nonmaleficence are challenged.
The South shore is the more affluent section of Staten Island and more economically
disadvantaged population with a more diverse population and larger percentage of people of
color is on the Island’s Manhattan-facing North shore (Milstein & Madden, 2017). Staten Island,
despite its relative affluence, had the highest all-cause mortality rate in New York City (Milstein
& Madden, 2017). Mortality here is driven by rates of cancer (lung cancer and breast cancer
among women), colorectal cancer, heart disease, and substance abuse respectively particularly in
the North shore area (Hinterland et al., 2018).
Staten Island has the highest rate of drug overdose deaths compared to the other New
York City boroughs (Serenity at Summit, 2022). Staten island was once the hot bed for the U.S.
opioid epidemic. Most hospitalizations for drug overdose in Staten Island are related to opioid
Telemental Health Services
16
use (Serenity at Summit, 2022). Based on Medicaid hospital admission data from 2016 to 2018,
there were 2,619 emergency department visits during this time attributed to opioid abuse and
1,851 of them where in people 18 to 44 years old (Serenity at Summit, 2022). Statistics indicate
that synthetic opioids accounted for 1,641 opioid-related overdose deaths in New York in 2016,
heroin caused 1,307 deaths, and 1,100 deaths were caused by prescription opioids (Serenity at
Summit, 2022). Out of the 306 overdose deaths in Staten Island from 2016 to 2018, 92 deaths
were in 2016, 108 deaths in 2017, and 105 deaths in 2018 (Serenity at Summit, 2022). Over 80%
of the fatal overdose in 2018 were seen in the Caucasian population and White males were more
likely to experience a mortal overdose (77%) (Serenity at Summit, 2022).
The rate of psychiatric hospitalizations in 2016 among adults in the North Shore is higher
than Staten Island and overall New York City (Richmond University Medical Center, 2019). The
North Shore of Staten Island in 2016, according to the Department of Health and Mental
Hygiene (DOHMH), had a rate of 963 psychiatric hospitalizations per 100, 000 compared to 707
Staten Island average, and the New York City average of 676 (The New York City Department
of health and Mental Hygiene, 2018). According to the County Health Rankings and Roadmaps
Richmond County of Staten Island’s premature death toll in 2020 was 5,600 compared to 5,500
with the top U.S. performers. Also, the ratio of mental health patients to mental health provider is
440:1, compared to 350:1 in New York City, and 290:1 with the top U.S. performers.
According to the DOHMH (2018), The New York State Department of Health, Statewide
Planning and Research Cooperative system, the rate of avoidable hospitalizations among adults
in the North Shore of Staten Island is higher than New York City rate in 2014. High psychiatric
hospitalization rates reflect the challenges residents in under-resourced neighborhoods face.
Telemental Health Services
17
These include difficulty accessing preventive services and early care, greater exposure to
stressors, and interruptions in health insurance coverage (DOHMH, 2018)
In New York City, the general wait times for psychiatry appointments were significantly
longer in the spring than in the summer (49.9 days versus 36.7 days) and the wait times of
therapy appointments were shorter in the community than the hospital clinics (19.1 days versus
35.3 days) (Olin et al., 2016). At the onset of the COVID-19 pandemic, when the clinic
adopted TMH services, there was a surge in mental health needs. Ways to access mental
health treatment in the North Shore community was initiated to meet the needs of the
community. As an Office of Mental Health (OMH) designated funding entity, there is annual
requirement to show effectiveness for increasing patient capacity within the clinic.
According to the New York State Office of Mental Health (OMH), there are clinic
services that must be available and offered as needed at any mental health clinic licensed by
OMH. These required services, as noted in Table 1, include assessments, therapies, injections,
and enhanced services (New York Office of Mental health, 2021a). These requisite services
serve as the template for allowable billing. Assessments include initial assessment and
psychiatric evaluation, therapies include individual psychotherapy, family/ collateral
psychotherapy, group psychotherapy, and psychotropic medication treatment. In-person clinic
visit serving adults covers injectable psychotropic medication administration and injectable
medication administration with education and monitoring. The enhanced services include crisis
intervention and monitoring.
Telemental Health Services
18
Table 1
Definition of term
Billable Services
1. Assessment
2. Therapies
3. Injections
4. Enhanced
services
Optional clinic services
1. Testing
2. Physical health
3. Injections
4. Enhanced
Services
Other Terms
1. Other Service
type
2. Outreach
Definition of terms
Initial assessment and psychiatric assessment
Individual psychotherapy, family/ collateral psychotherapy, group
psychotherapy, and psychotropic medication treatment.
Injectable psychotropic medication administration and injectable
psychotropic medication administration with education and
monitoring (both for clinics serving adults)
Crisis intervention and complex care management. A crisis
intervention refers to activities, including medication and verbal
therapy, which are designed to address acute distress and
associated behaviors when the individual’s condition requires
immediate attention. Complex care management is a clinical level
service which may be necessary as a follow up to psychotherapy,
psychotropic medication treatment or crisis service for the purpose
of preventing a change in community status or as a response to
complex conditions.
Description of services
Developmental testing, psychological testing, and psychiatric
consultation
Health physicals and health monitoring
Injectable psychotropic medication administration for clinics
serving children and injectable psychotropic administration with
education and monitoring for clinics serving children.
smoking cessation treatment for individual and group, and
Screening, Brief intervention, and referral to treatment (SBIRT).
Documentation of pertinent patient information necessary for
treatment that is not billable
Different attempts to contact a patient in connection with his
appointment or related matter.
3. Other definition
of terms
4. High-Risk
It involves weekly clinical team meetings to discuss any patient
Criteria protocol put on the HRCP pathway, calling patients who are missing
(HRSCP)
appointments and could potentially be loss to follow up, reporting
patients who are in Assisted Outpatient Treatment (AOT) who are
court ordered or on probation and not following up with treatment,
patients who have reported suicidal or homicidal thoughts within
the past six months, aggressive or violent patients, sexual
predators, and frequently hospitalized patients within the last six
months.
5. Contact log
A flowchart showing the service types in a period.
Telemental Health Services
6. Census
7. Number of the
census report
per contact
8. The number of
hospitalizations
per contact
19
The number of active patients in the wellness and recovery
program in a period.
This is the percentage of billable services received by each patient
in a period
The percentage of services received by hospitalized patients in a
period
To achieve the standard of the New York OMH, the program must find ways to
communicate with providers and patients to maintain quality care. A High-Risk Criteria protocol
(HRCP) developed by the PH became useful in monitoring patients. This HRCP involves weekly
clinical team meetings to discuss any patient put on the HRCP pathway, calling patients who are
missing appointments and could potentially be loss to follow up, reporting patients who are in
Assisted Outpatient Treatment (AOT) who are court ordered or on probation and not following
up with treatment, and patients who have reported suicidal or homicidal thoughts within the past
six months. Additionally, those patients who are frequently hospitalized within the past six
months, patients who are aggressive, violent, or with sexually inappropriate behaviors, patients
and anyone involved in domestic violence where the perpetrator continues to have access to the
victim are also included in the HRCP.
Intended Improvement
Project Hospitality (PH) is a non-for-profit outpatient Wellness and Recovery program
in Staten Island, New York. Telehealth/ TMH did not exist at the clinic before the Corona Virus
pandemic and was initiated in March 2020. The implementation was ongoing and there was lots
of learning and teaching as a team.
The PH program, an OMH funded entity, needed to show the effectiveness of TMH
services by increasing patients’ assessments. A quality improvement initiative was
Telemental Health Services
20
implemented which monitored patient outcomes by showing lack of deterioration to the point
of hospitalization for patients with co-occurring mental health and substance use disorder.
This was done using the self-assessment tools PHQ-9, the GAD-7, and C-SSRS. While CSSRS screens the suicide risks of the clients, the PHQ-9 assessment determined level of
depression of clients, the GAD-7 assessment was for anxiety levels. These three-assessment
metric are done during initial intake and every 6 months and based on patient symptom
changes. These assessments are completed in the scope of practice of licensed social workers
(LSWs) and mental health counselors (MHCs). These assessments are performed frequently on
patients based on their needs. The patient risk assessment tool was created to monitor patients
either at high risk of decompensation and/or suicide or those lost to follow up and are identified
by running the contact lapse report and then efforts are made to re-engage them in treatment.
Interventions to increase patient visits were initiated when the PHQ-9 score level was
ten and above or the GAD score is above 10 which indicates moderate to high anxiety. Also,
this is brought to the attention of the NPPs and other providers for a case conference and
modify medication management. The GAD-7 is associated with acceptable specificity and
sensitivity for detecting clinically significant anxiety symptoms (Mossman et al, 2018) like
the PHQ-9 which is sensitive and more specific for all ages from a cut off score of 10 or
above (Levis et al, 2019). The Columbia-Suicide Severity Rating Scale (C-SSRS) is a scale
designed to measure suicidal ideation severity and suicidal behaviors by capturing the pastmonth severity of suicidal ideation and measuring the past 3-month presence of actual and
aborted suicide attempts (Bjureberg et al., 2021). These screening tools PHQ-9, GAD-7, and
C-SSRS have been found to be valid and reliable in predicting level of depression, anxiety,
and adequate ability to detect suicidality in risk and behavior (Wood & Gupta, 2017).
Telemental Health Services
21
The hospitalization rate was checked monthly using the community based electronic
health records such as the PSYCKES and Healthix by the clinical supervisor. The current
emergency TMH guideline by the New York State Office of Mental Health (NYSOMH) utilized
verbal patient consent to provide services. Other verbal consents obtained included consent for
community based interoperable EHR, Psyckes, and Healthix. Psyckes is a HIPAA-compliant
web-based application designed to support clinical decision making, care coordination, and
quality improvement in New York State (NYSOMH, 2021b). While the Psyckes monitors
where and when a Medicaid patient receives mental health treatment, Healthix monitors all the
patients regardless of insurances and verbal consents are obtained from the patients to access
their information in these databases, allowing tracking of those patients who often use the
Emergency Department (ED) for care. The area of struggle for this organization is how to
monitor patients’ compliance to treatment regimen.
Project Purpose
The purpose for this DNP project was to evaluate the impact of TMH services in
increasing access to mental health services and reducing inpatient hospitalization because of
more frequent assessment usage. It was achieved by comparing records of in-person visits period
to the records of the emergency TMH guideline period.
Increasing access and preventing loss to patient follow up through TMH in outpatient
mental health wellness and recovery clinic was a new phenomenon in the PH program. The need
for access to mental health treatment grew throughout New York causing a shift to cater for the
mental health needs of the community at large. There were multiple systems needed to make the
system functional. Health care providers, insurers, patients, and others must work together to
ensure high quality, accessible treatment for all who need it (CVN & NCBH, 2018).
Telemental Health Services
22
Project Question
In an outpatient mental health wellness and recovery clinic in Staten Island New York,
does increased patient assessment via the implementation of TMH services lead to increased
access to mental health care and reduced hospitalization?
Project Objectives
The goal of this quality improvement project is to use findings to support the impact of
the New York emergency TMH guideline in increasing access to care and reducing inpatient
hospitalization.
The HRCP developed by the program was useful in evaluating patients’ level of risk
and assigning specialized treatment. Screening tools PHQ-9, GAD-7, and C-SSRS were used
to ascertain the severity of mental health symptoms. The use of these assessment tools would
be the primary source of data once a month or as needed based on patients’ symptom
presentation. The secondary source of data was obtained through the interoperable community
EHR like Psyckes and Healthix. These secondary sources of data indicated where and when a
client obtained care within and/ or outside the program within the last 90 days. These two
interoperable EHRs, Psyckes and Healthix, are community web-based applications in New York
state designed to obtain real-time patient data throughout the patients’ care used for care
coordination and quality improvement (Healthix, 2021; NYSOMH, 2021b).
It is important to note that the clinic initiated TMH under the emergency guideline in
April 2020 due to the extenuating circumstances posed by the COVID-19. There were
adjustments in the proposed implementation due to changes in New York State’s guidance,
which included loosening restrictions on duration of session for billing, allowing telephonic
services as audio only services, and protection of dual reimbursement where telehealth
Telemental Health Services
23
services not covered by Medicare but fully covered by Medicaid were allowed for
beneficiaries with both Medicare and Medicaid (Dorin, 2021). These guidelines allowed for
the provision of mental health services to continue despite health and safety concerns posed
by the COVID-19 pandemic.
Data obtained was compared between the office visit/ in-person office visit prepandemic metrics to those of the TMH. The outcome of TMH was evaluated by using the
assessment tools, organizational EHR (Awards) and interoperable community EHR.
Theoretical Framework
The project began with the goal of evaluating the efficiency, effectiveness, and experiences of
both the patients and the providers. An effective framework is required in knowing the impact of
the emergency TMH services in the clinic and how it impacts the patients, and their providers.
The evaluation approach utilized for this topic was the Effectiveness-Efficiency-Equity
conceptual framework. The 3 Es were important in how the plan came about. A framework
that reduces or eliminates disparity in mental health access as well as evaluate the usefulness of
the screening tools would be required explain the impact of the emergency TMH guideline. Each
component of this framework was examined and applied to the multiple facets implicit in
mental health services to determine the impact of TMH guideline on patients and how it
affected their access and outlook on mental health services.
Each of the three components in the Efficiency- Effectiveness- Equity conceptual
framework seeks to improve the provision of services. Sorato et al. (2020) define efficiency as “a
measure of the quality and/or quantity of output for a given level of input.” As applied to
healthcare and more specifically PH, it would refer to the reduction of wasted time between
seeing patients and providing services, particularly in the case of non-compliant patients who do
Telemental Health Services
24
not present for their appointments or those who are often rescheduled for their follow up
appointments. It also can refer to the length of time required between requesting and receiving an
appointment or travel time.
The emergency TMH guidelines allow for the provider to reach the patient over the
phone or through secure video services at the patients’ preferred location whereas the previous
guidelines dictated that patients had to be present at the physical clinic location to complete a
telehealth service. This perpetuated the challenges inherent with in-person care due to the need
for patients travel to a physical clinic setting where telehealth would take place. Due to the
shortage of psychiatrists and other mental health care providers, the emergency telehealth
guidelines eased the burden on providers allowing them to reach more patients with little
additional overhead cost. This eliminated the need for full-time clinic patient visits in the
traditional clinical setting.
Equity is defined by Sorato et al. (2020) as “the absence of avoidable or remediable
differences among groups of people whether those groups are defined socially, economically,
demographically, or geographically”. In the population involved in this study, there is a high
incidence of poverty and reliance on public assistance programs. Resources are limited and
negatively impact a person’s ability to attend healthcare appointments or to present to a clinic to
engage in telehealth as governed by the previous policy. Limitations include the lack of access to
a vehicle, travel costs, or access to affordable childcare.
On Staten Island, many providers in the South shore accept primarily private insurances
or cash payments, while the providers on the North shore more readily accept Medicare and
Medicaid. Access to telehealth can be afforded by out-of-pocket payment whereas those with
Medicare and Medicaid insurances must conform to the policy defined for telehealth for a
Telemental Health Services
25
session to be billable. Limitations in billable mental health services has created disparity in
access to telehealth based on insurance and affordability.
Effectiveness is the ability to produce desired results. The effectiveness of TMH can be
measured through indicators such as “waiting time to receive mental health treatment, the
availability of resources, the cost of access, availability of medical personnel, infrastructure and
equipment, and the cost of travel” (Sorato et al., 2020). Effectiveness also relies on the delivery
of care, improvement of standard screening scores such as PHQ-9, GAD and preventing hospital
admissions. The effectiveness of TMH services can be evident when patients are readily
connected with mental health providers instead of using the emergency department (ED) to
receive mental health services.
When effectiveness is paired with efficiency and equity, a health care system with little
waste and improved access can be established and meet patient care goals and achieve better
outcomes (Sylvia & Terhaar, 2018).
Telemental Health Services
26
Table 2
The benefit of the Effectiveness-Efficiency-Equity conceptual framework
Efficiency
Equity
A measure of quality and/
Absence of disparity
or quantity of output
1. Increase access to
1. Reduction of wasted
mental health services
time between seeing
by eliminating direct
patients and
patient cost.
provided services.
2. Reduces financial
2. Help manage
burden to patients as
patients’ time.
cost of access.
3. Ease the burden on
3. Eliminate the socioproviders by
economical,
reducing the impact
geographical, and
of shortage of mental
demographical impact
health workers.
of seeking mental
4. Eliminate overhead
health treatment
cost incurred in
clinic visits
Effectiveness
Ability to produce desired
results
1. Reduced waiting time
to receive mental
health treatment
2. Reduced cost of
access
3. Availability of mental
health workers,
infrastructure, and
equipment.
4. Eliminate cost of
travelling for
appointments.
Literature Synthesis
Mental health services in the U.S. are insufficient despite more than half of Americans
(56%) seeking help (National Council for Mental wellbeing, 2018). Physical limitations,
distance, and time are major obstacles to access mental health services (Haghnia et al., 2019).
Other limitations to receiving mental health services include inadequate specialty services,
maldistribution issues, and inadequate funding (Mace et al., 2018). These challenges have
plagued the behavioral health services and created the demand and need for an innovative
approach of providing care to individuals (Mace et al., 2018). Community-based agencies are
uniquely positioned to fill this gap in care; providing individualized care across settings,
coordinating services across agencies, and assisting individuals to transition at various stages of
Telemental Health Services
27
treatment (Williams, 2020). TMH can be used to further reduce the treatment gap and overcome
the challenges in accessing mental health services.
Reimbursement parity is one of the biggest barriers to TMH expansion (Warren &
Smalley, 2020). Lack of parity results from different rates for in-person office visits compared to
telehealth. In-person office visits are reimbursed fully for mental health services compared to
telehealth services. Currently, among the U.S. states and territories, there are twenty-three with
telehealth wavers, twenty-seven without wavers or that have closed wavers, and nineteen with
long-term or permanent interstate telemedicine including Guam, Commonwealth of the Northern
Mariana Islands, Puerto Rico, and the U.S. Virgin Islands (Federation of State Medical Boards,
2022). Due to the regulatory waiver, Executive Order 202 section 596 of Title 14, issued by the
New York state governor, the term TMH was temporarily expanded for Medicaid-reimbursable
services to include telephonic and/ or video, technology commonly available on smart phones
and other devices (Cuomo et al., 2020).
Less than 10% of the U.S. population used telehealth for a clinical encounter and only
18% of physicians provided such services prior to COVID-19 (Warren & Smalley, 2020).
Limited diagnosis, usually requiring physical comorbidities, would cover home-based telehealth
services (Warren & Smalley, 2020). For instance, radiological and critical care services were
reimbursable by telehealth when the patient or provider was at home. PH, a community-based
organization, did not apply for the TMH hub site license pre-pandemic despite eligibility due to
the difference in reimbursement fees. The hub site (provider location) also known as the distant
site is the consulting site where the provider is seeing the patient at a distance or consulting with
the patient’s provider, while the spoke site (patient location) also known as the originating site is
Telemental Health Services
28
where the patient or patient’s physician is located during the telehealth encounter (Medicare
Learning Network, 2021).
Smartphones are helping to bridge the disparity in access to treatment. While Black
Americans and Hispanics own smartphones at similar shares to whites, roughly 82% Whites
report owning a desktop or laptop computer compared to 58% of Black people and 57% of
Hispanics (Perrin & Turner, 2019). There are still concerns regarding disparities in access to
mental health services for those in rural, suburban, and urban communities. Technological
infrastructures like owning a desktop or laptop computer in these communities would ensure
improved access to TMH services.
Evidence Search
The evidence obtained for this review was from a computerized search of the library
databases. Health science databases reviewed include Cumulative Index to Nursing and Allied
Health literature (CINAHL), Centers for Disease Control and Prevention (CDC), PubMed,
Academic Search Premier (EBSCO), New York State Office of Mental Health (NYSOMH), and
the American Psychological Association (APA PsychInfo). The keywords used to retrieve
documents were “telepsychiatry,” “telemedicine,” “teleconferencing,” “telepsychology,”
“psychiatric clinics,” “access to mental health,” and “hospital readmission.” Alternate terms
include “telehealth,” “telemental health,” “community mental health centers,” “mental health
programs,” “walk in clinics,” “increased access to mental health,” “and decreasing
hospitalization readmission.” Limiters included last 10 years, English, primary research (clinical
study, clinical trial, clinical controlled trial, meta-analysis, randomized controlled trial). To
determine appropriateness, titles and abstracts of the results were screened. Out of the 240
articles resulted, ten studies were deemed appropriate via literature review and appraisal.
Telemental Health Services
29
Comprehensive Appraisal of Evidence
Knowledge of Telemental Health
Telemental health is the use of two-way interactive audio and video equipment used in
real time, to support clinical psychiatric care (New York Office of Mental Health, 2020). On July
3, 2019, the New York State Office of Mental Health (NYSOMH), changed the term
“telepsychiatry” to “telemental health” to demonstrate their broader use of technology and
expansion of “telemental health” services delivery (NYSOMH, 2020). This expansion enabled
additional providers: psychologists, social workers, mental health counselors, marriage and
family therapists, creative arts therapists, and psychoanalysts (NYSOMH, 2020) to deliver
psychiatric care remotely. To fully understand the need for TMH services, it is necessary to
understand the nature of mental disorders, costs, available resources, and treatment (Eisman et al,
2020)
Most mental disorders are a result of a combination of several “biopsychosocial” factors
rather than a single factor (Bashshur et al., 2016). According to the National Institute of Mental
Health (2021), mental illnesses include those that are diagnosable currently or within the past
year lasting a sufficient duration to meet diagnostic criteria within the Diagnostic and Statistical
manual of Mental Disorders (DSM-V) and excludes a developmental and or substance use
disorder. Age 14 is the beginning of 50 % of most lifetime mental illness and 75% have
manifested by age 24 (National Alliance on mental Illness (NAMI), 2021). Mental disorders
cause substantial distress or disruption in social, occupational, and/or family relationships, and
people with such disorders fail to meet important obligations in everyday life, work, or school
(Bashshur et al., 2016). Mental disorders are common, and it is difficult to determine the
prevalence of mental disorders in any population (Bashshur et al., 2016). Among U.S. adults,
Telemental Health Services
30
21% (52.9 million people) experienced mental illness in 2020, 5.6% (14.2 million people)
experienced serious mental illness, and 6.7% (17 million people) experienced a co-occurring
substance use disorder and mental illness in 2020 (NAMI, 2021). In 2016, there were 16.5% (7.7
million people) of U.S. youth aged 6 to 17 that experienced a mental health disorder (NAMI,
2021). In 2020, more than 12 million U.S adults had serious thoughts of suicide, 1 in 5
experienced mental illness, 1 in 20 experienced serious mental illness, and 1 in 15 experienced
substance use disorder and mental illness.
There are an estimated 12 million emergency department visits (1 out of every 8) by U.S.
adults related to mental illness and substance use disorders (NAMI, 2021). In 2020, 17 million
U.S. adults (32.1%) with mental illness experienced a concurrent substance use disorder (NAMI,
2021). Students aged 6 to 17 with mental, emotional, or behavioral concerns are three times more
likely to repeat a grade and high school students with significant symptoms of depression are
more than twice as likely to drop out of compared to their peers (NAMI 2021). The
unemployment rate among U.S. adults with mental illness is 6.4% compared to 5.1% of those
without (NAMI, 2021). There are at least 8.4 million people in the U.S. providing an average of
32 hours per week of unpaid care to adults with mental or emotional health issues (NAMI, 2021).
Among the people experiencing homelessness in the U.S., 20.8% have serious mental
illness and 37% of the adults incarcerated in the state and federal prison systems have a
diagnosed mental illness (NAMI, 2021). About 2 in 5 incarcerated people have mental illness
and an estimated 4,000 people with serious mental illness are held in solitary confinement in
U.S. prisons (NAMI, 2021). One in four people shot and killed in the U.S. by police officers
between 2015 and 2020 had mental health conditions (NAMI, 2021). Suicide is the second
leading cause of death among people aged 10 to 34 and the 10th leading cause of death in the
Telemental Health Services
31
U.S. (NAMI, 2021). Transgender adults are twelve times more likely to attempt suicide than the
general population (NAMI, 2021). Lesbian, gay, and bisexual youth are four times more likely to
attempt suicide than straight youth, and 78 % of people who die by suicide are male (NAMI,
2021).
Serious mental illness causes $193.2 billion in lost earnings in the U.S. economy each
year and mood disorders are the most common cause of hospitalization for all people in the U.S.
under the age 45 (NAMI, 2021). The global cost of depression and anxiety is $1 trillion in lost
productivity each year, and depression is the leading cause of disability worldwide (NAMI,
2022). The global cost of mental health disorders was estimated at $2.5 trillion per year in poor
health and reduced productivity in 2010 with a projected cost increase to $6 trillion by 2030 (The
Lancet Global health, 2020).
In 2020, 46.2% of U.S. adults with mental illness and 64.5% of those with serious mental
illness received treatment (NAMI, 2021). Among U.S. adults with mental illness, the annual rate
of treatment indicates that females (51.2%) received mental health services more than males
(37.4%); lesbian, gay and bisexual (54.3%) more than non-Hispanic White (51.8%), more than
non-Hispanic Black or African American (37.1%), and more than Hispanic or Latino (35.1%)
(NAMI, 2021).
The year 2020 was typified by uncertainty and loss brought about by the Coronavirus
(COVID-19) pandemic causing an impact on mental health of individuals around the world.
Manifest as Pronounced fear and anxiety about COVID-19, emotional distress resulting from
illness, bereavement, unemployment, income loss, and loneliness due to social isolation (The
Lancet Global health, 2020). Among adults in the U.S. in 2020, 1 in 5 report that the pandemic
had a significant impact on their mental health, 45% of those adults have mental health illness
Telemental Health Services
32
and 55% serious mental illness (NAMI, 2021). In 2020, among U.S. adults who received mental
health services, 17.7 million experienced delays or cancellation in appointments, 7.3 million
experienced delays in getting prescriptions and 4.9 million were unable to access needed care
(NAMI, 2021). Approximately 11% of adults with mental illness are without insurance coverage,
134 million people were living in a designated mental health professional shortage area, and 55%
of U.S. counties did not have a single practicing psychiatrist in 2020 (NAMI, 2021).
With the current shortage in the number of psychiatrists, and the predicted future shortage
of 15,600 to 16,060 by 2025, other mental health professionals are increasingly assuming
primary treatment roles (Harrar, 2021). This includes psychologists, nurse practitioners, mental
health counselors, and psychiatric social workers. As the number of inpatient psychiatric beds
decreases, the shortage of psychiatrists along with health insurance limitations of coverage,
determining which provider can be seen or the amount of help that can be received, has led to
long wait times for appointment, shorter doctor visits for psychiatric issues, less monitoring of
patients on prescriptions, and long wait times in the hospital (Harrar, 2021). The trends in mental
health treatment along with shortage of mental health personnel support the need for TMH
services.
Effects of telehealth health in increasing access to mental health services.
A case study report within a community mental health center in suburban Illinois
describes the planning, implementation, and evaluation of a telepsychiatry program (Mahmoud
et al., 2020). Identified telemental access barriers included clinician shortage, patient and staff
resistance, reimbursement, funding, technological challenges, infrastructure, patient privacy,
concerns about cyber security, and misinformation and misconception of telepsychiatry
(Mahmoud et al., 2020). A variety of access challenges were mitigated by defining clinician
Telemental Health Services
33
roles and developing clear workflow, increasing workers’ buy-in on telehealth by providing
training, assessing patients’ preferences as well as providing on-site alternatives, and enhancing
technology preparedness by adding a hotspot in case of internet failure (Mahmoud et al., 2020).
A double-blinded randomized control trial (RCT) examined the use of multimedia
applications (IMO voice calls, text messaging, Telegram, and Skype) for follow up treatment of
Iranian war veterans with PTSD (Haghnia et al., 2019). A significant positive relationship exists
between use and patient satisfaction. Additionally, there were benefits in reduced treatment costs
and completion rates.
Mace et al. (2018) conducted a mixed methods review of data collected from 329
behavioral health organizations to investigate the utilization patterns and barriers of telehealth
among behavioral health providers. The responding organizations represented all fifty states, the
District of Columbia and Puerto Rico. Half of the respondents (n=153; 47%) used telehealth
specifically for behavioral health services and the telehealth services most used by behavioral
health providers is videoconferencing (40%) followed by telephone (11%) (Mace et al., 2018).
Among the behavioral services respondents, psychiatrists are the largest group of users (78%) of
telehealth followed by mental health counselors (33%) (Mace et al., 2018).
The study also revealed the three most cited barriers to telehealth as; financing, including
the lack of reimbursement by insurers (59%), cost of implementing the infrastructure (56%), and
the cost of maintaining telehealth (56%) (Mace et al., 2018). The cost of teleconferencing
technology continues to decline because of innovations in the telecommunication media making
TMH services more cost effective (Guaiana et al, 2021). Other barriers highlighted by
respondents include the lack of leadership regarding telehealth, staffing challenges, client
challenges, and technical assistance barriers. The study also explored the primary reasons for
Telemental Health Services
34
implementing telehealth services. Study respondents agreed that telehealth is important to an
organization for reaching more patients, as it allows for the organizations to reach patients over a
geographical distance, and permits a more efficient workflow (Mace et al., 2018).
However, there are limitations to the generalizability of this study because of the sample
size. Even though there was representation from all the fifty states, there was unequal
representation in the responses to the electronic survey (i.e., NY 29.9%, PA 16.5% and six states/
area with only one response) (Mace et al., 208). Applicability of individual state-based findings
to other states may be challenging. Also, the sample for qualitative interviews was limited to
only five states compared to the RCT. (Mace et al., 2018).
An observational cross-sectional study by Cantor et al., 2021, reported that among the
8860-outpatient mental health treatment facilities in the United States, 3848 (43.4%) offered
telehealth services at the onset of the COVID-19 pandemic on April 16, 2020. This data was
from the National Mental health Services Survey (N-MHSS) of 2019 with response rate of 91%
(Cantor et al., 2021). The lack of telehealth access indicates that more needs to be done to
enhance telehealth policies and ensure continuous access to treatment even through another
pandemic. This study reveals limited telehealth infrastructure to meet the expected increase in
demand for mental health care during the pandemic. limitations of this study include
approximately 9.3% of the eligible facilities did not respond to the 2019 N-MHSS survey, the
behavioral treatment services locator did not contain treatment capacity or quality, and the data
collected did not reflect changes in telehealth services after the March 13, 2020, declared
national emergency (Cantor et al., 2021).
Effect of Telehealth in Reducing Hospitalization
Telemental Health Services
35
Brearly et al. (2020), investigated the implementation and outcome of a clinical quality
improvement project, Allied Transitional Telehealth Encounters post-inpatient Discharge
(ATTEND), which targeted veterans transitioning from inpatient to outpatient mental healthcare.
Here, Veterans, who met criteria, were provided 4G Android tablets with which they were
contacted via Clinical Video Telehealth (CVT) by a pharmacist to discuss their medications from
any location and at their convenience. Noted factors for a decrease in standard outpatient followup care were housing instability, transportation problems, distance from the clinic, forgetting to
attend appointments, driving related anxiety due to posttraumatic stress disorder (PTSD), poor
referral communication, and difficulty arranging time off work (Brearly et al., 2020).
Post inpatient mental health follow-up provided by the ATTEND clinic, helped veterans
to overcome barriers to care, increased patient satisfaction through prompt access, reduced the
likelihood of inpatient readmission, and lowered recidivism rate (Brearly et al., 2020). The
limitation of this implementation study model is the sample size of twenty patients. Although the
authors denied conflict of interest, this study was supported by the DVA Office of Academic
Affiliations Advanced Fellowship Program in Mental illness, Research, and Treatment and the
Mid-Atlantic Mental Illness Research, Education, and Clinical Center (Brearly et al., 2020).
Strengths of Evidence
The adoption of TMH services allows providers to increase continuity of care, extend
access beyond normal clinic hours, reduce patient travel burden, and help overcome clinician
shortages in rural and underserved populations which helps practices to enhance patient
wellness, improve efficiency, provide higher quality of care, and increase patient satisfaction
(American Medical Association [AMA], 2020). TMH has ensured that patients with mental
health problems, like depression, anxiety, attention-deficit/hyperactivity disorder (ADHD),
Telemental Health Services
36
bipolar disorder, schizophrenia, and other serious mental health conditions, who are adjusting to
new medications maintain contact with their mental health providers for their routine follow-ups.
It has enabled patients to participate in their routine virtual psychotherapy appointments.
Transportation barriers can be overcome when patients are able to access care
from their convenient locations.
Weakness of Evidence
Although there are benefits to TMH implementation, there are also challenges to
adopting this innovative technology. Some of the barriers include inconsistent reimbursement,
concerns over security, privacy, and confidentiality, concerns of providing safe and effective
care, and lack of evidence about impact on health care costs, utilization, or outcomes (AMA,
2020).
Gaps and Limitations
The resounding theme in these published studies discussed above is the correlation of
TMH with increased access to mental health services. There is increase in provider and patient
satisfaction while using TMH services in an industry that is overburdened by shortage of staff.
Though there is increased access of patients to their mental health providers for alleviation of
symptoms and preventing mental health crisis, there is little evidence discussed in the studies
about decreasing hospitalization. There is need for more current academic research studies to
evaluate the impact of TMH on reducing patients’ hospitalization readmission. There are not
enough studies that sufficiently examine the effect of telehealth on decreasing hospitalization.
There is the limitation to duplicability of studies using some of these studies because of the small
sample size.
Telemental Health Services
37
There are gaps in the use of TMH services. These studies have highlighted the
differences between states in their adoption of specific TMH policies. This makes the
duplicability of study difficult. A standardized national TMH policy would expand TMH access
at the end of the COVID-19 pandemic for regular use and maintain the infrastructures in place
should there be another pandemic. There is need to incentivize the training and recruitment of
mental health providers to help reduce the personnel shortage.
METHOD
This QI project aims to assess the ability of increased assessment via TMH to increase
access to care and decrease hospitalizations. This will be assessed by evaluating the impact of
increased monitoring and evaluation of anxiety, depression, and suicidality on hospitalization
rates and. The methodology section focused on project design, the model for implementation,
setting and stakeholders, planning the intervention, ethical consideration, data collection and
analysis of data collected.
Project design
Quality improvement in health care involves the analysis of performance in a system
and the development of a system designed to systematically improve care. It is a process of
systematic and continuous actions, which cause measurable improvements in the health status
of patient populations and health care services (Hickey & Brosnan, 2017). The goal here is to
utilize instruments to assess well-being and mental health status and their impact of health
care delivery and ensuring patient safety.
This QI project aims to assess the impact of increased assessment while using the current
New York State emergency TMH guidance. The current effects of TMH are being experienced
in real time while using the emergency guidance as a policy guide. However, the implementation
Telemental Health Services
38
of this QI project was designed via gap analysis to make course corrections by organizational
measures by looking at what was missing and determining ways to improve those faults. Areas
for correction include monitoring of patients’ remission of symptoms by frequently utilizing the
assessment tools, reducing patient loss to follow up by increasing contact with patients,
monitoring the effect of safety plan implementations by using the HRCP, and reducing inpatient
hospitalization.
This vulnerable population includes patients over the age of eighteen with mental illness,
some of whom have co-occurring substance use disorder. Ninety days retrospective data, for
three years prior to the roll out, was used as a comparison. The data in the retrospective timeline
is compared to the period of TMH implementation after the Institutional Review Board (IRB)
approval from September 15, 2021, until December 13, 2021.
This QI project utilizes a retrospective chart review of assessment tools. The degree of
change in mental health symptoms were measured by the pre and post interventions for both new
and established patients. Patients’ mental health interventions varied from patient to patient due
to the degree of their symptoms. In this QI project, interventions included easy access of patients
to their providers via telehealth, frequent therapy sessions based on assessed needs and/ or
medication adjustments determined by the HRCP.
The team reached a consensus to implement screening tools upon intake and at least once
a month during the implementation phase of data collection. The team includes the one clerk,
one intake specialist, four licensed Social Workers (LMSW) and two fee-for-service LMSW, two
Mental Health Counselors (MHC), two Nurse Practitioners in psychiatry (NPPs), a per diem
internal medicine physician, two substance abuse counselors, two billing personnel, the
Recovery Program director, and the Associate Area Director also filling in the role of Wellness
Telemental Health Services
39
Program director. The implementation timeline is from September 15, 2021, until December 15,
2021. The team discussed variables which could potentially impact the TMH services include
varying degree of serious mental illnesses, frequency of therapy already available, buy in of the
staff to assess patients by using the assessment tools, appointment kept by patients for their
therapy sessions and medication management sessions, and access to good working electronic
devices. We agreed that more frequent use of screening tools than once a month could potentially
skew the result of data collected.
Model for Implementation
The Plan-DO-Study-Act (PDSA) is a suitable implementation model to assess the stages
of change that is being implemented (AHRQ, 2020). Using the four-cycle model of testing
change/ improvement, the PDSA cycle develops a plan to evaluate the change (Plan), carrying
out the test (Do), observing and learning from the consequences (Study), and determining what
modifications should be made to the test (Act) (IHI, 2021).
We planned to use the mental health assessment tools PHQ-9, GAD, and C-SSRS to
assess for rate of symptoms of depression, anxiety, and suicidality, respectively. The rate of
patient hospitalization was assessed by checking the community based PSYCKES and Healthix
EHRs. To execute the plan, the PHQ-9, GAD, and C-SSRS were administered upon intake and at
least every 4 weeks for up to 90 days. After 90 days, gather data from the PSYCKES and
Healthix EHRs both pre and post intervention.
The assessment tools data were collected by the MHC and LMSW in the Wellness clinic
upon new patient intake and at least once every 90 days. The 90 days was the program policy for
completing a treatment plan for patients to accommodate non-Medicaid patients with
commercial insurances. Treatment planning is an ongoing process of assessing mental health
Telemental Health Services
40
status and needs of the patient, establishing his/ her treatment and rehabilitative goals, and
determining what services may be provided by the clinic to assist the patient in accomplishing
those goals (NYSOMH, 2021a). According the NYSOMH (2021a), the initial treatment plan
must be completed thirty calendar days from the admission date and reviewed annually after
admission or from the most recently completed treatment plan review unless the services are
covered by a third-party payer with a different requirement. Treatment plan is reviewed when
there are significant clinical changes like behavioral or medical diagnosis, changes in risk level,
increased or new symptoms, alterations in functioning, stressors, needs, circumstances, and
reviews may be required more frequently as determined by the treating clinician, patient, or
family member (NYSOMH, 2021a).
The implementation phase of this model is the “Do” phase. The plan was executed on
September 15, 2021. In this QI project, a majority of the TMH services provided were through
the telephone. This was the most readily available resource for the population under study. There
were cases in which ZOOM videoconferencing was utilized. There were other instances in which
patients received hybrid services of TMH and in-person visits.
Patients with worsening symptoms or high risk, as determined through use of the
assessment tools, received increased contact to their mental health providers. This increased
access continued until there was improvement as evidenced by low scores of the screening tools
or resolution of symptoms. There is increased number of new patient intakes who would be
screened using the assessment tools, both in-person visits and TMH.
The study model of the PDSA in the implementation of TMH can better be understood
using a fishbone diagram to capture the Effectiveness-Efficiency-Equity conceptual framework
discussed above.
Telemental Health Services
41
Figure 1
Telemental health fishbone diagram. (Adapted from Chang (2015). No permission needed to use
and adapt. Writer modified to focus on topic).
Human
Environment
System
Service Provider
Organization
Comfort w/ work flow
Leadership
Expertise on ICT
3rd party payers
Clinical information
systems
Education & Training
Work flow
re-engineering
Location / Travel time
Patient awareness
Disease characteristics
ICT skills
& knowledge
Reliability
of technology
Patient/ client
Social norms and values
(Stigma)
Training & support
OUTCOME
• Decreased
hospitalization
• Cost effective
• Increase access TO
___ health
Data quality
Medical liability
Transmission
Storage
ICT equipment
Interoperability
System Speed
Medical cost
(out-of-pocket)
User habit
Insurance fee schedule
ICT infrastructure
Budget
Resistance to change
Routine psychotherapy
opportunity, routine follow
ups with patient’s who are
adjusting to new medications
Reimbursement
Change
management
Comfort with patient
communication
Comfort with
Provider Interaction
Society
Organizational Culture
Privacy & security rules
• Enhance patient
wellness
• Provide higher
quality of care
Practice Certification
& license
Interface standards
Information Security
User Interface
Technology
Government authority
Rules /policies
To completely evaluate the “Act” phase of this implementation model, the
implementation data is compared to data obtained from the pre-pandemic, in-person visits period
of 2019. The total patient census at the given period would be compared. The frequency of
patient contact and assessment would be compared using service contact logs. Hospitalization
rate would be compared at both periods and length of hospitalization stays would be analyzed.
Settings and Stakeholders
The QI project was implemented in Project Hospitality (PH), an outpatient mental health
Wellness and Recovery Clinic in Staten Island, Richmond County of New York City. The larger
program consists of inpatient respite and long-term substance abuse program, housing programs,
shelters, case management and youth services. This outpatient Wellness and Recovery clinic is in
Telemental Health Services
42
a county with a population of 476,143 in 2019 (United States Census Bureau, 2019). About
59.6% of the county population were listed as White non-Hispanic, 11.6% as Black or African
American, 18.6% as Hispanic or Latino, and 10.9% as Asian alone (United States Census
Bureau, 2019).
The clinic was staffed with a clerk, an intake specialist, two billing personnel each for
recovery and wellness, two Licensed Mental Health Counselors (LMHC), four Licensed
Master Social workers (LMSW), two of them are fee-for-service, two Credentialed
Alcoholism and Substance Abuse counselors (CASAC), two Nurse practitioners in psychiatry
(NPPs), a per diem internal medicine physician, Program Director in wellness, and Program
Director in Recovery. This QI project was led by a nurse practitioner employed at the site as
partial fulfillment of a Doctor of Nursing Practice school project. The other NPP is the
clinical expert who helped in rolling out the TMH services within the organization. Common
stakeholders in healthcare improvement include, but not limited to patients, clinicians,
managers, executives, clinical assistants, and payers.
Planning the Intervention
The assessment tools used as intervention for the implementation of this QI project
include the PHQ-9, GAD, C-SSRS, and the community-based EHRs (Psyckes and Healthix). All
of these were used to assess patients’ status and frequency of hospitalization readmissions. After
a careful internet search, it was identified that the PHQ-9, GAD, and C-SSRS are all publicly
available and no written consent required (Wood & Gupta, 2017). The Psyckes and Healthix
required a written consent until the pandemic when a waiver was issued for obtaining verbal
consent, allowing for verbal consents to be entered into the PH’s EHR, Awards. Unlike the
Telemental Health Services
43
Psyckes, Healthix consent is traditionally obtained via verbal consent and entered electronically
by the intake specialist or the mental health workers.
The primary areas of knowing the effectiveness of the emergency TMH guideline is by
initiating a QI initiative to demonstrate effectiveness to support continued adoption of TMH
services during the pandemic and to provide support for a permanent adoption of the guideline
by showing effectiveness through reduction in recidivism, decreased inpatient hospitalization,
and improvement in patient outcomes. Recidivism rate is monitored using the High-Risk
Criteria Protocol (HRCP) adopted by the clinic and expanded from the New York State Office
of Mental Health Clinic Standards of care anchor (NYSOMH, 2021a). The “High Risk”
criteria include anyone that has had suicide attempt within the past six months, anyone
hospitalized for psychiatric reasons for the past six months, anyone who has been to the
emergency room due to psychiatric reasons within the past three months, anyone on AOT,
those experiencing suicidal and/ or homicidal ideation with a plan and deviating from their
normal baseline within the past three months, anyone that experienced an overdose within the
past six months, anyone reported to be engaged in violent behaviors or committed a violent
crime and has been released from prison within the past six months or is currently on parole
or probation for violent crime, anyone actively using drugs and/ or alcohol and exhibiting
behaviors perceived to be a potential danger to self or others, and anyone that is the victim of
a domestic violence situation where the perpetrator continues to have access to the victim
(Project Hospitality, n.d.).
The HRCP indicates that the clinical team should discuss and assess any patient put on
the high-risk list at the team meetings once a week. Patients on the high-risk list who missed
an appointment must be contacted via phone to ascertain the reason and rescheduled. Any
Telemental Health Services
44
patient reporting suicidal or homicidal ideation must go into the HRCP and CSSRS
administered at every session. If an Assisted Outpatient Treatment (AOT) patient, who is
court ordered to take medications, disappears, the legal system must be notified immediately,
and lack of attendance must be reported.
Improvement of symptoms was monitored using the assessment tools PHQ-9, GAD-7,
and C-SSRS. While the PHQ-9 assessment determines severity of depression of patients, the
GAD-7 tool is used to measure the level of general anxiety disorder, and the C-SSRS is a
systematic suicidal screening tool. This three-assessment metric was completed during initial
patient intake and every 30 days. Interventions to increase patient visits were initiated when
the PHQ-9 score level reached ten and above, the GAD score of ten or above, or moderate to
high suicidal ideation and behavior on the C-SSRS. The PHQ-9, GAD-7 and the C-SSRS are
valid and reliable in detecting depression, anxiety, and suicidal risks, respectively (Wood &
Gupta, 2017). The NPPs and other providers were notified by any mental health worker who
assessed the patient and identified these scores. Alerting the team to call for a case conference
of treating providers allows for modification of treatment plans in the form of medication
management adjustment and/ or increase in frequency of therapy.
Another way of tracking the effectiveness of increased access is by tracking
hospitalization through Psyckes and Healthix community based electronic health records
(EHRs). Psyckes is a HIPAA-compliant web-based application designed to support clinical
decision making, care coordination, and quality improvement in New York State (NYSOMH,
2021b). It is a community database that tracks patients’ care and shows the different programs
in which the patient receives care. Consents are sought from the patients to access their
Telemental Health Services
45
information in this database. Those patients who often use the Emergency Department (ER)
for care can successfully be tracked.
Participants and Recruitment
A convenience sample aged 18 years old and older, English speaking, with mental health
illness and/ or SUD attending PH were utilized. The exclusion criteria include patients who are
younger than 18 years old, those adult patients with serious mental health illness who are not of
sound mind, those unable to make decisions, and non-English speaking.
Consents and Ethical Considerations
The investigator requested and received permission from the Director of Policy and
Compliance of the Project Hospitality organization. There is no IRB approval requirement at the
organization. Also, the investigator obtained university IRB approval.
This QI project involves the use of the current emergency TMH guideline to improve
the mental health outcome of this population. Hence, there is no experimentation involved and
there is no increased potential for direct patient risk. The data needed for this QI project was
accessed or obtained through retrospective data collection from patients’ EHR and the
electronic databases (Awards). The ethical concerns involved in this project includes patient
safety, confidentiality, respect for persons, autonomy, and justice to the participants in this
project (Liem et al, 2020). These concerns are mitigated by using a flash drive and storage
space with a lock for locking up the devise in the office. A request for a password secured
laptop with Virtual Private Network (VPN) set up by the System Administrator was granted by
PH and provided for this QI project. Consents obtained in the physical chart or the PH’s EHR,
Awards, are confirmed by the investigator before accessing patient information in the Psyckes
and Healthix databases.
Telemental Health Services
46
Data Collection
Once the IRB was approved, the Psyckes and Healthix EHRs were reviewed at the onset
of data collection and at the end of the 90 days period. Team meetings were held weekly to
review the high-risk list, discuss any added information, review the treatment plan, and update
the list. The assessment tools were obtained from the organization’s EHR to assess depression,
anxiety, and suicidal scales using the PHQ-9, GAD-7, and C-SSRS, respectively. Data for PHQ9, GAD, and the C-SSRS were obtained every 4 weeks for a period of 90 days from PH’s EHR,
Awards.
Information obtained when accessing the EHR included diagnosis, medical diagnoses,
reasons for hospitalization, medical hospitalizations, gender, age, and the assessment tool scale.
Patients’ names were deidentified. Other data obtained from the high-risk protocol spreadsheet
included risk levels, proposed plans, and case review dates. The data obtained was reviewed by
the investigator to assess inclusion criteria, saved in flash drive, and locked at the program site in
the office where only the investigator could access it. Additionally, a password secured laptop
with the Virtual Private Network (VPN) was set up by the System Administrator of PH.
Data Analysis
The outcome of this QI project was analyzed by comparing the metrics of patients in
the control 90-day period in 2019 and those from the implementation period of 2021. To
measure access to mental health services, the census of active patients during in-person visits
pre-implementation, was compared to the number of active TMH visits during implementation
of 2021. Resolution of symptoms was determined by patients’ reports and measured by the
screening tools (PHQ-9, GAD-7, and C-SSRS) upon intake and every six months, patients
referred for the high-risk protocol. There was a change in the implementation of the screening
Telemental Health Services
47
tools because of the change in guidance proposed by the NYSOMH to implement treatment
plan upon intake and annually unless clinically necessary due to changes in mental health
symptoms, hospitalization and/ or deterioration of health. The change in guidance was
compensated by implementing the assessment tools upon intake, every six months and as
needed depending on patients’ presentations. Access was also measured in the overall number
of mental health service types provided within each 30-day period to indicate the form of
patient-provider interactions noted in the contact log report.
According to the NYSOMH (2021a), there are ten mental health clinic services that
must be available and offered at any mental health clinic licensed by the NYSOMH. The
required services fall under the following categories: assessments, therapies, injections, and
enhanced services (Table 1). While assessments involve interaction between a patient and a
mental health provider including initial assessment and psychiatric assessments, therapies
include individual psychotherapy, family/ collateral psychotherapy, group psychotherapy, and
psychotropic medication treatment (NYSOMH, 2021a). Injection services involve
administration of injectable psychotropic medication administration with education and
monitoring. Enhanced services include crisis intervention and complex care management
(NYSOMH, 2021a). There are eight optional services which include developmental testing,
psychological testing, psychiatric consultation, health physical, health monitoring, injectable
psychotropic medication administration for clinics serving children, injectable psychotropic
administration with education and monitoring for clinics serving children, and enhanced
services for smoking cessation treatment for individuals and group (NYSOMH, 2021a).
Patients placed on the HRCP were monitored frequently, at every patient contact with
their provider, weekly or biweekly, using the screening tools (PHQ-9, GAD-7, and C-SSRS)
Telemental Health Services
48
to monitor severity of symptoms, improvements, and viability of individualized treatment
plan. While the patients were under the HRCP, the average length of stay in the HRCP
pathway was measured along with revisions in treatment plan. Individualized treatment plans
were revised in the form of frequency for therapy/ medication management and/ or changes to
type of medication management.
The hospitalization rate was determined by the total number of psychiatric and
substance abuse hospitalizations within the 90 days in the control period of 2019 and that of
the study period of 2021. The length of hospital stay was evaluated in the control group and
study group and differentiated between inpatient hospitalization (> 24 hours stay) and ED
visits (< 24 hours stay). The reason for hospitalization was also evaluated to affirm
psychiatric and substance abuse related conditions for the purpose of this QI project.
RESULTS
The intervention was the ongoing implementation of telemental health and patient
assessment using the screening tools. This QI project used the convenience sample of patients
who are both in the Recovery and Wellness programs of the clinic and who are 18 years and
older. The PICOT questions whether increased patient access to mental health services and
increased assessment using the screening tools, PHQ-9, GAD-7, and C-SSRS via TMH enabled
decreased frequency of hospitalization for patients within the clinic. The data for the screening
tools were to be accessed monthly. The intake process in these programs involved the
assignment of risk levels based on screening in pre and post implementation as well as
implementation of resources which are specific for individualized patient care and based on these
assessments.
Telemental Health Services
49
During planning, there was a consensus by the team of providers to implement the
screening tools every four weeks for each patient as part of the NYSOMH requirement to update
treatment plans every six months. The program took it further to assess the treatment plans every
three months. Updates by the NYSOMH then changed requirement to perform screening using
assessment tools at intake and every 12 months, and with change in status. The team determined
to use tools at intake, every six months, and with change in status, and to continue with the
HRCP which was initiated before TMH.
Figure 2
Screening tool implementation process
Intake
Specialist
High Risk
Criteria
Protocol
OMH Guideline
(PHQ-9, GAD-7,
C-SSRS)
Within 30
days of
intake
Every 6
months
As needed
upon contact
(PHQ-9,
GAD-7, CSSRS)
1st Intake
2nd Intake
LMSW
LMSW
(LMSW,
(LMSW,
LMHC
LMHC
LMHC) Patients placed
on the high-risk criteria protocol have a GAD-7 score greater
than 10, or
LMHC)
PHQ-9 score greater than 10 and/ or have had suicidal behavior within previous three months,
Telemental Health Services
50
and/ or recent psychiatric hospitalization. Other criteria of suicidality, aggression, sexual
misconduct, victims of domestic violence, and assisted patient treatment for court mandated
patients are some examples of diagnostic categories of patients placed on the high-risk protocol.
The risk levels instituted included level 1 through level 4. While the low risk (level 1)
involved patients who wished to die without plan, intent, or suicidal behavior, Level 2 Moderate
risk patients are those with suicidal ideation more than a month ago without a plan, intent, or
suicidal behavior. For the low risk level 1 patients, the CSSRS was completed every 30 days and
the moderate risk level 2 patients had a CSSRS done biweekly.
Table 3
High Risk Criteria Protocol
Level 1
Low Risk
Patients who wished
to die without plan,
intent, or suicidal
behavior
Level 2
Moderate Risk
Patients with
suicidal ideation
more than a
month ago
without a plan,
intent, or
suicidal
behavior.
Level 3
Moderate Risk
Patients with
suicidal
behavior more
than 3 months
ago
Frequency of
CSSRS
Every 30 days
Biweekly
Disposition/
plan
Address
individualized patient
needs,
Case conferencing
among providers to
address symptoms,
weekly psychotherapy
Address
Individualized
patient needs,
Case
conferencing
among
providers to
address
Close
monitoring
with the CSSRS done
with every
patient contact.
Case
conferencing
among
providers.
Referral to
Health Homes
for home
checks, therapy
twice a week
Criteria
Level 4
High Risk
Patients with
suicidal ideation
with intent or
intent with a
plan in the past
month, and
recent
psychiatric
hospitalization.
CSSRS done on
every contact
Referred to the
NPP for
biweekly
assessment and
the patients’
family members
are involved as
collateral
protective factor.
Telemental Health Services
51
symptoms,
weekly therapy
The level 3 moderate risk involved those patients with suicidal behavior more than 3
months ago and they required close monitoring with the C-SSRS done with every patient
contact. Level 4 High risk involved patients with suicidal ideation with intent or intent with a
plan in the past month, and recent psychiatric hospitalization. The level 4 high risk patients had
their C-SSRS done on every contact, was referred to the NPP for biweekly assessment, and the
patients’ family members are involved as collateral protective factor. Systematic C-SSRS
screening while using the HRCP helped identify patients who were at risk for suicide in the
general clinic population and focused services on patients with the most need (Pumariega et al,
2020).
The initial plan was to obtain assessment data every 30 days, from the population sample,
during the intervention period. Data collected in the pre and post intervention period but did not
depict the overall sample but a sub-group of the sample. There were challenges encountered in
obtaining the necessary data for this QI project. The physical charts and Awards EHR were
reviewed for Psyckes and Healthix consents to enable the researcher entry into the interoperative
EHR to verify hospitalizations. While entering the hospitalizations from the Psyckes and
Healthix databases into Awards EMR, there was omission of pertinent mental health data such as
the reason for hospitalization or visiting the hospital emergency department by the personnel
entering the patients’ data. There were various categories of the C-SSRS assessment tool in
Awards and there are four separate places to obtain reports. These categories were listed as C-
Telemental Health Services
52
SSRS, C-SSRS, C-SSRS-Ongoing Weekly, and C-SSRS Intake/ Initial. There were no data in
the first two categories and minimal data in the C-SSRS- Ongoing Weekly. C-SSRS Intake/
Initial category contained more data than the other categories and sorting through the results was
confusing.
Outcome
TMH enabled PH to increased patient contact, mental health assessment, and decrease
hospital length of stay. The number on the overall patient census report increased by 21% after
the implementation of the TMH Services. However, when accounting for the number of patient
contacts with mental health workers, there was a 25% decrease in census per individual contacts
with providers after the implementation of the TMH Services. The number of patient contacts
with their providers increased by 61% after the implementation of the TMH Services. This
indicates that increased access to mental health services which allowed mental health providers
to be easily accessible by their patients as indicated by the service types provided at the TMH
period. The distribution of service types was different between (pre and post) time periods with
the largest increase within “Individual psychotherapy” (increased by >1000 contacts, an 8%
increase).
The number of psychiatric and substance abuse hospitalizations increased by 83% after the
implementation of the TMH Services. However, when accounting for the number of contacts, the
number of substance abuse hospitalizations per contact were not different between time periods.
The percent with >1 day length of stay was lower after the implementation of the TMH Services.
Although there were more hospital services needed at the implementation of the TMH, only 20%
of the total hospitalization were inpatient compared 46% during in-person visits.
Telemental Health Services
53
After statistical adjustment for gender identity, those in the post-TMH Services period
tended to have lower anxiety levels. Those in the post-TNH Services period tended to have lower
depression levels. It should be noted that while the number of patient contacts with wellness
(service) for mental health recovery increased after the implementation of TMH services, the
number of patient contacts with recovery (service) for patients with chemical dependency
decreased by 17% in the last month of data collection. The decreased contact by patients enrolled
in recovery could indicate that they may have dropped out of the program. There were twice the
number of suicidal responses following C-SSRS assessment in-person compared to the TMH
period. This increased number of responses in the in-person period could have several causes:
there were either increased patient intake (admission) into PH within the control period, or there
were patients placed on the HRCP within the same period, or there were just increased assessments
to update the treatment plan.
Contextual Elements
The independent variables in this QI project included the TMH services provided and the
frequency of assessments which were used to determine increased mental health access and
reduced inpatient hospitalization. Appointments kept by patients for their therapy sessions and
medication management sessions are other independent variables. Hospitalization is dependent on
TMH and assessment tools. Other dependent variables include buy-in of staff to assess patients
using assessment tools and access to good working electronic devices.
Contact Log
Contact logs were reviewed to describe the number of contacts by service type during the
period before and at the implementation of the TMH services. The table below includes a
summary of overall counts and percentages of total visits for the period under investigation. The
Telemental Health Services
54
number of contacts increased by 66% after implementation of the TMH services (calculated as
(4086-2532)/2532). The distribution of service types was different between time periods (chisquare p-value <0.0001), with the largest increase within “Therapies” (See Table 2, increased by
>1000 contacts, an 80% increase).
Table 4
Service types/ Contact logs
Required Services (Billable services)
Assessment
Therapies
Injections
Enhanced services
Optional Services (Non-billable)
Outreach (Non-billable)
In person OV
N=2466
37
1.5%
1707
69.2%
41
1.7%
189
7.7%
10
0.4%
482
19.5%
Telemental Health
N=4094
14
0.3%
3072
75.0%
31
0.8%
421
10.3%
13
0.3%
543
13.3%
Figure 3
Number of contacts by service type during the pre-period (in person office visits only, September
15, 2019 – December 13, 2019) and post-period (Telemental Health, September 15, 2021 –
December 13, 2021)
Telemental Health Services
55
3500
3072
In person OV
Telemental Health
Number of contacts
3000
2500
2000
1707
1500
1000
0
482 543
421
500
41 31
37 14
Billable
services:
Assessment
Billable
services:
Therapies
Billable
services:
Injections
187
Billable
services:
Enhanced
services
10 13
Optional non- Non-billable
billable
Outreach
services
Census
There were total of 280 on the census report in the period prior to introduction of TMH
services. This increased by 21% (up to 340 patients) in the period after introduction of TMH
services (See Figure 4).
However, the number of the census report per contact was 25% lower after introduction of
TMH services periods (Pre: 318 of 2532, 12.6%, Post: 374 of 4086, 9.2%; chi-square p-value
<0.0001). It implies the interaction of patient with their mental health providers while accessing
mental health treatment via TMH services is replicable and highly statistically significant.
Substance abuse and psychiatric related hospitalizations
There were a total of 48 substance abuse and psychiatric hospitalizations in the period prior
to introduction of TMH services. This increased by 83% (up to 88 cases) in the period after
introduction of TMH Services (See figure 4). Although this was a large increase, the number of
Telemental Health Services
56
substance abuse and psychiatric hospitalizations per contact was not statistically significant
between time periods (Pre: 48 of 2532, 1.9%, Post: 88 of 4086, 2.2%; chi-square p-value = 0.472).
Finally, the percent of psychiatric and substance abuse hospitalizations that required >1
day length of stay was lower after introduction of TMH Services (Pre: 45.8%, Post: 26.1%; chisquare p-value = 0.020). This implies that increased assessment in TMH period is effective because
of the decrease in prolonged inpatient hospitalization. It could also imply that the increased contact
of SUD patients who are receiving MAT kept them current with treatment and reduced
hospitalization.
Figure 4
Number in census report and number of substance abuse and psychiatric related hospitalizations
during the pre-period (in person visits only, September 15, 2019 – December 13, 2019) and post-
Number of patients
period (TMH, September 15, 2021 – December 13, 2021)
400
In person OV
Telemental Health
300
200
100
0
Census report
Substance abuse and psychiatric
hospitalizations
GAD7
Scores of GAD-7 range from zero to 21. The cut off points of five, ten, and 15 represent
mild, moderate, and severe levels of anxiety respectively, on the GAD-7 scoring scale (Spitzer et
al, 2006). The above categories were used to compare the percentage within each anxiety level
Telemental Health Services
57
between the pre-TMH period and TMH period using Ordinal Logistic Regression. Although there
was a slight trend towards lower anxiety in the post-period, the differences were not significant
(p=0.191).
However, there were differences in characteristics between the pre and TMH periods,
including gender identity (35% female in pre-period and 73% female in the post period). When
adjusting for gender identity within ordinal logistic regression, those female and male patients in
the post-TMH Services period tended to have lower anxiety levels (p=0.030) compared to those in
the in-person visits.
Figure 5
GAD-7 anxiety level before and after introduction of TMH services
GAD-7 anxiety level before and after introduction of
Telemental Health Services
100%
Severe, 13%
80%
Severe, 35%
60%
Moderate, 10%
40%
Mild, 35%
20%
0%
Moderate, 17%
Mild, 39%
None, 20%
Female
In person OV
Severe, 16%
Severe, 12%
Moderate, 6%
Moderate, 19%
Mild, 18%
Mild, 22%
None, 65%
None, 30%
Female
Telemental Health
None, 43%
Male
In person OV
Male
Telemental Health
Overall p-value comparing In person OV versus Telemental Health = 0.030
PHQ9
The analysis for differences in PHQ-9 used the established cutoff of <5 (no depression),
5-9 mild, 10-14 moderate, 15-19 moderately severe, and 20-27 severe. These categories were
used to compare the percentage within each depression level between the pre-TMH period and
Telemental Health Services
58
TMH period using Ordinal Logistic Regression. There was a significant trend towards lower
depression level in the post-period (p=.033).
Figure 6
PHQ-9 depression before and after the introduction of TMH services
100%
PHQ-9 depressionlevel before and after
introduction of Telemental Health Services
Any Severe, 13%
80%
Any Severe, 35%
60%
Moderate, 15%
40%
Mild, 24%
20%
0%
Moderate, 23%
Mild, 30%
None, 34%
None, 27%
In person OV
p-value = 0.033
Telemental Health
Note: Ordinal logistic regression adjusted for other factors (e.g., gender identity) did not change the conclusion
C-SSRS
The responses of individual items from the C-SSRS were compared between in-person
OV and TMH using Chi-square tests or Fisher’s exact test (when expected counts were <5). In
general, the rates of suicidal responses were higher during in-person visits, but these differences
were not significant. However, the small number of responses may lead to unstable or unreliable
results. Caution should be used when interpreting these results.
Table 5
C-SSRS suicidality before and after the introduction of TMH services
C-SSRS
In-person OV
Telemental health
p-value
Telemental Health Services
Wish to be dead
Suicidal thoughts (General
non-specific thoughts)
Suicidal thoughts with
method (w/o Specific plan or
intent)
Suicidal intent (without
specific plan)
Suicidal intent (with specific
plan)
Suicidal Behavior
Suicidal attempt during
lifetime
59
23%
14%
4
2
N=44
9%
5%
6
7%
0
0%
0.1771
6
7%
1
2%
0.4251
4
4%
0
0%
0.3021
4
30
4%
33%
1
13
2%
30%
0.9991
0.6592
21
13
N= 90
0.0591
0.1431
Note: ¹Fisher’s exact test, ²Chi-square test
DISCUSSION
Summary
During utilization of the emergency TMH services the number of contacts increased by
61%. The distribution of service types was different between the in-person visits and TMH with
the largest increase being within “Therapies” (increased by >1000 contacts, an 80% increase as
displayed in Table 4). The number on the census report increased by 21% after the implementation
of the TMH Services. However, when accounting for the number of contacts, there was a 25%
decrease in census per contact after the implementation of the TMH Services. Those patients who
needed increased contacts with their provider received it.
The number of psychiatric and substance abuse hospitalizations increased by 83% after the
implementation of the TMH Services. However, when accounting for the number of contacts, the
number of psychiatric and substance abuse hospitalizations per contact were not different between
time periods. The percent with >1 day length of stay was lower after the implementation of the
TMH Services. After statistical adjustment for gender identity, those in the post-TMH Services
period tended to have lower anxiety levels. Those in the post-TMH Services period tended to have
Telemental Health Services
60
lower depression levels as seen on Figure 6 and evidenced by the Ordinal Logistic Regression
(p=.033) when the percentages of each depression level are compared between the pre and TMH
period. Those patients assessed for suicidality were 51% less in the post-TMH period and 57%
less likely to attempt suicide compared to the pre-period.
The strengths of the project
The benefit of this QI project lies in the increased communication among team members
that resulted in the implementation of TMH services. It was difficult at first to get the team
working cohesively together. It was imperative for the project leader to remain vulnerable to
assemble a formidable team. The first stage was developing a functional team which included
building rapport among the team members and understanding each person’s strengths and their
experiences with regards to TMH. There was agreement by the team to attend weekly
interprofessional team meetings via Zoom. Team members then volunteered to take other roles.
A constructivist approach of learning and teaching enabled the collaborative effort of the team of
mental health workers to facilitate development of this project. Everyone, both trainer/ trainees,
relied on their knowledge in providing mental health services. Weekly interprofessional zoom
meetings, a PowerPoint presentation, role playing, and Inservice-type lectures were utilized to
provide organized information for the team.
The team of clinicians, both fee-for-service and direct employees contributed data and
their various experiences in the implementation of the TMH service in this organization were
another strength. These professionals influenced collaborative team building by forming the
changes that directly affected them, implementing the changes, and identifying the impact of
the changes they contributed, both positive and negative. This design creates a sense of
ownership, professional empowerment, and commitment, impacting the buy-in and
Telemental Health Services
61
implementation of TMH. This collaborative effort by the stakeholders increased job
satisfaction, mutual respect for the value of each discipline, and patient outcomes.
Interpretation
According to the AMA, the adoption of TMH services allowed mental health providers to
increase continuity of care, extend access beyond normal clinic hours, reduce patient travel
burden, and help overcome clinician shortages in rural and underserved populations helping
practices to enhance patient wellness, improve efficiency, provide higher quality of care, and
increase patient satisfaction (AMA, 2020). This was evident in PH in the following ways: TMH
ensured that patients with mental health problems, like depression, anxiety, ADHD, bipolar
disorder, schizophrenia, and other serious mental health conditions, who were adjusting to new
medications, maintained contact with their mental health providers for their routine follow-ups. It
enabled patients to participate in their routine virtual psychotherapy appointments.
Transportation barriers were overcome when patients accessed care from the convenience of
their current location.
After these screening tools were applied for the implementation period of three months,
there was decreased inpatient hospitalization rates for mental health and SUD admissions by
stratifying patients’ level of mental health decompensation risks using the HRCP and increasing
access to mental health treatment. These high-risk protocol levels included low, moderate, and
high risks and treatment was implemented according to patients’ needs. The type of treatment
changes ranged from increased frequency of therapy, type of therapy, and medication
adjustments. Access to mental health treatment was measured by frequency of contact with
mental health providers and number of follow up appointments kept.
Telemental Health Services
62
There were decreases in the metrics of the assessment tools (GAD-7 scores and the PHQ9) in the TMH period compared to the in-person visits indicating improved mental health status.
The number of patients screened for suicidality decreased in the TMH period despite increase in
the overall patient census of TMH period indicating fewer changes in mental health status or
mental health stability. There were increases in the census of the active wellness and recovery
patients, the contacts of patients with their providers, and an increase in the number of
psychiatric and substance abuse hospitalizations (Emergency Department utilization) with
decreased inpatient hospitalization. When considering the billable therapies, there was 135%
increase in individual psychotherapy, a 37% increase in group therapy and medication assisted
treatments (MAT), indicated as the chemical dependency groups, and a 33% increase in
psychotropic medications treatment.
Implications: Impact of the project on people and systems
This project impacts both the patients and the providers, due to increased access of
patients to providers and substance abuse counselors. The flow of communication between the
two parties generated individualized treatment plans and aided in improvement of symptoms.
The implementation of this QI project has affected system-wide workflow within the
Wellness and Recovery program. There is better flow of communication between the employees.
At the weekly interprofessional team meetings, team members provide informed contributions on
specific subject areas regarding patients’ treatments, engagement in therapy, and potential
patients’ risks. In these meetings each of the mental health workers are prepared to discuss and
resolve pertinent patient issues regarding patients’ welfare. The office served as central hub for
patient scheduling and intake. Patients’ calls, complaints, and referrals were overseen by these
two personnel in the office.
Telemental Health Services
63
Initially, just the clerk and intake specialist were present in the office to maintain social
distancing mandates. Systematically, each of the NPPs followed by coming into the clinic once a
week to administer injections, since there was no Registered Nurse to administer injections.
Patients were scheduled such that there were only two to three patients in the waiting area at a
given time, per CDC mandates.
The paper scheduling was replaced by the Award EHR patient scheduling. Notes were
written at the times of scheduled appointments, or an outreach note was written for those patients
who could not be reached at their scheduled time. The clerk reviewed notes and rescheduled the
patients as indicated by the mental health workers. Mental health providers like the LMSW,
LMHC, CASAC, and NPPs were expected to work remotely using private work designated
phone numbers to contact patients.
Reasons for any differences between observed and anticipated outcomes, including the
influence of context
The outcome of this QI project anticipated a decrease in mental health and substance
abuse related hospitalization, which did not occur. There was an increase in number of patient
hospitalizations, with a decrease in length of stay from an average of 42 days to < 1 day. These
findings could be due to the psychological effects of the sudden surge of the COVID-19 Delta
and Omicron variants and more people using the ER for COVID testing. It could also be because
of the declining participation in the number of patients in the recovery participating in the
chemical dependency groups and/ or MAT during the last month of the data collection in the
TMH period despite 37% overall increase.
There are people who know relatives, friends, and neighbors impacted by the COVID-19
pandemic. The COVID-19 pandemic brought many changes like social isolation, financial
Telemental Health Services
64
pressures, life uncertainty, and changes in routines. These changes led to increased life stress,
anxiety, and depression which can worsen. Others may have chosen to self-medicate by
increasing alcohol and/ or other drugs to cope with their mental health symptoms exacerbated by
the pandemic.
While patients were actively participating in their Wellness individual therapy and
medication management sessions, the number was declining in the Recovery program. Although
there was 37% growth in group therapy and MAT participation in the TMH period, there was
increased hospitalization. There were systemic changes which may have affected the results of
the study period. The only clerk who schedules clients’ appointment with their mental health
providers was out on extended sick leave and a replacement was in training. There was a period
between the new clerk replacement and the job training where we may have lost some patients to
follow up. There were also new Recovery and Wellness program directors who were adjusting to
their positions. Two Licensed Social workers resigned about a month apart before and at the
onset of the TMH study period. It was noted during data analysis that some of the patients were
not reassigned during the data collection phase and may have been lost to follow up after they
separated from their therapist.
According to the substance abuse counselors at Project Hospitality, patients struggling
with substance abuse recovery need the physical and emotional support of their peers and
counselors. There was a 31% increase of TMH services within the same monitoring period
(November 14 into December 13) compared to pre TMH. There was also 9% decline compared
to the previous 30-day period. As a result, we may have lost some patients within the study
period, which resulted in increased hospitalization. However, among the hospitalizations
recorded in the TMH period, only 20% were in the hospital for more than 24-hours compared to
Telemental Health Services
65
46% in the in-person visits. There was 22% decline in inpatient hospitalizations (greater than 24
hours) in TMH period.
Table 6
Hospitalizations greater than 24 hours
Hospitalizations
September
October
November
Total
In-Person visits (pre-TMH)
10
7
5
22/Total # 48 46%
TMH
3
9
6
18/total #88
20%
Costs and strategic trade-offs, including opportunity costs
Telepsychiatry is the most developed, innovative model in expanding access to
psychiatric providers. According to the National Council for Behavioral Health, the expansion of
telepsychiatry has been disadvantaged by burdensome regulations and limits of reimbursement.
Deliberately increasing the number of mental health providers that are well trained in TMH
services may impact the number of long-term hospital readmissions. Utilization of TMH may
impact substance use disorder patients with co-occurring mental illness who use the Emergency
Departments for stabilization.
The lack of timely access to mental health services affects private clinics, group
practices, individual offices, emergency departments in hospitals and community behavioral
health centers. Insufficient symptom management and non-compliance with medication can
occur with extended times between appointments, leading to frequent Emergency Departments
visits and more hospitalizations. This gap in mental health and substance use disorder services
significantly delays treatment and reduces the quality of treatment, resulting in unacceptable
patient experiences in care, poor outcomes, and higher costs (National Council for Behavioral
Health, 2017). Limited timely access to mental health care further impacts the providers’ ability
Telemental Health Services
66
to address other social determinants of health such as housing, employment, education, and
family engagement.
The direct cost of mental health treatment is the cost of mental health hospitalizations and
outpatient treatments, including medication and counseling, for the captured population. Indirect
cost includes, income loss, disability, the cost of incarceration for prisoners with serious mental
health problems and frequent emergency room visits. The incremental cost would be the cost of
training new providers to telepsychiatry and providing TMH communication models to patients.
Opportunity cost in telepsychiatry/ TMH implementation is the number of people lost to follow
up due to lack of continuous access to mental health treatment, the reimbursement cost per
treatment for in-person visits, and increased workload for providers. These results in high cost
for mental health treatment compared to low cost to train physicians in TMH services.
Limitations to the generalizability of the work
There were limitations experienced during this QI project. TMH was initiated via an
emergency order, prior to the beginning of this project, and at present remains in place. After
project initiation, there was a change from the New York Office of Mental Health, as to how to
evaluate the treatment plans of patients. The project initially proposed patient assessment with
screening tools (GAD-7, PHQ-9, and C-SSRS) by the team of providers within the program
(mental health workers), every 30 days within the 90 days study period. This study period was
from September 15 through December 13, 2021. However, the guidelines later extended
evaluations for all Medicaid patients to be completed at intake, then every 6 months or as needed
depending on presentation of symptoms. A measure was in place at PH to monitor severity of
symptoms using the high-risk protocol criteria. This consequently affected the implementation of
Telemental Health Services
67
the screening tools utilized to assess patients’ mental health such that the entire patient
population in the census was not assessed every 30 days using the screening tools as planned.
Secondly, there was significant turnover of staffing that affected patient contact and
implementation of the assessment tools. For instance, the clinic has gone through three wellness
program directors and one recovery program director since 2020. There were two licensed social
workers who resigned, one clerk who was out on leave, and one credentialed Alcoholism and
Substance Abuse Counselor (CASAC) who resigned during the study phase of this QI project in
2021.
Another limitation is the study design of using the convenience sample of patients from
both the wellness and recovery programs. The results may not be an accurate representation of
all the patients in both programs. The results of the screening tools do not truly reflect the total
number of patients assessed. Given the fluid nature of psychiatry, the frequency of screening
may have varied. For example, some patients may have more screening than normal based on
acute changes in their clinical course.
There were variations in the service types seen in the contact logs between the in-person
office visits of 2019 and the TMH study period of 2021. Some services were either not provided
in 2019 or were documented in the wrong service type. An argument could be made that the
same may have occurred in of the TMH period.
Psyckes EHR serves only Medicaid insurance, and it has delayed data update. Hence, it
may not have captured recent hospital visits/ admissions at the end of the 90 days. The Healthix
EHR is not limited to Medicaid and captures hospitalization of all patients with various
insurances.
Telemental Health Services
68
Although some providers may struggle due to the inadequate quality of reception of some
patients’ phone lines, overall communication has been improved and made easier. Patients who
may have been non-compliant were reengaged and follow up more readily available from their
preferred location.
A difficult area for this organization was how to monitor patients’ compliance to their
treatment regimen. This would necessitate running the patients contact lapse report at least
monthly to enquire about patients who have missed more than two appointments with their
providers, therapy, and medication management sessions. There should be consideration to
discharge those patients who may not comply with medications and TMH visits. Also, there
should be recommendations for easy admissions and access for the discharged clients to return to
the program.
The study was not fully able to meet the exclusion criteria by including those with serious
mental illness who are not of sound mind and unable to make decision, and non-English
speaking. This is because the data pulled from the Awards EHR was clustered and was unable to
differentiate individual patients who received services and how many times the patient received
the services. However, there were no patients younger than 18 years old included in the study.
Within the period of TMH implementation, the (PH) program secured a grant for interpretive
services for Spanish speaking patients and for those patients (documented and undocumented)
without insurance. Hence, Spanish speaking patients were not excluded from this project.
Although there are benefits to TMH implementation, there are also challenges to
adopting this innovative technology. These barriers include inconsistent reimbursement,
concerns over security, privacy, and confidentiality, concerns of clinical duty to provide safe and
Telemental Health Services
69
effective care, and lack of evidence about impact on health care costs, utilization, or outcomes
(AMA, 2020).
The data obtained from the study design of a convenience sample of patients from the
Wellness and Recovery programs may not fully depict the characteristics of the entire
population. The psychiatric and substance abuse hospitalization report obtained from the Award
EHR clustered medical, psychiatric, and substance abuse hospitalizations. The investigator
removed all medically indicated hospitalizations. However, there are cases where neither
psychiatric nor substance abuse hospitalizations could be determined. Therefore, the data is
clustered and cannot be further separated for analysis. The investigator accessed the Psyckes and
Healthix to verify the reasons for these hospitalizations.
The data obtained from the assessment tools are not the cumulative number of patients
assessed, but the results (scores) of assessments and indicates the number of times the
assessments were administered. A patient could be screened more than once in each period,
depending on needs and risks of the patients. The same for contact logs where a given patient
could be provided different service types and/ or same service in each period of data collection.
Efforts made to minimize and adjust for limitations
A cohort of the high-risk patients would have been more appropriate than using the
convenient sample of patients. Investing in the educational training of personnel that document
patients’ hospitalizations in Award her, as indicated in the Psyckes and Healthix her, would
increase the accuracy of the hospitalization report. The billing team participated in the planning
and implementation phase of TMH to increase understanding of federal, state, and private payer
requirements and regulations. The billing team was also provided education regarding the
documentation that captures the billable and other service types.
Telemental Health Services
70
DNP Essentials Addressed
The implementation of this QI project utilized all the essentials of doctoral of advanced
nursing practice (DNP). DNP essential I was utilized when the investigator incorporated
innovative ideas to improve care delivery for patients with mental health and co-occurring
substance abuse disorder. Essential II was incorporated when the investigator evaluated effective
strategies to enhance patient care and enhance the workflow within the PH organization. The
DNP Essential III incorporates nursing theories and frameworks that allowed methodical data
gathering which were utilized to implement the QI project. Essential IV was applied to data
extraction from practice information systems (Awards) and databases (Psyckes and Healthix)
and Essential V is by analyzing TMH guidelines and advocating for equitable policies to enhance
patient outcomes.
There has been interprofessional collaboration throughout the implementation of this QI
project that satisfies the DNP Essential VI. There have been collaborations with LCSWs, MHCs,
substance abuse counselors, the leadership in PH, and consultation with a biostatistician.
Essential VII was incorporated with the implementation of this QI project which aimed to
increase patient access to mental health care and reduce hospitalization by utilizing the TMH
emergency guideline to increase patient assessment. Essential VIII was employed when the
investigator utilized analytical skills in evaluating the current TMH emergency guideline and
providing education to the team of mental health professionals at PH.
Conclusions
Plan for Sustainability
This QI project would be useful for monitoring patients in high-risk situations and track
patients who could be lost to follow up. The sustainability of this QI project is dependent on the
continuous insurance reimbursement related to New York emergency TMH guidelines.
Telemental Health Services
71
Additionally, mental health workers spent time in addition to their scheduled work hours to
address patient needs. The mental health workers spent a vast amount of time overseeing billable
and unbillable services. This project may be beneficial to patients, but it could increase the
workload and stress of mental health providers.
While assessing the utilization of emergency TMH services to increase patient access to
mental health assessments and counseling, and decreasing hospitalization, our findings suggests
that there is need to adopt this guideline into policy and a hybrid approach would better serve the
patients in the recovery program. The hybrid approach entails a combination of in-person office
visits (for group therapy) for peer support and TMH (individual counseling) therapy. Future
studies could investigate a control sample of patients who are seen in in-person visits compared
to those patients who are solely receiving TMH services in real time.
Plan for Dissemination
The results of this QI project were presented to the doctoral committee and the team of
mental health workers at Project Hospitality using a poster and PowerPoints. There is need for
more in-service type education for the staff of Project hospitality on proper documentation of
service types to capture billable services.
Furthermore, there is a need to maintain TMH services as an option for patients in New
York City at the end of the COVID-19 pandemic. A robust TMH legislation which offers equal
reimbursement for TMH, and in-person visits is necessary to capture the increasing number of
patients seeking mental health services in an industry plagued with personnel shortage. This
legislation should increase and standardize the reimbursement of TMH services by Medicaid and
managed care organizations. Payment should be as much as the in-person visits to entice those
cash-only providers to start accepting Medicaid and Managed Care insurances. There is need for
Telemental Health Services
72
support for the legislation and provision of funding to recruit and train more mental health
providers in the North Shore. This would help to ensure the sustainability of the TMH services.
A next step to obtaining legislation of TMH services will be accomplished by contacting the
Council member for the 49th District of the New York City Council. Finally, Telecommunication
aid needs to be provided and adopted as a medical necessity to fully adopt this innovation.
Telemental Health Services
APPENDIX A:
SITE APPROVAL/AUTHORIZATION LETTER
73
Telemental Health Services
74
Telemental Health Services
APPENDIX B:
CONSENT DOCUMENT (DISCLOSURE FORM, CONSENT FORM,
ETC.)
75
Telemental Health Services
76
Telemental Health Services
77
Telemental Health Services
APPENDIX C:
78
EVALUATION INSTRUMENTS (STUDENT CREATED DATA
COLLECTION TOOLS – SURVEYS, QUESTIONNAIRES, INTERVIEW
QUESTIONS, ETC.)
Telemental Health Services
79
Telemental Health Services
80
Telemental Health Services
APPENDIX D:
PROJECT TIMELINE
81
Telemental Health Services
APPENDIX E:
82
OTHER DOCUMENTS AS APPLICABLE TO THE PROJECT (SUCH AS
BUDGET)
Telemental Health Services
83
Telemental Health Services
84
References
Agency for Healthcare Research and Quality. (2020). Plan-Do-Study-Act (PDSA) directions and
examples.
https://www.ahrq.gov/health-literacy/improve/precaustions/tool2b.html
American Medical Association (2020). Telehealth Implementation Playbook.
https://www.ama-assn.org/system/files/2020-04/ama-telehealth-implementationplaybook.pdf
Bashshur, R. L., Shannon, G. W., Bashshur, N., & Yellowlees, P. M. (2016). The empirical
evidence for telemedicine interventions in mental disorders. Telemedicine Journal and EHealth, 22(2), 87-113.
https://doi.org/10.1089/tmj.2015.0206
Brearly, T., Goodman, C., Haynes, C., McDermott, K., & Rowland, J. (2020). Improvement of
post inpatient psychiatric follow-up for veterans using telehealth. American Journal of
Health-System Pharmacy, 77(4), 288-294.
https://doi.org/10.1093/ajhp/zxz314
Bjureberg, J., Dahlin, M., Carlborg, A., Edberg, H., Haglund, A., & Runeson, B. (2021).
Columbia-Suicide Severity Rating Scale Screen Version: Initial screening for suicide risk in
a psychiatric emergency department. Psychological Medicine, 1-9.
doi:10.1017/S0033291721000751
Cantor, J. H., McBain, R. K., Kofner, A., Ma, Stein, B. D., & Yu, H. (2021). Availability of
outpatient telemental health services in the United States at the outset of the COVID-19
pandemic. Medical care, 59(4), 319-323.
https://doi.org/10.1097/MLR.0000000000001512
Telemental Health Services
85
Center for Disease Control and Prevention. (2020). National Center for health statistics: Healthy
people 2020. U.S. Department of Health & Human Services.
https://www.cdc.gov/nchs/healthy_people/hp2020.htm
Chang, Hyejung. (2015). Evaluation Framework for Telemedicine Using the Logical Framework
Approach and a Fishbone Diagram. Healthcare Informatics Research. 21. 230. 10.4258/hi
Cohen Veterans Network & National Council for Behavioral Health. (2018). America’s mental
health 2018.
https://www.cohenveteransnetwork.org/wp-content/uploads/2018/10/Research-Summary10-10-2018.pdf
County Health Rankings & Roadmaps. (2020). Health Outcomes: Overall rank of Richmond
County, New York. A Robert Wood Johnson Foundation Program.
https://www.countyhealthrankings.org/app/newyork/2020/rankings/richmond/county/outco
mes/overall/snapshot
Cuomo, A. M., Sullivan, A. M. T., & Tavella, C., & Revised March 17, 2. (2020). Supplemental
guidance regarding use of telehealth for people served by OMH licensed or designated
programs affected by the disaster emergency. Albany, New York.
https://omh.ny.gov/omhweb/guidance/supplemental-guidance-use-of-telemental-healthdisaster-emergnecy.pdf
Cuomo, A. M., Sullivan, A. M. T., & Tavella, C., & Revised March 17, 2. (2020). Use of
Telephone and Two-way Video Technology by OMH-Licensed, Funded or Designated
Providers and Clients Affected by the COVID-19 Pandemic. Albany, New York.
https://ahihealth.org/wp-content/uploads/2020/04/NYS-OMH-Consolidated-COVID-19Telemental-Health-Guidance.pdf
Telemental Health Services
86
Dorin, A. (2021). Behavioral health advocates support Governor Cuomo’s legislation to
expand access to telehealth in mental health and substance use services for all New
Yorkers. The Coalition for Behavioral Health. New York, NY.
Eisman, A. B., Kilbourne, A., M., Dopp, A. R., Saldana, L., Eisenberg, D. (2020). Economic
evaluation in implementation science: Making the business case for implementation
strategies. Psychiatry Research, 283.
https://doi.org/10.1016/j.psychres.2019.06.008.
Federation of State Medical Boards. (2022). U.S. States and Territories Modifying Requirements
for Telehealth in Response to COVID-19: Out-of-states physicians; preexisting providerpatient relationships; audio-only requirements, etc.
fsmb.org/siteassets/advocacy/pdf/states-waving-licensure-requirements-for-telehealth-inresponse-to-covid-19.pdf
Guaiana, G., Mastrangelo, J., Hendrikx, S., & Barbui, C. (2021). A systematic review of the use
of telepsychiatry in depression. Community Mental Health Journal, 57(1), 93-100.
https://doi.org/10.1007/s10597-020-00724-2
Healthix. (2021). Ixchange Blog: Healthix VIPs deliver healthcare interoperability solutions.
https://healthix.org/ixchange/healthix-vips-deliver-healthcare-interoperability-solutions/
Haghnia, Y., Samad-Soltani, T., Yousefi, M., Sadr, H., & Rezaei-Hachesu, P. (2019).
Telepsychiatry- based care for the treatment follow-up of Iranian war veterans with posttraumatic stress disorder: A randomized controlled trial. Iranian Journal of Medical
Sciences, 44(4), 291-298.
https://doi.org/10.30476/IJMS.2019.44944
Telemental Health Services
87
Harrar, S. (2021). Inside America’s Psychiatrist Shortage: An in-depth look at why it’s
happening, how bad it is, where it’s worst, and how to find the mental health help you need,
despite it. PSYCOM.
https://www.psycom.net/inside-americas-psychiatrist-shortage
Hickey, J. V. & Brosnan, C. A. (2017). Evaluation of health care quality for DNPs (2nd ed.).
Springer Publishing Company.
Hinterland, K., Naidoo, M., King, L., Lewin, V., Myerson, G., Noumbissi, B., Woodward, M.,
Gould, L. H., Gwynn, R. C., Barbot, O., Bassett, MT. (2018). Community Health Profiles
2018, Staten Island Community District 1: St. George and Stapleton. NYC.gov k57(59):1-20.
https://www1.nyc.gov/assets/doh/downloads/pdf/data/2018chp-si1.pdf
Institute for Healthcare Improvement. (2021). Tools: Plan-Do-Study-Act (PDSA) worksheet.
http://www.ihi.org/resources/Pages/Tools/PlanDoStudyActWorksheet.asp
Levis, B., Benedetti, A., Thombs, B. D. (2019). Accuracy of patient health questionnaire-9
(PHQ-9) for screening to detect major depression: Individual participant data metaanalysis. Quebec, Canada. BMJ 2019; 365:1476
https://doi.org/10.1136/bmj.l1781
Liem, A., Sit, H. F., Arjadi, R., Patel, A. R., Elhai, J. D., & Hall, B. J. (2020). Ethical
standards for telemental health must be maintained during the COVID-19
pandemic. Asian journal of psychiatry, 53, 102218.
https://doi.org/10.1016/j.ajp.2020.102218
Mace, S., Boccanelli, A., & Dormond, M. (2018). The use of telehealth within behavioral health
settings: Utilization, opportunities, and challenges. Behavioral Health Workforce Research
Center, University of Michigan.
Telemental Health Services
88
https://behavioralhealthworkforce.org/wp-content/uploads/2018/05/Telehealth-FullPaper_5.17.18-clean.pdf
Mahmoud, H., Naal, H., & Cerda, S. (2021). Planning and implementing telepsychiatry in a
community mental health setting: A case study report. Springer Science and Business Media
LLC.
https://doi.org/10.1007/s10597-020-00709-1
Medicare Learning Network. (2021). Telehealth services. Centers for Medicare & Medicaid
Services.
https://www.cms.gov/Outreach-and-Education/Medicare-Learning-NetworkMLN/MLNProducts/Downloads/TelehealthSrvcsfctsht.pdf
Milstein, D. & Madden, S. L. (2017). Tackling Youth Substance Abuse on Staten Island: A
collective impact project. Harvard T. H. Chan School of Public Health.
http://sipcw.org/wp-content/uploads/2019/09/Harvard-University-Case-Study-TacklingYouth-Substance-Abuse-on-Staten-Island.pdf
Mossman, S. A., Luft, M. J., Schroeder, H. K., Varney, S. T., Fleck, D. E., Barzman, D. H.,
Gilman, R., DelBello, M. P., & Strawn, J. R. (2017). The Generalized Anxiety Disorder
7-item scale in adolescents with generalized anxiety disorder: Signal detection and
validation. Annals of clinical psychiatry: official journal of the American Academy of
Clinical Psychiatrists, 29(4), 227–234A.
National Council for Behavioral Health. (2017). The Psychiatric Shortage: Causes and solutions.
Washington, DC.
https://www.thenationalcouncil.org/wp-content/uploads/2017/03/PsychiatricShortage_National-Council-.pdf?daf=375ateTbd56
Telemental Health Services
89
New York Office of Mental Health. (2020). Telemental health services: Interim guidance for
telemental health during the COVID-19 outbreak.
https://omh.ny.gov/omhweb/clinic_restructuring/telepsychiatry.html
New York State Office of Mental Health (2021a, January 1). New York State Office of Mental
health 14 NYCRR Part 599: “Clinic Treatment Programs” interpretive/ implementation
guidance. https://omh.ny.gov/omhweb/clinic_restructuring/part599/part-599.pdf
New York State Office of Mental Health. (2021b, February 1). PSYCKES. New York.
https://omh.ny.gov/omhweb/psyckes_medicaid/psyckes_medicaid_release_notes_710_fe
b_2021.pdf
Olin, S. C., O'Connor, B. C., Storfer-Isser, A., Clark, L. J., Perkins, M., Hudson Scholle, S.,
Whitmyre, E. D., Hoagwood, K., & Horwitz, S. M. (2016). Access to Care for Youth in a
State Mental Health System: A Simulated Patient Approach. Journal of the American
Academy of Child and Adolescent Psychiatry, 55(5), 392–399.
https://doi.org/10.1016/j.jaac.2016.02.014
Perrin, A. & Turner, E. (2019). Smartphones help blacks, Hispanics bridge some-but not alldigital gaps with whites. Pew Research Center.
https://www.pewresearch.org/fact-tank/2019/08/20/smartphones-help-blacks-hispanicsbridge-some-but-not-all-digital-gaps-with-whites/
Project Hospitality. (n.d.). High Risk Criteria Protocol
Pumariega, A. J., Posner, K., Millsaps, U., Romig, B., Starvarski, D., Gehret, M. J., Riley, K.,
Wasser, T. E., Walsh, G., Yarger, H. (2020). Systematic Suicide Screening in a General
Hospital Setting: Process and initial Results. World Soc. Psychiatry 2020; 2:31-42.
https://www.worldsocpsychiatry.org/text.asp?2020/2/1/31/281135
Telemental Health Services
90
Richmond University Medical Center. (2019). Community health needs assessment &
community service plan for Richmond County (Staten Island).
https://www.rumcsi.org/wp-content/uploads/2020/09/2019-CHNA-Web-site-FinalReport-2019-10-30.pdf
Serenity at Summit (2022). Addiction statistics and treatment in Staten Island. Retrieved from
https://serenityatsummit.com/new-york/staten-island/
Spitzer, R. L., Kroenke, K., Williams, J. B. W., Lowe, B. (2006). A brief measure for assessing
generalized anxiety disorder: The GAD-7. Arch Intern Med. 166(10):1092–1097.
doi:10.1001/archinte.166.10.1092
Sylvia, M. L., & Terhaar, M. F. (2018). Clinical Analytics and Data Management for the
DNP (2nd ed.). Springer Publishing Company.
The Lancet Global Health. (2020). Mental health matters. Elsevier Ltd 8(11).
https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(20)304320/fulltext?rss=yes
United States Census Bureau. (2019). QuickFacts: Richmond County (Staten Island Borough),
New York.
https://www.census.gov/quickfacts/richmondcountystatenislandboroughnewyork
Warren, J. C. & Smalley, K. B. (2020). Using telehealth to meet mental health needs during the
COVID-19 crisis. The Commonwealth Fund.
https://www.commonwealthfund.org/blog/2020/using-telehealth-meet-mental-health-needsduring-covid-19-crisis
Telemental Health Services
91
Williams, C. (2021). Using the hub and spoke model of telemental health to expand the reach of
community-based care in the United States. Community Mental Health Journal, 57(1), 4956.
https://doi.org/10.1007/s10597-020-00675-8
Wood, J. M. & Gupta, S. (2017). Using rating scale in a clinical setting. Clinical Reviews.
https://cdn.mdedge.com/files/s3fs-public/Document/August-2017/CR02709028.PDF
Wood, P., Burwell, J., Rawlett, K. (2018). New Study Reveals Lack of Access as Root Cause
for mental Health Crisis in America. National Council for Behavioral Health.
https://www.thenationalcouncil.org/press-releases/new-study-reveals-lack-of-access-asroot-cause-for-mental-health-crisis-in-america/