BLOOMSBURG UNNERSITY OF PENNSYLVANIA As members of the DNP (Doctor of Nursing Practice) Project Committee, we certify that we have read the DNP project prepared by Emmanuel Ahipue, titled "Telemental health: Increasing access to mental health services and decreasing hospitalization" and recommend that it be accepted as fulfilling the DNP project requirement for the Degree of Doctor of Nursing Practice. [Faculty of record] Peggy Z. Shipley PhD, RN [Faculty Mentor] �� j);,:Jrf, F/JF· G �ember] Date: 5'//S / Zt.fZ--z. 5/13/2022 Date: ____ _ Date: Final approval and acceptance of this DNP project is contingent upon the candidate's submission of the final copies of the DNP project to the Graduate College. I hereby certify that I have read this DNP project prepared under my direction and recommend that it be accepted as fulfilling the DNP project requirement. DNP · oordinator, Chair, or Dean [Graduate Nursing Department] Date: �/:w. Telemental Health Services 1 TELEMENTAL HEALTH SERVICES: INCREASING ACCESS TO MENTAL HEALTH SERVICES AND DECREASING HOSPITALIZATION by Emmanuel Ahipue ________________________ Copyright © Emmanuel Ahipue 2022 A DNP Project Submitted to the Faculty of the COLLEGE OF NURSING In Partial Fulfillment of the Requirements For the Degree of DOCTOR OF NURSING PRACTICE BLOOMSBURG UNIVERSITY 2022 Telemental Health Services 2 BLOOMSBURG UNIVERSITY OF PENNSYLVANIA As members of the DNP Project Committee, we certify that we have read the DNP project prepared by Emmanuel Ahipue, titled Decreasing hospitalization readmissions and increasing access to mental health services through increased assessment via telemental health in a Wellness and Recovery clinic in Staten Island, Richmond County of New York City: A Quality Improvement Project and recommend that it be accepted as fulfilling the DNP project requirement for the Degree of Doctor of Nursing Practice. _________________________________________________________________ Date: ____________ _________________________________________________________________ Date: ____________ _________________________________________________________________ Date: ____________ [Committee Chair Name] [Committee Member Name] [Committee Member Name] Final approval and acceptance of this DNP project is contingent upon the candidate’s submission of the final copies of the DNP project to the Graduate College. I hereby certify that I have read this DNP project prepared under my direction and recommend that it be accepted as fulfilling the DNP project requirement. _________________________________________________________________ [Committee Chair Name] DNP Project Committee Chair [Academic Department] Date: ____________ Telemental Health Services 3 ACKNOWLEDGMENTS I wish first to acknowledge the grace of God for allowing me to complete this quality improvement project. Despite the challenges in time management, travelling time to my clinical site, and long working hours, you have continued to inspire and strengthen me to complete this project. I am forever grateful and indebted to you. I wish to acknowledge Dr. Shipley, my Clinical mentor. I have nothing but respect and appreciation for your encouragement and prompt response to my questions. I also admire your wealth of knowledge and constructive criticism during the phases of this QI project. You edited my proposal for this project. I wish to acknowledge Dr. Jackson and Dr. Parke for their patience and encouragement throughout the DNP courses. I would not have asked for better professors. I wish to acknowledge G. Craig Wood, the biostatistical consultant, for processing the data. I also wish to acknowledge Mr. Bello Muftau, my colleague at Project hospitality. As a fellow psychiatric NP, I admire your insight and the experience in implementing the telemental health at Project Hospitality. I would also like to acknowledge Mr. Gary Seigel for your extraordinary leadership that you showed in bringing the team of mental health workers together at Project Hospitality at the onset of the Corona virus pandemic. Your leadership set the stage for the implementation of the telemental health in the facility. I can’ t help but acknowledge the amazing staff of Project Hospitality. The implementation of this telemental health in our facility has bonded us together and has brought out the best in all of us. I appreciate your dedication and kindness to our patients. You are all amazing and have formed a formidable team in the Staten Island District 49. I am immensely proud of the work we have done increasing access to mental health in our community. Telemental Health Services 4 DEDICATION This project is dedicated to my wife, Ulonna, my sons, Justin and Joshua, and my daughter, Alexandra. Thank you for your support, understanding and enduring love. You were all there at every bump in the road and I appreciate you all endlessly. This project is also dedicated to my mother, Justina Ahipue, and my father Paul Ahipue. Thank you, mom, for your prayers and encouragement to keep moving beyond your fourth-grade education. Also, for babysitting the grandchildren without complaining whenever I needed you. Dad, I know your spirit is with me and happy with me in this accomplishment. Mom and Dad, you are both my rock. Telemental Health Services 5 TABLE OF CONTENTS LIST OF FIGURES .........................................................................................................................7 LIST OF TABLES ...........................................................................................................................8 ABSTRACT.....................................................................................................................................9 INTRODUCTION........................................................................................................................11 Background Knowledge/Significance.........................................................................................12 Local Problem ..............................................................................................................................14 Intended Improvement ................................................................................................................19 Project Purpose ................................................................................................................21 Project Question ...............................................................................................................22 Project Objectives ............................................................................................................22 Theoretical Framework ...............................................................................................................23 Literature Synthesis .....................................................................................................................26 Evidence Search ...............................................................................................................28 Comprehensive Appraisal of Evidence ..........................................................................29 Strengths of Evidence ......................................................................................................35 Weaknesses of Evidence ..................................................................................................36 Gaps and Limitations ......................................................................................................36 METHODS ...................................................................................................................................37 Project Design...............................................................................................................................37 Model for Implementation ..........................................................................................................39 Setting and Stakeholders .............................................................................................................41 Planning the Intervention ...........................................................................................................42 Participants and Recruitment.....................................................................................................45 Consent and Ethical Considerations ..........................................................................................45 Data Collection .............................................................................................................................46 Data Analysis ................................................................................................................................46 RESULTS .....................................................................................................................................48 Outcomes ......................................................................................................................................52 Telemental Health Services 6 TABLE OF CONTENTS - Continued DISCUSSION ..............................................................................................................................59 Summary.......................................................................................................................................59 Interpretation ...............................................................................................................................61 Implications (Practice, Education, Research and Policy) ........................................................62 Limitations ....................................................................................................................................66 DNP Essentials Addressed .........................................................................................................69 Conclusions ...................................................................................................................................70 Plan for Sustainability .....................................................................................................70 Plan for Dissemination ....................................................................................................71 APPENDIX A: SITE APPROVAL/AUTHORIZATION LETTER .........................................73 APPENDIX B: CONSENT DOCUMENT (DISCLOSURE FORM, CONSENT FORM, ETC.)................................................................................................................75 APPENDIX C: EVALUATION INSTRUMENTS (STUDENT CREATED DATA COLLECTION TOOLS – SURVEYS, QUESTIONNAIRES, INTERVIEW QUESTIONS, ETC.) .......................................................................................78 APPENDIX D: PROJECT TIMELINE .....................................................................................81 APPENDIX E: OTHER DOCUMENTS AS APPLICABLE TO THE PROJECT (SUCH AS BUDGET) ........................................................................................................82 REFERENCE…………………………………………………………………………………….84 Telemental Health Services 7 LIST OF FIGURES Figure 1. Telemental health fishbone diagram. (Adapted from Chang (2015). No permission needed to use and adapt……………………………………….……………………………………41 Figure 2. Screening tool implementation process………………………………………………49 Figure 3: Number of contacts by service type during the pre-TMH (in person office visits only, September 15, 2019 – December 13, 2019) and post-TMH (Telemental Health, September 15, 2021 – December 13, 2021) ……………………………………………………………………54 Figure 4: Number in census report and number of substance abuse and psychiatric related hospitalizations during the pre-period (in person visits only, September 15, 2019 – December 13, 2019) and post-period (TMH, September 15, 2021 – December 13, 2021) …………………….56 Figure 5. GAD-7 anxiety level before and after introduction of TMH services…………………57 Figure 6. PHQ-9 depression before and after the introduction of TMH services………………58 Telemental Health Services 8 LIST OF TABLES Table 1. Definition of terms……………………………………………………………. 18 Table 2. The benefit of the Effectiveness-Efficiency-Equity conceptual framework….26 Table 3. High Risk Criteria Protocol……………………………………………………50 Table 4. Service types/ Contact logs…………………………………………………….54 Table 5. C-SSRS suicidality and after the introduction of TMH services………………58 Table 6. Hospitalizations greater than 24 hours……………………………………….65 Telemental Health Services 9 ABSTRACT Purpose: Assess the ability of increased assessment via telemental health (TMH) to increase access to care and decrease hospitalizations. Background: There is mental health crisis in the U.S. and mental health services are insufficient despite more than half of Americans (56%) seeking help. Patients seeking mental health treatment have had to deal with the limited options in treatment and long waiting periods with 46% of Americans who have or know someone who has had to drive more than an hour roundtrip to seek treatment, and 38% of Americans reporting that they have waited longer than one week for mental health treatments. Methods: This quality improvement (QI) project utilized retrospective chart audit at Project Hospitality (PH) outpatient clinic to compare patients’ hospitalizations, readmissions, and access to care between in-person pre-pandemic metrics with those occurring after institution of the New York emergency TMH guideline initiated during the COVID-19 pandemic. Data was obtained from PH’s electronic health record (EHR), Awards, evaluating these screening tools (Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder 7-item scale (GAD) and the Columbia-Suicidal Severity Rating Scale (C-SSRS)). Data was obtained from interoperative community EHR indicating where and when a patient obtained mental health treatment within and/ or outside the program within the last 90 days. Results: Availability of TMH through the emergency guideline has resulted in improved access to mental health providers leading to improvement of symptoms, reduction in in-patient hospitalization, and improved patient outcomes. Telemental Health Services 10 Conclusion: The implementation of TMH has helped to improve patient access to mental health, reduced loss to patient follow-up, and reduced in-patient hospitalization. Additional work is needed to adopt this guideline into policy ensuring that patients have flexible access to care when it is needed. Telecommunication aid, broadband, and internet access required and must be adopted as medical necessity to fully adopt this innovation. Telemental Health Services 11 INTRODUCTION As the emergency TMH guidelines are entering into the second year, evaluation of the effectiveness of TMH to increase patient access to their providers and decrease hospitalization at PH is important. TMH use focuses on the second overarching goals of the Healthy People 2020 which aimed to achieve health equity, eliminate disparities, and improve the health of all groups (Center for Disease Control and Prevention, 2020). There are several problems facing the use of TMH/ telepsychiatry in non-emergent times. There is lack of nationwide policy governing the provision of TMH care. Secondly, there is reimbursement disparity between traditional in-person office visits and TMH services. Finally, there are common restrictions placed on locations where providers and patients must be located to engage in TMH. This project is currently operating under an emergency executive order, with month-to-month extensions and revisions due to the COVID-19 pandemic. These revisions create uncertainty among different organizations and the population being served. At the onset of the COVID-19 pandemic, the request for TMH services, by both providers and patients, increased significantly. A regulatory waiver was thereby issued to maintain mental health services to vulnerable citizens throughout the duration of the declared emergency. As a result, TMH was temporarily expanded for Medicaid-reimbursable services to include telephonic and/ or video technology, including tools commonly available on smart phones and other devices (Cuomo et al., 2020). Additional roles were included in this emergency guidance as the definition of TMH practitioners were expanded to include any professional, paraprofessional, or unlicensed behavioral health staff who deliver a qualified service via TMH (Cuomo et al., 2020). TMH refers to “the use of technologies such as phone or video calls between mental health professionals and other mental health professional, Telemental Health Services 12 patients, service users, family members, or care givers to deliver mental health care” (Barnett et al., 2021). Services includes delivering remote psychiatric evaluations, psychotherapy, and medication management. The goal of this study is to use the findings to support the need for a more permanent and effective TMH policy. Background Knowledge/Significance There is mental health crisis in America. Mental health services in the U.S. are insufficient despite 56% of Americans seeking help (Wood et al., 2018). The attitudes of Americans have improved towards receiving mental health services and 76 % of Americans are now seeing mental health as important as physical health (Wood et al., 2018). This has further increased the demand for mental health services which had already been shown to be insufficient. Lack of access and long waiting periods have become the norm. Despite the high demand for mental health services and societal awareness for mental health in the U.S., 74% of Americans do not believe that these services are available for everyone and 47 % believe that the options are limited (Wood et al., 2018). Medicaid-covered patients in publicly funded community health centers experience the most extended delays in timely access (National Council for Behavioral Health, 2017). The disparity in access to mental health care is pronounced with patients who have low level income (annual household salary less than $49,999), reside in under-resourced neighborhoods, and with insufficient insurance coverage (Cohen Veterans Network & National Council for Behavioral Health, 2018). About 53% of Americans who have sought mental health treatment were in low-income households (National Council for Mental Wellbeing, 2018). The largest proportions of those seeking mental health treatment are millennials (37%), those in lower income households (53%), and those involved with the military either directly or Telemental Health Services 13 through a secondary relationship (66%) (CVN & NCBH, 2018). Younger Americans, despite having some of the more critical needs for mental health treatment, are unsure of their resources and are more impacted by the stigma of mental health care (CVN & NCBH, 2018). The average American is seeking mental health treatment and the need to receive care has grown (National Council for Behavioral Health, 2017). There are factors that prevent timely access to treatment for those who have established mental health treatment. These factors include cost of mental health care for those with insufficient insurance, social stigmas attached to mental health treatment, lack of access to mental health facility or knowledge of where to find one, concern relating to quality of care, and travelling associated with follow up appointments (Wood et al., 2018). Cost or poor insurance coverage constitutes 42% of the barriers for Americans seeking mental health, 15% lack information on where to start, 17% related to social stigma around seeking treatment, and 7% ascribes to the quality of care as a barrier (CVN & NCBH, 2018). Patients are losing time while waiting for treatment and traveling to their appointments. Patients seeking mental health treatment have had to deal with the limited options in treatment and long waiting periods, with 46% of Americans who have or know someone who has had to drive more than an hour roundtrip to seek treatment, and 38% of Americans reporting that they have waited longer than one week for mental health treatments (Wood et al., 2018). With expanded access under the Affordable Health Care reform, demand for mental health care is high, adding to the problem of timely access to mental health treatment. The National Council of Behavioral Health (2017) report indicates that the demand for psychiatry will outstrip supply by 15,600 psychiatrists in 2025, based on estimates of retirement and new entries into the workforce. Access is further diminished due to increasing numbers of private Telemental Health Services 14 practice psychiatrists accepting cash or private insurance only (National Council for Behavioral Health, 2017). There are factors affecting the shortage of psychiatrists in both the rural and urban areas across in the country. The National Council for Behavioral Health Report (2017) highlighted these reasons to include aging of the current workforce, low rates of reimbursement, burnout, burdensome documentation requirements, and restrictive regulations around sharing clinical information necessary to coordinate care. Innovative techniques are needed to solve the problem of timely access to care (National Council for Behavioral Health, 2017). This is necessary to capture the considerable number of people suffering from a range of psychiatric disorders like schizophrenia, bipolar disorder, major depression, anxiety, and other mood disorders, who needs mental health services. High quality care can be maintained while patients are reached, regardless of the location of the patients. The solutions to the shortage of psychiatry and timely access can only be achieved by a combination of interrelated solutions and not exclusively on recruiting more psychiatrists or increasing the payment or reimbursement rate (National Council for Behavioral Health, 2017). Telepsychiatry is the most developed, innovative model in expanding access to psychiatric provider (National Council for Behavioral Health, 2017). Local Problem PH has multiple programs that provide services to the community of the 49th district of Staten Island, New York also known as the North shore of Staten Island. The served population includes individuals with mental illness and some with co-occurring substance use disorder. The challenges faced in providing services to this population includes keeping the patients engaged in treatment long enough for recovery, reducing patients lost to follow up, and providing treatment Telemental Health Services 15 for those who are uninsured or underinsured. Most of the patient population at PH have Medicaid insurance. This population often face health inequity due to mental illness and addiction. These disparities can be related to unemployment, homelessness, and multiple health issues, including addiction and mental illness. There are scenarios where there is complete disregard for an established care or resources by the patients. Other instances, there is an issue of abandonment of needed resources like shelter due to impulsive judgment and isolation from familial support. These situations create ethical dilemmas for mental health workers when patients are not engaging with treatment. Ethical dilemmas arise in areas related to continuing treatment when a provider knows that the patient is still actively using drugs or whether to discharge a patient who has disengaged from treatment by missing appointments yet knowing that he/ she needs the medications for mental stability and functioning. Based on the nursing code of ethics, the components of beneficence, justice, and nonmaleficence are challenged. The South shore is the more affluent section of Staten Island and more economically disadvantaged population with a more diverse population and larger percentage of people of color is on the Island’s Manhattan-facing North shore (Milstein & Madden, 2017). Staten Island, despite its relative affluence, had the highest all-cause mortality rate in New York City (Milstein & Madden, 2017). Mortality here is driven by rates of cancer (lung cancer and breast cancer among women), colorectal cancer, heart disease, and substance abuse respectively particularly in the North shore area (Hinterland et al., 2018). Staten Island has the highest rate of drug overdose deaths compared to the other New York City boroughs (Serenity at Summit, 2022). Staten island was once the hot bed for the U.S. opioid epidemic. Most hospitalizations for drug overdose in Staten Island are related to opioid Telemental Health Services 16 use (Serenity at Summit, 2022). Based on Medicaid hospital admission data from 2016 to 2018, there were 2,619 emergency department visits during this time attributed to opioid abuse and 1,851 of them where in people 18 to 44 years old (Serenity at Summit, 2022). Statistics indicate that synthetic opioids accounted for 1,641 opioid-related overdose deaths in New York in 2016, heroin caused 1,307 deaths, and 1,100 deaths were caused by prescription opioids (Serenity at Summit, 2022). Out of the 306 overdose deaths in Staten Island from 2016 to 2018, 92 deaths were in 2016, 108 deaths in 2017, and 105 deaths in 2018 (Serenity at Summit, 2022). Over 80% of the fatal overdose in 2018 were seen in the Caucasian population and White males were more likely to experience a mortal overdose (77%) (Serenity at Summit, 2022). The rate of psychiatric hospitalizations in 2016 among adults in the North Shore is higher than Staten Island and overall New York City (Richmond University Medical Center, 2019). The North Shore of Staten Island in 2016, according to the Department of Health and Mental Hygiene (DOHMH), had a rate of 963 psychiatric hospitalizations per 100, 000 compared to 707 Staten Island average, and the New York City average of 676 (The New York City Department of health and Mental Hygiene, 2018). According to the County Health Rankings and Roadmaps Richmond County of Staten Island’s premature death toll in 2020 was 5,600 compared to 5,500 with the top U.S. performers. Also, the ratio of mental health patients to mental health provider is 440:1, compared to 350:1 in New York City, and 290:1 with the top U.S. performers. According to the DOHMH (2018), The New York State Department of Health, Statewide Planning and Research Cooperative system, the rate of avoidable hospitalizations among adults in the North Shore of Staten Island is higher than New York City rate in 2014. High psychiatric hospitalization rates reflect the challenges residents in under-resourced neighborhoods face. Telemental Health Services 17 These include difficulty accessing preventive services and early care, greater exposure to stressors, and interruptions in health insurance coverage (DOHMH, 2018) In New York City, the general wait times for psychiatry appointments were significantly longer in the spring than in the summer (49.9 days versus 36.7 days) and the wait times of therapy appointments were shorter in the community than the hospital clinics (19.1 days versus 35.3 days) (Olin et al., 2016). At the onset of the COVID-19 pandemic, when the clinic adopted TMH services, there was a surge in mental health needs. Ways to access mental health treatment in the North Shore community was initiated to meet the needs of the community. As an Office of Mental Health (OMH) designated funding entity, there is annual requirement to show effectiveness for increasing patient capacity within the clinic. According to the New York State Office of Mental Health (OMH), there are clinic services that must be available and offered as needed at any mental health clinic licensed by OMH. These required services, as noted in Table 1, include assessments, therapies, injections, and enhanced services (New York Office of Mental health, 2021a). These requisite services serve as the template for allowable billing. Assessments include initial assessment and psychiatric evaluation, therapies include individual psychotherapy, family/ collateral psychotherapy, group psychotherapy, and psychotropic medication treatment. In-person clinic visit serving adults covers injectable psychotropic medication administration and injectable medication administration with education and monitoring. The enhanced services include crisis intervention and monitoring. Telemental Health Services 18 Table 1 Definition of term Billable Services 1. Assessment 2. Therapies 3. Injections 4. Enhanced services Optional clinic services 1. Testing 2. Physical health 3. Injections 4. Enhanced Services Other Terms 1. Other Service type 2. Outreach Definition of terms Initial assessment and psychiatric assessment Individual psychotherapy, family/ collateral psychotherapy, group psychotherapy, and psychotropic medication treatment. Injectable psychotropic medication administration and injectable psychotropic medication administration with education and monitoring (both for clinics serving adults) Crisis intervention and complex care management. A crisis intervention refers to activities, including medication and verbal therapy, which are designed to address acute distress and associated behaviors when the individual’s condition requires immediate attention. Complex care management is a clinical level service which may be necessary as a follow up to psychotherapy, psychotropic medication treatment or crisis service for the purpose of preventing a change in community status or as a response to complex conditions. Description of services Developmental testing, psychological testing, and psychiatric consultation Health physicals and health monitoring Injectable psychotropic medication administration for clinics serving children and injectable psychotropic administration with education and monitoring for clinics serving children. smoking cessation treatment for individual and group, and Screening, Brief intervention, and referral to treatment (SBIRT). Documentation of pertinent patient information necessary for treatment that is not billable Different attempts to contact a patient in connection with his appointment or related matter. 3. Other definition of terms 4. High-Risk It involves weekly clinical team meetings to discuss any patient Criteria protocol put on the HRCP pathway, calling patients who are missing (HRSCP) appointments and could potentially be loss to follow up, reporting patients who are in Assisted Outpatient Treatment (AOT) who are court ordered or on probation and not following up with treatment, patients who have reported suicidal or homicidal thoughts within the past six months, aggressive or violent patients, sexual predators, and frequently hospitalized patients within the last six months. 5. Contact log A flowchart showing the service types in a period. Telemental Health Services 6. Census 7. Number of the census report per contact 8. The number of hospitalizations per contact 19 The number of active patients in the wellness and recovery program in a period. This is the percentage of billable services received by each patient in a period The percentage of services received by hospitalized patients in a period To achieve the standard of the New York OMH, the program must find ways to communicate with providers and patients to maintain quality care. A High-Risk Criteria protocol (HRCP) developed by the PH became useful in monitoring patients. This HRCP involves weekly clinical team meetings to discuss any patient put on the HRCP pathway, calling patients who are missing appointments and could potentially be loss to follow up, reporting patients who are in Assisted Outpatient Treatment (AOT) who are court ordered or on probation and not following up with treatment, and patients who have reported suicidal or homicidal thoughts within the past six months. Additionally, those patients who are frequently hospitalized within the past six months, patients who are aggressive, violent, or with sexually inappropriate behaviors, patients and anyone involved in domestic violence where the perpetrator continues to have access to the victim are also included in the HRCP. Intended Improvement Project Hospitality (PH) is a non-for-profit outpatient Wellness and Recovery program in Staten Island, New York. Telehealth/ TMH did not exist at the clinic before the Corona Virus pandemic and was initiated in March 2020. The implementation was ongoing and there was lots of learning and teaching as a team. The PH program, an OMH funded entity, needed to show the effectiveness of TMH services by increasing patients’ assessments. A quality improvement initiative was Telemental Health Services 20 implemented which monitored patient outcomes by showing lack of deterioration to the point of hospitalization for patients with co-occurring mental health and substance use disorder. This was done using the self-assessment tools PHQ-9, the GAD-7, and C-SSRS. While CSSRS screens the suicide risks of the clients, the PHQ-9 assessment determined level of depression of clients, the GAD-7 assessment was for anxiety levels. These three-assessment metric are done during initial intake and every 6 months and based on patient symptom changes. These assessments are completed in the scope of practice of licensed social workers (LSWs) and mental health counselors (MHCs). These assessments are performed frequently on patients based on their needs. The patient risk assessment tool was created to monitor patients either at high risk of decompensation and/or suicide or those lost to follow up and are identified by running the contact lapse report and then efforts are made to re-engage them in treatment. Interventions to increase patient visits were initiated when the PHQ-9 score level was ten and above or the GAD score is above 10 which indicates moderate to high anxiety. Also, this is brought to the attention of the NPPs and other providers for a case conference and modify medication management. The GAD-7 is associated with acceptable specificity and sensitivity for detecting clinically significant anxiety symptoms (Mossman et al, 2018) like the PHQ-9 which is sensitive and more specific for all ages from a cut off score of 10 or above (Levis et al, 2019). The Columbia-Suicide Severity Rating Scale (C-SSRS) is a scale designed to measure suicidal ideation severity and suicidal behaviors by capturing the pastmonth severity of suicidal ideation and measuring the past 3-month presence of actual and aborted suicide attempts (Bjureberg et al., 2021). These screening tools PHQ-9, GAD-7, and C-SSRS have been found to be valid and reliable in predicting level of depression, anxiety, and adequate ability to detect suicidality in risk and behavior (Wood & Gupta, 2017). Telemental Health Services 21 The hospitalization rate was checked monthly using the community based electronic health records such as the PSYCKES and Healthix by the clinical supervisor. The current emergency TMH guideline by the New York State Office of Mental Health (NYSOMH) utilized verbal patient consent to provide services. Other verbal consents obtained included consent for community based interoperable EHR, Psyckes, and Healthix. Psyckes is a HIPAA-compliant web-based application designed to support clinical decision making, care coordination, and quality improvement in New York State (NYSOMH, 2021b). While the Psyckes monitors where and when a Medicaid patient receives mental health treatment, Healthix monitors all the patients regardless of insurances and verbal consents are obtained from the patients to access their information in these databases, allowing tracking of those patients who often use the Emergency Department (ED) for care. The area of struggle for this organization is how to monitor patients’ compliance to treatment regimen. Project Purpose The purpose for this DNP project was to evaluate the impact of TMH services in increasing access to mental health services and reducing inpatient hospitalization because of more frequent assessment usage. It was achieved by comparing records of in-person visits period to the records of the emergency TMH guideline period. Increasing access and preventing loss to patient follow up through TMH in outpatient mental health wellness and recovery clinic was a new phenomenon in the PH program. The need for access to mental health treatment grew throughout New York causing a shift to cater for the mental health needs of the community at large. There were multiple systems needed to make the system functional. Health care providers, insurers, patients, and others must work together to ensure high quality, accessible treatment for all who need it (CVN & NCBH, 2018). Telemental Health Services 22 Project Question In an outpatient mental health wellness and recovery clinic in Staten Island New York, does increased patient assessment via the implementation of TMH services lead to increased access to mental health care and reduced hospitalization? Project Objectives The goal of this quality improvement project is to use findings to support the impact of the New York emergency TMH guideline in increasing access to care and reducing inpatient hospitalization. The HRCP developed by the program was useful in evaluating patients’ level of risk and assigning specialized treatment. Screening tools PHQ-9, GAD-7, and C-SSRS were used to ascertain the severity of mental health symptoms. The use of these assessment tools would be the primary source of data once a month or as needed based on patients’ symptom presentation. The secondary source of data was obtained through the interoperable community EHR like Psyckes and Healthix. These secondary sources of data indicated where and when a client obtained care within and/ or outside the program within the last 90 days. These two interoperable EHRs, Psyckes and Healthix, are community web-based applications in New York state designed to obtain real-time patient data throughout the patients’ care used for care coordination and quality improvement (Healthix, 2021; NYSOMH, 2021b). It is important to note that the clinic initiated TMH under the emergency guideline in April 2020 due to the extenuating circumstances posed by the COVID-19. There were adjustments in the proposed implementation due to changes in New York State’s guidance, which included loosening restrictions on duration of session for billing, allowing telephonic services as audio only services, and protection of dual reimbursement where telehealth Telemental Health Services 23 services not covered by Medicare but fully covered by Medicaid were allowed for beneficiaries with both Medicare and Medicaid (Dorin, 2021). These guidelines allowed for the provision of mental health services to continue despite health and safety concerns posed by the COVID-19 pandemic. Data obtained was compared between the office visit/ in-person office visit prepandemic metrics to those of the TMH. The outcome of TMH was evaluated by using the assessment tools, organizational EHR (Awards) and interoperable community EHR. Theoretical Framework The project began with the goal of evaluating the efficiency, effectiveness, and experiences of both the patients and the providers. An effective framework is required in knowing the impact of the emergency TMH services in the clinic and how it impacts the patients, and their providers. The evaluation approach utilized for this topic was the Effectiveness-Efficiency-Equity conceptual framework. The 3 Es were important in how the plan came about. A framework that reduces or eliminates disparity in mental health access as well as evaluate the usefulness of the screening tools would be required explain the impact of the emergency TMH guideline. Each component of this framework was examined and applied to the multiple facets implicit in mental health services to determine the impact of TMH guideline on patients and how it affected their access and outlook on mental health services. Each of the three components in the Efficiency- Effectiveness- Equity conceptual framework seeks to improve the provision of services. Sorato et al. (2020) define efficiency as “a measure of the quality and/or quantity of output for a given level of input.” As applied to healthcare and more specifically PH, it would refer to the reduction of wasted time between seeing patients and providing services, particularly in the case of non-compliant patients who do Telemental Health Services 24 not present for their appointments or those who are often rescheduled for their follow up appointments. It also can refer to the length of time required between requesting and receiving an appointment or travel time. The emergency TMH guidelines allow for the provider to reach the patient over the phone or through secure video services at the patients’ preferred location whereas the previous guidelines dictated that patients had to be present at the physical clinic location to complete a telehealth service. This perpetuated the challenges inherent with in-person care due to the need for patients travel to a physical clinic setting where telehealth would take place. Due to the shortage of psychiatrists and other mental health care providers, the emergency telehealth guidelines eased the burden on providers allowing them to reach more patients with little additional overhead cost. This eliminated the need for full-time clinic patient visits in the traditional clinical setting. Equity is defined by Sorato et al. (2020) as “the absence of avoidable or remediable differences among groups of people whether those groups are defined socially, economically, demographically, or geographically”. In the population involved in this study, there is a high incidence of poverty and reliance on public assistance programs. Resources are limited and negatively impact a person’s ability to attend healthcare appointments or to present to a clinic to engage in telehealth as governed by the previous policy. Limitations include the lack of access to a vehicle, travel costs, or access to affordable childcare. On Staten Island, many providers in the South shore accept primarily private insurances or cash payments, while the providers on the North shore more readily accept Medicare and Medicaid. Access to telehealth can be afforded by out-of-pocket payment whereas those with Medicare and Medicaid insurances must conform to the policy defined for telehealth for a Telemental Health Services 25 session to be billable. Limitations in billable mental health services has created disparity in access to telehealth based on insurance and affordability. Effectiveness is the ability to produce desired results. The effectiveness of TMH can be measured through indicators such as “waiting time to receive mental health treatment, the availability of resources, the cost of access, availability of medical personnel, infrastructure and equipment, and the cost of travel” (Sorato et al., 2020). Effectiveness also relies on the delivery of care, improvement of standard screening scores such as PHQ-9, GAD and preventing hospital admissions. The effectiveness of TMH services can be evident when patients are readily connected with mental health providers instead of using the emergency department (ED) to receive mental health services. When effectiveness is paired with efficiency and equity, a health care system with little waste and improved access can be established and meet patient care goals and achieve better outcomes (Sylvia & Terhaar, 2018). Telemental Health Services 26 Table 2 The benefit of the Effectiveness-Efficiency-Equity conceptual framework Efficiency Equity A measure of quality and/ Absence of disparity or quantity of output 1. Increase access to 1. Reduction of wasted mental health services time between seeing by eliminating direct patients and patient cost. provided services. 2. Reduces financial 2. Help manage burden to patients as patients’ time. cost of access. 3. Ease the burden on 3. Eliminate the socioproviders by economical, reducing the impact geographical, and of shortage of mental demographical impact health workers. of seeking mental 4. Eliminate overhead health treatment cost incurred in clinic visits Effectiveness Ability to produce desired results 1. Reduced waiting time to receive mental health treatment 2. Reduced cost of access 3. Availability of mental health workers, infrastructure, and equipment. 4. Eliminate cost of travelling for appointments. Literature Synthesis Mental health services in the U.S. are insufficient despite more than half of Americans (56%) seeking help (National Council for Mental wellbeing, 2018). Physical limitations, distance, and time are major obstacles to access mental health services (Haghnia et al., 2019). Other limitations to receiving mental health services include inadequate specialty services, maldistribution issues, and inadequate funding (Mace et al., 2018). These challenges have plagued the behavioral health services and created the demand and need for an innovative approach of providing care to individuals (Mace et al., 2018). Community-based agencies are uniquely positioned to fill this gap in care; providing individualized care across settings, coordinating services across agencies, and assisting individuals to transition at various stages of Telemental Health Services 27 treatment (Williams, 2020). TMH can be used to further reduce the treatment gap and overcome the challenges in accessing mental health services. Reimbursement parity is one of the biggest barriers to TMH expansion (Warren & Smalley, 2020). Lack of parity results from different rates for in-person office visits compared to telehealth. In-person office visits are reimbursed fully for mental health services compared to telehealth services. Currently, among the U.S. states and territories, there are twenty-three with telehealth wavers, twenty-seven without wavers or that have closed wavers, and nineteen with long-term or permanent interstate telemedicine including Guam, Commonwealth of the Northern Mariana Islands, Puerto Rico, and the U.S. Virgin Islands (Federation of State Medical Boards, 2022). Due to the regulatory waiver, Executive Order 202 section 596 of Title 14, issued by the New York state governor, the term TMH was temporarily expanded for Medicaid-reimbursable services to include telephonic and/ or video, technology commonly available on smart phones and other devices (Cuomo et al., 2020). Less than 10% of the U.S. population used telehealth for a clinical encounter and only 18% of physicians provided such services prior to COVID-19 (Warren & Smalley, 2020). Limited diagnosis, usually requiring physical comorbidities, would cover home-based telehealth services (Warren & Smalley, 2020). For instance, radiological and critical care services were reimbursable by telehealth when the patient or provider was at home. PH, a community-based organization, did not apply for the TMH hub site license pre-pandemic despite eligibility due to the difference in reimbursement fees. The hub site (provider location) also known as the distant site is the consulting site where the provider is seeing the patient at a distance or consulting with the patient’s provider, while the spoke site (patient location) also known as the originating site is Telemental Health Services 28 where the patient or patient’s physician is located during the telehealth encounter (Medicare Learning Network, 2021). Smartphones are helping to bridge the disparity in access to treatment. While Black Americans and Hispanics own smartphones at similar shares to whites, roughly 82% Whites report owning a desktop or laptop computer compared to 58% of Black people and 57% of Hispanics (Perrin & Turner, 2019). There are still concerns regarding disparities in access to mental health services for those in rural, suburban, and urban communities. Technological infrastructures like owning a desktop or laptop computer in these communities would ensure improved access to TMH services. Evidence Search The evidence obtained for this review was from a computerized search of the library databases. Health science databases reviewed include Cumulative Index to Nursing and Allied Health literature (CINAHL), Centers for Disease Control and Prevention (CDC), PubMed, Academic Search Premier (EBSCO), New York State Office of Mental Health (NYSOMH), and the American Psychological Association (APA PsychInfo). The keywords used to retrieve documents were “telepsychiatry,” “telemedicine,” “teleconferencing,” “telepsychology,” “psychiatric clinics,” “access to mental health,” and “hospital readmission.” Alternate terms include “telehealth,” “telemental health,” “community mental health centers,” “mental health programs,” “walk in clinics,” “increased access to mental health,” “and decreasing hospitalization readmission.” Limiters included last 10 years, English, primary research (clinical study, clinical trial, clinical controlled trial, meta-analysis, randomized controlled trial). To determine appropriateness, titles and abstracts of the results were screened. Out of the 240 articles resulted, ten studies were deemed appropriate via literature review and appraisal. Telemental Health Services 29 Comprehensive Appraisal of Evidence Knowledge of Telemental Health Telemental health is the use of two-way interactive audio and video equipment used in real time, to support clinical psychiatric care (New York Office of Mental Health, 2020). On July 3, 2019, the New York State Office of Mental Health (NYSOMH), changed the term “telepsychiatry” to “telemental health” to demonstrate their broader use of technology and expansion of “telemental health” services delivery (NYSOMH, 2020). This expansion enabled additional providers: psychologists, social workers, mental health counselors, marriage and family therapists, creative arts therapists, and psychoanalysts (NYSOMH, 2020) to deliver psychiatric care remotely. To fully understand the need for TMH services, it is necessary to understand the nature of mental disorders, costs, available resources, and treatment (Eisman et al, 2020) Most mental disorders are a result of a combination of several “biopsychosocial” factors rather than a single factor (Bashshur et al., 2016). According to the National Institute of Mental Health (2021), mental illnesses include those that are diagnosable currently or within the past year lasting a sufficient duration to meet diagnostic criteria within the Diagnostic and Statistical manual of Mental Disorders (DSM-V) and excludes a developmental and or substance use disorder. Age 14 is the beginning of 50 % of most lifetime mental illness and 75% have manifested by age 24 (National Alliance on mental Illness (NAMI), 2021). Mental disorders cause substantial distress or disruption in social, occupational, and/or family relationships, and people with such disorders fail to meet important obligations in everyday life, work, or school (Bashshur et al., 2016). Mental disorders are common, and it is difficult to determine the prevalence of mental disorders in any population (Bashshur et al., 2016). Among U.S. adults, Telemental Health Services 30 21% (52.9 million people) experienced mental illness in 2020, 5.6% (14.2 million people) experienced serious mental illness, and 6.7% (17 million people) experienced a co-occurring substance use disorder and mental illness in 2020 (NAMI, 2021). In 2016, there were 16.5% (7.7 million people) of U.S. youth aged 6 to 17 that experienced a mental health disorder (NAMI, 2021). In 2020, more than 12 million U.S adults had serious thoughts of suicide, 1 in 5 experienced mental illness, 1 in 20 experienced serious mental illness, and 1 in 15 experienced substance use disorder and mental illness. There are an estimated 12 million emergency department visits (1 out of every 8) by U.S. adults related to mental illness and substance use disorders (NAMI, 2021). In 2020, 17 million U.S. adults (32.1%) with mental illness experienced a concurrent substance use disorder (NAMI, 2021). Students aged 6 to 17 with mental, emotional, or behavioral concerns are three times more likely to repeat a grade and high school students with significant symptoms of depression are more than twice as likely to drop out of compared to their peers (NAMI 2021). The unemployment rate among U.S. adults with mental illness is 6.4% compared to 5.1% of those without (NAMI, 2021). There are at least 8.4 million people in the U.S. providing an average of 32 hours per week of unpaid care to adults with mental or emotional health issues (NAMI, 2021). Among the people experiencing homelessness in the U.S., 20.8% have serious mental illness and 37% of the adults incarcerated in the state and federal prison systems have a diagnosed mental illness (NAMI, 2021). About 2 in 5 incarcerated people have mental illness and an estimated 4,000 people with serious mental illness are held in solitary confinement in U.S. prisons (NAMI, 2021). One in four people shot and killed in the U.S. by police officers between 2015 and 2020 had mental health conditions (NAMI, 2021). Suicide is the second leading cause of death among people aged 10 to 34 and the 10th leading cause of death in the Telemental Health Services 31 U.S. (NAMI, 2021). Transgender adults are twelve times more likely to attempt suicide than the general population (NAMI, 2021). Lesbian, gay, and bisexual youth are four times more likely to attempt suicide than straight youth, and 78 % of people who die by suicide are male (NAMI, 2021). Serious mental illness causes $193.2 billion in lost earnings in the U.S. economy each year and mood disorders are the most common cause of hospitalization for all people in the U.S. under the age 45 (NAMI, 2021). The global cost of depression and anxiety is $1 trillion in lost productivity each year, and depression is the leading cause of disability worldwide (NAMI, 2022). The global cost of mental health disorders was estimated at $2.5 trillion per year in poor health and reduced productivity in 2010 with a projected cost increase to $6 trillion by 2030 (The Lancet Global health, 2020). In 2020, 46.2% of U.S. adults with mental illness and 64.5% of those with serious mental illness received treatment (NAMI, 2021). Among U.S. adults with mental illness, the annual rate of treatment indicates that females (51.2%) received mental health services more than males (37.4%); lesbian, gay and bisexual (54.3%) more than non-Hispanic White (51.8%), more than non-Hispanic Black or African American (37.1%), and more than Hispanic or Latino (35.1%) (NAMI, 2021). The year 2020 was typified by uncertainty and loss brought about by the Coronavirus (COVID-19) pandemic causing an impact on mental health of individuals around the world. Manifest as Pronounced fear and anxiety about COVID-19, emotional distress resulting from illness, bereavement, unemployment, income loss, and loneliness due to social isolation (The Lancet Global health, 2020). Among adults in the U.S. in 2020, 1 in 5 report that the pandemic had a significant impact on their mental health, 45% of those adults have mental health illness Telemental Health Services 32 and 55% serious mental illness (NAMI, 2021). In 2020, among U.S. adults who received mental health services, 17.7 million experienced delays or cancellation in appointments, 7.3 million experienced delays in getting prescriptions and 4.9 million were unable to access needed care (NAMI, 2021). Approximately 11% of adults with mental illness are without insurance coverage, 134 million people were living in a designated mental health professional shortage area, and 55% of U.S. counties did not have a single practicing psychiatrist in 2020 (NAMI, 2021). With the current shortage in the number of psychiatrists, and the predicted future shortage of 15,600 to 16,060 by 2025, other mental health professionals are increasingly assuming primary treatment roles (Harrar, 2021). This includes psychologists, nurse practitioners, mental health counselors, and psychiatric social workers. As the number of inpatient psychiatric beds decreases, the shortage of psychiatrists along with health insurance limitations of coverage, determining which provider can be seen or the amount of help that can be received, has led to long wait times for appointment, shorter doctor visits for psychiatric issues, less monitoring of patients on prescriptions, and long wait times in the hospital (Harrar, 2021). The trends in mental health treatment along with shortage of mental health personnel support the need for TMH services. Effects of telehealth health in increasing access to mental health services. A case study report within a community mental health center in suburban Illinois describes the planning, implementation, and evaluation of a telepsychiatry program (Mahmoud et al., 2020). Identified telemental access barriers included clinician shortage, patient and staff resistance, reimbursement, funding, technological challenges, infrastructure, patient privacy, concerns about cyber security, and misinformation and misconception of telepsychiatry (Mahmoud et al., 2020). A variety of access challenges were mitigated by defining clinician Telemental Health Services 33 roles and developing clear workflow, increasing workers’ buy-in on telehealth by providing training, assessing patients’ preferences as well as providing on-site alternatives, and enhancing technology preparedness by adding a hotspot in case of internet failure (Mahmoud et al., 2020). A double-blinded randomized control trial (RCT) examined the use of multimedia applications (IMO voice calls, text messaging, Telegram, and Skype) for follow up treatment of Iranian war veterans with PTSD (Haghnia et al., 2019). A significant positive relationship exists between use and patient satisfaction. Additionally, there were benefits in reduced treatment costs and completion rates. Mace et al. (2018) conducted a mixed methods review of data collected from 329 behavioral health organizations to investigate the utilization patterns and barriers of telehealth among behavioral health providers. The responding organizations represented all fifty states, the District of Columbia and Puerto Rico. Half of the respondents (n=153; 47%) used telehealth specifically for behavioral health services and the telehealth services most used by behavioral health providers is videoconferencing (40%) followed by telephone (11%) (Mace et al., 2018). Among the behavioral services respondents, psychiatrists are the largest group of users (78%) of telehealth followed by mental health counselors (33%) (Mace et al., 2018). The study also revealed the three most cited barriers to telehealth as; financing, including the lack of reimbursement by insurers (59%), cost of implementing the infrastructure (56%), and the cost of maintaining telehealth (56%) (Mace et al., 2018). The cost of teleconferencing technology continues to decline because of innovations in the telecommunication media making TMH services more cost effective (Guaiana et al, 2021). Other barriers highlighted by respondents include the lack of leadership regarding telehealth, staffing challenges, client challenges, and technical assistance barriers. The study also explored the primary reasons for Telemental Health Services 34 implementing telehealth services. Study respondents agreed that telehealth is important to an organization for reaching more patients, as it allows for the organizations to reach patients over a geographical distance, and permits a more efficient workflow (Mace et al., 2018). However, there are limitations to the generalizability of this study because of the sample size. Even though there was representation from all the fifty states, there was unequal representation in the responses to the electronic survey (i.e., NY 29.9%, PA 16.5% and six states/ area with only one response) (Mace et al., 208). Applicability of individual state-based findings to other states may be challenging. Also, the sample for qualitative interviews was limited to only five states compared to the RCT. (Mace et al., 2018). An observational cross-sectional study by Cantor et al., 2021, reported that among the 8860-outpatient mental health treatment facilities in the United States, 3848 (43.4%) offered telehealth services at the onset of the COVID-19 pandemic on April 16, 2020. This data was from the National Mental health Services Survey (N-MHSS) of 2019 with response rate of 91% (Cantor et al., 2021). The lack of telehealth access indicates that more needs to be done to enhance telehealth policies and ensure continuous access to treatment even through another pandemic. This study reveals limited telehealth infrastructure to meet the expected increase in demand for mental health care during the pandemic. limitations of this study include approximately 9.3% of the eligible facilities did not respond to the 2019 N-MHSS survey, the behavioral treatment services locator did not contain treatment capacity or quality, and the data collected did not reflect changes in telehealth services after the March 13, 2020, declared national emergency (Cantor et al., 2021). Effect of Telehealth in Reducing Hospitalization Telemental Health Services 35 Brearly et al. (2020), investigated the implementation and outcome of a clinical quality improvement project, Allied Transitional Telehealth Encounters post-inpatient Discharge (ATTEND), which targeted veterans transitioning from inpatient to outpatient mental healthcare. Here, Veterans, who met criteria, were provided 4G Android tablets with which they were contacted via Clinical Video Telehealth (CVT) by a pharmacist to discuss their medications from any location and at their convenience. Noted factors for a decrease in standard outpatient followup care were housing instability, transportation problems, distance from the clinic, forgetting to attend appointments, driving related anxiety due to posttraumatic stress disorder (PTSD), poor referral communication, and difficulty arranging time off work (Brearly et al., 2020). Post inpatient mental health follow-up provided by the ATTEND clinic, helped veterans to overcome barriers to care, increased patient satisfaction through prompt access, reduced the likelihood of inpatient readmission, and lowered recidivism rate (Brearly et al., 2020). The limitation of this implementation study model is the sample size of twenty patients. Although the authors denied conflict of interest, this study was supported by the DVA Office of Academic Affiliations Advanced Fellowship Program in Mental illness, Research, and Treatment and the Mid-Atlantic Mental Illness Research, Education, and Clinical Center (Brearly et al., 2020). Strengths of Evidence The adoption of TMH services allows providers to increase continuity of care, extend access beyond normal clinic hours, reduce patient travel burden, and help overcome clinician shortages in rural and underserved populations which helps practices to enhance patient wellness, improve efficiency, provide higher quality of care, and increase patient satisfaction (American Medical Association [AMA], 2020). TMH has ensured that patients with mental health problems, like depression, anxiety, attention-deficit/hyperactivity disorder (ADHD), Telemental Health Services 36 bipolar disorder, schizophrenia, and other serious mental health conditions, who are adjusting to new medications maintain contact with their mental health providers for their routine follow-ups. It has enabled patients to participate in their routine virtual psychotherapy appointments. Transportation barriers can be overcome when patients are able to access care from their convenient locations. Weakness of Evidence Although there are benefits to TMH implementation, there are also challenges to adopting this innovative technology. Some of the barriers include inconsistent reimbursement, concerns over security, privacy, and confidentiality, concerns of providing safe and effective care, and lack of evidence about impact on health care costs, utilization, or outcomes (AMA, 2020). Gaps and Limitations The resounding theme in these published studies discussed above is the correlation of TMH with increased access to mental health services. There is increase in provider and patient satisfaction while using TMH services in an industry that is overburdened by shortage of staff. Though there is increased access of patients to their mental health providers for alleviation of symptoms and preventing mental health crisis, there is little evidence discussed in the studies about decreasing hospitalization. There is need for more current academic research studies to evaluate the impact of TMH on reducing patients’ hospitalization readmission. There are not enough studies that sufficiently examine the effect of telehealth on decreasing hospitalization. There is the limitation to duplicability of studies using some of these studies because of the small sample size. Telemental Health Services 37 There are gaps in the use of TMH services. These studies have highlighted the differences between states in their adoption of specific TMH policies. This makes the duplicability of study difficult. A standardized national TMH policy would expand TMH access at the end of the COVID-19 pandemic for regular use and maintain the infrastructures in place should there be another pandemic. There is need to incentivize the training and recruitment of mental health providers to help reduce the personnel shortage. METHOD This QI project aims to assess the ability of increased assessment via TMH to increase access to care and decrease hospitalizations. This will be assessed by evaluating the impact of increased monitoring and evaluation of anxiety, depression, and suicidality on hospitalization rates and. The methodology section focused on project design, the model for implementation, setting and stakeholders, planning the intervention, ethical consideration, data collection and analysis of data collected. Project design Quality improvement in health care involves the analysis of performance in a system and the development of a system designed to systematically improve care. It is a process of systematic and continuous actions, which cause measurable improvements in the health status of patient populations and health care services (Hickey & Brosnan, 2017). The goal here is to utilize instruments to assess well-being and mental health status and their impact of health care delivery and ensuring patient safety. This QI project aims to assess the impact of increased assessment while using the current New York State emergency TMH guidance. The current effects of TMH are being experienced in real time while using the emergency guidance as a policy guide. However, the implementation Telemental Health Services 38 of this QI project was designed via gap analysis to make course corrections by organizational measures by looking at what was missing and determining ways to improve those faults. Areas for correction include monitoring of patients’ remission of symptoms by frequently utilizing the assessment tools, reducing patient loss to follow up by increasing contact with patients, monitoring the effect of safety plan implementations by using the HRCP, and reducing inpatient hospitalization. This vulnerable population includes patients over the age of eighteen with mental illness, some of whom have co-occurring substance use disorder. Ninety days retrospective data, for three years prior to the roll out, was used as a comparison. The data in the retrospective timeline is compared to the period of TMH implementation after the Institutional Review Board (IRB) approval from September 15, 2021, until December 13, 2021. This QI project utilizes a retrospective chart review of assessment tools. The degree of change in mental health symptoms were measured by the pre and post interventions for both new and established patients. Patients’ mental health interventions varied from patient to patient due to the degree of their symptoms. In this QI project, interventions included easy access of patients to their providers via telehealth, frequent therapy sessions based on assessed needs and/ or medication adjustments determined by the HRCP. The team reached a consensus to implement screening tools upon intake and at least once a month during the implementation phase of data collection. The team includes the one clerk, one intake specialist, four licensed Social Workers (LMSW) and two fee-for-service LMSW, two Mental Health Counselors (MHC), two Nurse Practitioners in psychiatry (NPPs), a per diem internal medicine physician, two substance abuse counselors, two billing personnel, the Recovery Program director, and the Associate Area Director also filling in the role of Wellness Telemental Health Services 39 Program director. The implementation timeline is from September 15, 2021, until December 15, 2021. The team discussed variables which could potentially impact the TMH services include varying degree of serious mental illnesses, frequency of therapy already available, buy in of the staff to assess patients by using the assessment tools, appointment kept by patients for their therapy sessions and medication management sessions, and access to good working electronic devices. We agreed that more frequent use of screening tools than once a month could potentially skew the result of data collected. Model for Implementation The Plan-DO-Study-Act (PDSA) is a suitable implementation model to assess the stages of change that is being implemented (AHRQ, 2020). Using the four-cycle model of testing change/ improvement, the PDSA cycle develops a plan to evaluate the change (Plan), carrying out the test (Do), observing and learning from the consequences (Study), and determining what modifications should be made to the test (Act) (IHI, 2021). We planned to use the mental health assessment tools PHQ-9, GAD, and C-SSRS to assess for rate of symptoms of depression, anxiety, and suicidality, respectively. The rate of patient hospitalization was assessed by checking the community based PSYCKES and Healthix EHRs. To execute the plan, the PHQ-9, GAD, and C-SSRS were administered upon intake and at least every 4 weeks for up to 90 days. After 90 days, gather data from the PSYCKES and Healthix EHRs both pre and post intervention. The assessment tools data were collected by the MHC and LMSW in the Wellness clinic upon new patient intake and at least once every 90 days. The 90 days was the program policy for completing a treatment plan for patients to accommodate non-Medicaid patients with commercial insurances. Treatment planning is an ongoing process of assessing mental health Telemental Health Services 40 status and needs of the patient, establishing his/ her treatment and rehabilitative goals, and determining what services may be provided by the clinic to assist the patient in accomplishing those goals (NYSOMH, 2021a). According the NYSOMH (2021a), the initial treatment plan must be completed thirty calendar days from the admission date and reviewed annually after admission or from the most recently completed treatment plan review unless the services are covered by a third-party payer with a different requirement. Treatment plan is reviewed when there are significant clinical changes like behavioral or medical diagnosis, changes in risk level, increased or new symptoms, alterations in functioning, stressors, needs, circumstances, and reviews may be required more frequently as determined by the treating clinician, patient, or family member (NYSOMH, 2021a). The implementation phase of this model is the “Do” phase. The plan was executed on September 15, 2021. In this QI project, a majority of the TMH services provided were through the telephone. This was the most readily available resource for the population under study. There were cases in which ZOOM videoconferencing was utilized. There were other instances in which patients received hybrid services of TMH and in-person visits. Patients with worsening symptoms or high risk, as determined through use of the assessment tools, received increased contact to their mental health providers. This increased access continued until there was improvement as evidenced by low scores of the screening tools or resolution of symptoms. There is increased number of new patient intakes who would be screened using the assessment tools, both in-person visits and TMH. The study model of the PDSA in the implementation of TMH can better be understood using a fishbone diagram to capture the Effectiveness-Efficiency-Equity conceptual framework discussed above. Telemental Health Services 41 Figure 1 Telemental health fishbone diagram. (Adapted from Chang (2015). No permission needed to use and adapt. Writer modified to focus on topic). Human Environment System Service Provider Organization Comfort w/ work flow Leadership Expertise on ICT 3rd party payers Clinical information systems Education & Training Work flow re-engineering Location / Travel time Patient awareness Disease characteristics ICT skills & knowledge Reliability of technology Patient/ client Social norms and values (Stigma) Training & support OUTCOME • Decreased hospitalization • Cost effective • Increase access TO ___ health Data quality Medical liability Transmission Storage ICT equipment Interoperability System Speed Medical cost (out-of-pocket) User habit Insurance fee schedule ICT infrastructure Budget Resistance to change Routine psychotherapy opportunity, routine follow ups with patient’s who are adjusting to new medications Reimbursement Change management Comfort with patient communication Comfort with Provider Interaction Society Organizational Culture Privacy & security rules • Enhance patient wellness • Provide higher quality of care Practice Certification & license Interface standards Information Security User Interface Technology Government authority Rules /policies To completely evaluate the “Act” phase of this implementation model, the implementation data is compared to data obtained from the pre-pandemic, in-person visits period of 2019. The total patient census at the given period would be compared. The frequency of patient contact and assessment would be compared using service contact logs. Hospitalization rate would be compared at both periods and length of hospitalization stays would be analyzed. Settings and Stakeholders The QI project was implemented in Project Hospitality (PH), an outpatient mental health Wellness and Recovery Clinic in Staten Island, Richmond County of New York City. The larger program consists of inpatient respite and long-term substance abuse program, housing programs, shelters, case management and youth services. This outpatient Wellness and Recovery clinic is in Telemental Health Services 42 a county with a population of 476,143 in 2019 (United States Census Bureau, 2019). About 59.6% of the county population were listed as White non-Hispanic, 11.6% as Black or African American, 18.6% as Hispanic or Latino, and 10.9% as Asian alone (United States Census Bureau, 2019). The clinic was staffed with a clerk, an intake specialist, two billing personnel each for recovery and wellness, two Licensed Mental Health Counselors (LMHC), four Licensed Master Social workers (LMSW), two of them are fee-for-service, two Credentialed Alcoholism and Substance Abuse counselors (CASAC), two Nurse practitioners in psychiatry (NPPs), a per diem internal medicine physician, Program Director in wellness, and Program Director in Recovery. This QI project was led by a nurse practitioner employed at the site as partial fulfillment of a Doctor of Nursing Practice school project. The other NPP is the clinical expert who helped in rolling out the TMH services within the organization. Common stakeholders in healthcare improvement include, but not limited to patients, clinicians, managers, executives, clinical assistants, and payers. Planning the Intervention The assessment tools used as intervention for the implementation of this QI project include the PHQ-9, GAD, C-SSRS, and the community-based EHRs (Psyckes and Healthix). All of these were used to assess patients’ status and frequency of hospitalization readmissions. After a careful internet search, it was identified that the PHQ-9, GAD, and C-SSRS are all publicly available and no written consent required (Wood & Gupta, 2017). The Psyckes and Healthix required a written consent until the pandemic when a waiver was issued for obtaining verbal consent, allowing for verbal consents to be entered into the PH’s EHR, Awards. Unlike the Telemental Health Services 43 Psyckes, Healthix consent is traditionally obtained via verbal consent and entered electronically by the intake specialist or the mental health workers. The primary areas of knowing the effectiveness of the emergency TMH guideline is by initiating a QI initiative to demonstrate effectiveness to support continued adoption of TMH services during the pandemic and to provide support for a permanent adoption of the guideline by showing effectiveness through reduction in recidivism, decreased inpatient hospitalization, and improvement in patient outcomes. Recidivism rate is monitored using the High-Risk Criteria Protocol (HRCP) adopted by the clinic and expanded from the New York State Office of Mental Health Clinic Standards of care anchor (NYSOMH, 2021a). The “High Risk” criteria include anyone that has had suicide attempt within the past six months, anyone hospitalized for psychiatric reasons for the past six months, anyone who has been to the emergency room due to psychiatric reasons within the past three months, anyone on AOT, those experiencing suicidal and/ or homicidal ideation with a plan and deviating from their normal baseline within the past three months, anyone that experienced an overdose within the past six months, anyone reported to be engaged in violent behaviors or committed a violent crime and has been released from prison within the past six months or is currently on parole or probation for violent crime, anyone actively using drugs and/ or alcohol and exhibiting behaviors perceived to be a potential danger to self or others, and anyone that is the victim of a domestic violence situation where the perpetrator continues to have access to the victim (Project Hospitality, n.d.). The HRCP indicates that the clinical team should discuss and assess any patient put on the high-risk list at the team meetings once a week. Patients on the high-risk list who missed an appointment must be contacted via phone to ascertain the reason and rescheduled. Any Telemental Health Services 44 patient reporting suicidal or homicidal ideation must go into the HRCP and CSSRS administered at every session. If an Assisted Outpatient Treatment (AOT) patient, who is court ordered to take medications, disappears, the legal system must be notified immediately, and lack of attendance must be reported. Improvement of symptoms was monitored using the assessment tools PHQ-9, GAD-7, and C-SSRS. While the PHQ-9 assessment determines severity of depression of patients, the GAD-7 tool is used to measure the level of general anxiety disorder, and the C-SSRS is a systematic suicidal screening tool. This three-assessment metric was completed during initial patient intake and every 30 days. Interventions to increase patient visits were initiated when the PHQ-9 score level reached ten and above, the GAD score of ten or above, or moderate to high suicidal ideation and behavior on the C-SSRS. The PHQ-9, GAD-7 and the C-SSRS are valid and reliable in detecting depression, anxiety, and suicidal risks, respectively (Wood & Gupta, 2017). The NPPs and other providers were notified by any mental health worker who assessed the patient and identified these scores. Alerting the team to call for a case conference of treating providers allows for modification of treatment plans in the form of medication management adjustment and/ or increase in frequency of therapy. Another way of tracking the effectiveness of increased access is by tracking hospitalization through Psyckes and Healthix community based electronic health records (EHRs). Psyckes is a HIPAA-compliant web-based application designed to support clinical decision making, care coordination, and quality improvement in New York State (NYSOMH, 2021b). It is a community database that tracks patients’ care and shows the different programs in which the patient receives care. Consents are sought from the patients to access their Telemental Health Services 45 information in this database. Those patients who often use the Emergency Department (ER) for care can successfully be tracked. Participants and Recruitment A convenience sample aged 18 years old and older, English speaking, with mental health illness and/ or SUD attending PH were utilized. The exclusion criteria include patients who are younger than 18 years old, those adult patients with serious mental health illness who are not of sound mind, those unable to make decisions, and non-English speaking. Consents and Ethical Considerations The investigator requested and received permission from the Director of Policy and Compliance of the Project Hospitality organization. There is no IRB approval requirement at the organization. Also, the investigator obtained university IRB approval. This QI project involves the use of the current emergency TMH guideline to improve the mental health outcome of this population. Hence, there is no experimentation involved and there is no increased potential for direct patient risk. The data needed for this QI project was accessed or obtained through retrospective data collection from patients’ EHR and the electronic databases (Awards). The ethical concerns involved in this project includes patient safety, confidentiality, respect for persons, autonomy, and justice to the participants in this project (Liem et al, 2020). These concerns are mitigated by using a flash drive and storage space with a lock for locking up the devise in the office. A request for a password secured laptop with Virtual Private Network (VPN) set up by the System Administrator was granted by PH and provided for this QI project. Consents obtained in the physical chart or the PH’s EHR, Awards, are confirmed by the investigator before accessing patient information in the Psyckes and Healthix databases. Telemental Health Services 46 Data Collection Once the IRB was approved, the Psyckes and Healthix EHRs were reviewed at the onset of data collection and at the end of the 90 days period. Team meetings were held weekly to review the high-risk list, discuss any added information, review the treatment plan, and update the list. The assessment tools were obtained from the organization’s EHR to assess depression, anxiety, and suicidal scales using the PHQ-9, GAD-7, and C-SSRS, respectively. Data for PHQ9, GAD, and the C-SSRS were obtained every 4 weeks for a period of 90 days from PH’s EHR, Awards. Information obtained when accessing the EHR included diagnosis, medical diagnoses, reasons for hospitalization, medical hospitalizations, gender, age, and the assessment tool scale. Patients’ names were deidentified. Other data obtained from the high-risk protocol spreadsheet included risk levels, proposed plans, and case review dates. The data obtained was reviewed by the investigator to assess inclusion criteria, saved in flash drive, and locked at the program site in the office where only the investigator could access it. Additionally, a password secured laptop with the Virtual Private Network (VPN) was set up by the System Administrator of PH. Data Analysis The outcome of this QI project was analyzed by comparing the metrics of patients in the control 90-day period in 2019 and those from the implementation period of 2021. To measure access to mental health services, the census of active patients during in-person visits pre-implementation, was compared to the number of active TMH visits during implementation of 2021. Resolution of symptoms was determined by patients’ reports and measured by the screening tools (PHQ-9, GAD-7, and C-SSRS) upon intake and every six months, patients referred for the high-risk protocol. There was a change in the implementation of the screening Telemental Health Services 47 tools because of the change in guidance proposed by the NYSOMH to implement treatment plan upon intake and annually unless clinically necessary due to changes in mental health symptoms, hospitalization and/ or deterioration of health. The change in guidance was compensated by implementing the assessment tools upon intake, every six months and as needed depending on patients’ presentations. Access was also measured in the overall number of mental health service types provided within each 30-day period to indicate the form of patient-provider interactions noted in the contact log report. According to the NYSOMH (2021a), there are ten mental health clinic services that must be available and offered at any mental health clinic licensed by the NYSOMH. The required services fall under the following categories: assessments, therapies, injections, and enhanced services (Table 1). While assessments involve interaction between a patient and a mental health provider including initial assessment and psychiatric assessments, therapies include individual psychotherapy, family/ collateral psychotherapy, group psychotherapy, and psychotropic medication treatment (NYSOMH, 2021a). Injection services involve administration of injectable psychotropic medication administration with education and monitoring. Enhanced services include crisis intervention and complex care management (NYSOMH, 2021a). There are eight optional services which include developmental testing, psychological testing, psychiatric consultation, health physical, health monitoring, injectable psychotropic medication administration for clinics serving children, injectable psychotropic administration with education and monitoring for clinics serving children, and enhanced services for smoking cessation treatment for individuals and group (NYSOMH, 2021a). Patients placed on the HRCP were monitored frequently, at every patient contact with their provider, weekly or biweekly, using the screening tools (PHQ-9, GAD-7, and C-SSRS) Telemental Health Services 48 to monitor severity of symptoms, improvements, and viability of individualized treatment plan. While the patients were under the HRCP, the average length of stay in the HRCP pathway was measured along with revisions in treatment plan. Individualized treatment plans were revised in the form of frequency for therapy/ medication management and/ or changes to type of medication management. The hospitalization rate was determined by the total number of psychiatric and substance abuse hospitalizations within the 90 days in the control period of 2019 and that of the study period of 2021. The length of hospital stay was evaluated in the control group and study group and differentiated between inpatient hospitalization (> 24 hours stay) and ED visits (< 24 hours stay). The reason for hospitalization was also evaluated to affirm psychiatric and substance abuse related conditions for the purpose of this QI project. RESULTS The intervention was the ongoing implementation of telemental health and patient assessment using the screening tools. This QI project used the convenience sample of patients who are both in the Recovery and Wellness programs of the clinic and who are 18 years and older. The PICOT questions whether increased patient access to mental health services and increased assessment using the screening tools, PHQ-9, GAD-7, and C-SSRS via TMH enabled decreased frequency of hospitalization for patients within the clinic. The data for the screening tools were to be accessed monthly. The intake process in these programs involved the assignment of risk levels based on screening in pre and post implementation as well as implementation of resources which are specific for individualized patient care and based on these assessments. Telemental Health Services 49 During planning, there was a consensus by the team of providers to implement the screening tools every four weeks for each patient as part of the NYSOMH requirement to update treatment plans every six months. The program took it further to assess the treatment plans every three months. Updates by the NYSOMH then changed requirement to perform screening using assessment tools at intake and every 12 months, and with change in status. The team determined to use tools at intake, every six months, and with change in status, and to continue with the HRCP which was initiated before TMH. Figure 2 Screening tool implementation process Intake Specialist High Risk Criteria Protocol OMH Guideline (PHQ-9, GAD-7, C-SSRS) Within 30 days of intake Every 6 months As needed upon contact (PHQ-9, GAD-7, CSSRS) 1st Intake 2nd Intake LMSW LMSW (LMSW, (LMSW, LMHC LMHC LMHC) Patients placed on the high-risk criteria protocol have a GAD-7 score greater than 10, or LMHC) PHQ-9 score greater than 10 and/ or have had suicidal behavior within previous three months, Telemental Health Services 50 and/ or recent psychiatric hospitalization. Other criteria of suicidality, aggression, sexual misconduct, victims of domestic violence, and assisted patient treatment for court mandated patients are some examples of diagnostic categories of patients placed on the high-risk protocol. The risk levels instituted included level 1 through level 4. While the low risk (level 1) involved patients who wished to die without plan, intent, or suicidal behavior, Level 2 Moderate risk patients are those with suicidal ideation more than a month ago without a plan, intent, or suicidal behavior. For the low risk level 1 patients, the CSSRS was completed every 30 days and the moderate risk level 2 patients had a CSSRS done biweekly. Table 3 High Risk Criteria Protocol Level 1 Low Risk Patients who wished to die without plan, intent, or suicidal behavior Level 2 Moderate Risk Patients with suicidal ideation more than a month ago without a plan, intent, or suicidal behavior. Level 3 Moderate Risk Patients with suicidal behavior more than 3 months ago Frequency of CSSRS Every 30 days Biweekly Disposition/ plan Address individualized patient needs, Case conferencing among providers to address symptoms, weekly psychotherapy Address Individualized patient needs, Case conferencing among providers to address Close monitoring with the CSSRS done with every patient contact. Case conferencing among providers. Referral to Health Homes for home checks, therapy twice a week Criteria Level 4 High Risk Patients with suicidal ideation with intent or intent with a plan in the past month, and recent psychiatric hospitalization. CSSRS done on every contact Referred to the NPP for biweekly assessment and the patients’ family members are involved as collateral protective factor. Telemental Health Services 51 symptoms, weekly therapy The level 3 moderate risk involved those patients with suicidal behavior more than 3 months ago and they required close monitoring with the C-SSRS done with every patient contact. Level 4 High risk involved patients with suicidal ideation with intent or intent with a plan in the past month, and recent psychiatric hospitalization. The level 4 high risk patients had their C-SSRS done on every contact, was referred to the NPP for biweekly assessment, and the patients’ family members are involved as collateral protective factor. Systematic C-SSRS screening while using the HRCP helped identify patients who were at risk for suicide in the general clinic population and focused services on patients with the most need (Pumariega et al, 2020). The initial plan was to obtain assessment data every 30 days, from the population sample, during the intervention period. Data collected in the pre and post intervention period but did not depict the overall sample but a sub-group of the sample. There were challenges encountered in obtaining the necessary data for this QI project. The physical charts and Awards EHR were reviewed for Psyckes and Healthix consents to enable the researcher entry into the interoperative EHR to verify hospitalizations. While entering the hospitalizations from the Psyckes and Healthix databases into Awards EMR, there was omission of pertinent mental health data such as the reason for hospitalization or visiting the hospital emergency department by the personnel entering the patients’ data. There were various categories of the C-SSRS assessment tool in Awards and there are four separate places to obtain reports. These categories were listed as C- Telemental Health Services 52 SSRS, C-SSRS, C-SSRS-Ongoing Weekly, and C-SSRS Intake/ Initial. There were no data in the first two categories and minimal data in the C-SSRS- Ongoing Weekly. C-SSRS Intake/ Initial category contained more data than the other categories and sorting through the results was confusing. Outcome TMH enabled PH to increased patient contact, mental health assessment, and decrease hospital length of stay. The number on the overall patient census report increased by 21% after the implementation of the TMH Services. However, when accounting for the number of patient contacts with mental health workers, there was a 25% decrease in census per individual contacts with providers after the implementation of the TMH Services. The number of patient contacts with their providers increased by 61% after the implementation of the TMH Services. This indicates that increased access to mental health services which allowed mental health providers to be easily accessible by their patients as indicated by the service types provided at the TMH period. The distribution of service types was different between (pre and post) time periods with the largest increase within “Individual psychotherapy” (increased by >1000 contacts, an 8% increase). The number of psychiatric and substance abuse hospitalizations increased by 83% after the implementation of the TMH Services. However, when accounting for the number of contacts, the number of substance abuse hospitalizations per contact were not different between time periods. The percent with >1 day length of stay was lower after the implementation of the TMH Services. Although there were more hospital services needed at the implementation of the TMH, only 20% of the total hospitalization were inpatient compared 46% during in-person visits. Telemental Health Services 53 After statistical adjustment for gender identity, those in the post-TMH Services period tended to have lower anxiety levels. Those in the post-TNH Services period tended to have lower depression levels. It should be noted that while the number of patient contacts with wellness (service) for mental health recovery increased after the implementation of TMH services, the number of patient contacts with recovery (service) for patients with chemical dependency decreased by 17% in the last month of data collection. The decreased contact by patients enrolled in recovery could indicate that they may have dropped out of the program. There were twice the number of suicidal responses following C-SSRS assessment in-person compared to the TMH period. This increased number of responses in the in-person period could have several causes: there were either increased patient intake (admission) into PH within the control period, or there were patients placed on the HRCP within the same period, or there were just increased assessments to update the treatment plan. Contextual Elements The independent variables in this QI project included the TMH services provided and the frequency of assessments which were used to determine increased mental health access and reduced inpatient hospitalization. Appointments kept by patients for their therapy sessions and medication management sessions are other independent variables. Hospitalization is dependent on TMH and assessment tools. Other dependent variables include buy-in of staff to assess patients using assessment tools and access to good working electronic devices. Contact Log Contact logs were reviewed to describe the number of contacts by service type during the period before and at the implementation of the TMH services. The table below includes a summary of overall counts and percentages of total visits for the period under investigation. The Telemental Health Services 54 number of contacts increased by 66% after implementation of the TMH services (calculated as (4086-2532)/2532). The distribution of service types was different between time periods (chisquare p-value <0.0001), with the largest increase within “Therapies” (See Table 2, increased by >1000 contacts, an 80% increase). Table 4 Service types/ Contact logs Required Services (Billable services) Assessment Therapies Injections Enhanced services Optional Services (Non-billable) Outreach (Non-billable) In person OV N=2466 37 1.5% 1707 69.2% 41 1.7% 189 7.7% 10 0.4% 482 19.5% Telemental Health N=4094 14 0.3% 3072 75.0% 31 0.8% 421 10.3% 13 0.3% 543 13.3% Figure 3 Number of contacts by service type during the pre-period (in person office visits only, September 15, 2019 – December 13, 2019) and post-period (Telemental Health, September 15, 2021 – December 13, 2021) Telemental Health Services 55 3500 3072 In person OV Telemental Health Number of contacts 3000 2500 2000 1707 1500 1000 0 482 543 421 500 41 31 37 14 Billable services: Assessment Billable services: Therapies Billable services: Injections 187 Billable services: Enhanced services 10 13 Optional non- Non-billable billable Outreach services Census There were total of 280 on the census report in the period prior to introduction of TMH services. This increased by 21% (up to 340 patients) in the period after introduction of TMH services (See Figure 4). However, the number of the census report per contact was 25% lower after introduction of TMH services periods (Pre: 318 of 2532, 12.6%, Post: 374 of 4086, 9.2%; chi-square p-value <0.0001). It implies the interaction of patient with their mental health providers while accessing mental health treatment via TMH services is replicable and highly statistically significant. Substance abuse and psychiatric related hospitalizations There were a total of 48 substance abuse and psychiatric hospitalizations in the period prior to introduction of TMH services. This increased by 83% (up to 88 cases) in the period after introduction of TMH Services (See figure 4). Although this was a large increase, the number of Telemental Health Services 56 substance abuse and psychiatric hospitalizations per contact was not statistically significant between time periods (Pre: 48 of 2532, 1.9%, Post: 88 of 4086, 2.2%; chi-square p-value = 0.472). Finally, the percent of psychiatric and substance abuse hospitalizations that required >1 day length of stay was lower after introduction of TMH Services (Pre: 45.8%, Post: 26.1%; chisquare p-value = 0.020). This implies that increased assessment in TMH period is effective because of the decrease in prolonged inpatient hospitalization. It could also imply that the increased contact of SUD patients who are receiving MAT kept them current with treatment and reduced hospitalization. Figure 4 Number in census report and number of substance abuse and psychiatric related hospitalizations during the pre-period (in person visits only, September 15, 2019 – December 13, 2019) and post- Number of patients period (TMH, September 15, 2021 – December 13, 2021) 400 In person OV Telemental Health 300 200 100 0 Census report Substance abuse and psychiatric hospitalizations GAD7 Scores of GAD-7 range from zero to 21. The cut off points of five, ten, and 15 represent mild, moderate, and severe levels of anxiety respectively, on the GAD-7 scoring scale (Spitzer et al, 2006). The above categories were used to compare the percentage within each anxiety level Telemental Health Services 57 between the pre-TMH period and TMH period using Ordinal Logistic Regression. Although there was a slight trend towards lower anxiety in the post-period, the differences were not significant (p=0.191). However, there were differences in characteristics between the pre and TMH periods, including gender identity (35% female in pre-period and 73% female in the post period). When adjusting for gender identity within ordinal logistic regression, those female and male patients in the post-TMH Services period tended to have lower anxiety levels (p=0.030) compared to those in the in-person visits. Figure 5 GAD-7 anxiety level before and after introduction of TMH services GAD-7 anxiety level before and after introduction of Telemental Health Services 100% Severe, 13% 80% Severe, 35% 60% Moderate, 10% 40% Mild, 35% 20% 0% Moderate, 17% Mild, 39% None, 20% Female In person OV Severe, 16% Severe, 12% Moderate, 6% Moderate, 19% Mild, 18% Mild, 22% None, 65% None, 30% Female Telemental Health None, 43% Male In person OV Male Telemental Health Overall p-value comparing In person OV versus Telemental Health = 0.030 PHQ9 The analysis for differences in PHQ-9 used the established cutoff of <5 (no depression), 5-9 mild, 10-14 moderate, 15-19 moderately severe, and 20-27 severe. These categories were used to compare the percentage within each depression level between the pre-TMH period and Telemental Health Services 58 TMH period using Ordinal Logistic Regression. There was a significant trend towards lower depression level in the post-period (p=.033). Figure 6 PHQ-9 depression before and after the introduction of TMH services 100% PHQ-9 depressionlevel before and after introduction of Telemental Health Services Any Severe, 13% 80% Any Severe, 35% 60% Moderate, 15% 40% Mild, 24% 20% 0% Moderate, 23% Mild, 30% None, 34% None, 27% In person OV p-value = 0.033 Telemental Health Note: Ordinal logistic regression adjusted for other factors (e.g., gender identity) did not change the conclusion C-SSRS The responses of individual items from the C-SSRS were compared between in-person OV and TMH using Chi-square tests or Fisher’s exact test (when expected counts were <5). In general, the rates of suicidal responses were higher during in-person visits, but these differences were not significant. However, the small number of responses may lead to unstable or unreliable results. Caution should be used when interpreting these results. Table 5 C-SSRS suicidality before and after the introduction of TMH services C-SSRS In-person OV Telemental health p-value Telemental Health Services Wish to be dead Suicidal thoughts (General non-specific thoughts) Suicidal thoughts with method (w/o Specific plan or intent) Suicidal intent (without specific plan) Suicidal intent (with specific plan) Suicidal Behavior Suicidal attempt during lifetime 59 23% 14% 4 2 N=44 9% 5% 6 7% 0 0% 0.1771 6 7% 1 2% 0.4251 4 4% 0 0% 0.3021 4 30 4% 33% 1 13 2% 30% 0.9991 0.6592 21 13 N= 90 0.0591 0.1431 Note: ¹Fisher’s exact test, ²Chi-square test DISCUSSION Summary During utilization of the emergency TMH services the number of contacts increased by 61%. The distribution of service types was different between the in-person visits and TMH with the largest increase being within “Therapies” (increased by >1000 contacts, an 80% increase as displayed in Table 4). The number on the census report increased by 21% after the implementation of the TMH Services. However, when accounting for the number of contacts, there was a 25% decrease in census per contact after the implementation of the TMH Services. Those patients who needed increased contacts with their provider received it. The number of psychiatric and substance abuse hospitalizations increased by 83% after the implementation of the TMH Services. However, when accounting for the number of contacts, the number of psychiatric and substance abuse hospitalizations per contact were not different between time periods. The percent with >1 day length of stay was lower after the implementation of the TMH Services. After statistical adjustment for gender identity, those in the post-TMH Services period tended to have lower anxiety levels. Those in the post-TMH Services period tended to have Telemental Health Services 60 lower depression levels as seen on Figure 6 and evidenced by the Ordinal Logistic Regression (p=.033) when the percentages of each depression level are compared between the pre and TMH period. Those patients assessed for suicidality were 51% less in the post-TMH period and 57% less likely to attempt suicide compared to the pre-period. The strengths of the project The benefit of this QI project lies in the increased communication among team members that resulted in the implementation of TMH services. It was difficult at first to get the team working cohesively together. It was imperative for the project leader to remain vulnerable to assemble a formidable team. The first stage was developing a functional team which included building rapport among the team members and understanding each person’s strengths and their experiences with regards to TMH. There was agreement by the team to attend weekly interprofessional team meetings via Zoom. Team members then volunteered to take other roles. A constructivist approach of learning and teaching enabled the collaborative effort of the team of mental health workers to facilitate development of this project. Everyone, both trainer/ trainees, relied on their knowledge in providing mental health services. Weekly interprofessional zoom meetings, a PowerPoint presentation, role playing, and Inservice-type lectures were utilized to provide organized information for the team. The team of clinicians, both fee-for-service and direct employees contributed data and their various experiences in the implementation of the TMH service in this organization were another strength. These professionals influenced collaborative team building by forming the changes that directly affected them, implementing the changes, and identifying the impact of the changes they contributed, both positive and negative. This design creates a sense of ownership, professional empowerment, and commitment, impacting the buy-in and Telemental Health Services 61 implementation of TMH. This collaborative effort by the stakeholders increased job satisfaction, mutual respect for the value of each discipline, and patient outcomes. Interpretation According to the AMA, the adoption of TMH services allowed mental health providers to increase continuity of care, extend access beyond normal clinic hours, reduce patient travel burden, and help overcome clinician shortages in rural and underserved populations helping practices to enhance patient wellness, improve efficiency, provide higher quality of care, and increase patient satisfaction (AMA, 2020). This was evident in PH in the following ways: TMH ensured that patients with mental health problems, like depression, anxiety, ADHD, bipolar disorder, schizophrenia, and other serious mental health conditions, who were adjusting to new medications, maintained contact with their mental health providers for their routine follow-ups. It enabled patients to participate in their routine virtual psychotherapy appointments. Transportation barriers were overcome when patients accessed care from the convenience of their current location. After these screening tools were applied for the implementation period of three months, there was decreased inpatient hospitalization rates for mental health and SUD admissions by stratifying patients’ level of mental health decompensation risks using the HRCP and increasing access to mental health treatment. These high-risk protocol levels included low, moderate, and high risks and treatment was implemented according to patients’ needs. The type of treatment changes ranged from increased frequency of therapy, type of therapy, and medication adjustments. Access to mental health treatment was measured by frequency of contact with mental health providers and number of follow up appointments kept. Telemental Health Services 62 There were decreases in the metrics of the assessment tools (GAD-7 scores and the PHQ9) in the TMH period compared to the in-person visits indicating improved mental health status. The number of patients screened for suicidality decreased in the TMH period despite increase in the overall patient census of TMH period indicating fewer changes in mental health status or mental health stability. There were increases in the census of the active wellness and recovery patients, the contacts of patients with their providers, and an increase in the number of psychiatric and substance abuse hospitalizations (Emergency Department utilization) with decreased inpatient hospitalization. When considering the billable therapies, there was 135% increase in individual psychotherapy, a 37% increase in group therapy and medication assisted treatments (MAT), indicated as the chemical dependency groups, and a 33% increase in psychotropic medications treatment. Implications: Impact of the project on people and systems This project impacts both the patients and the providers, due to increased access of patients to providers and substance abuse counselors. The flow of communication between the two parties generated individualized treatment plans and aided in improvement of symptoms. The implementation of this QI project has affected system-wide workflow within the Wellness and Recovery program. There is better flow of communication between the employees. At the weekly interprofessional team meetings, team members provide informed contributions on specific subject areas regarding patients’ treatments, engagement in therapy, and potential patients’ risks. In these meetings each of the mental health workers are prepared to discuss and resolve pertinent patient issues regarding patients’ welfare. The office served as central hub for patient scheduling and intake. Patients’ calls, complaints, and referrals were overseen by these two personnel in the office. Telemental Health Services 63 Initially, just the clerk and intake specialist were present in the office to maintain social distancing mandates. Systematically, each of the NPPs followed by coming into the clinic once a week to administer injections, since there was no Registered Nurse to administer injections. Patients were scheduled such that there were only two to three patients in the waiting area at a given time, per CDC mandates. The paper scheduling was replaced by the Award EHR patient scheduling. Notes were written at the times of scheduled appointments, or an outreach note was written for those patients who could not be reached at their scheduled time. The clerk reviewed notes and rescheduled the patients as indicated by the mental health workers. Mental health providers like the LMSW, LMHC, CASAC, and NPPs were expected to work remotely using private work designated phone numbers to contact patients. Reasons for any differences between observed and anticipated outcomes, including the influence of context The outcome of this QI project anticipated a decrease in mental health and substance abuse related hospitalization, which did not occur. There was an increase in number of patient hospitalizations, with a decrease in length of stay from an average of 42 days to < 1 day. These findings could be due to the psychological effects of the sudden surge of the COVID-19 Delta and Omicron variants and more people using the ER for COVID testing. It could also be because of the declining participation in the number of patients in the recovery participating in the chemical dependency groups and/ or MAT during the last month of the data collection in the TMH period despite 37% overall increase. There are people who know relatives, friends, and neighbors impacted by the COVID-19 pandemic. The COVID-19 pandemic brought many changes like social isolation, financial Telemental Health Services 64 pressures, life uncertainty, and changes in routines. These changes led to increased life stress, anxiety, and depression which can worsen. Others may have chosen to self-medicate by increasing alcohol and/ or other drugs to cope with their mental health symptoms exacerbated by the pandemic. While patients were actively participating in their Wellness individual therapy and medication management sessions, the number was declining in the Recovery program. Although there was 37% growth in group therapy and MAT participation in the TMH period, there was increased hospitalization. There were systemic changes which may have affected the results of the study period. The only clerk who schedules clients’ appointment with their mental health providers was out on extended sick leave and a replacement was in training. There was a period between the new clerk replacement and the job training where we may have lost some patients to follow up. There were also new Recovery and Wellness program directors who were adjusting to their positions. Two Licensed Social workers resigned about a month apart before and at the onset of the TMH study period. It was noted during data analysis that some of the patients were not reassigned during the data collection phase and may have been lost to follow up after they separated from their therapist. According to the substance abuse counselors at Project Hospitality, patients struggling with substance abuse recovery need the physical and emotional support of their peers and counselors. There was a 31% increase of TMH services within the same monitoring period (November 14 into December 13) compared to pre TMH. There was also 9% decline compared to the previous 30-day period. As a result, we may have lost some patients within the study period, which resulted in increased hospitalization. However, among the hospitalizations recorded in the TMH period, only 20% were in the hospital for more than 24-hours compared to Telemental Health Services 65 46% in the in-person visits. There was 22% decline in inpatient hospitalizations (greater than 24 hours) in TMH period. Table 6 Hospitalizations greater than 24 hours Hospitalizations September October November Total In-Person visits (pre-TMH) 10 7 5 22/Total # 48 46% TMH 3 9 6 18/total #88 20% Costs and strategic trade-offs, including opportunity costs Telepsychiatry is the most developed, innovative model in expanding access to psychiatric providers. According to the National Council for Behavioral Health, the expansion of telepsychiatry has been disadvantaged by burdensome regulations and limits of reimbursement. Deliberately increasing the number of mental health providers that are well trained in TMH services may impact the number of long-term hospital readmissions. Utilization of TMH may impact substance use disorder patients with co-occurring mental illness who use the Emergency Departments for stabilization. The lack of timely access to mental health services affects private clinics, group practices, individual offices, emergency departments in hospitals and community behavioral health centers. Insufficient symptom management and non-compliance with medication can occur with extended times between appointments, leading to frequent Emergency Departments visits and more hospitalizations. This gap in mental health and substance use disorder services significantly delays treatment and reduces the quality of treatment, resulting in unacceptable patient experiences in care, poor outcomes, and higher costs (National Council for Behavioral Health, 2017). Limited timely access to mental health care further impacts the providers’ ability Telemental Health Services 66 to address other social determinants of health such as housing, employment, education, and family engagement. The direct cost of mental health treatment is the cost of mental health hospitalizations and outpatient treatments, including medication and counseling, for the captured population. Indirect cost includes, income loss, disability, the cost of incarceration for prisoners with serious mental health problems and frequent emergency room visits. The incremental cost would be the cost of training new providers to telepsychiatry and providing TMH communication models to patients. Opportunity cost in telepsychiatry/ TMH implementation is the number of people lost to follow up due to lack of continuous access to mental health treatment, the reimbursement cost per treatment for in-person visits, and increased workload for providers. These results in high cost for mental health treatment compared to low cost to train physicians in TMH services. Limitations to the generalizability of the work There were limitations experienced during this QI project. TMH was initiated via an emergency order, prior to the beginning of this project, and at present remains in place. After project initiation, there was a change from the New York Office of Mental Health, as to how to evaluate the treatment plans of patients. The project initially proposed patient assessment with screening tools (GAD-7, PHQ-9, and C-SSRS) by the team of providers within the program (mental health workers), every 30 days within the 90 days study period. This study period was from September 15 through December 13, 2021. However, the guidelines later extended evaluations for all Medicaid patients to be completed at intake, then every 6 months or as needed depending on presentation of symptoms. A measure was in place at PH to monitor severity of symptoms using the high-risk protocol criteria. This consequently affected the implementation of Telemental Health Services 67 the screening tools utilized to assess patients’ mental health such that the entire patient population in the census was not assessed every 30 days using the screening tools as planned. Secondly, there was significant turnover of staffing that affected patient contact and implementation of the assessment tools. For instance, the clinic has gone through three wellness program directors and one recovery program director since 2020. There were two licensed social workers who resigned, one clerk who was out on leave, and one credentialed Alcoholism and Substance Abuse Counselor (CASAC) who resigned during the study phase of this QI project in 2021. Another limitation is the study design of using the convenience sample of patients from both the wellness and recovery programs. The results may not be an accurate representation of all the patients in both programs. The results of the screening tools do not truly reflect the total number of patients assessed. Given the fluid nature of psychiatry, the frequency of screening may have varied. For example, some patients may have more screening than normal based on acute changes in their clinical course. There were variations in the service types seen in the contact logs between the in-person office visits of 2019 and the TMH study period of 2021. Some services were either not provided in 2019 or were documented in the wrong service type. An argument could be made that the same may have occurred in of the TMH period. Psyckes EHR serves only Medicaid insurance, and it has delayed data update. Hence, it may not have captured recent hospital visits/ admissions at the end of the 90 days. The Healthix EHR is not limited to Medicaid and captures hospitalization of all patients with various insurances. Telemental Health Services 68 Although some providers may struggle due to the inadequate quality of reception of some patients’ phone lines, overall communication has been improved and made easier. Patients who may have been non-compliant were reengaged and follow up more readily available from their preferred location. A difficult area for this organization was how to monitor patients’ compliance to their treatment regimen. This would necessitate running the patients contact lapse report at least monthly to enquire about patients who have missed more than two appointments with their providers, therapy, and medication management sessions. There should be consideration to discharge those patients who may not comply with medications and TMH visits. Also, there should be recommendations for easy admissions and access for the discharged clients to return to the program. The study was not fully able to meet the exclusion criteria by including those with serious mental illness who are not of sound mind and unable to make decision, and non-English speaking. This is because the data pulled from the Awards EHR was clustered and was unable to differentiate individual patients who received services and how many times the patient received the services. However, there were no patients younger than 18 years old included in the study. Within the period of TMH implementation, the (PH) program secured a grant for interpretive services for Spanish speaking patients and for those patients (documented and undocumented) without insurance. Hence, Spanish speaking patients were not excluded from this project. Although there are benefits to TMH implementation, there are also challenges to adopting this innovative technology. These barriers include inconsistent reimbursement, concerns over security, privacy, and confidentiality, concerns of clinical duty to provide safe and Telemental Health Services 69 effective care, and lack of evidence about impact on health care costs, utilization, or outcomes (AMA, 2020). The data obtained from the study design of a convenience sample of patients from the Wellness and Recovery programs may not fully depict the characteristics of the entire population. The psychiatric and substance abuse hospitalization report obtained from the Award EHR clustered medical, psychiatric, and substance abuse hospitalizations. The investigator removed all medically indicated hospitalizations. However, there are cases where neither psychiatric nor substance abuse hospitalizations could be determined. Therefore, the data is clustered and cannot be further separated for analysis. The investigator accessed the Psyckes and Healthix to verify the reasons for these hospitalizations. The data obtained from the assessment tools are not the cumulative number of patients assessed, but the results (scores) of assessments and indicates the number of times the assessments were administered. A patient could be screened more than once in each period, depending on needs and risks of the patients. The same for contact logs where a given patient could be provided different service types and/ or same service in each period of data collection. Efforts made to minimize and adjust for limitations A cohort of the high-risk patients would have been more appropriate than using the convenient sample of patients. Investing in the educational training of personnel that document patients’ hospitalizations in Award her, as indicated in the Psyckes and Healthix her, would increase the accuracy of the hospitalization report. The billing team participated in the planning and implementation phase of TMH to increase understanding of federal, state, and private payer requirements and regulations. The billing team was also provided education regarding the documentation that captures the billable and other service types. Telemental Health Services 70 DNP Essentials Addressed The implementation of this QI project utilized all the essentials of doctoral of advanced nursing practice (DNP). DNP essential I was utilized when the investigator incorporated innovative ideas to improve care delivery for patients with mental health and co-occurring substance abuse disorder. Essential II was incorporated when the investigator evaluated effective strategies to enhance patient care and enhance the workflow within the PH organization. The DNP Essential III incorporates nursing theories and frameworks that allowed methodical data gathering which were utilized to implement the QI project. Essential IV was applied to data extraction from practice information systems (Awards) and databases (Psyckes and Healthix) and Essential V is by analyzing TMH guidelines and advocating for equitable policies to enhance patient outcomes. There has been interprofessional collaboration throughout the implementation of this QI project that satisfies the DNP Essential VI. There have been collaborations with LCSWs, MHCs, substance abuse counselors, the leadership in PH, and consultation with a biostatistician. Essential VII was incorporated with the implementation of this QI project which aimed to increase patient access to mental health care and reduce hospitalization by utilizing the TMH emergency guideline to increase patient assessment. Essential VIII was employed when the investigator utilized analytical skills in evaluating the current TMH emergency guideline and providing education to the team of mental health professionals at PH. Conclusions Plan for Sustainability This QI project would be useful for monitoring patients in high-risk situations and track patients who could be lost to follow up. The sustainability of this QI project is dependent on the continuous insurance reimbursement related to New York emergency TMH guidelines. Telemental Health Services 71 Additionally, mental health workers spent time in addition to their scheduled work hours to address patient needs. The mental health workers spent a vast amount of time overseeing billable and unbillable services. This project may be beneficial to patients, but it could increase the workload and stress of mental health providers. While assessing the utilization of emergency TMH services to increase patient access to mental health assessments and counseling, and decreasing hospitalization, our findings suggests that there is need to adopt this guideline into policy and a hybrid approach would better serve the patients in the recovery program. The hybrid approach entails a combination of in-person office visits (for group therapy) for peer support and TMH (individual counseling) therapy. Future studies could investigate a control sample of patients who are seen in in-person visits compared to those patients who are solely receiving TMH services in real time. Plan for Dissemination The results of this QI project were presented to the doctoral committee and the team of mental health workers at Project Hospitality using a poster and PowerPoints. There is need for more in-service type education for the staff of Project hospitality on proper documentation of service types to capture billable services. Furthermore, there is a need to maintain TMH services as an option for patients in New York City at the end of the COVID-19 pandemic. A robust TMH legislation which offers equal reimbursement for TMH, and in-person visits is necessary to capture the increasing number of patients seeking mental health services in an industry plagued with personnel shortage. This legislation should increase and standardize the reimbursement of TMH services by Medicaid and managed care organizations. Payment should be as much as the in-person visits to entice those cash-only providers to start accepting Medicaid and Managed Care insurances. There is need for Telemental Health Services 72 support for the legislation and provision of funding to recruit and train more mental health providers in the North Shore. This would help to ensure the sustainability of the TMH services. A next step to obtaining legislation of TMH services will be accomplished by contacting the Council member for the 49th District of the New York City Council. Finally, Telecommunication aid needs to be provided and adopted as a medical necessity to fully adopt this innovation. Telemental Health Services APPENDIX A: SITE APPROVAL/AUTHORIZATION LETTER 73 Telemental Health Services 74 Telemental Health Services APPENDIX B: CONSENT DOCUMENT (DISCLOSURE FORM, CONSENT FORM, ETC.) 75 Telemental Health Services 76 Telemental Health Services 77 Telemental Health Services APPENDIX C: 78 EVALUATION INSTRUMENTS (STUDENT CREATED DATA COLLECTION TOOLS – SURVEYS, QUESTIONNAIRES, INTERVIEW QUESTIONS, ETC.) Telemental Health Services 79 Telemental Health Services 80 Telemental Health Services APPENDIX D: PROJECT TIMELINE 81 Telemental Health Services APPENDIX E: 82 OTHER DOCUMENTS AS APPLICABLE TO THE PROJECT (SUCH AS BUDGET) Telemental Health Services 83 Telemental Health Services 84 References Agency for Healthcare Research and Quality. (2020). Plan-Do-Study-Act (PDSA) directions and examples. https://www.ahrq.gov/health-literacy/improve/precaustions/tool2b.html American Medical Association (2020). Telehealth Implementation Playbook. https://www.ama-assn.org/system/files/2020-04/ama-telehealth-implementationplaybook.pdf Bashshur, R. L., Shannon, G. W., Bashshur, N., & Yellowlees, P. M. (2016). The empirical evidence for telemedicine interventions in mental disorders. Telemedicine Journal and EHealth, 22(2), 87-113. https://doi.org/10.1089/tmj.2015.0206 Brearly, T., Goodman, C., Haynes, C., McDermott, K., & Rowland, J. (2020). Improvement of post inpatient psychiatric follow-up for veterans using telehealth. American Journal of Health-System Pharmacy, 77(4), 288-294. https://doi.org/10.1093/ajhp/zxz314 Bjureberg, J., Dahlin, M., Carlborg, A., Edberg, H., Haglund, A., & Runeson, B. (2021). Columbia-Suicide Severity Rating Scale Screen Version: Initial screening for suicide risk in a psychiatric emergency department. Psychological Medicine, 1-9. doi:10.1017/S0033291721000751 Cantor, J. H., McBain, R. K., Kofner, A., Ma, Stein, B. D., & Yu, H. (2021). Availability of outpatient telemental health services in the United States at the outset of the COVID-19 pandemic. Medical care, 59(4), 319-323. https://doi.org/10.1097/MLR.0000000000001512 Telemental Health Services 85 Center for Disease Control and Prevention. (2020). National Center for health statistics: Healthy people 2020. U.S. Department of Health & Human Services. https://www.cdc.gov/nchs/healthy_people/hp2020.htm Chang, Hyejung. (2015). Evaluation Framework for Telemedicine Using the Logical Framework Approach and a Fishbone Diagram. Healthcare Informatics Research. 21. 230. 10.4258/hi Cohen Veterans Network & National Council for Behavioral Health. (2018). America’s mental health 2018. https://www.cohenveteransnetwork.org/wp-content/uploads/2018/10/Research-Summary10-10-2018.pdf County Health Rankings & Roadmaps. (2020). Health Outcomes: Overall rank of Richmond County, New York. A Robert Wood Johnson Foundation Program. https://www.countyhealthrankings.org/app/newyork/2020/rankings/richmond/county/outco mes/overall/snapshot Cuomo, A. M., Sullivan, A. M. T., & Tavella, C., & Revised March 17, 2. (2020). Supplemental guidance regarding use of telehealth for people served by OMH licensed or designated programs affected by the disaster emergency. Albany, New York. https://omh.ny.gov/omhweb/guidance/supplemental-guidance-use-of-telemental-healthdisaster-emergnecy.pdf Cuomo, A. M., Sullivan, A. M. T., & Tavella, C., & Revised March 17, 2. (2020). Use of Telephone and Two-way Video Technology by OMH-Licensed, Funded or Designated Providers and Clients Affected by the COVID-19 Pandemic. Albany, New York. https://ahihealth.org/wp-content/uploads/2020/04/NYS-OMH-Consolidated-COVID-19Telemental-Health-Guidance.pdf Telemental Health Services 86 Dorin, A. (2021). Behavioral health advocates support Governor Cuomo’s legislation to expand access to telehealth in mental health and substance use services for all New Yorkers. The Coalition for Behavioral Health. New York, NY. Eisman, A. B., Kilbourne, A., M., Dopp, A. R., Saldana, L., Eisenberg, D. (2020). Economic evaluation in implementation science: Making the business case for implementation strategies. Psychiatry Research, 283. https://doi.org/10.1016/j.psychres.2019.06.008. Federation of State Medical Boards. (2022). U.S. States and Territories Modifying Requirements for Telehealth in Response to COVID-19: Out-of-states physicians; preexisting providerpatient relationships; audio-only requirements, etc. fsmb.org/siteassets/advocacy/pdf/states-waving-licensure-requirements-for-telehealth-inresponse-to-covid-19.pdf Guaiana, G., Mastrangelo, J., Hendrikx, S., & Barbui, C. (2021). A systematic review of the use of telepsychiatry in depression. Community Mental Health Journal, 57(1), 93-100. https://doi.org/10.1007/s10597-020-00724-2 Healthix. (2021). Ixchange Blog: Healthix VIPs deliver healthcare interoperability solutions. https://healthix.org/ixchange/healthix-vips-deliver-healthcare-interoperability-solutions/ Haghnia, Y., Samad-Soltani, T., Yousefi, M., Sadr, H., & Rezaei-Hachesu, P. (2019). Telepsychiatry- based care for the treatment follow-up of Iranian war veterans with posttraumatic stress disorder: A randomized controlled trial. Iranian Journal of Medical Sciences, 44(4), 291-298. https://doi.org/10.30476/IJMS.2019.44944 Telemental Health Services 87 Harrar, S. (2021). Inside America’s Psychiatrist Shortage: An in-depth look at why it’s happening, how bad it is, where it’s worst, and how to find the mental health help you need, despite it. PSYCOM. https://www.psycom.net/inside-americas-psychiatrist-shortage Hickey, J. V. & Brosnan, C. A. (2017). Evaluation of health care quality for DNPs (2nd ed.). Springer Publishing Company. Hinterland, K., Naidoo, M., King, L., Lewin, V., Myerson, G., Noumbissi, B., Woodward, M., Gould, L. H., Gwynn, R. C., Barbot, O., Bassett, MT. (2018). Community Health Profiles 2018, Staten Island Community District 1: St. George and Stapleton. NYC.gov k57(59):1-20. https://www1.nyc.gov/assets/doh/downloads/pdf/data/2018chp-si1.pdf Institute for Healthcare Improvement. (2021). Tools: Plan-Do-Study-Act (PDSA) worksheet. http://www.ihi.org/resources/Pages/Tools/PlanDoStudyActWorksheet.asp Levis, B., Benedetti, A., Thombs, B. D. (2019). Accuracy of patient health questionnaire-9 (PHQ-9) for screening to detect major depression: Individual participant data metaanalysis. Quebec, Canada. BMJ 2019; 365:1476 https://doi.org/10.1136/bmj.l1781 Liem, A., Sit, H. F., Arjadi, R., Patel, A. R., Elhai, J. D., & Hall, B. J. (2020). Ethical standards for telemental health must be maintained during the COVID-19 pandemic. Asian journal of psychiatry, 53, 102218. https://doi.org/10.1016/j.ajp.2020.102218 Mace, S., Boccanelli, A., & Dormond, M. (2018). The use of telehealth within behavioral health settings: Utilization, opportunities, and challenges. Behavioral Health Workforce Research Center, University of Michigan. Telemental Health Services 88 https://behavioralhealthworkforce.org/wp-content/uploads/2018/05/Telehealth-FullPaper_5.17.18-clean.pdf Mahmoud, H., Naal, H., & Cerda, S. (2021). Planning and implementing telepsychiatry in a community mental health setting: A case study report. Springer Science and Business Media LLC. https://doi.org/10.1007/s10597-020-00709-1 Medicare Learning Network. (2021). Telehealth services. Centers for Medicare & Medicaid Services. https://www.cms.gov/Outreach-and-Education/Medicare-Learning-NetworkMLN/MLNProducts/Downloads/TelehealthSrvcsfctsht.pdf Milstein, D. & Madden, S. L. (2017). Tackling Youth Substance Abuse on Staten Island: A collective impact project. Harvard T. H. Chan School of Public Health. http://sipcw.org/wp-content/uploads/2019/09/Harvard-University-Case-Study-TacklingYouth-Substance-Abuse-on-Staten-Island.pdf Mossman, S. A., Luft, M. J., Schroeder, H. K., Varney, S. T., Fleck, D. E., Barzman, D. H., Gilman, R., DelBello, M. P., & Strawn, J. R. (2017). The Generalized Anxiety Disorder 7-item scale in adolescents with generalized anxiety disorder: Signal detection and validation. Annals of clinical psychiatry: official journal of the American Academy of Clinical Psychiatrists, 29(4), 227–234A. National Council for Behavioral Health. (2017). The Psychiatric Shortage: Causes and solutions. Washington, DC. https://www.thenationalcouncil.org/wp-content/uploads/2017/03/PsychiatricShortage_National-Council-.pdf?daf=375ateTbd56 Telemental Health Services 89 New York Office of Mental Health. (2020). Telemental health services: Interim guidance for telemental health during the COVID-19 outbreak. https://omh.ny.gov/omhweb/clinic_restructuring/telepsychiatry.html New York State Office of Mental Health (2021a, January 1). New York State Office of Mental health 14 NYCRR Part 599: “Clinic Treatment Programs” interpretive/ implementation guidance. https://omh.ny.gov/omhweb/clinic_restructuring/part599/part-599.pdf New York State Office of Mental Health. (2021b, February 1). PSYCKES. New York. https://omh.ny.gov/omhweb/psyckes_medicaid/psyckes_medicaid_release_notes_710_fe b_2021.pdf Olin, S. C., O'Connor, B. C., Storfer-Isser, A., Clark, L. J., Perkins, M., Hudson Scholle, S., Whitmyre, E. D., Hoagwood, K., & Horwitz, S. M. (2016). Access to Care for Youth in a State Mental Health System: A Simulated Patient Approach. Journal of the American Academy of Child and Adolescent Psychiatry, 55(5), 392–399. https://doi.org/10.1016/j.jaac.2016.02.014 Perrin, A. & Turner, E. (2019). Smartphones help blacks, Hispanics bridge some-but not alldigital gaps with whites. Pew Research Center. https://www.pewresearch.org/fact-tank/2019/08/20/smartphones-help-blacks-hispanicsbridge-some-but-not-all-digital-gaps-with-whites/ Project Hospitality. (n.d.). High Risk Criteria Protocol Pumariega, A. J., Posner, K., Millsaps, U., Romig, B., Starvarski, D., Gehret, M. J., Riley, K., Wasser, T. E., Walsh, G., Yarger, H. (2020). Systematic Suicide Screening in a General Hospital Setting: Process and initial Results. World Soc. Psychiatry 2020; 2:31-42. https://www.worldsocpsychiatry.org/text.asp?2020/2/1/31/281135 Telemental Health Services 90 Richmond University Medical Center. (2019). Community health needs assessment & community service plan for Richmond County (Staten Island). https://www.rumcsi.org/wp-content/uploads/2020/09/2019-CHNA-Web-site-FinalReport-2019-10-30.pdf Serenity at Summit (2022). Addiction statistics and treatment in Staten Island. Retrieved from https://serenityatsummit.com/new-york/staten-island/ Spitzer, R. L., Kroenke, K., Williams, J. B. W., Lowe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Arch Intern Med. 166(10):1092–1097. doi:10.1001/archinte.166.10.1092 Sylvia, M. L., & Terhaar, M. F. (2018). Clinical Analytics and Data Management for the DNP (2nd ed.). Springer Publishing Company. The Lancet Global Health. (2020). Mental health matters. Elsevier Ltd 8(11). https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(20)304320/fulltext?rss=yes United States Census Bureau. (2019). QuickFacts: Richmond County (Staten Island Borough), New York. https://www.census.gov/quickfacts/richmondcountystatenislandboroughnewyork Warren, J. C. & Smalley, K. B. (2020). Using telehealth to meet mental health needs during the COVID-19 crisis. The Commonwealth Fund. https://www.commonwealthfund.org/blog/2020/using-telehealth-meet-mental-health-needsduring-covid-19-crisis Telemental Health Services 91 Williams, C. (2021). Using the hub and spoke model of telemental health to expand the reach of community-based care in the United States. Community Mental Health Journal, 57(1), 4956. https://doi.org/10.1007/s10597-020-00675-8 Wood, J. M. & Gupta, S. (2017). Using rating scale in a clinical setting. Clinical Reviews. https://cdn.mdedge.com/files/s3fs-public/Document/August-2017/CR02709028.PDF Wood, P., Burwell, J., Rawlett, K. (2018). New Study Reveals Lack of Access as Root Cause for mental Health Crisis in America. National Council for Behavioral Health. https://www.thenationalcouncil.org/press-releases/new-study-reveals-lack-of-access-asroot-cause-for-mental-health-crisis-in-america/