OMB Part A Supporting Statement 05122023

Supporting Statement A

Enhancing HIV Care of Women, Infants, Children and Youth Building Capacity through Communities of Practice

OMB Control No. 0915-XXXX-New

Terms of Clearance: None.

A. Justification

1. Circumstance of Information Collection

The Health Resources and Services Administration (HRSA) HIV/AIDS Bureau’s (HAB) Division of Community HIV/AIDS Programs (DCHAP) is requesting approval from the Office of Management and Budget (OMB) for data collection activities related to implementation of the Enhancing HIV Care of Women, Infants, Children and Youth Building Capacity through Communities of Practice (CoPs) contract. The project will implement CoPs and provide technical assistance (TA) with the aim to increase delivery of evidence-based interventions that enhance client outcomes, increase the skill level of HIV workforce providing care and treatment to Women, Infants, Children and Youth, and disseminate best practices through partnerships to RWHAP Part D participants.

Scope of the Issue and the Ryan White HIV/AIDS Program

In 2020, nearly 1.2 million people in the United States were living with HIV, with 30,635 individuals newly diagnosed. HIV/AIDS has a disproportionate impact on low-income and minority populations. The RWHAP is one of the largest providers of health services for those living with HIV in the United States. First authorized by Congress in 1990 and administered by the HRSA HAB, RWHAP provides medical support services for more than half of U.S. residents diagnosed with HIV, reaching an estimated 568,000 people in 2019. The disproportionate impact on low-income and minority populations makes this program vital to ensure that underserved populations and communities receive comprehensive, quality health care so they can attain optimal HIV health outcomes.

RWHAP accomplishes this by facilitating grants to community, State, and national organizations who provide HIV/AIDS care and treatment service. Services are intended to increase access to care, improve health outcomes, and reduce transmission rates amongst these disproportionately affected populations. The program integrates a public health approach in which organizations tailor services to the needs of their communities, furthering the larger mission of HRSA: to improve access to healthcare services for the uninsured, isolated, and medically vulnerable, ensuring greater health equity and quality in the nation. A portion of the funds are also used for TA, clinical training, and the development of innovative models of HIV care.

RWHAP has been successful in reaching diverse groups of people—with approximately 47 percent of their clients identifying as Black/African American, 23 percent as Hispanic/Latino, and 26 percent as White—a percentage distribution that has been consistent since 2016. In 2020, nearly two-thirds of RWHAP clients were living at or below 100 percent of the Federal poverty level (FPL), with a higher percentage of female (69.7 percent) than male clients (57.4 percent). RWHAP has made it possible for the most underserved and isolated Americans to access cutting edge information, treatment, and support, providing them with the tools they need to fight this devastating disease.

RWHAP has five parts (Part A, B, C, D, F) defined by statute that provide funding for medical/support services, TA, training, and developing innovative models of care for different communities and populations affected by HIV. Part D is focused on serving the needs of low-income WICY living with HIV. It funds family- centered, community-based outpatient HIV medical and support services. Part D also provides support services to affected family members. In 2020, just over a quarter (26 percent) of individuals served by RWHAP were female. While only a small percentage of RWHAP clients served were under 25 years (0.6 percent were under 13; 3.5 percent between the ages of 13 and 24 years), people ages 13 to 24 years accounted for 20 percent of all new HIV diagnoses in 2020. This is a key population to engage and retain in HIV services.

Great strides have been made in preventing HIV and suppressing the virus overall. Between 2015 and 2019, the rates of HIV diagnoses among women decreased by 6 percent, including for Black/African American women and women ages 13 to 24 years—among the groups most affected by HIV. RWHAP demonstrated effectiveness in improving outcomes for women and youth living with HIV. The program reported annual improvements in viral suppression among its participants, with rates increasing from 69.5 percent in 2010 to 89.4 percent in 2020. RWHAP plays a critical role in linking individuals living with HIV to medical care and ensuring retention.

Despite these strides, the rates of viral suppression in women who have HIV are lower compared to all people with HIV. Discrimination, stigma, poverty, depression, and anxiety can be barriers to accessing quality healthcare and other needed services for women with HIV. Approximately 22 percent of women with HIV reported needing but not receiving dental care, 11 percent reported needing but not receiving shelter or housing services, and 10 percent did not receive SNAP or WIC. Only 62 percent reported consistently taking their HIV medicine as prescribed over the last 30 days and 26 percent reported missed medical appointments. Failure to access needed services or adhere to treatment can lead to negative outcomes including lower rates of viral submission.

Social determinants of health can contribute to an individual’s capacity to participate in or benefit from HIV care interventions. For interventions to be successful they must address underlying social and structural factors, the social determinants of health, which are barriers to effective HIV health care and influence health outcomes.

Treatment adherence is also a critical strategy for preventing perinatal transmission of HIV. Most cases of HIV in children younger than age 13 were perinatal, contracted during pregnancy, childbirth, or breastfeeding. There are strategies that can minimize the risk of perinatal transmission. Taking HIV medicine as prescribed during pregnancy and childbirth and providing HIV medicine to a baby for 4 to 6 weeks after birth can reduce the HIV transmission risk to 1 percent or less. In addition, avoiding breast feeding can also reduce risk as breast milk contains HIV.

As noted earlier, youth ages 13 to 24 years accounted for 20 percent of new HIV diagnoses in 2020. This group also has the highest rates of undiagnosed HIV. Approximately half do not know they are living with HIV. Most youth who get HIV during adolescence contract it through sex (behavioral). In fact, young heterosexual women and transgender adolescents have the highest risk factors of behavioral contraction of HIV. HIV youth need comprehensive and multidisciplinary support integrated within HIV care that includes mental health services, sexual/reproductive health services, substance use disorder treatment and prevention, and social support.

The COVID-19 pandemic disrupted HIV testing and the use of pre-exposure prophylaxis (PrEP) and shined an uncomfortable light on inequities within the health and socioeconomic systems for marginalized populations, including those served by the RWHAP. However, these pressures also led to many innovations in prevention, treatment, and access to care within RWHAP-funded programs particularly in relation to telehealth, PrEP and HIV drug access; RWHAP Part D programs can capitalize on this forward momentum.

Opportunities for Expansion and Improved Outcomes

In a recent analysis of the program, RWHAP identified several ways the Part D program could expand their reach and improve outcomes. They identified three areas of focus: trauma-informed interventions, pre-conception counseling, and youth transition into adult RWHAP programs.

Trauma-Informed Interventions. Individuals with HIV are more likely to have significant trauma histories. Ninety five percent report severe traumatic stressor, 54 percent meet the criteria for PTSD, 30 percent experienced physical or sexual abuse before the age of 13, and 95 percent of women experienced intimate partner violence. Trauma-informed care (TIC) is a strength-based framework that recognizes that many people have experienced traumatic events that can lead to significant health problems and other adverse consequences and can influence how affected individuals engage with and experience health care. The TIC approach frames individuals as resilient and survivors, versus as victims. Clients are actively engaged in treatment, so they feel a sense of control and involvement and are empowered through collaboration and choices. TIC ensures that health care is delivered in an effective, safe, and inclusive manner that is oriented towards healing and recovery. The benefits for patients include increased engagement and adherence to their treatment and recovery, and improved health outcomes. TIC can benefit organizations, too, by reducing costs for unnecessary or avoidable health care and social services.

Pre-Conception Counseling. Research indicates that many women who are HIV positive have a high desire and intention to have biological children. Yet typically they do not receive reproductive counseling as it is often not integrated into HIV care. Pre-conception counseling (PCC) is an intervention that can improve perinatal outcomes and reduce the risk of transmission. The goal of PCC in HIV care settings is to help individuals make informed decisions about and during their pregnancy, optimize maternal and paternal health before pregnancy, improve pregnancy and fetal/newborn outcomes, and reduce the risk of HIV transmission.

Youth Transition. According to SAMHSA, 4 of 5 youth diagnosed with HIV are aged 20-24. Most youth transition from pediatric to adult care between the ages of 18-22 (or later if enrolled in college). About half of youth with HIV disengage from care at the time of transition (Philbin et al, 2017). There may be structural barriers such as issues with insurance eligibility or transportation; clinical barriers such as inadequate inter-clinic communication, and individual barriers regarding transition readiness or developmental capacity to transition. Supporting and engaging youth in continued treatment as they transition to adult care is critical to ensure continued positive healthcare and other outcomes.

Through this contract, RWHAP seeks to (1) increase the use of evidence- based, evidence-informed, and emerging interventions, (2) increase workforce skill level to provide care and treatment; and (3) support foster partner collaborations to disseminate best practices. With a focus on continuous quality improvement, RWHAP is committed to advancing best practices and addressing existing challenges that WICY with HIV experience, so it can expand its program reach and improve outcomes. However, adoption of evidence-based practices into routine care is tenuous. In recent years, research has examined how science-based findings can be integrated into public health delivery more quickly, effectively, and efficiently. Implementation science examines methods that promote the implementation of research and evidence-based practices into routine practice. As RWHAP seeks to increase the uptake of evidence-based, evidence-informed and emerging interventions, integrating findings from implementation science can elevate the uptake of these practices and be the driver for creating a learning healthcare system—one that engages in self-study and data analytics to implement changes and transform practice.

Process Improvement Model

The Institute for Healthcare Improvement (IHI) Collaborative Model is a proven framework that helps health care organizations close the gap between knowledge and practice (“what we know” and “what we do”) while reducing costs. Teams of health care service providers participate in collaborative learning with peers and subject matter experts on a focused topic area for 6 to 15 months. The goal is to learn from each other and experts to improve patient outcomes through innovation and quality improvement. This type of learning is considered one of the most successful for quality improvement and system change. It has garnered impressive results including reduced waiting times, costs, and staff absenteeism. The IHI Collaborative Model—known as the Breakthrough Series (BTS)—will be utilized for this quality improvement collaborative effort.

Communities of Practice

To increase delivery of evidence-based interventions that enhance client outcomes, increase the skill level of HIV workforce providing care and treatment to WICY, and involves partnerships for dissemination of best practices to Ryan White HIV/AIDS Program Part D participants, HRSA seeks to implement a Communities of Practice (CoP) platform for RWHAP Part D recipients. RWHAP will implement CoP for each of the three identified focus areas to achieve the contract’s goals. The domains for the proposed CoPs are Trauma Informed Care, Pre-conception Counseling and Sexual Health, and Youth Transitioning into Adult HIV Care Services. A CoP engages recipient teams in improvement learning sessions using subject matter experts along with application experts who help recipient teams select, test, and implement changes on the front line of care. CoP integrates the proven components of the BTS. Subject matter experts will be identified who have experience in their own practice with breakthrough performance. Once RWHAP practices are enrolled in the CoP, learning sessions will include multidisciplinary teams from participant organizations and will focus on specific changes for improved care in the focus area, as well as success, barriers and lessons learned as CoP participants test change ideas in actual practice.

Through organizational self-assessments, didactic learning on specific care topics, goals setting and work plan development, each team can strategically benefit their organization. CoPs afford participants the opportunity to work in a group to solve a recognized challenge related to a COP domain and support dialogue among participants and the consultant/subject matter experts. Recipient teams commit to working over a period 12 months, alternating between Learning Sessions in which teams come together to learn about the chosen topic and to plan changes, and Action Periods in which the teams return to their respective organizations and test those changes in their clinic settings.

Action periods between learning sessions are designated for testing and implementing changes in local settings. Follow-up and support are provided between learning sessions among participants and experts. The IHI Model for Improvement provides a tested and systematic approach for implementing changes locally, identifying and tracking measurable aims and changes over time, and the use of testing cycles to test key change ideas and implement as appropriate. Through the contract, TA will be provided to RWHAP Part D WICY grant recipients to facilitate their adoption and implementation of evidence based and best practices in the focus areas and to disseminate HIV quality improvement methodologies.

Track the performance of all CoPs

HRSA has developed process and impact evaluation measures to track the performance of all CoP teams over the duration of the period of performance. DCHAP wants to ensure dissemination of lessons learned, evidence- based interventions, evidence -informed interventions and emerging interventions used during each CoP to various relevant audiences throughout the course of the proposed project and how any tools and resources developed will be utilized to sustain interventions after the project has ended. The integrated dissemination plan shall document, communicate and present information from the CoP, including interim findings and project updates for system dissemination, including submission of work into the Best Practices Compilation portal and dissemination to external and internal stakeholders via presentations, conferences, and meetings.

Process and outcome evaluation is a critical part of all activities, ensuring that initiatives were implemented as planned and met their intended outcome. Evaluation of coaching and support depends on establishing clear goals and plans from the beginning of the process. This includes specifying the intended impact of the coaching and support with concrete, measurable objectives. To judge performance against goals, we will administer coaching and support evaluation surveys following offsite coaching and support and training, webinars, teleconferences, and meetings. Our findings drive quality improvement activities and reports.

The proposed evaluation of CoPs will use principles from the Kirkpatrick Evaluation Model. The components of the Kirkpatrick Evaluation model that we propose to apply are reaction, learning, and behavior. We have operationalized these components to include measures of satisfaction with the coaching and support (reaction), change in knowledge after the coaching and support (learning), and change in behavior or practice after the introduction of evidence-based interventions (behavior). More specifically, the evaluation plan includes (1) post coaching and support satisfaction measures, (2) pre-post measures of CoP staff knowledge about effective practices, (3) measures of coaching and support usefulness and impact on CoP performance, and (4) pre-post-follow-up measures of CoP adoption and demonstration of evidence-based practices and effectiveness. Sample copies of the survey tools for CoP and coaching and support participants are included in Appendix A of this document.

The evaluation plan for coaching and support provided to CoP participants will focus on the performance of all CoP Teams, including participant learning (individuals as well as organizations), participant satisfaction, and participant behavior (or reported behavior). We propose to evaluate CoP coaching and support requests/events to be sure the CoP learns from those requests/events and can incorporate improvements as they continue their work in the community. Our evaluation plan includes both rapid process as well as outcome evaluations for the coaching and support provided to CoP participants, depending upon the type of coaching and support provided.

2. Purpose and Use of Information

Purpose

HAB/DCHAP executed a contract from September 2018 – June 2020, focusing on understanding ways for the RWHAP Part D program to reach and serve more people with HIV and improve their outcomes. The project team identified future directions for the RWHAP Part D program. Upon analysis of the study data, RWHAP identified several ways the Part D program could expand their reach and improve outcomes. They identified three areas of focus: Trauma-informed Interventions, Pre-conception Counseling, and Youth Transition into Adult HIV Care Services RWHAP programs. RWHAP will implement CoP for each of the three identified focus areas. HRSA seeks to implement a CoP platform for RWHAP Part D recipients starting in 2023 with an end of 2026. The goal is to increase delivery of evidence-based interventions that enhance client outcomes, increase the skill level of HIV workforce providing care and treatment to WICY, and involves partnerships for dissemination of best practices to Ryan White HIV/AIDS Program Part D participants consisting of subject matter experts engaged in learning sessions (i.e., when teams come together to learn about the chosen topic and to plan changes) followed by action periods (i.e., the teams return to their respective organizations and test those changes in their clinic settings.)

To improve and account for activities conducted by the three planned CoPs, there is a need to systematically evaluate the CoPs by tracking the performance of all CoPs as well as the process, content and effectiveness. HRSA will conduct ongoing evaluations of the planned CoP approach for HIV WICY to determine precisely how well the approach is working to build relationships, foster collaborative work across public health practices, and improve capacities. The evaluation will also serve to conduct self-assessment to inform decision-making and demonstrate effectiveness of a CoP to community stakeholders. The use of CoPs is a promising new approach and information collected through evaluations will assist HRSA in modifying the approach.

DCHAP wants to ensure dissemination of lessons learned, evidence- based interventions, evidence -informed interventions and emerging interventions used during each CoP to various relevant audiences throughout the course of the proposed project and how any tools and resources developed will be utilized to sustain interventions after the project has ended. The integrated dissemination plan shall document, communicate and present information from the CoP, including interim findings and project updates for system dissemination, including submission of work into the Best Practices Compilation portal and dissemination to external and internal stakeholders via presentations, conferences, and meetings.

Need and Proposed Use of the Information

The three CoPs HRSA designed to coach the Part D recipients is being introduced for the first time. It is therefore imperative a thorough understanding of what works and what does not is attained. Process and outcome evaluation are critical for ensuring that initiatives were implemented as planned and met their intended outcome.

The information will be used in several ways. First, as the three CoPs are offered sequentially, information and lessons learned from preceding CoPs will be used by HRSA to calibrate subsequent CoP offerings. Importantly, information will be used by HRSA to craft future policies on supporting public health and specifically the RWHAP. In addition, information gathered by conducting the evaluation will be used to fill any effective HIV treatment information gap. The information may also be helpful in assessing the usefulness of the breakthrough series approach in the context of RWHAP treatment and support services. With a focus on continuous quality improvement, RWHAP is committed to advancing best practices and addressing existing challenges that WICY with HIV experience, so it can expand its program reach and improve outcomes. Finally, as concerns regarding the cost and effectiveness of public health activities continue to grow, evaluation can be a powerful tool to inform decision-making and build support for continued funding.

3. Use of Information Technology

Electronic Monitoring, Feedback Process, Data Collection, and Data Analysis

HRSA will co-create an electronic monitoring and feedback process so that RWHAP Part D WICY recipients and federal staff may provide feedback on each CoP and every coaching and support activity (including quality of coaching and support and value of the coaching and support to recipients in supporting program implementation).

Data Collection and Data Analysis

This web-based solution will be used to collect the data using various assessment tools. All data collection and analysis approaches will be reviewed and approved by the COR or designated HRSA staff. We will target an 80 percent response rate to account for course attrition for evaluation activities. We will develop an observational checklist to assess whether principles and evidence-based and evidence-informed practices are integrated into practices. We will use the SurveyMonkey platform for CoP Team members to provide all the data needed to conduct the process and outcomes evaluation. The SurveyMonkey platform link for each of the assessment tools will be embedded on the project website with appropriate access and security controls. The data from each assessment provided will be stored in a database, analyzed using simple descriptive analysis (e.g., numbers, percentages, means, and ranges), and displayed in a visually engaging manner. The SurveyMonkey platform allows for real-time descriptive data analysis. However, if the need for conducting multivariate analysis is required, we will export the data from the platform into such statistical software as SAS or SPSS. We will take advantage of the simple graphical representation capabilities provided by the platform. But for more elaborate, visual depiction of the findings of the data analysis, we will export the data into our Tableau data visualization software.

The data collection strategy will leverage familiar, widely used information technology readily available online and at no-cost to the public. SurveyMonkey will be used to build, test, and administer the survey; implement quality control procedures that support participant engagement and a high response rate; collect responses; and generate reports to support analysis of the data.

This approach offers several advantages. When constructing the survey, it offers question-format templates, making it simple to match the question type (e.g., multiple choice, matrix of selections) to the question design. The implementation of skip logic is also easily accomplished, ensuring that the participant follows the correct path through the series of questions. The survey can be previewed and pilot-tested in this environment by multiple stakeholders, with all responses centralized and immediately available, and none of the limitations imposed by time and geography.

The online format supports the rapid initiation and closing of the survey, with multiple options for engaging participants. The survey itself can be accessed by the participant using the survey link. Only potential participants will receive the survey link and password. The approach supports monitoring and controlling of the technological aspects of the survey administration by the BHITs project team members, which also supports data integrity.

The security and privacy of this online survey tool was also assessed. SurveyMonkey is most able to address these concerns when compared to other “free” online tools available. For example, the SurveyMonkey business model is not based on archiving and then selling participant information and survey data. It is based on upselling the free product to a paid subscription version. It is its own business entity, not a subsidiary of a corporate entity that sells data collected through their “free” online service. In addition, it is a member of the privacy-seal program TRUSEe and is self-certified to the U.S.-E.U. Safe Harbor standards.

There are also specific activities the project team will take to better ensure participant security and privacy. While SurveyMonkey allows the end-user to send invitations via a participant email list input into their system, this would make the participants somewhat more vulnerable to spam and other types of unwelcome email. We will protect participant information by emailing the invitations via the project team’s secured email system.

Access to the data is another important consideration. The ability to generate reports from the data defines the overall utility of the data collected. SurveyMonkey helps the novice develop a wide range of different types of reports for organizing and interpreting the data. The level of effort that is usually expended in organizing and analyzing data is minimized, and the resources that will be necessary are easily anticipated. The effort of skilled data analysts will be better spent in implementing the plan for the manipulation and analysis of the data, rather than developing sets of queries and “scrubbing” the database contents to ensure meaningful answers.

4. Efforts to Identify Duplication and Use of Similar Information

This evaluation is collecting information unique to the three CoP domains (Trauma Informed Care, Pre-conception Counseling and Sexual Health, and Youth Transitioning into Adult HIV Care Services) that is otherwise not available. Information being collected by the HRSA Enhancing HIV Care Evaluation Assessment tools is unique and specific to the current project and cannot be obtained through other sources.

5. Impact on Small Businesses or Other Small Entities

Participation in this evaluation will not impose a significant impact on small entities. RWHAP Part D recipients and their partners may be small entities; however, the surveys are designed to request only the most pertinent information needed to be able to carry out the evaluation effectively, and their impact will not be significant.

6. Consequences of Collecting the Information Less Frequently

The multiple data collection points for the HRSA Part D CoP evaluation are necessary to track and evaluate grantees’ and subrecipient communities’ progress and change over time. In addition to the purposes of the HRSA Part D CoP evaluation, HRSA will use these data to monitor grantee performance, and grantee and subrecipient communities will use these data to track changes in their ongoing implementation. Less frequent reporting will affect HRSA’s and the grantees’ ability to do so effectively. The HRSA Part D CoP evaluation has made every effort to ensure that data are collected only when necessary and that extraneous collection will not be conducted.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This data information collection complies fully with 5 CFR 1320.5(d)(2).

8. Comments in Response to the Federal Register Notice/Outside Consultation

Section 8A:

A 60-day notice published in the Federal Register on February 27, 2023, vol. 88, No. 38; pp. 12386-12387. The one comment received was outside the scope of the ICR, and therefore no changes to the information collection were made as a result of this comment.

Section 8B:

The HRSA Part D CoP recipient evaluation survey of grantee participation in CoPs represents a cross-center collaboration among the HRSA offices. The process was informed by their comments on both the survey design and the survey questions. Outside consultation included the senior Evaluation and Technical Assistance experts from the Bizzell US and their subcontractor partner, Advocates for Human Potential, contractor on this project, HAB/DCHAP COR and Senior Leadership Team, and the HAB Office of General Counsel. Additionally, 9 of the 15 participants of the preconception counseling and sexual health community of practice were contacted to complete the assessment tools to understand the burden in February 2023. 5 of the 9 completed the assessment tools and provided feedback. This feedback was included in the 60 day and 30 day FRNs.

1. The year in which the consultation(s) took place: 2023

2. A summary of any major problems that could not be resolved during consultation.: Not Applicable

3. A description of other public contacts and opportunities for public comment, and a summary of the comments received: Not Applicable

9. Payment to Respondents

No incentives or gifts will be given to respondents.

10. Assurance of Confidentiality Provided to Respondents

The survey does not collect data that requires assurances of confidentiality. Therefore, the participants will not receive this assurance. To support candid and complete responses, the participants will be informed that the survey report will not identify or associate any individual response with a specific grantee. Responses will be analyzed, and the analysis presented on groupings of aggregate data. If an individual response is used to illustrate these data, it will not be attributed to a specific grantee or organization.

11. Justification for Sensitive Questions

No questions of a sensitive nature will be included in the data collection process or survey instrument.

12. Estimates of Annualized Hour and Cost Burden

The initial burden assessment conducted in February 2023 include 5 respondents who are participants of the preconception counseling and sexual health community of practice. Each respondent was asked the following:

1. Are the instructions in all the assessment tools clear?

2. Have you experienced any challenges with completing any of the assessment tools? If so, could you describe these challenges?

3. On average, how long (in hours) in total do you estimate it would take you or your organization to complete the assessment tools?

4. Is there anything else you would like to share regarding your experience(s) completing the assessment tools?

Participants of each CoP will complete their own specific CoP evaluation instruments. In addition, we will not evaluate TA until the second CoP. As a result, the burden and total respondent costs vary by year. The duration of each survey was confirmed in a pilot study of five volunteers. The instruments are provided in Appendix A. Estimated annual burden was calculated by estimating the average amount of time to complete each instrument, multiplied by the average hourly wage for participants of CoP teams. A typical CoP team will consist of a project director or manager, an HIV case manager, a clinical provider (physician, physician assistant, nurse practitioner, or registered nurse), up to two Part D participants that may be staff members (such as community health workers), and a quality assurance specialist. Average hourly wages were determined by finding the average hourly wage for each title using the latest data from the Bureau of Labor Statistics (May 2021), and then determining the average hourly wage for each CoP member. There are no direct costs to respondents other than the time to complete the instruments.

12A. Estimated Annualized Burden Hours

Table 1: Estimated Annualized Burden Hours

Form Name	Number of Respondents	Number of Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours
Pre-conception Counseling Community of Practice Retrospective Pretest-Post Assessment	90	1	90	0.4733	42.6
Community of Practice Pre-Assessment	180	1	180	0.2900	52.2
Community of Practice Post-Assessment	180	1	180	0.3767	67.8
Community of Practice Session Assessment	270	6	1,620	0.0767	124.3
Targeted and Intensive TA Assessment	120	1	120	0.0833	10.0
Foundational TA Assessment	150	1	150	0.0616	9.2
Total	990		2,340		306.1

12B.

The estimated total response burden and total respondent cost for the six assessment tools is projected to be $11,720.49 for 2023-2026 and is displayed in Table 2 below, which is what will be reflected in ROCIS. The average annual response burden is estimated to be $3,906.83. The average response burden respondent cost will vary over the 3.5-year period (2023-2026) and the variance is due to Community of Practice (CoP) #1 only completing four of the six assessment tools (Pre-conception Counseling Community of Practice Retrospective Pretest-Post Assessment, Community of Practice Session Assessments, Targeted and Intensive TA Assessments and Foundational TA Assessments). The major contributor to the variance is OMB approval was not established before year 1 began. The average burden estimated for CoPs #2 (2024-2025) and #3 (2025-2026) are about the same due to the anticipation of OMB clearance being established before CoPs #2 and #3 begin.

Tables 3, 4, and 5 display additional data on how the estimated average annual burden and total respondent costs vary each year.

Table 2. Estimated Total Response Burden and Total Respondent Cost for Surveys, 2023-2026

Form Name	Number of Respondents	Number of Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours	Average Hourly Wage	Total Respondent Costa
Pre-conception Counseling Community of Practice Retrospective Pretest-Post Assessment	90	1	90	0.4733	42.6	$36.82	$1568.42
Community of Practice Pre-Assessment	180	1	180	0.2900	52.2	$36.82	$1922.00
Community of Practice Post-Assessment	180	1	180	0.3767	67.8	$36.82	$2496.40
Community of Practice Session Assessment	270	6	1,620	0.0767	124.3	$36.82	$4576.73
Targeted and Intensive TA Assessment	120	1	120	0.0833	10.0	$36.82	$368.20
Foundational TA Assessment	150	1	150	0.0616	9.2	$36.82	$338.74
Total	990		2340		306.1		$11,720.49

^a Total respondent cost is calculated as total burden hours x average hourly wage.

Table 3. Estimated Annual Response Burden and Total Respondent Cost for Surveys, 2023-2024

Form Name	Number of Respondents	Number of Responses per Respondent	Total Responses	Average Burden per Response (in hours)	Total Burden Hours	Average Hourly Wage	Total Respondent Costa
Pre-conception Counseling Community of Practice Retrospective Pretest-Post Assessment	90	1	90	0.4733	42.6	$36.82	$1,568.42
Community of Practice Pre-Assessment	90	1	90	0.2900	26.1	$36.82	$961.00
Community of Practice Session Assessment	90	6	540	0.0767	41.418	$36.82	$1,525.01
Targeted and Intensive TA Assessment	26	1	26	0.0833	2.17	$36.82	$79.74
Foundational TA Assessment	33	1	33	0.0616	2.03	$36.82	$74.85
Total	329		779		114.318		$4209.02

^a Total respondent cost is calculated as total burden hours x average hourly wage.

Table 4. Estimated Annual Response Burden and Total Respondent Cost for Surveys, 2024-2025

Instrument	Number of Respondents	Responses per Respondent	Total Number of Responses	Hours per Response	Total Burden Hours	Average Hourly Wage	Total Respondent Costa
Community of Practice Post-Assessment	90	1	90	0.3767	33.903	$36.82	$1,248.31
Community of Practice Pre-Assessment	90	1	90	0.29	26.1	$36.82	$961.00
Community of Practice Session Assessment	90	6	540	0.0767	41.418	$36.82	$1,525.01
Targeted and Intensive TA Assessment	47	1	47	0.0833	3.915	$36.82	$144.15
Foundational TA Assessment	59	1	59	0.0616	3.6344	$36.82	$133.82
TOTAL	376		826		108.97		$4012.29

^a Total respondent cost is calculated as total burden hours x average hourly wage.

Table 5. Estimated Annual Response Burden and Total Respondent Cost for Surveys, 2025- 2026

Instrument	Number of Respondents	Responses per Respondent	Total Number of Responses	Hours per Response	Total Burden Hours	Average Hourly Wage	Total Respondent Costa
Community of Practice Post-Assessment	90	1	90	0.3767	33.903	$36.82	$1,248.31
Community of Practice Session Assessment	90	2	180	0.0767	13.806	$36.82	$508.34
Targeted and Intensive TA Assessment	47	1	47	0.0833	3.915	$36.82	$144.15
Foundational TA Assessment	58	1	17	0.0616	1.0472	$36.82	$38.56
TOTAL	285		334		52.6712		$1939.36

^a Total respondent cost is calculated as total burden hours x average hourly wage.

13. Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/Capital Costs

There are no respondent costs for capital, start-up, operations, or maintenance associated with this data collection.

14. Annualized Cost to Federal Government

The total estimated 4-year cost to the government for the data collection is $528,139.21. This includes approximately $510,221 for developing the instruments; programming and maintaining the online data collection system; providing data collection training to grantees and subrecipients; processing, cleaning, and housing data; and analyzing and reporting data. HRSA costs to monitor and manage the data collection and analysis project is approximately $17,918 for all four years. The overall annualized cost is approximately $132,034.80.

15. Explanation for Program Changes or Adjustments

There are no changes to the information collection. This is a new collection of information.

16. Plans for Tabulation, Publication, and Project Time Schedule

A.16.a. Time Schedule

Table 5 outlines the key time points for the evaluation and data collection.

Exhibit 5. Time Schedule for Data Collection

Activity	Time Schedule
Prepare for data collection including programming web system	April 2023 – December 2023
Obtain OMB approval for data collection	April 2023 – December 2023 (Estimated)
Collect data	February 2024 February 2026
Collect Pre-conception Counseling Community of Practice (CoP)#1 Pre and post Assessments	February 2024
Collect CoP#2-Pre-Assessment	March 2024
Collect CoP#2 Post-Assessment	February 2025
Collect CoP#3 Pre-Assessment	March 2025
Collect CoP#3 Post-Assessment	February 2026
Collect C0P#1 Session Assessment	December 2023-February 2024
Collect C0P#2 Session Assessment	March 2024-February 2025
Collect C0P#3 Session Assessment	March 2024-February 2025
Collect Targeted and Intensive TA Assessments	September 2023-February 2026
Collect Foundational TA Assessments	September 2023-February 2026
Analyze data	February 2024-May 2026
Disseminate findings	March 2024-July 2026

16.b. Analyses and Publication

Publication

The HRSA Part D Recipient CoP evaluation will help HRSA reach its diverse stakeholders through targeted products and innovative dissemination venues. The evaluation’s objective for all reports and dissemination products is to provide user-friendly documents and presentations that help HRSA successfully disseminate and explain the findings. The dissemination plan includes products in a variety of formats for a variety of target audiences. Audiences for these reports will include Congress, the ONDCP, SAMHSA Centers, the evaluation’s HRSA Contracting Officer’s Representatives (CORs), HRSA Part D Recipient CoP grantees, and the broader HIV/AIDS prevention field (e.g., academia, researchers, policymakers, providers).

The HRSA Part D Recipient CoP evaluation recognizes that different audiences are best reached by different types of report formats. For example, reports to Congress and Federal agencies will require materials that are concise but offer policy-relevant recommendations. Reports created for HRSA Offices and Bureaus and the CORs will require more in-depth information, such as substantive background and discussion sections, to supplement the analytic approach. Reports created for HRSA Part D Recipient CoP grantees will be concise handouts with helpful and easy-to-read graphics on performance data rather than lengthy text. The HRSA Part D Recipient CoP evaluation will develop an assortment of disseminations products, including short and long analytic reports, congressional briefings, annual evaluation reports, research and policy briefs, ad hoc analytic reports, journal articles, best practice summaries, and conference or other presentations.

16.c. Analyses and Publication

Analysis

The HRSA Part D Recipient CoP evaluation uses a series of interdependent analysis frameworks that have been selected to maximize the coverage of the key Evaluation Questions posed for assessing the objectives of HRSA Part D Recipient Cops. The analysis plan proposes a series of analyses that move from basic descriptive analyses (e.g., means, frequencies, percentages) to the use of sophisticated quantitative analysis techniques.

The HRSA Part D Recipient CoP evaluation will use a pre/post design. Evaluation design and Evaluation Questions guided the selection of the analysis framework. In addition to descriptive analyses, appropriate statistical techniques to estimate effectiveness using repeated measures design will be used.

A repeated measures design involves measuring the same variable on the same subjects at multiple points in time or under multiple conditions. In a repeated measures ANOVA, the within-subjects variability is partitioned into different sources of variation, including the effect of the independent variable (such as the CoP coaching), the effect of time, and the interaction between the independent variable and time.

The repeated measures ANOVA has several advantages over other types of ANOVA, including increased power, reduced error variance, and the ability to control for individual differences between subjects. However, it also has some assumptions that need to be met, such as normality of the distribution of the outcome variable and sphericity (the equality of variances of the differences between all pairs of conditions or time points). Violations of these assumptions can affect the validity of the results.

If the assumptions of repeated measures ANOVA are violated, we will consider using other statistical approaches such as mixed-effects models or generalized estimating equations (GEE). These methods can provide more flexibility and can handle missing data, non-normality, and other issues that may arise in repeated measures designs.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

OMB approval expiration dates will be displayed.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

HRSA is able to certify compliance with all provisions under Item 19, “The Certification for Paperwork Reduction Act.” There are no exceptions to the certification.

10