CLEAR Evaluation Field Guide
Choosing Life: Empowerment! Action! Results!, CLEAR, treat, prevent, HIV, AIDS, Centers for Disease Control and Prevention, CDC, Effective Interventions, Ending the HIV Epidemic, EHE, persons with HIV, behavioral change, capacity building assistance, training
Centers for Disease Control Prevention (CDC)
CLEAR Evaluation Field Guide
cdc-hiv-ei-clear-evaluation-field-guide ACKNOWLEDGMENTS
The CLEAR Evaluation Field Guide was developed with funding from the Centers for Disease Control and Prevention (CDC). Dr. Aisha Gilliam provided leadership in the development of this document, reviewed the guide, and provided valuable recommendations to the content.
We wish to acknowledge the efforts of the development team of Macro International Inc. and the support of Macro's HIV Project Director, Dr. David Cotton.
It is hoped that this guide will prove useful to those implementing the CLEAR program across the nation. It is our goal to keep this guide and its information as current as possible. To achieve this, we welcome your comments. Please contact Dr. Gilliam, DHAP, CDC, via electronic mail at aisha.gilliam@cdc.hhs.gov with any comments or concerns.
CLEAR Evaluation Field Guide--September 2008
i
TABLE OF CONTENTS
INTRODUCTION........................................................................................................1 Purpose...................................................................................................................... 1 Modifying Materials .................................................................................................. 2 Organization of this Document ................................................................................ 2 Theoretical Basis and Core Elements...................................................................... 3
SECTION 1: REPORTING HIV PREVENTION PROGRAM INFORMATION TO CDC....................5 NHM&E Program Planning Data .............................................................................. 6 NHM&E Client Services Data .................................................................................18
SECTION 2: CLEAR OBJECTIVES AND EVALUATION QUESTIONS ................................... 19 CLEAR Program Objectives.....................................................................................19 Process Monitoring Questions ...............................................................................19
SECTION 3: DATA COLLECTION SCHEDULE AND ACTIVITIES.......................................... 25 Data Collection Activities........................................................................................25
SECTION 4: DATA COLLECTION PROTOCOLS.............................................................. 28 Program Enrollment Form Risk Reduction Interview Session Fidelity Forms (Core Skill Sessions 15) Client Participation Record Form Program Monitoring Summary
APPENDIX A CLEAR Behavioral Risk Analysis B CLEAR Conceptual Framework C CLEAR Logic Model D 2008 National HIV Prevention Program Monitoring and Evaluation Data Set (NHM&E DS) Variable Requirements E References
CLEAR Evaluation Field Guide--September 2008
ii
INTRODUCTION
PURPOSE
The CLEAR Evaluation Field Guide was developed to provide community-based organizations implementing CLEAR (Choosing Life: Empowerment, Action, Results!) with systematic methods to conduct evaluation processes and activities that will inform, guide, and assess their CLEAR activities and their effectiveness. The evaluation field guide recommends staff responsibilities; indicates how an agency should track intervention activities and collect and manage data; states how data could be analyzed; and suggests plans for the dissemination of the data to CLEAR stakeholders. This field guide is designed as a supplement to the Evaluation Capacity Building Guide developed for the Capacity Building Branch (CBB), Division of HIV/AIDS Prevention (DHAP), National Center for HIV, Hepatitis, STD, and TB Prevention (NCHHSTP), Centers for Disease Control and Prevention (CDC), under a contract with Macro International (CDC, 2008a).
This manual is one of several documents disseminated by DHAP to provide information and guidance on HIV prevention program evaluation, data collection, data utilization, and uses the variables included in CDC's National HIV Prevention Program Monitoring and Evaluation Data Set (NHM&E DS). Related documents include:
Evaluation Capacity Building Guide. This guide provides an overview of monitoring and evaluating evidence-based interventions, with particular focus on process monitoring and evaluation activities, tools, and templates (CDC, 2008a).
National Monitoring and Evaluation Guidance for HIV Prevention Programs (NMEG). This manual provides a framework and specific guidance on using NHM&E DS variables to monitor and evaluate HIV prevention programs (CDC, 2008b).
Program Evaluation and Monitoring (PEMS) User Manual. This how-to manual describes the functionality within the application and provides step-by-step instructions for each module within the Web-based software tool. Screenshots, example extracts of data, and reports are used to illustrate key features included in the PEMS software. You can download this manual at the PEMS Web site (http://team.cdc.gov) under Trainings/PEMS User Manual (CDC, 2008c).
National HIV Prevention Program Monitoring and Evaluation Data Set. The complete list and description of all M&E variables required for reporting to CDC and optional for local M&E and specific to certain interventions (CDC, 2008d).
Disclaimer: The reporting requirements for the National HIV Prevention Program Monitoring and Evaluation Data Set presented in this document are current as of September 2008. Please refer to the PEMS Web site (https://team.cdc.gov) for the most current reporting requirements.
These documents provide a foundation for monitoring and evaluating HIV prevention programs and reporting required data using PEMS software. Health departments and organizations directly funded by CDC can request monitoring and evaluation technical assistance through the Capacity Building Branch's Web-based system, Capacity Request Information System (CRIS). For more information about and access to CRIS, visit http://www.cdc.gov/hiv/cba. Additional information or technical assistance for the National HIV Prevention Program Monitoring and
CLEAR Evaluation Field Guide--September 2008
1
Evaluation Plan and the PEMS software may be accessed through the Program Evaluation Branch's National HIV Prevention Program Monitoring and Evaluation Service Center, which you can reach by calling 1-888-PEMS-311 (1-888-736-7311) or e-mailing pemsservice@cdc.gov; visiting the PEMS Web site (https://team.cdc.gov); or contacting the DHAP Help Desk (1-877659-7725 or dhapsupport@cdc.gov).
MODIFYING MATERIALS
The evaluation questions and data collection forms contained in this document are very general in nature. These questions and data collection forms reflect the reporting requirements of CDC1 and the basic monitoring and evaluation requirements of CLEAR. Your agency may have additional reporting requirements or you may have information needs within your organization that are not reflected in the evaluation questions or data collection forms. The data collection forms and questions can be modified to reflect the needs of your organization. The Evaluation Capacity Building Guide provides additional information on developing an agency specific evaluation plan (CDC, 2008a).
ORGANIZATION OF THIS DOCUMENT
Section 1 of the document contains an overview of CDC's reporting requirements for CLEAR. Section 2 contains the evaluation objectives, followed by evaluation questions. A brief narrative that describes the relevance of the question follows each question. The table below each question provides a list of data that would answer the question, methods that can be used to obtain the data, and recommendations on how to analyze the data so that you can use the information to enhance your implementation of CLEAR and plan future implementation. Section 3 has data collection tables that summarize the data collection activities (arranged by primary activities), recommend data collection schedules, provide a brief description of agency resources needed, and suggest ways to use the data. Section 4 includes all the required and optional CLEAR instruments. Each evaluation instrument is arranged by CLEAR activity. The appendixes consist of the CLEAR behavioral risk analysis (Appendix A), conceptual framework (Appendix B), logic model (Appendix C), and a list of the NHM&E DS variables (not all of which are required for this intervention) (Appendix D).2
The development of the CLEAR Evaluation Field Guide was informed by the development of a behavioral risk analysis, conceptual framework, and logic model. The risk analysis explores possible circumstances that may place members of the target population at risk for acquiring or transmitting HIV and factors that may contribute to that risk. The conceptual framework links the types of intervention activities to the risk and protective factors identified in the behavioral risk analysis. The logic model describes the relationships between risk behaviors, the activities of the intervention, and the intended outcomes. These appendices are based on program materials and consultations with members of the Science Application Team within the Capacity Building Branch.
1 NHM&E DS program planning, HIV testing, and agency data variables were finalized for January 1, 2008, reporting per the Dear Colleague Letter. The evaluation instruments in this guide are templates designed to capture data for evaluating the CLEAR in its entirety. They are also designed to capture most program planning and client services NHM&E DS variables. Agencies should check with their CDC Project Officer or other contract monitors specific reporting requirements for CLEAR.
2 The variable requirements in Appendix D are for the January 1 and July 1, 2008, data collection periods, excluding variable requirements for HIV Testing and Partner Counseling and Referral Services (PCRS). Since this document only provides a summary of the requirements, please refer to the NHM&E DS (CDC, 2008d) for a more detailed description of definitions and value choices.
CLEAR Evaluation Field Guide--September 2008
2
THEORETICAL BASIS AND CORE ELEMENTS
CLEAR is an individual-level intervention that was designed for individuals ages 16 and older who are living with HIV/AIDS or at high risk for HIV. The intervention provides them with the skills necessary to live their best life and to make healthy choices. The objectives of CLEAR are to increase behaviors that promote
Healthy living
Effectively facing the challenges of daily living
Positive feelings, thoughts, and actions
Developing daily routines to stay healthy
The intervention is based on Social Action Theory. Social Action Theory asserts that a person's ability to change behaviors that endanger his or her health is influenced by the individual's cognitive capability (ability to think, reason, imagine, etc.), as well as environmental factors and social interactions that encourage or discourage the change process. Social Action Theory incorporates the principles that are expressed in traditional social-cognitive models of healthbehavior change. These models include Social Cognitive Theory, the Health Belief Model, and the Transtheoretical Model (Stages of Change), as well as theories related to social context, interpersonal relationships, and environmental influences.
With Social Action Theory as its foundation, CLEAR applies cognitive-behavioral strategies to maintaining health, reducing the risk for HIV and STI transmission or reinfection, and improving the quality of life of youth and adults living with HIV/AIDS. Strategies in the intervention include role-playing as a means of learning new skills and improving old ones, building client's belief that he or she can change a behavior (self-efficacy); and instilling the belief that changing behaviors will result in a desired outcome (response efficacy). The cognitivebehavioral strategies used in the intervention are introduced within the framework of the intervention's core elements (Rotheram-Borus, Swendeman, Comulada, Weiss, Lee, & Lightfoot, 2004).
The original CLEAR intervention was demonstrated to be effective with HIV-positive youth over a 15-month period by increasing their likelihood of engaging in safer sex behaviors, specifically condom use, and by reducing the number of sexual partners. During its preparation for use in the field, CLEAR was modified in several ways to make implementation easier. More detail is available in the implementation manual (Rotheram-Borus et al., 2004). CLEAR is one of the interventions developed by the CDC Replication of Effective Programs (REP). There are five core elements of the intervention (Table 1). "Core elements are those parts of an intervention that must be done and cannot be changed. They come from the behavioral theory upon which the intervention or strategy is based; they are thought to be responsible for the intervention's effectiveness. Core elements are essential and cannot be ignored, added to, or changed." (CDC, April 2006).
CLEAR Evaluation Field Guide--September 2008
3
TABLE 1. THE CORE ELEMENTS OF CLEAR 1. Development of emotional awareness through use of a Feeling Thermometer and identification of
the link among feelings, thoughts, and actions (F-T-D Framework) 2. Identification of Ideal Self to help motivate and personalize behavior change 3. Teaching, modeling, and practicing Short-and Long-Term Goal Setting 4. Teaching, modeling, and practicing SMART Problem-Solving 5. Teaching, modeling, and practicing Assertive Behavior and Communication
In addition to core elements, there are four key characteristics of CLEAR (Table 2). Key characteristics are activities and delivery methods for conducting an intervention that, while considered of great value to the intervention, can be altered without changing the outcome of the intervention. They can be adapted and tailored for your agency or target populations (CDC, 2003).
TABLE 2. THE KEY CHARACTERISTICS OF CLEAR* 1. Use of incentives to encourage clients to return to sessions. It is up to each implementing agency to
decide whether or not to use incentives, what kind to use, and the estimated value of an incentive. The most appropriate incentive strategies are those that the agency's community advisory group and client pool think will work best to encourage attendance and participation. 2. Time. With practice, all sessions can be finished in the 60- to 75-minute time period indicated in the script of each session. It is recommended that the sessions be kept to the amount of time allocated for each session as often as possible. 3. Intervals between sessions can be tailored to the needs and capacity of the agency and population. A general rule of thumb is to conduct sessions once a week. A biweekly schedule may also work, although monthly sessions are not recommended except in very unusual situations. When planning for the session frequency, there are several things to consider: Time for clients to think about what they have experienced Ability to retain clients Availability of both clients and counselors It is not recommended that an agency conduct all core or menu sessions in 1 day or a weekend 4. Location: CLEAR can be held anywhere there is a private room. The venue and room should be handicapped accessible. For some communities, venues that advertise services for people living with HIV/AIDS are not good places to hold CLEAR sessions. Some clients have not disclosed their status and therefore would not attend sessions at a place that would compromise their privacy.
* These key characteristics bring immediate credibility and access to groups.
CLEAR Evaluation Field Guide--September 2008
4
SECTION 1: REPORTING HIV PREVENTION PROGRAM INFORMATION TO CDC
CDC has undertaken significant efforts to ensure that the HIV prevention programs it funds are effective in preventing the spread of HIV (Thomas, Smith, & Wright-DeAgüero, 2006). One strategy employed by CDC to strengthen HIV prevention is improving organizational capacity to monitor and evaluate prevention programs (CDC, 2007). The National HIV Prevention Program Monitoring and Evaluation Data Set (NHM&E DS) is a major component of this strategy.
The NHM&E DS is the complete set of CDC's HIV prevention monitoring and evaluation (M&E) variables, including required variables for reporting to CDC and optional variables specific to an intervention or for local M&E. Implementation of NHM&E DS makes it possible for CDC to answer critical national questions about the following:
Demographic and risk behavior of clients being served by its grantees Resources used to provide these services Effectiveness of these services in preventing HIV infection and transmission
All HIV prevention grantees funded by CDC are required to collect and report data using the NHM&E DS. CDC has provided various M&E resources to assist grantees in this effort, including the following:
National Monitoring and Evaluating Guidance for HIV Prevention Programs (NMEG)--describes how to use the NHM&E DS to improve program, inform programmatic decisions, and answer local M&E questions (CDC, 2008b).
Program Evaluation and Monitoring System (PEMS) software--an optional, secure, browser-based software that allows for data management and reporting. PEMS includes all required and optional NHM&E DS variables (CDC, 2008c).
Disclaimer: The reporting requirements for the National HIV Prevention Program Monitoring and Evaluation Data Set presented in this document are current as of September 2008. Please refer to the PEMS Web site (https://team.cdc.gov) for the most current reporting requirements.
The NHM&E DS is organized into a series of data tables with specific variables. Variables from these tables are captured in the PEMS software in different modules according to categories, (e.g., information about your agency, your HIV prevention programs, and the clients you serve). You should be familiar with following key elements in the NHM&E DS:
Variables required for reporting to CDC and optional variables needed for the CLEAR intervention or for local M&E
Variable name Variable number Definition of each variable
CLEAR Evaluation Field Guide--September 2008
5
This evaluation field guide is designed to help your agency monitor and evaluate your day-to-day implementation of CLEAR. Collecting and analyzing CLEAR data will help you improve your implementation of CLEAR and provide you with information to guide future planning. This section details only those tables and associated NHM&E DS modules you will use to collect and report information specific to CLEAR. Though the data you collect will include NHM&E DS variables, you will collect and use more data than actually submitted to CDC. Please refer to the National HIV Prevention Program Monitoring and Evaluation Data Set (NHM&E DS) for the complete list and description of all M&E variables required for reporting to CDC and optional variables for local M&E.
NHM&E PROGRAM PLANNING DATA
NHM&E DS program planning data provide information about what you intend to do. Your program plan describes the following:
The population you will serve with CLEAR The name you will use for CLEAR within your agency The activities within CLEAR you will deliver The funds available to support delivery of the intervention Staff who will deliver the intervention How the interventions will be delivered How many times the intervention will be delivered
Carefully describing your program is a process that will help your agency determine how to best implement and monitor CLEAR. A clearly described and well-thought-out program plan will allow you to use your process monitoring data to conduct process evaluations. Please refer to CDC's Evaluation Capacity Building Guide (CDC, 2008a) for additional information on conducting process evaluations and using that information to plan and improve your implementation of CLEAR.
Recommended Activity
Review your client intake and session record forms to ensure you are gathering all the required NHM&E DS variables and the optional variables specific to CLEAR.
Figure 1 illustrates how CLEAR is organized in the NHM&E DS.
CLEAR Evaluation Field Guide--September 2008
6
Figure 1. Organization of CLEAR in NHM&E DS
Program Model: Project CLEAR
Intervention type: Individual-level intervention (NHM&E DS variable F01 = 06)
1 cycle = 1 client
Intervention Plan Ch aracteristics:
PreImplementation
/Intake
Screen/Enrollment Interview
Intervention Plan Ch aracteristics:
Core Skill Sessions (1-5) Individual sessions that focus on the Core elements through instruction, discussion, and
role-plays
Intervention Plan Characteristics: Menu Sessions
(1-21) Individual sessions
that focus on 6 possible topics that
advance clients' risk -related
personal goals
Intervention Plan Characteristics:
Wrap-Up Session Individual session that focuses on behavior-change maintenance
Client Level Data: Client Characteristics demographic and risk profile (NHM&E DS table G)
Intervention session details (NHM&E DS table H)
Client Behavioral Characteristics
Client Level Data: Update client characteristics as needed
(NHM&E DS table G) Intervention details for each client session
(NHM&E DS table H)
Referrals*
Client Level Data: Update client characteristics
(NHM&E DS table G)
Intervention Session Details
Client Behavioral Characteristics
* In NHM&E DS, reporting on referral information is required when agency staff provide a formal referral for which they intend to conduct a referral follow up.
Table 3 provides guidance on selecting NHM&E DS variables you can use to describe your intervention as you develop your program plan. This table depicts program information variables that are applicable to CLEAR and identifies which variables are required by CDC. For instance, Program Model Name (NHM&E DS E101) is labeled "Agency Determined" because the name of your program model can be CLEAR or any other name determined by your agency. The Evidence Base (NHM&E DS E102) variable, however, specifies a particular variable code ("1.15") because, regardless of what you have named your program, it is based on the CLEAR model, one of CDC's Effective Behavioral Interventions.
Note that the variables presented in the table include only those specific to CLEAR; additional, agency-specific variables are required. Please refer to the National HIV Prevention Program Monitoring and Evaluation Data Set (CDC, 2008d) or the for the complete list and description of all M&E variables required for reporting to CDC and optional variables for local M&E or the 2008 National HIV Prevention Program Monitoring and Evaluation Data Set Variable Requirements (Appendix D).
CLEAR Evaluation Field Guide--September 2008
7
VARIABLE Program Model Name
Evidence Base Target Population
NHM&E DS NUMBER
E101
E102 E105
TABLE 3. PROGRAM INFORMATION
VARIABLE CODE
GUIDANCE
Agency determined
The name of the Program Model can be CLEAR or any other name determined by the agency. See the National Monitoring and Evaluation Guidance for HIV Prevention Programs (CDC, 2008b) for additional information if you are implementing more than one CLEAR within the same program.
1.15
CLEAR (variable value code: 1.15).*
Agency determined
CLEAR was designed for individuals ages 16+ who are living with HIV/AIDS or are at high risk for HIV. If you are targeting a different population with CLEAR, select the appropriate variable code.
* Organizations funded directly by CDC to implement CLEAR are required to adhere to the core elements of the intervention. Other organizations may alter or not follow the core elements at the discretion of their funding agency; however the program can no longer be called CLEAR. If you intend to drop or change a core element of CLEAR to meet the needs of your priority populations, use the fields provided in E104 to describe the changes to the core elements.
Intervention Plan Characteristics provide information about what you plan to do in your implementation of CLEAR. It describes the activities you intend to implement, the planned number of cycles and sessions, the duration of the cycles, whether client services data will be collected at the aggregate or individual level from CLEAR participants. Table 4 lists the NHM&E DS Intervention Plan variables with the variable number and code, variables required to be reported to CDC and guidance to help you understand how to apply these variables when implementing CLEAR.
Note that the variables presented in Table 4 include only those specific to monitoring CLEAR. Additional, agency-specific variables are required. The complete list and description of all M&E variables required for reporting to CDC and optional variables for local M&E or the 2008 National HIV Prevention Program Monitoring and Evaluation Data Set Variable Requirements can be found in Appendix D. Please refer to the National HIV Prevention Program Monitoring and Evaluation Data Set (CDC, 2008d) for further information and updates.
CLEAR Evaluation Field Guide--September 2008
8
VARIABLE Intervention Type Total Number of Clients Planned Number of Cycles Number of Sessions
Unit of Delivery Activity
Activity
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
F01
06
CLEAR is a Health Education/Risk Reduction intervention (variable value choice: 06).
F05
Agency
The total number of clients equals the total number of
determined individuals expected to be served by CLEAR.
F07
Agency
A cycle is the complete delivery of an intervention to its
determined intended audience. For CLEAR, because it is an individual-
level session, provide the number of individuals expected
to receive the intervention (same as F05 above).
F08
Agency
A response of "unknown" is appropriate: the number of
determined sessions in CLEAR is not standard across all clients;
instead, it is determined individually during service delivery.
You should instead note the number of sessions in variable H02: Intended Number of Sessions for each client enrolled in the intervention.
F09
09
CLEAR is delivered to individuals (variable value code: 09).
CORE SKILL SESSION 1: GETTING TO KNOW EACH OTHER
F10
08.15
What Is Our
11.66 Discussion--Other
11.12
Commitment?
11.19 11.66
How Do I Feel About Living With HIV/AIDS?
11.12 Discussion--Living with HIV/AIDS
What Are Good Goals?
08.15 11.19
Information--Decision making
Discussion--Decision making
CORE SKILL SESSION 2: CREATING A VISION FOR THE FUTURE
F10
08.15
What Is My Ideal 08.15 Information--Decision
08.66
Self?
11.19 making
10.66 11.19
Discussion--Decision making
How Can I Create a 11.19 Discussion--Decision
Vision for My
making
Future?
How Can I Relax? 08.66 Information--Other
10.66 Practice--Other
CLEAR Evaluation Field Guide--September 2008
9
VARIABLE Activity
Activity
Activity
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
CORE SKILL SESSION 3: STRESSORS AND SMART PROBLEM-SOLVING
F10
08.15
What Are My
10.04 Practice--Decision making
08.66
Current Stressors? 11.19 Discussion--Decision
10.04
making
10.66
What Is CLEAR
08.15 Information--Decision
11.19
Thinking?
11.19
making Discussion--Decision
making
What Is SMART Problem-Solving?
10.04 Practice--Decision making
How Can I Relax? 08.66 Information--Other
10.66 Practice--Other
CORE SKILL SESSION 4: EXPLORING DIFFERENT TYPES OF COMMUNICATION
F10
08.66
How Is
08.66 Information--Other
10.66
Communication Related to F-T-D?
What Are Different Types of Communication?
08.66
Information--Other
How Can I Apply Assertive Communication in My Life?
10.66
Practice--Other
How Can I Relax? 08.66 Information--Other
10.66 Practice--Other
CORE SKILL SESSION 5: PUTTING IT ALL TOGETHER
F10
05.00
What Prevention
05.00 Personalized Risk
Steps Have I
Assessment
Already Taken?
What Prevention Goals Do I Want to Start Working on as a Part of My Prevention Plan?
05.00
Personalized Risk Assessment
CLEAR Evaluation Field Guide--September 2008
10
VARIABLE Activity Activity
Activity
Activity
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
SEXUAL RISK 1: UNDERSTANDING WHY I HAVE UNSAFE SEX
F10
05.00
What Leads Me to 05.00 Personalized Risk
11.01
Have Unsafe Sex? 11.01 Assessment
11.21
11.21 Discussion--Sexual risk reduction
Discussion--Alcohol and drug use prevention
How Do I Handle My Unsafe Sex Triggers
11.01 Discussion--Sexual risk reduction
SEXUAL RISK 2: HOW TO USE CONDOMS (CORRECTLY)
F10
05.00
What Are My
05.00 Personalized Risk
08.13 09.01 11.01
Thoughts About Condoms?
11.01
Assessment
Discussion--Sexual risk reduction
How Do I Use a
08.13 Information--
Female Condom? 09.01 Condom/barrier use
Demonstration--
Condom/barrier use
How Do I Use a Male Condom?
08.13 09.01
Information-- Condom/barrier use
Demonstration-Condom/barrier use
SEXUAL RISK 3: CAN I INFLUENCE MY PARTNER TO USE CONDOMS?
F10
05.00
What Does It Mean 05.00 Personalized Risk
08.14 10.03 11.17 11.18
to Influence My Partner?
11.17 11.18
Assessment
Discussion-- Condom/barrier use
Discussion-- Negotiation/ communication
How Do I Influence My Partner to Accept Condoms?
08.14 10.03
Information-- Negotiation/ communication
Practice--Negotiation/ communication
SEXUAL RISK 4: CAN I INFLUENCE MY PARTNER TO ENGAGE IN SAFER SEX?
F10
08.10
What's Safe?
08.10 Information--Sexual risk
08.14
11.01 reduction
10.03 11.01
Discussion--Sexual risk reduction
How Do I
08.14 Information--
Communicate My Safer Sex Desires to My Partner?
10.03
Negotiation/ communication
Practice--Negotiation/
communication
CLEAR Evaluation Field Guide--September 2008
11
VARIABLE Activity
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
SEXUAL RISK 5: HOW DO I REFUSE UNSAFE SEX?
F10
08.14
How Do I Refuse
08.14 Information--
10.03
Unsafe Sex?
10.03 Negotiation/
communication
Practice--Negotiation/ communication
Activity
SEXUAL RISK 6: SHOULD I DISCLOSE MY STATUS TO MY SEXUAL PARTNER(S)?
F10
08.16
How Do I Feel
11.05 Discussion--Disclosure of
10.05
About Disclosure?
HIV status
11.05
The Pros and Cons of Sharing My Status
11.05
Discussion--Disclosure of HIV status
How Do I Tell a Partner I Am HIV Positive?
08.16 10.05
Information--Disclosure of HIV status
Practice--Disclosure of HIS status
Activity
SUBSTANCE USE RISK 1: SETTING A FOUNDATION FOR CHANGE
F10
08.15
What Are the Pros 11.21 Discussion--Alcohol and
08.21 11.19
and Cons of My Substance Use?
drug use prevention
11.21
How Comfortable Am I with the Cons of My Substance Use?
11.19 11.21
Discussion--Decision making
Discussion--Alcohol and drug use prevention
Keeping Track of Progress
08.15 08.21
Information--Decision making
Information--Alcohol and drug use prevention
Activity
SUBSTANCE USE RISK 2: WHAT ARE MY EXTERNAL DRUG AND ALCOHOL TRIGGERS?
F10
05.00
What Keeps My
08.21 Information--Alcohol and
08.21 11.21
Drug and Alcohol Use Going?
drug use prevention
What Are My
05.00 Personalized risk
External Triggers for Drug and Alcohol Use?
08.21
assessment
Information--Alcohol and drug use prevention
How Can I Handle My External Triggers?
08.21 11.21
Information--Alcohol and drug use prevention
Discussion--Alcohol and drug use prevention
CLEAR Evaluation Field Guide--September 2008
12
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
VARIABLE
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
SUBSTANCE USE RISK 3: WHAT ARE MY INTERNAL DRUG AND ALCOHOL TRIGGERS?
Activity
F10
05.00 11.19 11.21
What Are My Internal Triggers for Drug and Alcohol Use?
05.00
Personalized risk assessment
How Can I Handle My Internal Triggers and Unhelpful Thoughts?
11.21
Discussion--Alcohol and drug use prevention
What If I Slip?
11.19 11.21
Discussion--Decision making
Discussion--Alcohol and drug use prevention
SUBSTANCE USE RISK 4: WHAT WILL HELP ME ACHIEVE MY GOAL AROUND INJECTION DRUG USE?
Activity
F10
05.00 08.11 11.02
What Do I Think About Changing My Injection Drug Use Behaviors
05.00
Personalized risk assessment
What Are My Injection Drug Use Triggers?
11.02
Discussion--IDU risk reduction
How Do I Handle My Triggers and Unhelpful Thoughts?
11.02 Discussion--IDU risk reduction
What's Harmful About Injecting Drugs?
08.11 Information--IDU risk reduction
Activity
F10
SUBSTANCE USE RISK 5: DRUGS, ALCOHOL, AND HIV
08.07
What Do People
08.07 Information--Living with
08.21 10.66 11.07 11.21
Living With HIV Need to Be Aware of When It Comes to Substance Use?
08.21
HIV/AIDS
Information--Alcohol and drug use prevention
How Do I Bring Up 08.21 Information--Alcohol and
My Drug Use With My Health Care Provider?
10.66
drug use prevention Practice--Other
CLEAR Evaluation Field Guide--September 2008
13
VARIABLE Activity Activity
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
SUBSTANCE USE RISK 5: DRUGS, ALCOHOL, AND HIV (CONTINUED)
How Can I Achieve Perfect Adherence When I'm High or Drunk?
11.07 11.21
Discussion--HIV medication therapy adherence
Discussion--Alcohol and drug use prevention
ADHERENCE 1: UNDERSTANDING MY MEDICATIONS AND ADHERENCE
F10
05.00
What Is My Current 05.00 Personalized Risk
11.07
Medication Regimen?
Assessment
How Is My Adherence?
05.00 Personalized Risk Assessment
What Affects the Way I Take My Medications?
11.07
Discussion--HIV medication therapy adherence
How Can I Use CLEAR Thinking to Improve My Adherence?
11.07
Discussion--HIV medication therapy adherence
ADHERENCE 2: WHAT AFFECTS THE WAY I TAKE MY MEDICATIONS?
F10
08.20
What Are My HIV 08.20 Information--HIV
10.04 11.07
Medications All About?
medication therapy adherence
How Can I Use
10.04 Practice--Decision Making
SMART ProblemSolving to Improve Adherence?
11.07
Discussion--HIV medication therapy adherence
How Can I Plan to Achieve My Adherence Goals?
11.07
Discussion--HIV medication therapy adherence
CLEAR Evaluation Field Guide--September 2008
14
VARIABLE Activity
Activity Activity Activity
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
ADHERENCE 3: WHAT AFFECTS THE WAY I TAKE MY MEDICATIONS?
F10
08.20
How Should I Talk 08.20 Information--HIV
10.66 11.07
to My Health Care Provider about Medication?
10.66
medication therapy adherence
Practice--Other
What Are the Barriers to Discussing Medication With My Health Care Provider?
11.07
Discussion--HIV medication therapy adherence
Practice Discussing Medications with My Health Care Provider
10.66
Practice--Other
STIGMA 1: HOW CAN I COPE WITH INTERNAL STIGMA?
F10
11.12
Who Am I?
11.12 Discussion--Living with
HIV/AIDS
How Do I Feel About Stigma?
11.12 Discussion--Living with HIV/AIDS
CLEAR Thinking
11.12 Discussion--Living with HIV/AIDS
STIGMA 2: HOW CAN I DEAL WITH EXTERNAL STIGMA?
F10
08.07
How Can I Use
10.66 Practice--Other
10.66 11.12
CLEAR Thinking to 11.12 Handle External HIV Stigma?
Discussion--Living with HIV/AIDS
What Are My Rights 08.07 as a Person Living With HIV or AIDS?
Information--Living with HIV/AIDS
DISCLOSURE 1: SHOULD I DISCLOSE MY STATUS?
F10
11.05
What Do I Feel
11.05 Discussion--Disclosure of
About HIV
HIV Status
Disclosure?
Disclosure: Advantages and Disadvantages
11.05 Discussion--Disclosure of HIV Status
Who Needs to Know?
11.05 Discussion--Disclosure of HIV Status
CLEAR Evaluation Field Guide--September 2008
15
VARIABLE Activity Activity
Activity
Activity
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
DISCLOSURE 2: WHEN AND HOW SHOULD I DISCLOSE MY HIV STATUS?
F10
08.66
Review
10.05
11.05 Discussion--Disclosure of HIV Status
10.66 11.05
Practicing Disclosure
Relaxation
10.05 08.66
Practice--Disclosure of HIV Status
Information--Other
10.66 Practice--Other
HEALTH CARE 1: MOTIVATION FOR CHANGE: WANTING TO STAY HEALTHY
F10
10.04
Why Should I Stay 11.12 Discussion--Living with
11.12
Healthy?
HIV/AIDS
What Does It Mean 11.12 Discussion--Living with
to Stay Healthy?
HIV/AIDS
What Are My Barriers to Staying Healthy and How Can I Cope With Them?
10.04 11.12
Practice--Decision Making
Discussion--Living with HIV/AIDS
HEALTH CARE 2: ATTENDING HEALTH CARE APPOINTMENTS
F10
10.04
What Weighs on My 11.12 Discussion--Living with
11.12
Decision to Keep or Skip
HIV/AIDS
Appointments?
Attending Medical Appointments: Breaking Down Barriers
10.04 11.12
Practice--Decision making
Discussion--Living with HIV/AIDS
HEALTH CARE 3: PARTNERING IN MY CARE AND TREATMENT
F10
10.04
How Do I
11.07 Discussion--HIV
11.07 11.12
Communicate What 11.12 I Need to My Health Care Provider?
medication therapy adherence
Discussion--Living with
HIV/AIDS
How Can I Promote a Productive Relationship With My Health Care Provider?
10.04 11.12
Practice--Decision making
Discussion--Living with HIV/AIDS
Activity
HEALTH CARE 3: PARTNERING IN MY CARE AND TREATMENT
F10
08.07
How Do I
10.04 Practice--Decision making
08.09 10.04 11.12
Communicate What I Need to My Health Care Provider?
How Can I Promote
11.12 10.04
Discussion--Living with HIV/AIDS
Practice--Decision making
a Productive Relationship With My Health Care
11.12
Discussion--Living with HIV/AIDS
Provider?
CLEAR Evaluation Field Guide--September 2008
16
VARIABLE
Activity
Delivery Method Level of Data Collection Specified Recall Period
TABLE 4. PROGRAM INFORMATION--INTERVENTION DETAILS (CONTINUED)
NHM&E DS NUMBER
VARIABLE CODE
GUIDANCE
What Are My Rights 08.07
and Responsibilities?
08.09
Information--Living with HIV/AIDS
Information--Availability of medical services
WRAP-UP SESSION: HOW DO I MAINTAIN THE CHANGES I HAVE MADE?
F10
11.19
What Do I Feel and 11.19 Discussion--Decision
Think About
Making
Maintaining the
Changes I've
Made?
How Do I Maintain the Changes I've Made?
11.19
Discussion--Decision making
F11
01.00
CLEAR is delivered to clients in person (variable value code:
01.00).
F14
1
Data in CLEAR is collected at the client-level (variable value code: 1).
F17
02
CLEAR uses a 90-day recall period on the initial and postassessment surveys (variable value code: 02).
CLEAR Evaluation Field Guide--September 2008
17
NHM&E CLIENT SERVICES DATA
Client services data provide information about the clients who are receiving services and information about each service session or encounter in which the client participates. Client services data describe the demographic and risk characteristics of individuals that participated in CLEAR, the sessions that clients participated in, and the activities implemented during each session. The client services data for CLEAR involve the collection of client level data for NHM&E DS tables H, G1, and G2.
Client-Level Data
Specific information is gathered about each client (e.g., the client was a male 19-year-old HIV-positive Hispanic MSM).
Client services data provide your agency process monitoring data. These data allow you to monitor whom you are serving and what you are doing. You can compare information from your implementation of CLEAR to what you included in your plan. This will help ensure that your activities and your participants are consistent with your plan.
CLEAR Evaluation Field Guide--September 2008
18
SECTION 2: CLEAR OBJECTIVES AND EVALUATION QUESTIONS
This section includes objectives relative to the intervention and related evaluation questions. The objectives and evaluation questions are organized by stage of evaluation--process monitoring, process evaluation, and outcome monitoring. Below each question is a brief rationale for why the question is important. Following the rationale is a table which describes the types of data needed, potential data sources, and how data may be analyzed to answer the question.
These questions will help your agency collect data that can be used for program planning and improvement. Your agency may choose to ask additional questions. As your agency and stakeholders develop and prioritize questions, it may be beneficial to define the importance of the question and use the table to identify data sources. This will help your agency determine the feasibility of answering questions.
CLEAR PROGRAM OBJECTIVES
The objectives that will be addressed as part of the CLEAR evaluation are as follows:
Process Objectives Recruit and retain client to CLEAR Implement CLEAR with fidelity to the intervention's core elements
Outcome Objectives Increase clients' emotional awareness Increase clients' problem-solving and goal-setting skills Increase clients' assertive behavior and communication skills Increase clients' motivation to change behavior Increase clients' intention and ability to decrease targeted risk behaviors
PROCESS MONITORING QUESTIONS
The following are potential process monitoring and evaluation questions that stakeholders may ask about your agency's implementation of CLEAR. Process monitoring information allows you to get a picture of the activities implemented, populations served, services provided, or resources used. This information can be used to inform program improvement and to conduct process evaluation. Process monitoring information often answers questions such as "What are the characteristics of the population served?" "What intervention activities were implemented?" and "What resources were used to deliver those activities?" Table 5 includes examples of process monitoring questions for CLEAR.
CLEAR Evaluation Field Guide--September 2008
19
TABLE 5. PROCESS MONITORING QUESTIONS CLIENT ENROLLMENT AND RETENTION
1. WHAT PROPORTION OF THE RECRUITED INDIVIDUALS WAS ENROLLED IN CLEAR?
Rationale: It is important to determine the number of clients recruited and the proportion of that population participating in the intervention. This information can be used to examine recruitment strategies and guide planning.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Number of individuals
Program Monitoring
Compute the proportion by
recruited
Summary
dividing the number of
Number of clients enrolled
enrolled by the number of
recruited.
2. WHAT PROPORTION OF THE ENROLLED CLIENTS COMPLETED ALL FIVE CORE SKILL SESSIONS?
Rationale: Data on the ability to retain clients and expose them to a "full dose" of the intervention has direct implications for planning (e.g., the use of incentives, client follow-up) and a direct bearing on the intervention's effectiveness and ability to yield the intended outcomes.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Number of clients enrolled Client Participation Record Compute the proportion by
Number of clients completed Program Monitoring
dividing the number of
all Core Skill Sessions
Summary
completed by the number of
enrolled.
3. WHAT PROPORTION OF THE ENROLLED CLIENTS COMPLETED ALL FIVE CORE SKILL SESSIONS AND AT
LEAST ONE DOMAIN FROM THE MENU SESSIONS?
Rationale: Data on the ability to retain clients and expose them to a "full dose" of the intervention has direct implications for planning (e.g., the use of incentives, client follow-up) and a direct bearing on the intervention's effectiveness and ability to yield the intended outcomes.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Number of clients enrolled Client Participation Record Compute the proportion by
Number of clients completed Program Monitoring
dividing the number of
all Core Skill Sessions
Summary
completed by the number of
Number of clients completed
enrolled.
a full domain of Menu
Sessions
4. WHAT WERE THE DEMOGRAPHIC CHARACTERISTICS OF THE CLIENTS SERVED?
Rationale: It is important to know whether the population served is the same as the population that the agency intended to serve and that CLEAR was designed and adapted for. This information can be used to examine recruitment strategies and guide planning.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Sociodemographic characteristics (age, race, ethnicity, gender, sexual orientation, HIV status, etc.)
Program Enrollment Form Program Monitoring
Summary
For each demographic indicator, aggregate the number of clients who fall within each category (e.g., African American for race)
5. WHAT WAS THE RISK PROFILE OF THE CLIENTS SERVED?
Rationale: It is important to know whether the population served is the same as the population that the agency intended to serve and that CLEAR was designed and adapted for. This information can be used to examine recruitment strategies and guide planning.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Clients' risk-related personal Client Participation Record
goals (identified initially and Program Monitoring
throughout the intervention)
Summary
For each risk category: count the number of clients who set a related personal goal (e.g., goal to increase treatment adherence)
CLEAR Evaluation Field Guide--September 2008
20
Process Evaluation Questions
Process evaluation involves an analysis of process data that facilitates comparison between what was planned and what actually occurred during implementation. Process evaluation allows you to determine if your process objectives can be met and provides information that guides planning and improvement. Process evaluation questions address issues such as "Was the intervention implemented as planned?" "Did the intervention reach the intended audience?" and "What barriers were experienced by clients and staff during the course of the intervention?" Table 6 includes examples of process evaluation questions for CLEAR.
TABLE 6. PROCESS EVALUATION QUESTIONS ADHERENCE TO INTERVENTION DESIGN
1. WHICH OF THE FIVE CORE ELEMENTS WERE IMPLEMENTED?
Rationale: It is important to know whether all of the core elements of CLEAR were implemented as intended and consistent with the design of the intervention (fidelity). Fidelity has a direct bearing on an intervention's effectiveness and ability to yield the intended outcomes.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Fidelity to sessions and their Session Fidelity Form
Review the Session Fidelity
components
Forms for each of the five
Core Skill Sessions.
Qualitatively assess which
core elements were
implemented as designed,
and which were not
implemented or modified
2.
WHAT WERE THE BARRIERS TO AND FACILITATORS OF IMPLEMENTATION?
Rationale: Identifying the barriers to implementing CLEAR can help inform and enhance strategies used to implement the intervention. It is also important to identify facilitators to implementing CLEAR to recognize successful implementation activities and approaches.
MEASURES Barriers and facilitators
identified Data from clients Data from staff
DATA COLLECTION METHOD(S) Session Fidelity Form Client Participation Record Staff meetings Staff observations
ANALYSIS Identify barriers and
facilitators to implementation Qualitatively summarize
barriers and facilitators; organize the identified issues by theme (e.g., client transportation, project space, etc.)
CLEAR Evaluation Field Guide--September 2008
21
TABLE 6. PROCESS EVALUATION QUESTIONS ADHERENCE TO INTERVENTION DESIGN (CONTINUED)
3. HOW AND WHY WERE PROGRAM ACTIVITIES MODIFIED?
Rationale: You may modify activities on the basis of the characteristics of the target population, agency resources, or priorities, or in consideration of current activities, as long as the core elements are maintained. For example, incentives may or may not be used or the content/sequence of the menu sessions may be tailored or adjusted to the needs of the target group.
4. HOW AND WHY WERE PROGRAM ACTIVITIES MODIFIED? (CONTINUED)
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Number of sessions
Session Fidelity Form
conducted
Client Participation Record
Length of sessions
Staff meetings
Description of activities
Staff observations
conducted/material covered
during each session
Number and type of
materials disseminated
Other
Compare the activities conducted to those described in the CLEAR Implementation Manual
Document the rationale for the changes made
Identify trends (e.g., how participants responded to particular sessions, where more or less emphasis was needed) across clients
Outcome Monitoring Questions
Outcome monitoring involves reviewing and assessing changes that occurred after exposure to the intervention, such as changes in the knowledge, attitudes, behaviors, or service access of individuals who participated in the intervention; or changes in community norms or structural factors. Answers to outcome monitoring questions allow you to determine if your outcome objectives were met. Outcomes include changes in knowledge, attitudes, skills, or behaviors. Outcome monitoring answers the question, "Did the expected outcomes occur?" Table 7 includes examples of possible outcome monitoring questions for CLEAR.
TABLE 7. OUTCOME MONITORING QUESTIONS CLIENT-LEVEL CHANGE
1. TO WHAT EXTENT DID CLIENTS DEMONSTRATE AN UNDERSTANDING OF THE LINK BETWEEN THEIR FEELINGS, THOUGHTS, AND ACTIONS?
Rationale: The F-T-D grid is a central component of this intervention: many of the intervention's objectives require that clients understand the interconnection between their feelings, thoughts, and actions. Clients' grasp of it may be observed by facilitators or self-reported by participants.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Observations of clients'
Client Participation Record
behavior change motivation Process Notes
Outcome monitoring data Risk Reduction Interviews
from posttest interview
Compare Risk Reduction Interview responses at pretest to responses at posttest
Summarize observations and informal interview data
CLEAR Evaluation Field Guide--September 2008
22
TABLE 7. OUTCOME MONITORING QUESTIONS CLIENT-LEVEL CHANGE (CONTINUED)
2. WHAT CHANGES OCCURRED IN CLIENTS' EMOTIONAL AWARENESS?
Rationale: Changes might occur in participants' ability to recognize and identify their emotional state and level of discomfort. They may demonstrate an understanding and use of the F-T-D framework and Feeling Thermometer, as taught in Core Skill Sessions. This information, which may be observed by facilitators or self-reported by participants, indicates whether program objectives were realized.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Clients' risk-related
Risk Reduction Interviews
personal goals (identified Client Participation Record
initially and throughout the Process Notes
intervention)
Outcome monitoring data
from pre- and posttest
interviews
Observations of clients'
practice of emotional
awareness
3. WHAT CHANGES OCCURRED IN CLIENTS' PROBLEM-SOLVING SKILLS?
Compare Risk Reduction Interview responses at pretest to responses at posttest Summarize observations and informal interview data
Rationale: Changes might occur in participants' ability to solve problems and make decisions. They may demonstrate an understanding and use of SMART problem-solving and CLEAR thinking, as taught in Core Skill Sessions. This information, which may be observed by facilitators or self-reported by participants, indicates whether program objectives were realized.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Outcome monitoring data Risk Reduction Interviews
Compare Risk Reduction
from pre- and posttest
Client Participation Record
Interview responses at
interviews
Process Notes
pretest to responses at
Observations of clients'
posttest
practice of problem-solving
Summarize observations and
skills
informal interview data
4. WHAT CHANGES OCCURRED IN CLIENTS' GOAL-SETTING SKILLS?
Rationale: Changes might occur in participants' ability to set general life goals and prevention goals for
themselves. They may demonstrate an understanding and use of short- and long-term goal setting, as
taught in Core Skill Sessions. This information, which may be observed by facilitators or self-reported by
participants, indicates whether program objectives were realized.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Outcome monitoring data from pre- and posttest interviews
Observations of clients' practice of goal-setting skills
Risk Reduction Interviews Client Participation Record Process Notes
Compare Risk Reduction Interview responses at pretest to responses at posttest
Summarize observations and informal interview data
CLEAR Evaluation Field Guide--September 2008
23
TABLE 7. OUTCOME MONITORING QUESTIONS CLIENT-LEVEL CHANGE (CONTINUED)
5. WHAT CHANGES OCCURRED IN CLIENTS' ASSERTIVE BEHAVIOR AND COMMUNICATION SKILLS?
Rationale: Changes might occur in participants' ability to interact effectively with others. They may demonstrate an understanding and use of assertive communication and behavior, as taught in Core Skill Sessions. This information, which may be observed by facilitators or self-reported by participants, indicates whether program objectives were realized.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Outcome monitoring data from pre- and posttest interviews
Observations of clients' practice of assertive communication and behavior skills
Risk Reduction Interviews Client Participation Record Process Notes
Compare Risk Reduction Interview responses at pretest to responses at posttest
Summarize observations and informal interview data
6. WHAT CHANGES OCCURRED IN CLIENTS' OVERALL BEHAVIOR CHANGE MOTIVATION?
Rationale: Changes might occur in participants' overall behavior change motivation. They may demonstrate an understanding and use of the Ideal Self, as taught in Core Skill Sessions. This information, which may be observed by facilitators or self-reported by participants, indicates whether program objectives were realized.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Outcome monitoring data Risk Reduction Interviews
Compare Risk Reduction
from pre- and posttest
Client Participation Record
Interview responses at
interviews
Process Notes
pretest to responses at
Observations of clients'
posttest
behavior change motivation
Summarize observations and
informal interview data
7. TO WHAT EXTENT WAS THERE IMPROVEMENT IN CLIENTS' INTENTION TO CHANGE TARGETED RISK
BEHAVIORS FOR WHICH THEY RECEIVED MENU SESSIONS?
Rationale: Changes might occur in participants' intention to change relevant risk behaviors and
attitudes. They may demonstrate a willingness to employ concepts that are taught in Core Skill Sessions.
This information, which may be observed by facilitators or self-reported by participants, indicates
whether program objectives were realized.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Outcome monitoring data Risk Reduction Interview
Compare Risk Reduction
from pre- and posttest
Client Participation Record
Interview responses at
interviews
Process Notes
pretest to responses at
Observations of clients'
posttest
behavior change motivation
Summarize observations and
informal interview data
8. TO WHAT EXTENT WAS THERE IMPROVEMENT IN CLIENTS' KNOWLEDGE ABOUT TARGETED RISK
BEHAVIORS FOR WHICH THEY RECEIVED MENU SESSIONS?
Rationale: Changes might occur in participants' knowledge and understanding of their risk behaviors and
attitudes. They may demonstrate an understanding of the concepts that are taught in Core Skill
Sessions. This information, which may be observed by facilitators or self-reported by participants,
indicates whether program objectives were realized.
MEASURES
DATA COLLECTION METHOD(S)
ANALYSIS
Outcome monitoring data Risk Reduction Interview
from pre- and posttest
Client Participation Record
interviews
Process Notes
Observations of clients'
behavior change motivation
Compare Risk Reduction Interview responses at pretest to responses at posttest
Summarize observations and informal interview data
CLEAR Evaluation Field Guide--September 2008
24
SECTION 3: DATA COLLECTION SCHEDULE AND ACTIVITIES
This section describes the data collection processes and instruments for CLEAR. Table 8 indicates when each instrument should be administered, who administers the instruments, and who should complete the instrument. Subsequent tables (912) provide more detail regarding data collection activities and schedules for each component of CLEAR.
INSTRUMENT Risk Reduction Interview
Session Fidelity Forms
Client Participation Record Program Monitoring Summary
TABLE 8. CLEAR DATA COLLECTION SCHEDULE
WHEN TO USE
Prior to or during the first Core Skill Session, AND
During or shortly after the Wrap-Up Session
Following each session
ADMINISTERED BY Counselor Intake Staff
Counselor
Following each session
Counselor
On an agencydetermined timeline (e.g., quarterly)
Project Manager Counselor
COMPLETED BY Counselor
Counselor Counselor Project Director Counselor
DATA COLLECTION ACTIVITIES
Tables 912 are arranged by CLEAR activity. Each table indicates when data should be collected, resources needed to collect data, data provided by the instruments located later in this field guide, how the data can be analyzed, the evaluation questions the data will answer, and ways to use the data to plan, implement, and improve your implementation of CLEAR.
TABLE 9. DATA COLLECTION ACTIVITIES--ENROLLMENT AND RETENTION OF CLIENTS
DATA COLLECTION METHODS
INSTRUMENTS
WHEN TO COLLECT THE DATA
RESOURCES NEEDED DATA PROVIDED POSSIBLE USES OF DATA
Interviews Records Notes Risk Reduction Interview Client Participation Record Program Monitoring Summary Risk Reduction Interview: prior to or during first Core Skill Session Client Participation Record: daily, following each session Program Monitoring Summary: monthly or quarterly Staff time to record process data Staff time to organize and analyze data monthly or quarterly Number of clients recruited, enrolled, and retained Demographic and risk characteristics of clients Inform recruitment and screening strategies Inform retention strategies (e.g., incentives, follow-ups) Determine if intervention adaptation is required and appropriate Identify barriers and facilitators of implementation
CLEAR Evaluation Field Guide--September 2008
25
TABLE 10. DATA COLLECTION ACTIVITIES--DELIVERY OF CORE SKILL SESSIONS
DATA COLLECTION METHODS INSTRUMENTS
WHEN TO COLLECT THE DATA
RESOURCES NEEDED DATA PROVIDED
POSSIBLE USES OF DATA
Records Notes Session Fidelity Forms Client Participation Record Program Monitoring Summary Session Fidelity Form and Client Participation Record: daily, following
each session Program Monitoring Summary: monthly or quarterly Staff time to record process data Staff time to organize and analyze data monthly or quarterly Client participation Adherence to core elements Adherence to original design of sessions Ensure that counselors are sufficiently familiar with and trained in
CLEAR Assess intervention fidelity Determine if intervention adaptation is required and appropriate Identify barriers and facilitators of implementation
TABLE 11. DATA COLLECTION ACTIVITIES--DELIVERY OF TARGETED MENU SESSIONS
DATA COLLECTION METHODS INSTRUMENTS
WHEN TO COLLECT THE DATA
RESOURCES NEEDED DATA PROVIDED
POSSIBLE USES OF DATA
Records Notes Session Fidelity Forms Client Participation Record Program Monitoring Summary Session Fidelity Form and Client Participation Record: daily,
following each session Program Monitoring Summary: monthly or quarterly Staff time to record process data Staff time to organize and analyze data monthly or quarterly Client participation Adherence to core elements Adherence to original design of sessions Ensure that counselors are sufficiently familiar with and trained in
CLEAR Assess intervention fidelity Determine if intervention adaptation is required and appropriate Assess the risk behaviors and needs of the target population Identify barriers and facilitators of implementation
CLEAR Evaluation Field Guide--September 2008
26
TABLE 12. DATA COLLECTION ACTIVITIES--EVALUATION & MAINTENANCE
DATA COLLECTION METHODS INSTRUMENTS WHEN TO COLLECT THE DATA
RESOURCES NEEDED DATA PROVIDED POSSIBLE USES OF DATA
Interviews Records Notes Risk Reduction Interviews Program Monitoring Summary Reduction Interview (pretest): before the first session for each client Risk Reduction Interview (posttest): upon completion of the
intervention for each client Program Monitoring Summary: monthly or quarterly Staff time to enter data Staff time to organize and analyze data monthly or quarterly Client characteristics Client risk-related goals Change in clients' knowledge, behavior, attitudes, intentions Evaluate the intervention's ability to achieve intended outcomes Ensure that counselors are sufficiently familiar with and trained in
CLEAR Determine if intervention adaptation is required and appropriate Assess the risk behaviors and needs of the target population Identify barriers and facilitators of implementation
CLEAR Evaluation Field Guide--September 2008
27
SECTION 4: DATA COLLECTION PROTOCOLS
This section includes protocols for each of the data collection activities previously described. The data collection and reporting requirements of CDC are incorporated in the data collection forms. This field guide includes forms from the CLEAR Implementation Manual (Rotheram-Borus & Klosinski, 2008) that have been modified to include NHM&E DS data variables. These forms can be modified to meet your agency's specific information needs. There is no requirement to use the data collection forms included in this evaluation plan. However, it is important to make sure that any modifications to the instruments maintain the basic integrity of the original forms in order to fulfill reporting requirements of your funding agency. In other words, do not remove questions that provide information you will need to report to your funding agency or use in implementing your intervention. You may, however, rephrase the question so that your participants understand what you want to know.
Each of the four evaluation forms includes instructions and recommendations for administering and completing the form. Each form includes instructions and recommendations for administering and/or completing the form. Additionally, certain forms include items that collect NHM&E DS variables that will be submitted to CDC.3 Following the instructions for these forms is a table listing the NHM&E DS variables and the item on the form that corresponds to that variable.
3 NHM&E DS program planning, HIV testing, and agency data variables were finalized for January 1, 2008, reporting per the Dear Colleague Letter. The evaluation instruments in this guide are templates designed to capture data for evaluating the CLEAR in its entirety. They are also designed to capture most program planning and client services NHM&E DS variables. Agencies should check with their CDC Project Officer or other contract monitors specific reporting requirements for CLEAR.
CLEAR Evaluation Field Guide--September 2008
28
PROGRAM ENROLLMENT FORM
When to use:
During the Program Enrollment Session after the client has agreed to participate in CLEAR.
Administered by: Counselor/intake staff
Completed by:
Counselor/intake staff
Instructions:
The Program Enrollment Form should be part of the enrollment process to determine prospective clients' eligibility for, and interest in, the CLEAR. The form is set up as a structured interview and should be conducted individually with each client. The Program Enrollment Form is unsuitable for self-administration by clients.
When administering the form, the service provider should explain to the client the reasons for wanting such personal information and how it will be used to provide services. For example, "This information will be used by program staff to understand who is enrolling in this program and how the program can best meet your needs."
All respondents should be instructed to answer the questions as honestly and thoroughly as possible. It is important that the respondent be reminded that all answers will remain confidential to the extent allowed by law. Your agency may require clients to sign a Health Insurance Portability Accountability Act (HIPAA) waiver or consent form prior to participating in CLEAR.
Staff administering this interview should ask the respondent to listen to each question and the corresponding answer choices before responding. If the client is eligible for CLEAR, you continue the session with the Risk Reduction Interview.
The NHM&E DS variables listed in the table below are collected on the Program Enrollment Form. Note that the variables presented in the table include only those required variables captured on this instrument. Please refer to the National HIV Prevention Program Monitoring and Evaluation Data Set (CDC, 2008d) for the complete list and description of all M&E variables required for reporting to CDC, optional variables for local M&E, or the 2008 National HIV Prevention Program Monitoring and Evaluation Data Set Variable Requirements (Appendix D).
CDC'S NATIONAL HIV PREVENTION PROGRAM MONITORING AND EVALUATION DATA SET VARIABLES
NHM&E DS TABLE
NHM&E DS NUMBER
VARIABLE NAME
ITEM # ON FORM
01
Date collected (today's date)
02
PEMS Client Unique Key (system generated)
Client
12
Date of birth year
1
Characteristics
13
Age (system calculated)
Demographic (NHM&E DS Table G1)
14 16
Ethnicity Race
6 7
18
More than one race
7
20
State/territory of residence
2
23
Assigned sex at birth
4
24
Current gender
5
04
05
Client Characteristics
06 07
Risk Profile
08
(NHM&E DS
09
Table G2)
10
11
12
13
Previous HIV test HIV test result Date of last HIV-negative test (if HIV-) Date of first HIV-positive test (if HIV+) Medical care (if HIV+) Pregnant In prenatal care (if pregnant) Client risk factors Additional risk factors Resent STD (Not HIV) (9)
8 8a 8a 9 10 11 12 18 13-17 19
06
09
Client Intervention Characteristics (NHM&E DS Table
11 13
H)
18
21
Session date
Worker ID
Duration of session
Recruitment source
3
Recruitment source--service /intervention type (if agency referral)
3
Incentive provided
Program Enrollment Form
To be completed by agency staff
Staff Name:
Staff ID:
Today's Date
/
/
Month Day Year
Session Number:
Site ID:
Start Time
:
AM / PM End Time:
:
(circle one)
Client name (optional):
Client ID Code:
Is the client currently receiving services from this agency?
Yes (specify): No
Was an incentive provided? Yes (specify): No
AM / PM (circle one)
Interviewer: Please answer the following questions to help <Name of Implementing Agency> and its HIV prevention programs gather information to help with their HIV prevention efforts. Your answers are anonymous. Thanks for your help.
1. What is your birth date? ____ / ____ / ____ (month/day/year)
2. Please provide the following information for you and an emergency contact.
Client's Address (optional): _________________________________ _________
Street
Apt
_____________________________ ____ _____
City
State Zip code
Phone (optional): Home: _________________ Mobile:________________ E-mail (optional):______________________________________________________ Hangouts: (1) ___________________________ (2) ________________________ Contact name: __________________________ Relationship:________________
CLEAR Evaluation Field Guide--September 2008
1
Contact's Address (optional): _________________________________ _________
Street
Apt
_____________________________ ____ _________
City
State Zip code
Phone (optional): Home: _________________
Mobile: ______________
E-mail (optional):______________________________________________________
2a. Is it OK for us to leave a message for you with your emergency contact?
Yes
No
2b. When leaving a message with your contact, is it OK for us to mention this agency?
Yes
No
3. How did you hear about CLEAR?
Another agency (specify: _______________) Billboard Flyer Sexual partner Family member or friend Outreach worker Other (specify: ___________________)
4. Were you born as a male or female?
Male Female Don't know Did not ask Refused to answer
5. How do you view yourself now?
Male Female Transgender--Male to Female Transgender--Female to Male Don't know Did not ask Refused to answer
CLEAR Evaluation Field Guide--September 2008
2
6. What best describes your ethnicity?
Hispanic or Latino Non-Hispanic or Latino Don't know Did not ask Refused to answer
7. What best describes your race? (select all that apply)
American Indian/Alaska Native Asian African American/Black White Native Hawaiian/Pacific Islander Don't know Did not ask Refused to answer
8. When was your last HIV test? ____ / ____ (month/year) Don't Know
8a. What was the result of your last HIV test?
Positive
Negative
Don't Know
9. When did you first test positive for HIV? ____ / ____ (month/year)
Never
Don't Know
10. Are you currently receiving medical care or treatment for HIV?
Yes
No
11. Are you currently pregnant?
Yes No (skip to question 13) Don't know (skip to question 13) Did not ask (skip to question 13) Refused to answer (skip to question 13)
12. Are you receiving prenatal care?
Yes No Don't know Did not ask Refused to answer
CLEAR Evaluation Field Guide--September 2008
3
Interviewer: To help prevent the spread of HIV, the <Name of Implementing Agency> needs to know about risk behaviors of young people. Some of these questions are personal. You may choose not to answer any questions. We appreciate your cooperation in answering the following questions. Please check the box next to the response which best reflects your answer.
13. In the last 3 months, have you had sex?
Yes No (skip to Question #14) Refused to Answer (skip to Question #14)
13a. If yes, how many sex partners did you have?
Number of men ______ Number of women ______ Don't Know Refused to Answer
14. In the last 3 months, how often did you or your partner(s) use condoms for sex?
Always Most of the time Sometimes Never Don't Know Refused to Answer Not Applicable
15. In the past 3 months, have you had unprotected sex with someone whom you knew had HIV/AIDS?
Yes No Don't Know Refused to Answer
16. In the past 3 months, did you use? (Check all that apply)
Crystal Ecstasy Cocaine Crack Heroin Amphetamine/speed (pills) Downers or tranquilizers (Valium, etc.) Nitrites LSD Inhalants Alcohol Other: (Specify): __________________
CLEAR Evaluation Field Guide--September 2008
4
17. In the last 3 months, did you have sex with someone while you were high on drugs and/or alcohol?
Yes No Don't Know Refused to Answer
18. Please indicate if you have engaged in the following behaviors in the past 12 months:
a. Injection drug use (including skin popping) b. Share injection drug equipment c. Sex with transgender d. Sex with female e. Sex with male f. Oral sex with female g. Oral sex with male h. Exchanged sex for drugs/money/or something
you need i. Sex while intoxicated and/or high on drugs j. Sex with an injection drug user k. Sex with someone who is HIV positive l. Sex with someone of unknown HIV status m. Sex with a person who exchanges sex for
drug/money n. Sex with a man who has sex with men (MSM) o. Sex with an anonymous partner p. Sex with a person who has hemophilia or a
transfusion/transplant recipient q. Sex without using a condom
Yes
No
Did Not Refused to
Ask
Answer
CLEAR Evaluation Field Guide--September 2008
5
19. In the past 12 months, have you been diagnosed with syphilis, gonorrhea, or Chlamydia?
Yes (specify type) a) Syphilis Self report Laboratory confirmed b) Chlamydia: Self report Laboratory confirmed c) Gonorrhea: Self report Laboratory confirmed
No Did Not Ask Refused to Answer
Interviewer: Those are all of my questions. Thank you for your patience and sharing this information with me.
CLEAR Evaluation Field Guide--September 2008
6
RISK REDUCTION INTERVIEW
When to use:
Pretest: Prior to or during the first session Posttest: during or following the wrap-up session
Administered by: Counselor
Completed by:
Counselor
Instructions:
The Risk Reduction Interview is used to assess attitudes and behaviors before and after participation in CLEAR. Consistent with the intervention's Menu Sessions, six areas are assessed in this interview through 17 specific behaviors and attitudes. The skills that are covered during the Core Skill Sessions are integrated incorporated throughout.
Before administering the Risk Reduction Interview, your agency will need to decide whether to administer the pre-test interview during the enrollment phase (e.g. during a screening or enrollment meeting), or during the first Core Skill Session of CLEAR. It is preferable that this interview be administered by itself, on a different day from any intervention activities. If, for some reason, your agency plans to have clients complete a written form, please revise the document to exclude non-response categories such as "Did not ask" and "Refused to answer."
This template includes items for the NHM&E DS variables related to clients' demographic, risk profile, and risk behavior characteristics. It is very important for your agency to identify which items answer your evaluation questions and which data elements are required by your funding agency. Your agency may choose not to ask questions that do not meet your information needs or reporting requirements--be careful not to delete questions that provide data required by your funding agency.
Administration: Read each question or statement to the client exactly as it is written. Do not change the wording of the items. Text that should be read aloud to the client is shown in bold. Record the client's responses by checking the appropriate box following each question or statement. Some of the risk reduction behaviors may be skipped, as determined by the client's response to the four general risk questions that are administered first.
For each one of the risk reduction behaviors listed, read the behavior aloud to the client (e.g., "using condoms with my sexual partner"), then read each of the statements below it and mark "Yes," "Somewhat," or "No" for each statement according to the client's response. Do not let the client fill out the form him or herself. Be sure that the client responds to all of the statements in each block that is administered. As each block of statements is administered, check for obvious inconsistencies between the client's responses (e.g., saying "No" to "I have tried doing this in the last 90 days "and" Yes" to "I have had 100% success doing this in the last 90 days"), and bring these to the attention of the client. Resolve response inconsistencies as they are encountered.
NOTES: The Sexual Risk (16) and Substance Use Risk (7 and 8) tables should
only be asked if the client reports having had sex and having used substances within the last 90 days, respectively.
The Disclosure, Stigma, Health Care, and Adherence tables (917) should only be asked of HIV-positive clients.
Risk Reduction Interview
Interviewer Read Aloud: Now we're going to find out where you stand on a number of issues that impact people's lives. I am going to give you a topic (read the first topic aloud.), and then I am going to read you some statements on that topic. Please tell how much you agree or disagree with the following statement: (Read the first opinion statement aloud.) Say "Yes" if you agree with the statement and believe it is true for you, "No" if you disagree with the statement and believe it is not true for you, and "Somewhat" if your opinion is somewhere in between. So, for the statement I have read, would you say "Yes," "No," or "Somewhat" so far as [risk reduction behavior] is concerned for you?
Now, here is the next statement. Tell me "Yes," "No," or "Somewhat" depending on how you believe it applies to you. (Read the second opinion statement aloud. Follow the same procedure for the remaining seven statements.)
Sexual Risk
1. Using condoms with my sexual partner (Skip if no sex during last 90 days: Q1=No)
Score ____
a. I believe doing this can keep me from getting or giving HIV.
b. Doing this has more positives than negatives in my mind.
c. I believe that I am ready to do this.
d. I am confident in my ability to do this.
e. I have planned how to go about doing this.
f. I have tried doing this in the last 90 days.
g. I have been able to do this in the last 90 days.
h. I have had 100% success doing this in the last 90 days.
i. I feel certain I will be able to continue doing this for the next 3 months.
2. Disclosing my HIV status to my sexual partner (Skip if no sex during last 90 days: Q1=No)
Score ____
a. I believe doing this can keep me from getting or giving HIV.
b. Doing this has more positives than negatives in my mind.
c. I believe that I am ready to do this.
d. I am confident in my ability to do this.
e. I have planned how to go about doing this.
f. I have tried doing this in the last 90 days.
g. I have been able to do this in the last 90 days.
h. I have had 100% success doing this in the last 90 days.
i. I feel certain I will be able to continue doing this for the next 3 months.
Continued on next page
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
CLEAR Evaluation Field Guide--September 2008
1
3. Discussing condom use with my sexual partner (Skip if no sex during last 90 days: Q1=No)
Score ____
a. I believe doing this can keep me from getting or giving HIV.
b. Doing this has more positives than negatives in my mind.
c. I believe that I am ready to do this.
d. I am confident in my ability to do this.
e. I have planned how to go about doing this.
f. I have tried doing this in the last 90 days.
g. I have been able to do this in the last 90 days.
h. I have had 100% success doing this in the last 90 days.
i. I feel certain I will be able to continue doing this for the next 3 months.
4. Refusing to have unsafe sex, even if I am pressured
(Skip if no sex during last 90 days: Q1=No)
Score ____
a. I believe doing this can keep me from getting or giving HIV.
b. Doing this has more positives than negatives in my mind.
c. I believe that I am ready to do this.
d. I am confident in my ability to do this.
e. I have planned how to go about doing this.
f. I have tried doing this in the last 90 days.
g. I have been able to do this in the last 90 days.
h. I have had 100% success doing this in the last 90 days.
i. I feel certain I will be able to continue doing this for the next 3 months.
5. Having fewer sex partners (Skip if no sex during last 90 days: Q1=No)
Score ____
a. I believe doing this can keep me from getting or giving HIV.
b. Doing this has more positives than negatives in my mind.
c. I believe that I am ready to do this.
d. I am confident in my ability to do this.
e. I have planned how to go about doing this.
f. I have tried doing this in the last 90 days.
g. I have been able to do this in the last 90 days.
h. I have had 100% success doing this in the last 90 days.
i. I feel certain I will be able to continue doing this for the next 3 months.
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Continued on next page
CLEAR Evaluation Field Guide--September 2008
2
6. Having sex when drunk or high (Skip if no sex during last 90 days: Q1=No)
Score ____
a. I believe doing this can keep me from getting or giving HIV.
b. Doing this has more positives than negatives in my mind.
c. I believe that I am ready to do this.
d. I am confident in my ability to do this.
e. I have planned how to go about doing this.
f. I have tried doing this in the last 90 days.
g. I have been able to do this in the last 90 days.
h. I have had 100% success doing this in the last 90 days.
i. I feel certain I will be able to continue doing this for the next 3 months.
Substance Use Risk
7. Knowing what triggers my drug use (Skip if no substance use during last 90 days: Q1=No) Score ____
a. I believe knowing this can help me stay healthy. b. Being aware of my triggers has more positives than negatives in my
mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
8. Stopping myself from using drugs (Skip if no substance use during last 90 days: Q1=No) Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
Continued on next page
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
CLEAR Evaluation Field Guide--September 2008
3
Disclosure of HIV Positive Status
9. Disclosing my HIV status to others
Score ____
a. I believe doing this can keep me from getting or giving HIV. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
Coping with Stigma
10. Coping with the stigmas and negative images out there about people
who are HIV positive
Score ____
a. I believe doing this can help me stay healthy.
b. Doing this has more positives than negatives in my mind.
c. I believe that I am ready to do this.
d. I am confident in my ability to do this.
e. I have planned how to go about doing this.
f. I have tried doing this in the last 90 days.
g. I have been able to do this in the last 90 days.
h. I have had 100% success doing this in the last 90 days.
i. I feel certain I will be able to continue doing this for the next 3 months.
11. Coping with my own feelings about being HIV positive Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Continued on next page
CLEAR Evaluation Field Guide--September 2008
4
Health Care
12. Attending health care appointments
Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
13. Developing relationships with health care providers Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
14. Overall staying motivated and dedicated to my health Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Continued on next page
CLEAR Evaluation Field Guide--September 2008
5
Treatment Adherence
15. Understanding my medications and adherence
Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
16. Taking my medications as prescribed
Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
17. Discussing my medications with my health care provider Score ____
a. I believe doing this can help me stay healthy. b. Doing this has more positives than negatives in my mind. c. I believe that I am ready to do this. d. I am confident in my ability to do this. e. I have planned how to go about doing this. f. I have tried doing this in the last 90 days. g. I have been able to do this in the last 90 days. h. I have had 100% success doing this in the last 90 days. i. I feel certain I will be able to continue doing this for the next 3
months.
Continued on next page
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
Yes Somewhat No
(2)
(1)
(0)
CLEAR Evaluation Field Guide--September 2008
6
Interviewer Read Aloud: I realize that there were a lot of personal questions. Thank you for answering them honestly. Is there anything else you would like to say or do you have any questions that you would like to ask me?
Thank you again for taking the time to complete this interview.
CLEAR Evaluation Field Guide--September 2008
7
SESSION FIDELITY FORMS
When to use:
After each session with a client
Administered by: Counselor
Completed by:
Counselor
Instructions: .
The Session Fidelity Forms are for the counselor to complete. They ask for feedback on the ways each component was implemented during a session.
Provide as much feedback as possible. The more feedback you provide, the more helpful this evaluation tool will be in future implementations of the program. Please explain any changes made to the session in the "Session Notes" section, as well as ways in which the session can be enhanced. Comments and suggestions concerning the program content, structure, and clarity of the materials are particularly helpful and should be shared with your supervisor.
Note: templates of this form are only provided for the 5 Core Skill Sessions in this CLEAR Evaluation Field Guide. Agencies are encouraged to develop similar forms for the menu sessions that are implemented with clients, since fidelity to the menu sessions should be monitored as well.
The NHM&E DS variables listed in the table below are collected on the Session Fidelity Form. Note that the variables presented in the table include only those required variables captured on this instrument. Please refer to the National HIV Prevention Program Monitoring and Evaluation Data Set (CDC, 2008d) for the complete list and description of all M&E variables required for reporting to CDC, optional variables for local M&E, or the 2008 National HIV Prevention Program Monitoring and Evaluation Data Set Variable Requirements (Appendix D).
CDC'S NATIONAL HIV PREVENTION PROGRAM MONITORING AND EVALUATION SYSTEM DATA SET (NHM&E DS) VARIABLES
NHM&E DS TABLE
NHM&E DS NUMBER
VARIABLE NAME
01
Intervention Name/ID
04a
Form ID
05
Session Number
Client Intervention
06
Characteristics
09
10
(NHM&E DS Table H)
11
Session Date Worker ID Site ID Duration of Session
12
Unit of Duration
20
Activities (14)
21
Incentive Provided
Core Skill Session 1: Getting to Know Each Other
Counselor: Date of Session: Time Started:
__________________ __________________ __________________
Client name/ID: _____________________ Today's Date: _____________________ Time Ended: _____________________
Session Notes
Describe here reasons for eliminating, adding, or modifying activities, and suggested changes:
Session Activities: Check one box for each activity
1. What can CLEAR do for me?
2. What is our commitment?
Facilitated as suggested Facilitated with changes Did not teach
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
Remarks:
3. How do I feel about living with HIV?
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
4. What are good goals?
Assigned as suggested Assigned with changes Did not assign
Remarks:
CLEAR Evaluation Field Guide--September 2008
1
Core Skill Session 2: Creating a Vision for the Future
Counselor: Date of Session: Time Started:
__________________ __________________ __________________
Client name/ID: _____________________ Today's Date: _____________________ Time Ended: _____________________
Session Notes
Describe here reasons for eliminating, adding, or modifying activities, and suggested changes:
Session Activities: Check one box for each activity
1. Check-in
2. What is my Ideal Self?
Facilitated as suggested Facilitated with changes Did not teach
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
Remarks:
3. How can I create a vision for my future?
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
4. How can I relax?
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
5. What's next?
Assigned as suggested Assigned with changes Did not assign
Remarks:
CLEAR Evaluation Field Guide--September 2008
2
Core Skill Session 3: Stressors and SMART Problem-Solving
Counselor: Date of Session: Time Started:
__________________ __________________ __________________
Client name/ID: _____________________ Today's Date: _____________________ Time Ended: _____________________
Session Notes
Describe here reasons for eliminating, adding, or modifying activities, and suggested changes:
Session Activities: Check one box for each activity
1. Check-in
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
2. What are my current stressors?
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
3. What is CLEAR thinking?
Taught as suggested Taught with changes Did not teach
Remarks:
4. What is SMART problemsolving?
Taught as suggested Taught with changes Did not teach
Remarks:
5. How can I relax?
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
6. What's next?
Assigned as suggested Assigned with changes Did not assign
Remarks:
CLEAR Evaluation Field Guide--September 2008
3
Core Skill Session 4: Exploring Different Types of Communication
Counselor: Date of Session: Time Started:
__________________ __________________ __________________
Client name/ID: _____________________ Today's Date: _____________________ Time Ended: _____________________
Session Notes
Describe here reasons for eliminating, adding, or modifying activities, and suggested changes:
Session Activities: Check one box for each activity
1. Check-in
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
2. How is communication related to F-T-D?
Taught as suggested Taught with changes Did not teach
Remarks:
3. What are different types of communication?
Taught as suggested Taught with changes Did not teach
Remarks:
4. How can I apply assertive communication in my life?
Taught as suggested Taught with changes Did not teach
Remarks:
5. How can I relax?
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
6. What's next?
Assigned as suggested Assigned with changes Did not assign
Remarks:
CLEAR Evaluation Field Guide--September 2008
4
Core Skill Session 5: Getting to Know Each Other
Counselor: Date of Session: Time Started:
__________________ __________________ __________________
Client name/ID: _____________________ Today's Date: _____________________ Time Ended: _____________________
Session Notes
Describe here reasons for eliminating, adding, or modifying activities, and suggested changes:
Session Activities: Check one box for each activity
1. What can CLEAR do for me?
2. What prevention steps have I already taken?
Facilitated as suggested Facilitated with changes Did not teach
Facilitated as suggested Facilitated with changes Did not teach
Remarks:
Remarks:
3. What prevention goals do I want to start working on as a part of my prevention plan?
Facilitated as suggested Facilitated with changes Did not teach
4. What's next?
Assigned as suggested Assigned with changes Did not assign
Remarks:
Remarks:
CLEAR Evaluation Field Guide--September 2008
5
CLIENT PARTICIPATION RECORD FORM
When to use:
Updated after each session for each client
Administered by: Counselor
Completed by:
Counselor
Instructions:
After each CLEAR session, the counselor should update the Client Participation Record.
Client Participation Record Form
Client Name: ___________________
Client I.D.: ________________________
Enrollment Date: ___ / ___ / ___
Program Enrollment Date: ___ / ___/ ___
Recruitment Source(s): _________________________________________________________
Risk-Related Personal Goals: Sexual risk Substance use risk Treatment adherence
Checklist of Sessions Completed: Core 1 Core 2
Core Skill Sessions Core 3 Core 4 Core 5
HIV stigma Disclosure of HIV status Health care and self care
Substance Use Risk Sessions
Substance Use 1 Substance Use 2 Substance Use 3 Substance Use 4 Substance Use 5
Sexual Risk Sessions
Sexual 1 Sexual 2 Sexual 3 Sexual 4 Sexual 5 Sexual 6
Adherence 1 Adherence Adherence 2
Sessions Adherence 3
Stigma 1 Stigma Sessions Stigma 2
Disclosure Disclosure 1 Sessions Disclosure 1
CLEAR Evaluation Field Guide--September 2008
1
CORE SKILL session 1:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
CORE SKILL SESSIONS Date ___ / ___ / ______ (month/day/year)
CORE SKILL session 2:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
CORE SKILL session 3:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
2
CORE SKILL session 4:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
CORE SKILL session 5:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year) Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
3
SEXUAL RISK MENU SESSIONS
SEXUAL RISK session 1:
Date ___ / ___ / ______ (month/day/year)
Weekly goal:
Progress toward goal:
Barriers or reasons goal not achieved:
SEXUAL RISK session 2:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
SEXUAL RISK session 3:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
4
SEXUAL RISK session 4:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
SEXUAL RISK session 5:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
SEXUAL RISK session 6:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year) Date ___ / ___ / ______ (month/day/year) Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
5
SUBSTANCE USE RISK MENU SESSIONS
SUBSTANCE USE session 1:
Date ___ / ___ / ______ (month/day/year)
Weekly goal:
Progress toward goal:
Barriers or reasons goal not achieved:
SUBSTANCE USE session 2:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
SUBSTANCE USE session 3:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
6
SUBSTANCE USE session 4:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
SUBSTANCE USE session 5:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year) Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
7
ADHERENCE MENU SESSIONS
ADHERENCE session 1:
Date ___ / ___ / ______ (month/day/year)
Weekly goal:
Progress toward goal:
Barriers or reasons goal not achieved:
ADHERENCE session 2:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
ADHERENCE session 3:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
8
STIGMA session 1:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
STIGMA session 2:
Weekly goal: Progress toward goal: Barriers or reasons goal not achieved:
STIGMA MENU SESSIONS Date ___ / ___ / ______ (month/day/year)
Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
9
DISCLOSURE MENU SESSIONS
DISCLOSURE session 1:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
DISCLOSURE session 2:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
10
HEALTH CARE MENU SESSIONS
HEALTH CARE session 1:
Date ___ / ___ / ______ (month/day/year)
Weekly goal:
Progress toward goal:
Barriers or reasons goal not achieved:
HEALTH CARE session 2:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
HEALTH CARE session 3:
Weekly goal:
Progress toward goal: Barriers or reasons goal not achieved:
Date ___ / ___ / ______ (month/day/year)
CLEAR Evaluation Field Guide--September 2008
11
WRAP-UP SESSION
WRAP-UP SESSION:
Date ___ / ___ / ______ (month/day/year)
Long-term goals: ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________
Barriers and facilitators: ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________
CLEAR Evaluation Field Guide--September 2008
12
Notes:
__________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________
CLEAR Evaluation Field Guide--September 2008
13
PROGRAM MONITORING SUMMARY
When to use it:
Monthly or quarterly
Administered by: Project Manager; Counselor
Completed by:
Project Manager; Counselor
Instructions:
Identify a standard interval--e.g., monthly or quarterly--for completing this form.
For each item, review the data collected on the other monitoring and evaluation forms (Risk Reduction Interview, Client Participation Record). Enter the total number of each item on the line provided.
Program Monitoring Summary
Staff ID: ____________________
Date: ____ / ____/ ____
Number of Clients Enrolled Total number of clients recruited during the period. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Total number of clients enrolled during the period. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Number of clients with sex-related personal goals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Number of clients with drug-related personal goals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Number of clients with adherence-related personal goals. . . . . . . . . . . . . . . . . . . . . . . . . . Number of clients with stigma-related personal goals. . . . . . . . . . . . . . . . . . . . . . . . . . Number of clients with disclosure-related personal goals. . . . . . . . . . . . . . . . . . . . . . . . . . Number of clients with health care-related personal goals. . . . . . . . . . . . . . . . . . . . . . . . . Number of clients with two or more personal goals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Total
Number of Clients Attending Sessions Core Skill Session 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . Core Skill Session 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . Core Skill Session 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . Core Skill Session 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . Core Skill Session 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . Sexual Risk Session 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sexual Risk Session 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sexual Risk Session 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sexual Risk Session 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sexual Risk Session 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sexual Risk Session 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Substance Use Risk Session 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Substance Use Risk Session 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Substance Use Risk Session 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Substance Use Risk Session 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Substance Use Risk Session 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adherence Session 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adherence Session 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adherence Session 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stigma Session 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stigma Session 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disclosure Session 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disclosure Session 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Health Care Session 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Health Care Session 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Health Care Session 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wrap-up Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Total
CLEAR Evaluation Field Guide--September 2008
1
Number of Clients Completing: All Core Skill Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Sexual Risk Menu Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Substance Use Risk Menu Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Adherence Menu Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Stigma Menu Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Disclosure Menu Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Health Care Menu Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Core Skill Sessions & Wrap-up Session (but no Menu Sessions) . . . . . . . . . . . . . . . All Core Skill Sessions & One or More Menu Session Domain. . . . . . . . . . . . . . . . . . . .
Total
Comments:
CLEAR Evaluation Field Guide--September 2008
2
APPENDIX A: CLEAR BEHAVIORAL RISK ANALYSIS
This appendix provides a generic behavior risk analysis for the populations identified in CLEAR--HIV positive youth engaging in high risk behaviors. A behavioral risk analysis shows the relationships between the personal, interpersonal, societal, and environmental factors (also referred to as "behavioral determinants" or "determinants of risk") which facilitate high-risk behaviors. This information is used to understand why members of the target population engage in the identified risk behavior, and where CLEAR intervenes to protect individuals against the determinants of risk. Your agency should modify the risk analysis to illustrate the influencing factors specific to your target population and local environmental conditions. Use information obtained through a needs assessment of your target populations.
CLEAR Evaluation Field Guide--September 2008
Appendix A · Page 1
APPENDIX A: CLEAR BEHAVIORAL RISK ANALYSIS FLOWCHART
Limited verbal and behavioral Assertiveness
Low Emotional Awareness
Low SelfRegulation
Low self-efficacy
Does not disclose HIV+
status
Limited relationships with health
providers
Low medical treatment adherence
Difficulties coping with
stigma
Low levels of health and self-care
Substance use or abuse
Social isolation
Youth and young adults living with HIV/AIDS
struggle to lead healthy, fulfilling lives
Limited motivation for behavior change
Limited problemsolving skills
Overwhelmed by the challenges of
daily living
Limited goalsetting skills
Low perception of risk to self or others
Limited sense of an ideal self
Negative outcome expectancies
Underlined risks are directly targeted by CLEAR's five core elements and core sessions
CLEAR Evaluation Field Guide--September 2008
Appendix A · Page 2
APPENDIX B: CLEAR CONCEPTUAL FRAMEWORK
This appendix provides a conceptual framework for CLEAR. This framework depicts the influential relationship of intervention activities on determinants of risk to influence behavior change. Use information obtained through a needs assessment of your target population to modify this framework to illustrate the determinants of risk specific to your target population.
CLEAR Evaluation Field Guide--September 2008
Appendix B · Page 1
APPENDIX B: CLEAR CONCEPTUAL FRAMEWORK
Primary Program Activities and C o m ponents
Co re Skill Session 1: Introduce Feeling Thermometer & F-T-D Framework
Affect Risk and Protective Factors
Develop emotional a wareness
W h ich I n fluence Be h avior
To Help A c hieve O v erall
Goal
Co re Skill Session 2: Identify Ideal Self a nd Life Goals
Co re Skill Session 3: Identify life stressors a nd learn/practice SMART pro blemsolving
Co re Skill Session 4: Lea rn/practice a ssertive communication
Co re Skill Session 5: R eflect o n steps taken so far, initiate HIV risk reduction pla n a nd identify next s teps
CLE AR Menu Sessions: Address risk-related personal goals in the a reas of sexual risk, substance use, treatment a dherence, disclosure, stigma, and/or health care
Mo tivate behavioral change a nd establish personal life goals
Increase ability to problem-solve in the f ace of s tress
Increase assertive behavior & co mmunication
Ma intain behavioral & a ttitudinal cha nges a nd plan f or the future
Increase ability a nd motivation to meet s hort- a nd long-term, riskrela ted personal goals
Develop healthy feelings, thoughts, a nd routines
Increase ability to fa ce & res olve daily cha llenges
Increase healthy l i v i ng
Process Monitoring & Evaluation
CLEAR Evaluation Field Guide--September 2008
Outcome Monitoring & Evaluation
Implementation and Evaluation
Impact Evaluation
Appendix B · Page 2
APPENDIX C: CLEAR LOGIC MODEL
This section provides a generic logic model for CLEAR. The model reflects activities designed to affect the behaviors and attitudes of members of targeted communities and illustrates the relationship of the program's activities to the expected outputs and outcomes as described in the CLEAR Implementation Manual (Rotheram-Borus & Klosinski, 2008). As with the behavioral risk analysis, is important that you adapt and tailor this logic model to reflect your agency's implementation of CLEAR.
CLEAR Evaluation Field Guide--September 2008
Appendix C · Page 1
APPENDIX C: CLEAR BEHAVIOR CHANGE LOGIC MODEL
Problem Statement
Activities
Outcomes
Young HIV-positive individuals struggle to lead healthy, safe lives because of the following behavioral risk determinants:
Young HIV positive individuals participate in these Project CLEAR activities to address the behavioral risk determinants:
Expected changes as a result of activities targeting the behavioral risk determinants:
Immediate Outcomes
(immediately following or within 1-2 weeks of the intervention)
Intermediate Outcomes
(e.g., 1, 3, or 6 months following the intervention)
Long-Term Outcomes
(e.g., 6, 9, or 12 months following the intervention)
Low motivation for behavior change
Negative outcome expectancies
Low levels of selfregulation and emotional awareness
Low goal-setting skills
Low problem-solving skills
Low self-efficacy in assertive communication skills and behaviors
Low self-efficacy related to safer sex, substance use, maintaining treatment adherence, developing relationships with health care providers
Ideal self characteristics identified
Life goals identified
F-T-D framework reviewed and applied
Feeling Thermometer reviewed and applied
Goal setting guidelines reviewed and practiced
SMART problem-solving skills reviewed and practice
CLEAR thinking techniques reviewed and practiced
Assertive communication reviewed and practiced
Targeted menu sessions delivered that identify barriers, review appropriate models, practice strategies, and establish goals
Increased motivation for positive feelings, thoughts, and behaviors
Effective goal-setting and problem-solving skills
Ability to communicate assertively
Increased behaviors that promote positive feelings, thoughts, and behaviors
Consistent daily routines to stay healthy
Consistent emotional regulation
Effective coping with the challenges of daily living
IMPACT Emotional well-being
and healthy living
CLEAR Evaluation Field Guide--September 2008
Appendix C · Page 2
APPENDIX D: 2008 NATIONAL HIV PREVENTION PROGRAM MONITORING AND EVALUATION DATA SET (NHM&E DS) VARIABLE REQUIREMENTS
The table below presents a summary of the variable requirements for the data collection periods of January 1 and July 1, 2008, excluding variable requirements for HIV Testing and Partner Counseling and Referral Services (PCRS). HIV Testing variable requirements are currently specified in the HIV Testing Form and Variables Manual and the CDC HIV Testing Variables Data Dictionary (both are available on the PEMS Web site, https://team.cdc.gov). Requirements for PCRS will be released later in 2008. Since this document only provides a summary of the requirements, please refer to the NHM&E DS (CDC, 2008d) for a more detailed description of definitions and value choices.
VARIABLE NUMBER
A01 A01a A02 A03 A04 A05 A06 A08 A09 A10 A11 A12 A13 A14 A18 A21 A22 A23 A24 A25 A26
VARIABLE NAME
GENERAL AGENCY INFORMATION (TABLE A)
Agency Name PEMS Agency ID Community Plan Jurisdiction Employer Identification Number (EIN) Street Address 1 Street Address 2
City State ZIP code Agency Web site Agency DUNS Number Agency Type Faith-based Race/Ethnicity Minority Focused Directly Funded Agency Agency Contact Last Name Agency Contact First Name Agency Contact Title Agency Contact Phone Agency Contact Fax Agency Contact E-mail
HD & CDC REPORTED REQUIRED
Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required
CLEAR Evaluation Field Guide--September 2008
Appendix D· Page 1
VARIABLE NUMBER
B01 B02 B03 B04 B06 B06a B07 B08 B09 B10
C01 C04 C06 C07 C13 C14 C15 C16 C17 C18 C19 C20 C21 C22 C23 C25 C26 C27
S01 S03 S04 S08 S09 S10 S16
VARIABLE NAME
CDC PROGRAM ANNOUNCEMENT AWARD INFORMATION (TABLE B)
CDC HIV Prevention PA Number CDC HIV Prevention PA Budget Start Date CDC HIV Prevention PA Budget End Date CDC HIV Prevention PA Award Number Total CDC HIV Prevention Award Amount Annual CDC HIV Prevention Award Amount Expended Amount Allocated for Community Planning Amount Allocated for Prevention Services
Amount Allocated for Evaluation Amount Allocated for Capacity Building
CONTRACTOR INFORMATION (TABLE C)
Agency Name City State
ZIP code Employer Identification Number (EIN)
DUNS Number Agency Type Agency Activities Faith-based Race/Ethnicity Minority Focused Contract Start Date - Month Contract Start Date - Year Contract End Date - Month Contract End Date - Year Total Contract Amount Awarded CDC HIV Prevention Program Announcement Number CDC HIV Prevention PA Budget Start Date CDC HIV Prevention PA Budget End Date
SITE INFORMATION (TABLE S)
Site ID Site Name Site Type
County State ZIP Code Use of Mobile Unit
HD & CDC REPORTED REQUIRED
Required Required Required Required Required Required Required Required Required Required
Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required
Required Required Required Required Required Required Required
CLEAR Evaluation Field Guide--September 2008
Appendix D· Page 2
VARIABLE NUMBER
D01 D02 D03
E101 E102 E103 E104 E105 E107 E108 E109
F01 F02 F03 F04 F05 F06 F07 F08 F09 F11 F14
G101 G102 G112 G113 G114 G116 G120 G123 G124 G200 G204 G205 G208 G209
VARIABLE NAME
PROGRAM NAME - PLANNING (TABLE D)
Program Name Community Planning Jurisdiction
Community Planning Year
PROGRAM MODEL AND BUDGET - PLANNING (TABLE E1)
Program Model Name Evidence Base
CDC Recommended Guidelines Other Basis for Program Model
Target Population Program Model Start Date Program Model End Date Proposed Annual Budget
INTERVENTION PLAN CHARACTERISTICS (TABLE F)
Intervention Type Intervention Name/ID
HIV+ Intervention Perinatal Intervention Total Number of Clients Sub-Total Target Population Planned Number of Cycles Number of Sessions
Unit of Delivery Delivery Method Level of Data Collection
CLIENT CHARACTERISTICS (TABLE G)
Date Collected PEMS Client Unique Key
Date of Birth - Year Calculated Age Ethnicity Race
State/Territory of Residence Assigned Sex at Birth Current Gender Date Collected Previous HIV Test
Self Reported HIV Test Result In HIV Medical Care/Treatment (only if HIV+)
Pregnant (only if female)
CLEAR Evaluation Field Guide--September 2008
HD & CDC REPORTED REQUIRED
Required Required Required
Required Required Required Required Required Required Required Required
Required Required Required Required Required Required Required Required Required Required Required
Required Required Required Required Required Required Required Required Required Required Required Required Required Required
Appendix D· Page 3
VARIABLE NUMBER
VARIABLE NAME
CLIENT CHARACTERISTICS (TABLE G) (CONTINUED)
G210 G211 G212 G213
In Prenatal Care (only if pregnant) Client Risk Factors ***
Additional Client Risk Factors ^^^ Recent STD (Not HIV)
***Note: The recall period for client risk factors is 12 months. ^^^Note: Additional value choices for risk factors added:
Sex without using a condom Sharing drug injection equipment
CLIENT INTERVENTION CHARACTERISTICS (TABLE H)
H01 H01a H05 H06 H07 H08 H10 H13 H18 H21 H22 H23
Intervention Name/ID Cycle
Session Number Session Date-Month
Session Date-Day Session Date-Year
Site Name/ID Recruitment Source Recruitment Source - Service/Intervention Type Incentive Provided
Unit of Delivery Delivery Method
REFERRAL (TABLE X7)
X701 X702 X703 X706 X710
PEMS Referral Code Referral Date
Referral Service Type Referral Outcome Referral Close Date
AGGREGATE HE/RR AND OUTREACH (TABLE AG)
AG00 AG01 AG02 AG03 AG04 AG05a AG05c AG06 AG08a AG08b AG08c
Intervention Name Session Number Date of Event/Session Duration of Event/Session Number of Client Contacts Delivery Method Incentive Provided
Site Name/ID Client Primary Risk - MSM Client Primary Risk - IDU Client Primary Risk - MSM/IDU
CLEAR Evaluation Field Guide--September 2008
HD & CDC REPORTED REQUIRED
Required Required Required Required
Required Required Required Required Required Required Required Required Required Required Required Required
Required Required Required Required Required
Required Required Required Required Required Required Required Required Required Required Required
Appendix D· Page 4
VARIABLE NUMBER
AG08d AG08e AG08f AG09a AG09b AG09c AG09d AG10a AG10b AG11a AG11b AG11c AG11d AG11e AG12a AG12b AG12c AG12d AG12e AG12f AG14a AG14b AG14c AG14d AG14e AG14f AG14g AG14h
HC01 HC02 HC05 HC06 HC07 HC08 HC09 HC10 HC11 HC12
VARIABLE NAME
AGGREGATE HE/RR AND OUTREACH (TABLE AG) (CONTINUED)
Client Primary Risk - Sex Involving Transgender Client Primary Risk - Heterosexual Contact
Client Primary Risk - Other/Risk Not Identified Client Gender - Male Client Gender - Female
Client Gender - Transgender MTF Client Gender - Transgender FTM Client Ethnicity - Hispanic or Latino Client Ethnicity - Not Hispanic or Latino Client Race - American Indian or Alaska Native
Client Race - Asian Client Race - Black or African American Client Race - Native Hawaiian or Other Pacific Islander
Client Race - White Client Age - Under 13 years
Client Age - 1318 years Client Age - 1924 years Client Age - 2534 years Client Age - 3544 years Client Age - 45 years and older Materials Distributed - Male Condoms Materials Distributed - Female Condoms Materials Distributed - Bleach or Safer Injection Kits Materials Distributed - Education Materials Materials Distributed - Safe Sex Kits Materials Distributed - Referral list Materials Distributed - Role Model Stories Materials Distributed - Other (specify)
HEALTH COMMUNICATION / PUBLIC INFORMATION (TABLE HC)
Intervention Name HC/PI Delivery Method
Event Start Date Event End Date Total Number of Airings Estimated Total Exposures Number of Materials Distributed Total Number of Web Hits Total Number of Attendees Number of Callers
HD & CDC REPORTED REQUIRED
Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required Required
Required Required Required Required Required Required Required Required Required Required
CLEAR Evaluation Field Guide--September 2008
Appendix D· Page 5
VARIABLE NUMBER
HC13 HC14 HC15 HC16 HC17 HC18 HC19 HC20
CP-A01 CP-A02 CP-B01 CP-B02 CP-B03 CP-B04 CP-B05 CP-B06 CP-B07 CP-B08 CP-B09 CP-C01 CP-C02 CP-C04 CP-C05 CP-C06 CP-C07
HD & CDC
VARIABLE NAME
REPORTED
REQUIRED
HEALTH COMMUNICATION / PUBLIC INFORMATION (TABLE HC) (CONTINUED)
Number of Callers Referred
Required
Distribution - Male Condoms
Required
Distribution - Female Condoms
Required
Distribution - Lubricants
Required
Distribution - Bleach or Safer Injection Kits
Required
Distribution - Referral Lists
Required
Distribution - Safe Sex Kits
Required
Distribution - Other
COMMUNITY PLANNING LEVEL (TABLE CP-A/B/C)
Required
Name of HIV Prevention CPG
HD only
Community Plan Year
HD only
Priority Population
HD only
Rank
HD only
Age
HD only
Gender
HD only
Ethnicity
HD only
Race
HD only
HIV Status
HD only
Geo Location
HD only
Transmission Risk
HD only
Name of the Prevention Activity/Intervention
HD only
Prevention Activity/Intervention Type
HD only
Evidence Based
HD only
CDC Recommended Guidelines
HD only
Other Basis for Intervention
HD only
Activity
HD only
CLEAR Evaluation Field Guide--September 2008
Appendix D· Page 6
APPENDIX E: REFERENCES
1. Centers for Disease Control and Prevention (2003). Procedural guidance for selected strategies and interventions for community based organizations funded under program announcement 04064: Draft 9 Dec 03. Atlanta, GA: Author.
2. Centers for Disease Control and Prevention (2006). Provisional procedural guidance for communitybased organizations: Revised April 2006. Atlanta, GA: Author. Retrieved March 14, 2007, from http://www.cdc.gov/hiv/topics/prev_prog/AHP/resources/guidelines/pro_guidance.pdf
3. Centers for Disease Control and Prevention. (2007). HIV prevention strategic plan: extended through 2010. Retrieved April 2, 2008, from http://www.cdc.gov/hiv/resources/reports/psp/pdf/psp.pdf
4. Centers for Disease Control and Prevention (2008a). Evaluation capacity building guide. Draft in preparation. Developed for the Centers for Disease Control and Prevention under contract number 200-2006-18987. Atlanta, GA: Author.
5. Centers for Disease Control and Prevention (2008b). National monitoring and evaluation guidance for HIV prevention programs. Draft in preparation. Developed for the Centers for Disease Control and Prevention under contract number 200-2003-01926. Atlanta, GA: Author.
6. Centers for Disease Control and Prevention (2008c). Program Evaluation and Monitoring System (PEMS) user manual. Atlanta, GA: Author.
7. Centers for Disease Control and Prevention. (2008d). National HIV prevention program monitoring and evaluation data set. Retrieved September 16, 2008, from http://team.cdc.gov
8. Rotheram-Borus, M., Swendeman, D., Comulada, S., Weiss, R. E., Lee, M., & Lightfoot, M. (2004). Prevention for substance-using HIV positive young people: Telephone and in-person delivery. Journal of Acquired Immune Deficiency Syndrome, 37(2) S68S77.
9. Rotheram-Borus, M. J., & Klosinski, L. E. (2008). CLEAR: Choosing life: Empowerment, action, results! (implementation manual). Developed for the Centers for Disease Control and Prevention under cooperative agreement #200-2004-09778. Los Angeles: The University of California, Center for Community Health, Semel Institute for Neuroscience and Human Behavior.
10. Thomas, C. W., Smith, B. D., & Wright-DeAgüero, L. (2006). The Program Evaluation and Monitoring System: A key source of data for monitoring evidence-based HIV prevention program processes and outcomes. AIDS Education and Prevention, 18(Suppl. A), 7480.
CLEAR Evaluation Field Guide--September 2008
Appendix E · Page 1
Acrobat Distiller 9.0.0 (Windows)