Overview of APS National Evaluation
The Administration for Community Living (ACL) established the Adult Protective Service Technical Assistance Resource Center (APS TARC) to enhance the effectiveness of APS programs. As a first step in that process, the APS TARC is conducting the first-ever national evaluation of APS. The following provides an overview and approach of the plan.
Includes an APS Logic Model to use as a theoretical framework for conducting the program evaluation.
Includes a compilation of potential research questions, based on the APS Logic Model.
Presents a study comprised of three components to evaluate APS. A phased or stepwise approach is recommended to address the fact that there is currently only a basic understanding of the services provided by APS in each state, with little knowledge about the effectiveness of these services at reducing abuse and preventing re-referral to the APS system.
The APS Logic Model is a one-page depiction of the following elements of APS programs: context, inputs/resources, activities, activity metrics, and results. Activities, activity metrics, and results are divided into the typical case flow of intake (also often called “prescreening”), investigation, and post-investigation services. Quality assurance is also included and is comprised of a number of activities (e.g., documentation and supervisory review) that are critical aspects of APS programs.
Exhibit A, Program Operation Constructs
The identification of research questions is a critical step in the development of a program evaluation plan. Research questions help formulate what will be studied and evaluated, and provide a tool for prioritizing topics to be studied. The APS TARC went one step further and elaborated upon five areas or constructs of program operations, which were used to organize the research questions. The five constructs are policy, practice, personnel, partners, and performance. The constructs are defined in Exhibit A.
To develop the research questions, these constructs were cross-referenced with the critical elements of the APS Logic Model (context, intake, etc.) and research questions developed for each intersection. The result is a comprehensive set of potential broad research questions to guide the evaluation. Ultimately, based on feedback from ACL and development of research hypotheses, the research questions will be refined and focused on specific areas of inquiry consistent with the selected research methodology. Exhibit B show a summary of the cross-referencing and how it is addressed in the evaluation plan.
Exhibit B, Cross Referencing
The APS program evaluation is comprised of a three component study. The first two components are process evaluations, which build towards a more outcome-oriented evaluation. The process-oriented components start with a recognition, discussed with ACL, that knowledge-building is the first type of evaluation to undertake when evaluating APS. Out of this knowledge base, more sophisticated, outcome-oriented evaluations can be undertaken. Given this, the three evaluation components were developed. A summary of these components are as follows: