Skip To Main content Skip Global Navigation

How do you evaluate APS?

Overview of APS National Evaluation

The Administration for Community Living (ACL) established the Adult Protective Service Technical Assistance Resource Center (APS TARC) to enhance the effectiveness of APS programs. As a first step in that process, the APS TARC is conducting the first-ever national evaluation of APS. The following provides an overview and approach of the plan. 

The plan:

  • Includes an APS Logic Model to use as a theoretical framework for conducting the program evaluation.

  • Includes a compilation of potential research questions, based on the APS Logic Model.

  • Presents a study comprised of three components to evaluate APS. A phased or stepwise approach is recommended to address the fact that there is currently only a basic understanding of the services provided by APS in each state, with little knowledge about the effectiveness of these services at reducing abuse and preventing re-referral to the APS system.

The APS Logic Model is a one-page depiction of the following elements of APS programs: context, inputs/resources, activities, activity metrics, and results. Activities, activity metrics, and results are divided into the typical case flow of intake (also often called “prescreening”), investigation, and post-investigation services. Quality assurance is also included and is comprised of a number of activities (e.g., documentation and supervisory review) that are critical aspects of APS programs.

The identification of research questions is a critical step in the development of a program evaluation plan. Research questions help formulate what will be studied and evaluated, and provide a tool for prioritizing topics to be studied. The APS TARC went one step further and elaborated upon five areas or constructs of program operations, which were used to organize the research questions. The five constructs are policy, practice, personnel, partners, and performance. The constructs are defined in Exhibit A.

Exhibit A, Program Operation Constructs

Table-1-(2).PNG

To develop the research questions, these constructs were cross-referenced with the critical elements of the APS Logic Model (context, intake, etc.) and research questions developed for each intersection. The result is a comprehensive set of potential broad research questions to guide the evaluation. Ultimately, based on feedback from ACL and development of research hypotheses, the research questions will be refined and focused on specific areas of inquiry consistent with the selected research methodology. Exhibit B show a summary of the cross-referencing and how it is addressed in the evaluation plan. 

Exhibit B, Cross Referencing 
Table-2-(2).PNG
The APS program evaluation is comprised of a three component study.  The first two components are process evaluations, which build towards a more outcome-oriented evaluation. The process-oriented components start with a recognition, discussed with ACL, that knowledge-building is the first type of evaluation to undertake when evaluating APS. Out of this knowledge base, more sophisticated, outcome-oriented evaluations can be undertaken. Given this, the three evaluation components were developed. A summary of these components are as follows:

  • The first component is a detailed examination of extant policies of state APS agencies, which is needed for foundational knowledge-building. This consists of documenting state policies pertaining to APS programs, and developing and comparing state profiles. The emphasis of this component is: what are APS policies?
  • The second component, a comprehensive inventory of state agency APS practices of delivering client services, collects more specific information about how services are actually delivered and the parameters and restrictions on the scope of client services. This will result in an environmental scan of state practices under each of the major stages in the APS Logic Model: intake, investigation, post-investigation services, and quality assurance. The emphasis of this component is: what activities are conducted by APS and how are they conducted?
  • The third component, the linkage of state level client services surveys with NAMRS state-level data and Census data, will develop a rich database through which to understand client outcomes and the policies and practices that contribute to such outcomes. The emphasis of this evaluation component is: what policies, practices, and environmental factors are found to result in good client outcomes? Based on ongoing refinements in the APS Logic Model and stakeholder input, the program evaluation team will work to define appropriate outcomes.
 
 
Exhibit C provides a summary of the components.
 
Exhibit C, Summary of Study Components

Table-3-(1).PNG
ACL MISSION

All people, regardless of age or disability, should be able to live independently and participate fully in their communities, and have the right to make choices and control the decisions in an about their lives. Learn more about community living


Safe Exit