Improving Use Through Evaluation Participant Engagement

Abstract

When conducted with relevant participants evaluation has the potential to contribute to improved programs. By prioritizing learning, reflection, and growth in evaluation, not simply focussing on determining the value of a program, evaluation can contribute to social change. Evaluation participants are people who participate in and contribute to the programs being evaluated, many becoming key evaluation collaborators. To facilitate social change, understanding the influence of evaluation participant engagement (EPE) on use is critical. This study examined the beliefs, strategies, and practices that evaluation scholars, practitioners, and students affiliated with the ECN hold regarding EPE and its contribution to evaluation use. These research questions guided the study: (1) How do evaluators describe the influence of evaluation participant engagement on use? (2) What strategies of evaluation participant engagement do evaluators identify to increase use? (3) How can study participants benefit from the strategies of evaluation participant engagement identified to influence use? An exploratory qualitative bounded case study was conducted across three phases. Phase one consisted of focus groups with scholars, practitioners, and students of evaluation (N= 14). Phase two involved a nested case study of an in-progress program evaluation (N= 14). Phase three consisted of a data party with participants from phases one and two (N= 11). The first two phases included two cycles of inductive and deductive analysis. Phase three involved deductive analysis. Data from each phase was analyzed on its own and then merged to a final data set. Throughout the study, participants emphasized EPE and use as positively influenced by purposeful, front-end embedded acts of relationship building. Relationship building corresponds with the themes of contextual understanding and enhanced capacity to participate in evaluation activities, and to think evaluatively. Findings highlight the connection between effort placed on relationships and the extent of use. Results support evaluators seeking deeper understanding about how to build contextually responsive and inclusive evaluation relationships through purposeful and ongoing relationship building. Delving into emergence of evaluation practice partnerships will provide the field with new perspectives on EPE and use. Finally, building reflection into the evaluation process can support capacity building and facilitate both EPE and use.