Evaluation and Social Justice in Complex Sociopolitical Contexts - Barbara Rosenstein - E-Book

Evaluation and Social Justice in Complex Sociopolitical Contexts E-Book

Barbara Rosenstein

22,99 €


This volume is devoted to the theme of social responsibility, social justice, and evaluation. It examines the evaluation-social justice interface and: * shares a variety of options and examples from different settings, * gives voice to populations whose voices are rarely heard, and * contributes to fulfilling the potential of the significant role evaluation can have in promoting social change. First discussing issues related to evaluation, social responsibility, social justice, and marginalized populations in general, it goes on to address issues concerning populations marginalized due to health, psychological, and physical difficulties; their cultural or ethnic/national status; or the specific geopolitical context of Israel. This is the 146th issue in the New Directions for Evaluation series from Jossey-Bass. It is an official publication of the American Evaluation Association.

Sie lesen das E-Book in den Legimi-Apps auf:

von Legimi
zertifizierten E-Readern

Seitenzahl: 246


Number 146 Summer 2015 New Directions for Evaluation

Paul R. Brandon Editor-in-Chief

Evaluation and Social Justice in Complex Sociopolitical Contexts

Barbara Rosenstein

Helena Desivilya Syna


EVALUATION AND SOCIAL JUSTICE IN COMPLEX SOCIOPOLITICAL CONTEXTS Barbara Rosenstein, Helena Desivilya Syna (eds.) New Directions for Evaluation, no. 146 Paul R. Brandon, Editor-in-Chief

Copyright © 2015 Wiley Periodicals, Inc., A Wiley Company, and the American Evaluation Association. All rights reserved. No part of this publication may be reproduced in any form or by any means, except as permitted under sections 107 and 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher or authorization through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923; (978) 750-8400; fax (978) 646-8600. The copyright notice appearing at the bottom of the first page of a chapter in this journal indicates the copyright holder's consent that copies may be made for personal or internal use, or for personal or internal use of specific clients, on the condition that the copier pay for copying beyond that permitted by law. This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating collective works, or for resale. Such permission requests and other permission inquiries should be addressed to the Permissions Department, c/o John Wiley © Sons, Inc., 111 River Street, Hoboken, NJ 07030; (201) 748-6011, fax (201) 748-6008, www.wiley.com/go/permissions.

Microfilm copies of issues and articles are available in 16mm and 35mm, as well as microfiche in 105mm, through University Microfilms Inc., 300 North Zeeb Road, Ann Arbor, MI 48106-1346.

New Directions for Evaluation is indexed in Academic Search Alumni Edition (EBSCO Publishing), Education Research Complete (EBSCO Publishing), Higher Education Abstracts (Claremont Graduate University), SCOPUS (Elsevier), Social Services Abstracts (ProQuest), Sociological Abstracts (ProQuest), Worldwide Political Science Abstracts (ProQuest).

NEW DIRECTIONS FOR EVALUATION (ISSN 1097-6736, electronic ISSN 1534-875X) is part of The Jossey-Bass Education Series and is published quarterly by Wiley Subscription Services, Inc., A Wiley Company, at Jossey-Bass, One Montgomery Street, Suite 1200, San Francisco, CA 94104-4594.

SUBSCRIPTIONS for individuals cost $89 for U.S./Canada/Mexico/interna-tional. For institutions, $358 U.S.; $398 Canada/Mexico; $432 international. Electronic only: $89 for individuals all regions; $358 for institutions all regions. Print and electronic: $98 for individuals in the U.S., Canada, and Mexico; $122 for individuals for the rest of the world; $430 for institutions in the U.S.; $470 for institutions in Canada and Mexico; $504 for institutions for the rest of the world.

All issues are proposed by guest editors. For proposal submission guidelines, go to http://www.eval.org/p/cm/ld/fid=48. Editorial correspondence should be addressed to the Editor-in-Chief, Paul R. Brandon, University of Hawai‘i at Mānoa, 1776 University Avenue, Castle Memorial Hall Rm 118, Honolulu, HI 96822-2463.


Cover photograph by ©iStock.com/Smithore

New Directions for Evaluation

Sponsored by the American Evaluation Association


Paul R. Brandon


University of Hawai‘i at Mānoa

Associate Editors

J. Bradley Cousins

University of Ottawa

Lois-ellin Datta

Datta Analysis

Editorial Advisory Board

Anna Ah Sam

University of Hawai‘i at Mānoa

Michael Bamberger

Independent consultant

Gail Barrington

Barrington Research Group, Inc.

Fred Carden

International Development Research Centre

Thomas Chapel

Centers for Disease Control and Prevention

Leslie Cooksy

Sierra Health Foundation

Fiona Cram

Katoa Ltd.

Peter Dahler-Larsen

University of Southern Denmark

E. Jane Davidson

Real Evaluation Ltd.

Stewart Donaldson

Claremont Graduate University

Jody Fitzpatrick

University of Colorado Denver

Jennifer Greene

University of Illinois at Urbana-Champaign

Melvin Hall

Northern Arizona University

George M. Harrison

University of Hawai‘i at Mānoa

Gary Henry

Vanderbilt University

Rodney Hopson

George Mason University

George Julnes

University of Baltimore

Jean King

University of Minnesota

Saville Kushner

University of Auckland

Robert Lahey

REL Solutions Inc.

Miri Levin-Rozalis

Ben Gurion University of the Negev and Davidson Institute at the Weizmann Institute of Science

Laura Leviton

Robert Wood Johnson Foundation

Melvin Mark

Pennsylvania State University

Sandra Mathison

University of British Columbia

Robin Lin Miller

Michigan State University

Michael Morris

University of New Haven

Debra Rog

Westat and the Rockville Institute

Patricia Rogers

Royal Melbourne Institute of Technology

Mary Ann Scheirer

Scheirer Consulting

Robert Schwarz

University of Toronto

Lyn Shulha

Queen's University

Nick L. Smith

Syracuse University

Sanjeev Sridharan

University of Toronto

Monica Stitt-Bergh

University of Hawai‘i at Mānoa

Editorial Policy and Procedures

New Directions for Evaluation, a quarterly sourcebook, is an official publication of the American Evaluation Association. The journal publishes works on all aspects of evaluation, with an emphasis on presenting timely and thoughtful reflections on leading-edge issues of evaluation theory, practice, methods, the profession, and the organizational, cultural, and societal context within which evaluation occurs. Each issue of the journal is devoted to a single topic, with contributions solicited, organized, reviewed, and edited by one or more guest editors.

The editor-in-chief is seeking proposals for journal issues from around the globe about topics new to the journal (although topics discussed in the past can be revisited). A diversity of perspectives and creative bridges between evaluation and other disciplines, as well as chapters reporting original empirical research on evaluation, are encouraged. A wide range of topics and substantive domains is appropriate for publication, including evaluative endeavors other than program evaluation; however, the proposed topic must be of interest to a broad evaluation audience.

Journal issues may take any of several forms. Typically they are presented as a series of related chapters, but they might also be presented as a debate; an account, with critique and commentary, of an exemplary evaluation; a feature-length article followed by brief critical commentaries; or perhaps another form proposed by guest editors.

Submitted proposals must follow the format found via the Association's website at http://www.eval.org/Publications/NDE.asp. Proposals are sent to members of the journal's Editorial Advisory Board and to relevant substantive experts for single-blind peer review. The process may result in acceptance, a recommendation to revise and resubmit, or rejection. The journal does not consider or publish unsolicited single manuscripts.

Before submitting proposals, all parties are asked to contact the editor-in-chief, who is committed to working constructively with potential guest editors to help them develop acceptable proposals. For additional information about the journal, see the “Statement of the Editor-in-Chief” in the Spring 2013 issue (No. 137).

Paul R. Brandon, Editor-in-Chief University of Hawai‘i at Mānoa College of Education 1776 University Avenue Castle Memorial Hall, Rm. 118 Honolulu, HI 968222463 e-mail: [email protected]


From the Editor-in-Chief

Editors' Notes

The Role of Evaluators in Promoting Social Justice

Brief Overview of the Articles


1: A Different Light on Normalization: Critical Theory and Responsive Evaluation Studying Social Justice in Participation Practices


From Theory to Practice: Setting


First Analysis and Reflection


Taking a Closer Look: Secondary Analysis Through the Lens of Foucault



2: A Purpose-Driven Action: The Ethical Aspect and Social Responsibility of Evaluation


3: Social Space and Field as Constructs for Evaluating Social Inclusion

Evaluating Social Exclusion/Inclusion as a “Field” Phenomenon

Evaluating a Program for an Excluded Population

Assessing Social Inclusion as an Expansion of the Life Space



4: Social Justice in Action: The Contribution of Evaluation to Employment Integration of a Vulnerable Population—The Case of College Graduates With Learning Disabilities

The Characteristics of the “Brave” New Job Market

The Interface of Social Justice and Evaluation

The Evaluation Model: Enhancing Social Justice for Vulnerable Social Groups—Social Justice in Action

Implementation of the Social Justice in Action Evaluation Model

Conclusions: The Role of Evaluation in Fostering Social Justice


5: Integrating Human Rights in Program Evaluation: Lessons From Law and Health Programs in Kenya

Evaluation Background

Evaluation Overview

Incorporation of Human Rights Into the Evaluation

Challenges and Limitations



6: Promoting Social Justice Through a New Teacher Training Program for the Bedouin Population in the Negev: An Evaluation Case Study

The Bedouin Community in the Negev


The Status of Women in Bedouin Society

The New Program Design

The Evaluation Process and Contribution

Changes Made in the Program as a Response to Field Needs and to the Evaluation Recommendations




7: The Role of Evaluation in Affirmative Action–Type Programs

The Social and Political Contexts of the Study

The Two Case Studies


8: Evaluation in the Branco Weiss Institute: From Social Vision to Educational Practice

The Third Sector in Israel

Evaluation in the Third Sector

The Branco Weiss Institute in the Context of Israeli Society

The Evaluation Unit

Contributing to Second Order Change


An Example of the Model in Practice




9: Evaluation Under Occupation: The Role of Evaluators in Protecting and Promoting Social Justice and Equality in Conflict-Affected and Fragile Contexts (The Case of the Occupied Palestinian Territory)



Protecting and Promoting Social Justice and Equality




10: Evaluation of a Joint Israeli–Palestinian Project

Evaluation Helps Move Through a Crossroads

Responsiveness and Conceptualization

Participatory Evaluation






End User License Agreement

List of Tables

Chapter 5

Table 5.1

Chapter 6

Table 6.1

List of Illustrations

Chapter 3

Figure 3.1

Mapping the Life Space

Figure 3.2

Mapping Inclusion as Change in the Life Space

Chapter 5

Figure 5.1

Human Rights in Logic Model Development

Chapter 6

Figure 6.1

A Map of the Area Included in the Program

Figure 6.2

Partner Developmental School Design

Chapter 7

Figure 7.1

Undergraduate Arab Students at the Technion by Years

Figure 7.2

Arab Students by Gender

Figure 7.3

Dropout Rate Among Arab Students by Year

Figure 7.4

Distribution of Arab Students by Departments (Percentage)

Chapter 8

Figure 8.1

Known and Unknown Knowledge Areas in the Johari Window

Figure 8.2

Example of an Individual Diagram of a Participant in the Program, in Comparison With the Mean End-of-the-Year Scores of the Participants and Comparison Group

Figure 8.3

Example of Findings of the Mean Scores in the Beginning and End of the School Year Among Program Participants and the Comparison Group Regarding Learning Habits

Chapter 9

Figure 9.1

Map of the West Bank Illustrating Areas A, B, and C



Table of Contents





























































































































From the Editor-in-Chief

Evaluation is receiving considerable attention this year. Under the leadership of EvalPartners, the collaborative endeavor of professional evaluation associations and organizations around the world, 2015 has been declared the International Year of Evaluation. A formal United Nations resolution has acknowledged this status, and associations are symbolically passing an evaluation torch from conference to conference.

In recognition of the current heightened attention to evaluation internationally, two issues of the journal this year will be presented by guest editors and authors, mostly from outside of North America. The present issue is edited by Barbara Rosenstein, Chairperson of the Israeli Association for Program Evaluation, and Helena Desivilya Syna, Chair of the MA Program in Organizational Development and Consulting at the Max Stern Yezreel Valley College in Israel. It is the first issue in the journal's 37-year history to focus mostly on evaluation in the Middle East, as well as the first since 1990 to focus on issues of social justice.

Dr. Rosenstein and Dr. Desivilya Syna state in the Editors’ Notes that “the role of evaluation has expanded to include an almost ‘watchdog’ function of making sure that policy and programming protect the rights of all people and address issues affecting both marginalized and mainstream populations. Such a role is particularly important concerning programs and policies that do not refer to social justice explicitly.” The reader will find much in this issue that supports this assertion.

Paul R. Brandon Editor-in-Chief

Editors' Notes

Many significant changes have taken place in the theory and practice of evaluation in the last several decades. The issues of accountability and results-based programming were a major focus at the beginning of the 21st century, with random control trials taking center stage in the United States. Evaluation played a major role in contributing to these developments. However, the epistemology supporting the quantitative approach to evaluation was followed by concern for sustainability of “good” programs, greater use of mixed methods, and recognition of the complexity of both programs and evaluation (Bamberger & Vijayendra, 2010).

Alongside these changes in the field of evaluation, social scientists and evaluation practitioners became more aware of social justice and social responsibility due to socioeconomic and political transitions and their repercussions on social policies (Stuart, Grugulis, Tomlinson, Forde, & MacKenzie, 2013). These changes reflect widening gaps among social classes: the rich, the upper and lower middle classes, and the poor. The socioeconomic transformations are accompanied by vast reforms in the delivery of social services, which have swept the majority of the Western states. The most prominent change has been an application of market-type models to the provision of public services (Plantinga, de Ridder, & Corra, 2011). As a result of this transformation, the public sector has begun outsourcing social services to private and nongovernment organizations based on the premise that it would increase effectiveness and efficiency. Plantinga et al. cite research findings that seem to suggest that contracting out social services creates a quasi-market organizational environment, which in turn spurs competition and conflicting interests among various service providers. Such outcomes undermine equal opportunities and interfere with social justice. Thus, it is becoming increasingly necessary for those in positions of influence to join forces and take responsibility for promoting social justice. Evaluators are in a pivotal position to do so. Indeed, many of them are reemphasizing social justice and responsibility, as inspired by House's, Howe's, and Stake's frameworks (House & Howe, 1999; Stake, 2014). The current issue is on the forefront of this vanguard presenting a new direction: a pertinent and timely role of evaluation actively engaging social issues in the arena of increasing inequalities. We shed light on the evaluation–social justice interface in complex circumstances of conflict and social marginalization. Such an interface warrants special attention in a leading journal of the evaluation field.

The Role of Evaluators in Promoting Social Justice

We draw on Rawls's (1999) conceptualization of social justice, usually labeled “justice as fairness.” According to this scholar, social justice denotes ensuring and guarding equal access to civil freedoms, human rights, and opportunities, and protecting the least privileged members of society. Rawls's tenets concerning social justice refer to principles underlying the basic structures of society rather than principles that apply to institutions and associations in society or principles applying to international law. In this issue, we view social justice as a framework from which to address the evaluator's role in dealing with inequalities and power imbalance among social groups in society. Thus, in the field of evaluation, social responsibility is included in the concept of social justice.

In 2001, Mertens wrote: “the opportunity is upon us to engage in reciprocal learning and support, and to make a significant contribution to the amelioration of social and educational problems and the transformation of society to the end of greater justice and equality” (p. 373). Where are we now, almost 15 years after Mertens made her forecast? This issue of New Directions for Evaluation attempts to contribute to the fulfillment of that prophecy. Evaluators find themselves at the center of changing political agendas and in a position to influence these agendas by providing policy makers with evidence-based research to support and promote the merit and value of policy, programming, and project directions. We submit that the role of evaluation has expanded to include an almost “watchdog” function of making sure that policy and programming protect the rights of all people and address issues affecting both marginalized and mainstream populations. Such a role is particularly important concerning programs and policies that do not refer to social justice explicitly. Issues of inclusion and exclusion are implicit in all programming and we propose that it is the evaluator's role to ensure that these issues are placed firmly on the public agenda.

This issue reflects a substantive focus, honing in on this timely renewed role for evaluation from a variety of perspectives, stressing complex contexts that are characterized by social divisions among diverse social groups, as demonstrated in Israeli society. The main intricacies of such settings revolve around social divisions and intergroup tensions, features that accentuate the need to monitor social justice, especially as reflected in services provided to minorities and marginalized social groups.

The issue addresses these topics against the background of complex sociopolitical contexts attempting to answer the questions: How does the sociopolitical context foster or hinder more democratic and transformative evaluation approaches? And how does or can evaluation influence or have an impact on the sociopolitical context?

A range of evaluation approaches from responsive evaluation, democratic evaluation, social justice evaluation, as well as a variety of participatory forms of evaluation have proposed the social responsibility role of evaluation. These evaluations strive to give voice to the marginalized populations that are usually involved in a program or an intervention. Often, evaluators realize their increasing social responsibility through work in the field, reflect their understanding of the projects’ participants and staff within their specific contexts and present an overview of society and societal needs to those in a position to make changes. In some instances they attempt to advocate change actively. The current issue explores the following topics related to evaluation in the pursuit of social justice and responsibility:

The potential of evaluation to critique and transform the wider sociopolitical and socioeconomic context;

Theoretical foundations and methodological approaches of this role of evaluation; and

Possible ways whereby evaluation may promote social justice and responsibility for instance by implementing dialogical processes among stakeholders, especially in the context of asymmetric relationships, when strategic communication dominates and prevailing discourses silence certain voices.

In sum, the articles in this issue discuss (a) social justice methodologies in evaluation that use evaluation approaches which in and by themselves foster social justice; (b) evaluation use to promote social justice; and (c) evaluation of programs that emphasize social justice.

Brief Overview of the Articles

The articles in this issue concern various frequently marginalized social groups, exhibiting overt diversities as well as more hidden ones, such as national or ethnic minorities, women, and groups with special needs and disabilities. In many cases, the populations fit in several categories such as national minority women.

We have arranged the articles according to the population discussed in the article. Thus, the first two articles discuss issues related to evaluation, social responsibility, social justice, and marginalized populations in general. The third, fourth, and fifth articles address issues concerning populations marginalized due to health, psychological, and physical difficulties. The sixth, seventh, and eighth articles concern populations marginalized due to their cultural or ethnic/national status. And the final two articles confront issues involving the specific geopolitical context of the Israeli and Palestinian reality.

We begin with Woelders and Abma's discussion of an evaluation involving the marginalized population with disabilities in academia. The authors introduce Foucault's theory of normalization as a key to understanding social injustices and the role the evaluator can play in the process of using this theoretical framework. They discuss how a different perspective on “normal” can further inclusion. In the next article, Levin-Rozalis examines features of the evaluator's role that go beyond professional duties to conduct a good evaluation that meets professional standards, and answers these questions using Kant's categorical imperative, Ulrich's discussion on professionalism and systems, and Alexander's notion of a life worth living, among others. The third article addresses the issues of social space and social inclusion. Lapidot-Lefler, Friedman, Arieli, Haj, Sykes, and Kais demonstrate the need for new constructs to evaluate social inclusion and offer an innovative approach to programs involving disabled participants and their families. The article suggests that social space and field concepts can provide tools for generating “actionable” knowledge that can guide programs and practices aimed at inclusion. This article is followed by Desivilya Syna, Rottman, and Raz's illustration of a model used for promoting social justice in the increasingly competitive, insecure, and socially unjust field of employment. The proposed evaluation model draws on Abma and Widdershoven's (2008) typology concerning the relationship between evaluators and evaluees and is implemented in evaluation of a program attempting to integrate learning disabled college graduates in the labor market. In the fifth article, Gruskin, Waller, Safreed-Harmon, Ezer, Cohen, Gathumbi, and Kameri-Mbote present a mixed-methods model approach to assessing social justice aspects of evaluation in legalization of health programming. The authors show us a different setting, Kenya, demonstrating the global necessity to address the issues of social justice, responsibility in evaluation, and gender concerns with reference to vulnerable populations.

The six remaining articles provide case studies of the role of responsible evaluation in promoting social justice in a variety of settings in Israel and the Palestinian Authority. In the sixth article, Zamir and Abu Jaber discuss the ways in which evaluation pointed the Partner Development School program in a direction that included Bedouin women as teacher trainees. Through evaluation, cultural constraints that would have otherwise been ignored were incorporated into the program. Zoabi and Awad, in the seventh article, shine a new light on the role of evaluation in an affirmative action program within the framework of social justice and responsibility. Lustig, Ben Baruch-Koskas, Makhani-Belkin, and Hirsch present a dialog model of evaluation, focusing on a program with Ethiopian immigrants to Israel in the eighth article. The process promotes professional organizational discourse amongst several cycles, such as NGO commissioners, institute headquarters, the field practitioners, and the evaluation unit staff. In addition, the example illustrates how to build trust and partnership while turning organizational tacit knowledge into explicit knowledge. Bitar addresses the challenges of conducting a responsible, justice-oriented evaluation within a conflict setting in the ninth article. The article begins with a short history of the political situation and then examines and illustrates the impact of the Israeli occupation on the field of evaluation in the Occupied Palestinian Territories (oPt). More importantly, the article examines the role evaluators play in the oPt to promote social justice in the interventions that are taking place in the development arena and the challenges they face in so doing. The article concludes with the implications for and generalizability of the role of evaluators in promoting the social justice agenda in other conflict contexts. We conclude the issue with another discussion of the role of evaluation in a conflict setting. In the final article of the issue, Steinberg and Zamir tell the story of an Israeli–Palestinian partnership in constructing parallel narrative histories of the region and evaluation's contribution to this endeavor. This article opens a window revealing the challenging task of evaluating against the backdrop of complex, conflicting, and hostile interrelations.

It is important to mention that this issue of the journal is being prepared during the summer of 2014 at the height of the war in our region. These articles shine a ray of light into a seemingly dark intractable conflict. The discussion of narrative, challenges, hearing the voices of those who are rarely heard, and the possibility of working together adds a note of optimism at a generally pessimistic time.


Abma, T. A., & Widdershoven, G. A. M. (2008). Evaluation and/as social relation.




(2), 209–225.

Bamberger, M., & Vijayendra, R. (2010).

Using mixed methods in monitoring and evaluation. Experiences from international development

. The World Bank Development Research Group Poverty and Inequality Team.

House, E. R., & Howe, K. R. (1999).

Values in evaluation and social research

. Thousand Oaks, CA: Sage.

Mertens, D. M. (2001). Inclusivity and transformation: Evaluation in 2010.

American Journal of Evaluation



, 367–374.

Plantinga, M., de Ridder, K., & Corra, A. (2011). Choosing whether to buy or make: The contracting out of employment reintegration services by Dutch municipalities.

Social Policy and Administration



(3), 245–263.

Rawls, J. (1999).

A theory of justice

(rev. ed.). Cambridge, MA: Belknap.

Stake, R. E. (2014). Eisner's qualities and quality.

American Journal of Evaluation



, 453–454.

Stuart, M., Grugulis, I., Tomlinson, J., Forde, C., & MacKenzie, R. (2013). Reflections on work and employment into the 21st century: Between equal rights, force decides.

Work, Employment and Society



(3), 379–395.

Barbara RosensteinHelena Desivilya SynaEditors




Barbara Rosenstein

is a founding member and current chairperson of the Israeli Association for Program Evaluation. She was on the first board of the International Organization for Cooperation in Evaluation (IOCE), has taught evaluation theory and ethics, and currently conducts evaluations of social enterprise and social change programs.

Helena Desivilya Syna

is a social/organizational psychologist and the current chair of the MA program in organizational development and consulting at the Max Stern Yezreel Valley College. Her areas of expertise, research, and publications revolve around social relations and social issues, especially management of social conflict, building partnerships, and intergroup collaborations.

Woelders, S., & Abma, T. (2015). A different light on normalization: Critical theory and responsive evaluation studying social justice in participation practices. In B. Rosenstein & H. Desivilya Syna (Eds.), Evaluation and social justice in complex sociopolitical contexts. New Directions for Evaluation, 146, 9–18.

1A Different Light on Normalization: Critical Theory and Responsive Evaluation Studying Social Justice in Participation Practices

Susan Woelders, Tineke Abma


Responsive evaluation provides guidelines to include various stakeholders in dialogue. However, a substantial theory to understand power asymmetries and inequalities is lacking. The purpose of this article is to consider which theoretical framework for societal critique can be helpful to evaluate practices in relation to social justice. These questions will be addressed using fragments from a responsive evaluation study on the involvement of people with an intellectual disability in public policy. Our study shows that Foucault's framework on normalization was helpful. It revealed that the engagement and striving for equality and social justice can turn out to be disciplining itself. © 2015 Wiley Periodicals, Inc., and the American Evaluation Association.

Social justice has been explicitly addressed as a concern in the evaluation literature (Greene, 2006; Mertens, 2009; Schwandt, 1997). Responsive evaluation is an approach that aims to enhance the mutual understanding between stakeholder groups and value-driven transformations (Abma, 2005; Abma & Widdershoven, 2011; Guba & Lincoln, 1989). Responsive evaluation takes into account the issues and voices of as many stakeholders as possible, as well as those who are less heard in policymaking. It is an interactive, reflexive process on the meanings and values of a practice with and among all groups whose interests are involved. This is stimulated by dialogue between the different stakeholder groups. Dialogue is a learning process oriented toward mutual understanding (versus a debate focused on strategic action). In dialogue, people meet each other as persons with a name and face (and not as parties in a debate).

In responsive evaluation, the evaluator should create a power balance between the various stakeholders, leveling power differentials among groups. The empowerment of marginalized groups is in some sense an extension of this (Baur & Abma, 2011; Mertens, 2009). To level out the influence of all stakeholders, most of the time it is necessary to support the weaker voices. In the absence of the evaluator's advocacy for minority group interests, majority elite views can dominate (House, 1993).