Mixed Methods and Credibility of Evidence in Evaluation E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Bildung
• Serie: J-B PE Single Issue (Program) Evaluation
• Sprache: Englisch

Seitenzahl: 246

Veröffentlichungsjahr: 2013

Beschreibung

Mixed methods in evaluation have the potential to enhance the credibility of evaluation and the outcomes of evaluation. This issue explores advances in understanding mixed methods in philosophical, theoretical, and methodological terms and presents specific illustrations of the application of these concepts in evaluation practice. Leading thinkers in the mixed methods evaluation community provide frameworks and strategies that are associated with improving the probability of reaching the goals of enhanced credibility for evaluations, the evidence they produce, and the actions taken as a result of the evaluation findings. This is the 138th volume of the Jossey-Bass quarterly report series New Directions for Evaluation, an official publication of the American Evaluation Association.

Leseprobe

Contents

Editors’ Notes

Chapter 1: Mixed Methods and Credibility of Evidence in Evaluation

What Is Mixed Methods?

Synergy and Mixed Methods Evaluation Designs

Evaluation Paradigms and Mixed Methods

Chapter 2: Pragmatism, Evidence, and Mixed Methods Evaluation

Deweyan Pragmatism

Intelligent Mixed Methods Evaluation and Evidence

Putting Intelligent Action Into Practice

Conclusion

Chapter 3: What Does a Transformative Lens Bring to Credible Evidence in Mixed Methods Evaluations?

Transformative Paradigm

Transformative Axiological Assumption

Transformative Ontological Assumptions

Transformative Epistemological Assumption

Transformative Methodological Assumption

Conclusions

Chapter 4: Considering the Evidence-and-Credibility Discussion in Evaluation Through the Lens of Dialectical Pluralism

A Brief Overview of Dialectical Pluralism

Evidence-Based Practice and Its Critics

Practice-Based Evidence and Its Critics

Practice Research Networks and Communities of Practice

A Lingering Monism

A Circle of Scientific Evidence and Knowledge Model

Some Questions and Tentative Answers for Evidence-Based Evaluation

Chapter 5: Thinking Outside the Randomized Controlled Trials Experimental Box: Strategies for Enhancing Credibility and Social Justice

Mixed Methods and RCT

Re-Visioning RCT: Weaving and Shifting a Qualitative Component Into RCT Evaluation Designs

Prior to RCT

During the RCT

Post-RCT

Putting Things Together: Evaluation Case Study

Conclusions

Chapter 6: The Use of Mixed Methods in Randomized Control Trials

Conceptual Issues

Examples of Mixed Method Causal-Chain Analysis

Conclusions

Chapter 7: The Contribution of Pluralistic Qualitative Approaches to Mixed Methods Evaluations

Using Qualitative Methods in Evaluation

Pluralistic Qualitative Evaluation

Youth Participation in a Youth Inclusion Program

Discussion

Chapter 8: Establishing Interpretive Consistency When Mixing Approaches: Role of Sampling Designs in Evaluations

Interpretive Consistency

Challenges Affecting Interpretive Consistency

Application of Sampling Criteria: Heuristic Example

Conclusions

Chapter 9: Incorporating Qualitative Evidence in Systematic Reviews: Strategies and Challenges

The Structure of Systematic Reviews

Design as a Selection Criterion in Review Systems

The Meta-Evaluative Step: Assessing Quality

Movement Toward Mixed Methods Evidence-Based Reviews

Chapter 10: Reflections and Ruminations

Spinning the Kaleidoscope

Mixed Methods and Credible Evidence in Evaluation

Reprise

Index

Mixed Methods and Credibility of Evidence in Evaluation

Donna M. Mertens, Sharlene Hesse-Biber (eds.)

New Directions for Evaluation, no. 138

Paul R. Brandon, Editor-in-Chief

Copyright ©2013 Wiley Periodicals, Inc., A Wiley Company, and the American Evaluation Association. All rights reserved. No part of this publication may be reproduced in any form or by any means, except as permitted under sections 107 and 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher or authorization through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923; [phone]; fax [phone]. The copyright notice appearing at the bottom of the first page of a chapter in this journal indicates the copyright holder’s consent that copies may be made for personal or internal use, or for personal or internal use of specific clients, on the condition that the copier pay for copying beyond that permitted by law. This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating collective works, or for resale. Such permission requests and other permission inquiries should be addressed to the Permissions Department, c/o John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030; [phone], fax [phone], www.wiley.com/go/permissions.

Microfilm copies of issues and articles are available in 16mm and 35mm, as well as microfiche in 105mm, through University Microfilms Inc., 300 North Zeeb Road, Ann Arbor, MI 48106-1346.

New Directions for Evaluation is indexed in Education Research Complete (EBSCO Publishing), ERIC: Education Resources Information Center (CSC), Higher Education Abstracts (Claremont Graduate University), SCOPUS (Elsevier), Social Services Abstracts (ProQuest), Sociological Abstracts (ProQuest), and Worldwide Political Science Abstracts (ProQuest).

New Directions for Evaluation (ISSN 1097-6736, electronic ISSN 1534-875X) is part of The Jossey-Bass Education Series and is published quarterly by Wiley Subscription Services, Inc., A Wiley Company, at Jossey-Bass, One Montgomery Street, Suite 1200, San Francisco, CA 94104-4594.

Subscriptions for individuals cost $89 for U.S./Canada/Mexico; $113 international. For institutions, $313 U.S.; $353 Canada/Mexico; $387 international. Electronic only: $89 for individuals all regions; $313 for institutions all regions. Print and electronic: $98 for individuals in the U.S., Canada, and Mexico; $122 for individuals for the rest of the world; $363 for institutions in the U.S.; $403 for institutions in Canada and Mexico; $437 for institutions for the rest of the world.

Editorial correspondence should be addressed to the Editor-in-Chief, Paul R. Brandon, University of Hawai‘i at Mānoa, 1776 University Avenue, Castle Memorial Hall Rm 118, Honolulu, HI 96822-2463.

www.josseybass.com

New Directions for Evaluation

Sponsored by the American Evaluation Association

EDITOR-IN-CHIEF

Paul R. Brandon

University of Hawai‘i at Mānoa

EDITORIAL ADVISORY BOARD

Anna Ah Sam

University of Hawai‘i at Mānoa

Michael Bamberger

Independent consultant

Gail Barrington

Barrington Research Group, Inc.

Fred Carden

International Development Research Centre

Thomas Chapel

Centers for Disease Control and Prevention

Leslie Cooksy

Sierra Health Foundation

Fiona Cram

Katoa Ltd.

Peter Dahler-Larsen

University of Southern Denmark

E. Jane Davidson

Real Evaluation Ltd.

Stewart Donaldson

Claremont Graduate University

Jody Fitzpatrick

University of Colorado Denver

Jennifer Greene

University of Illinois at Urbana-Champaign

Melvin Hall

Northern Arizona University

Gary Henry

Vanderbilt University

Rodney Hopson

Duquesne University

George Julnes

University of Baltimore

Jean King

University of Minnesota

Saville Kushner

University of Auckland

Robert Lahey

REL Solutions Inc.

Miri Levin-Rozalis

Ben Gurion University of the Negev and Davidson Institute at the Weizmann Institute of Science

Laura Leviton

Robert Wood Johnson Foundation

Melvin Mark

Pennsylvania State University

Sandra Mathison

University of British Columbia

Robin Lin Miller

Michigan State University

Michael Morris

University of New Haven

Debra Rog

Westat and the Rockville Institute

Patricia Rogers

Royal Melbourne Institute of Technology

Mary Ann Scheirer

Scheirer Consulting

Robert Schwarz

University of Toronto

Lyn Shulha

Queen’s University

Nick L. Smith

Syracuse University

Sanjeev Sridharan

University of Toronto

Monica Stitt-Bergh

University of Hawai‘i at Mānoa

Editorial Policy and Procedures

New Directions for Evaluation, a quarterly sourcebook, is an official publication of the American Evaluation Association. The journal publishes works on all aspects of evaluation, with an emphasis on presenting timely and thoughtful reflections on leading-edge issues of evaluation theory, practice, methods, the profession, and the organizational, cultural, and societal context within which evaluation occurs. Each issue of the journal is devoted to a single topic, with contributions solicited, organized, reviewed, and edited by one or more guest editors.

The editor-in-chief is seeking proposals for journal issues from around the globe about topics new to the journal (although topics discussed in the past can be revisited). A diversity of perspectives and creative bridges between evaluation and other disciplines, as well as chapters reporting original empirical research on evaluation, are encouraged. A wide range of topics and substantive domains is appropriate for publication, including evaluative endeavors other than program evaluation; however, the proposed topic must be of interest to a broad evaluation audience. For examples of the types of topics that have been successfully proposed, go to http://www.josseybass.com/WileyCDA/Section/id-155510.html.

Journal issues may take any of several forms. Typically they are presented as a series of related chapters, but they might also be presented as a debate; an account, with critique and commentary, of an exemplary evaluation; a feature-length article followed by brief critical commentaries; or perhaps another form proposed by guest editors.

Submitted proposals must follow the format found via the Association’s website at http://www.eval.org/Publications/NDE.asp. Proposals are sent to members of the journal’s Editorial Advisory Board and to relevant substantive experts for single-blind peer review. The process may result in acceptance, a recommendation to revise and resubmit, or rejection. The journal does not consider or publish unsolicited single manuscripts.

Before submitting proposals, all parties are asked to contact the editor-in-chief, who is committed to working constructively with potential guest editors to help them develop acceptable proposals. For additional information about the journal, see the “Statement of the Editor-in-Chief” in the Summer 2013 issue (No. 138).

Paul R. Brandon, Editor-in-Chief

University of Hawai‘i at Mānoa

College of Education

1776 University Avenue

Castle Memorial Hall, Rm. 118

Honolulu, HI 96822–2463

e-mail: [email protected]

Editors’ Notes

An old Italian proverb reads, “What’s old is new, what’s new is old” (Melfi, 2011). This quote characterizes the story of mixed methods in the evaluation community in that mixed methods have been used by evaluators for many years. Many evaluators intuitively came to the conclusion that evaluations on complex social programs could be enhanced by the use of multiple methods; hence the combination of both quantitative and qualitative data in the same study is nothing new. Attention to mixed methods in evaluation was apparent in the New Directions for Evaluation (NDE) edited by Jennifer Greene and Valerie Caracelli in 1997 (Greene & Caracelli, 1997). Since that time, attention to mixed methods has increased exponentially, as evidenced by the launch of the in 2007, which had an initial impact factor of 2.219 and ranked fifth out of 83 journals in the social sciences, interdisciplinary category, according to the 2010 Journal Citation Reports by Thomson Reuters (2011). The American Evaluation Association (AEA) Topical Interest Group (TIG): Mixed Methods in Evaluation was founded in 2010 and quickly became one of the largest of AEA’s TIGs. And, the (Tashakkori & Teddlie, 2010) is in its second edition.