Evidence and Evaluation in Social Policy E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Geisteswissenschaft
• Serie: Broadening Perspectives in Social Policy
• Sprache: Englisch

Seitenzahl: 317

Veröffentlichungsjahr: 2013

Beschreibung

By examining the role of evidence in social policymaking and the extent of its influence, Evidence and Evaluation in Social Policy delves deeply into one of the central questions of the field for the last 20 years.

• Chronicles the trend towards evidence-based policy over the last decade
• Assesses the ways in which scarce resources can best be used for the best care, particularly in times of austerity
• Describes methodological innovation, the ways in which researchers and politicians are working together effectively, and suggestions for future improvement
• Covers topics such as the role of randomized controlled trials in shaping public policy; the pitfalls of evidence-based policy as a prescriptive ideal; the challenges of measuring public support for policy interventions; and the benefits of engaging local government decision-makers with evaluation research

Leseprobe

Table of Contents

Broadening Perspectives on Social Policy

Title page

Copyright page

List of Contributors

Introduction: Evidence and Evaluation in Social Policy

1: Trials and Tribulations: The ‘Use’ (and ‘Misuse’) of Evidence in Public Policy

Introduction

Public Policy for Development: The Rise of Behavioural Economics

Welfare Transformed: ‘Workfare’ States in the Developed World

Arguments Concerning the ‘Use’ and ‘Misuse’ of Evidence and Evaluation

Discussion

Acknowledgements

2: Understanding the Influence of Evidence in Public Health Policy: What Can We Learn from the ‘Tobacco Wars’?

Introduction

Four Ways of Understanding the UK's ‘Tobacco Wars’

Concluding Discussion

Acknowledgements

3: Caught in the Same Frame? The Language of Evidence-based Policy in Debates about the Australian Government ‘Intervention’ into Northern Territory Aboriginal Communities

Introduction

Critiques of Evidence-based Policy Discourse

Evidence as Knowledge in the Intervention

Evidence as Politics in the Intervention

Evidence-based Policy as a Critical Framework? Limitations and Risks

Conclusion: Alternative Strategies

4: A Systematic Review of Comparative Studies of Attitudes to Social Policy

Introduction

Systematic Review

Illustration of Strengths and Weaknesses: The Attitude Survey Study

A Demonstration of the Value of Systematic Review in Social Science and of the Issues of Quality and Bias

Conclusions

5: Public Opinion and Policy-making

Introduction

Overview of the Evidence

Uncertainties in the Data

Dealing with Uncertainty

Evidence for Policy

Acknowledgement

6: Obstacles to Evidence-based Policy-making in the EU Enlargement Countries: The Case of Skills Policies

Introduction

The Origins of Evidence-based Policy-making

The Policy Process in Transition Economies

The Policy Process in the EU Enlargement Countries

International Agencies and Skills Policy in the Western Balkans

Conclusions

Acknowledgements

7: Understanding Employment Barriers for Lone Parents in Great Britain: Research Gaps and Missed Opportunities

Evidence and Policy-making

Aims and Objectives

Research on Lone Parents not in Work under the Previous Labour Government

What Do We Know about Lone Parents not in Work after 13 Years of Research?

What Went Wrong?

The Role of Research and Researchers

Lessons for Evidence-based Policy-making

By Way of Conclusion: Why it Matters …

8: Putting the Research Boot on the Policymakers' Foot: Can Participatory Approaches Change the Relationship between Policymakers and Evaluation?

Introduction

The Policy Process, Social Science and Politics

Participatory Research Approaches and Politics

A Case Study Example: Self-evaluation by Politicians in Local Government

Evaluation Done by Local Policymakers: Findings

Discussion

Conclusion

Index

Broadening Perspectives on Social Policy

Series Editor: Bent Greve

The object of this series, in this age of re-thinking on social welfare, is to bring fresh points of view and to attract fresh audiences to the mainstream of social policy debate.

The choice of themes is designed to feature issues of major interest and concern, such as are already stretching the boundaries of social policy.

This is the seventeenth collection of papers in the series. Previous volumes include:

Crime and Social Policy H. KemshallThe Times They Are Changing? Crisis and the Welfare State B. GreveReforming Long-term Care in Europe J. Costa-FontChoice: Challenges and Perspectives for the European Welfare States B. GreveLiving in Dangerous Times: Fear, Insecurity, Risk and Social Policy D. DenneyReforming the Bismarckian Welfare Systems B. Palier and C. MartinChallenging Welfare Issues in the Global Countryside G. GiarchiMigration, Immigration and Social Policy C. Jones FinerOverstretched: European Families Up Against The Demands of Work and Care T. Kröger and J. SipiläMaking a European Welfare State?: Convergences and Conflicts over European Social Policy P. Taylor-GoobyThe Welfare of Food: Rights and Responsibilities in a Changing World E. Dowler and C. Jones FinerEnvironmental Issues and Social Welfare M. Cahill and T. FitzpatrickThe Business of Research: Issues of Policy and Practice C. Jones Finer and G. LewandoHundtNew Risks, New Welfare: Signposts for Social Policy N. Manning and I. ShawTransnational Social Policy C. Jones FinerCrime & Social Exclusion C. Jones Finer and M. Nellis

This edition first published 2014

Originally published as Volume 47, Issue 4 of Social Policy & Administration

Book compilation © 2014 John Wiley & Sons Ltd

Registered Office

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

Editorial Offices

350 Main Street, Malden, MA 02148-5020, USA

9600 Garsington Road, Oxford, OX4 2DQ, UK

The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

For details of our global editorial offices, for customer services, and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com/wiley-blackwell.

The rights of Ian Greener and Bent Greve to be identified as the authors of the editorial material in this work have been asserted in accordance with the UK Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and editors have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. It is sold on the understanding that the publisher is not engaged in rendering professional services and neither the publisher nor the author shall be liable for damages arising herefrom. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data applied for

Paperback ISBN: 978-1-118-81654-7

A catalogue record for this book is available from the British Library.

Cover image: © Dumitru Gornet/Alamy

Cover design by Design Deluxe

List of Contributors

Will Bartlett, European Institute, London School of Economics and Political Science, London, UK.

Christopher Deeming, Geographical Sciences, University of Bristol, Bristol, UK.

Ian Greener, Durham University, Durham, UK.

Bent Greve, Roskilde University, Roskilde, Denmark.

Tina Haux, University of Lincoln, Lincoln, UK.

Emma Partridge, Social Policy Research Centre, University of New South Wales, Sydney, Australia.

Ray Pawson, School of Sociology and Social Policy, University of Leeds, Leeds, UK.

Liz Richardson, School of Social Sciences, University of Manchester, Manchester, UK.

K. E. Smith, Global Public Health Unit, School of Social and Political Science, University of Edinburgh, Edinburgh, UK.

Trude Sundberg, University of Kent, Kent, UK.

Peter Taylor-Gooby, University of Kent, Kent, UK.

Geoff Wong, Centre for Primary Care and Public Health, Queen Mary, University of London, London, UK.

Introduction: Evidence and Evaluation in Social Policy

Ian Greener and Bent Greve

What counts as evidence in social policy, and how evidence does, or perhaps more often, does not, influence policy-making, have become central questions in the last 20 years. The evidence-based medicine movement gathered momentum in the 1980s and 1990s (Sackett et al. 1997), creating a framework for the assessment of research in that field and how it might lead to a more robust basis for clinical decision-making, and even health policy. This led to policymakers, especially the New Labour government in the UK, suggesting that the ideological and interest-based politics of the past were now to be jettisoned in favour of an approach that was based instead on a pragmatic, ‘what works’ basis instead (Davies et al. 1999).

Still, in times of austerity, policy suggestions that are based on solid evidence, especially where they offer opportunities for efficiency savings, will, all other things being equal, have a higher chance of being accepted. The counter-argument naturally being that this also implies a risk that policymakers will only use evidence pointing in the same direction as their existing biases, especially given that evidence is seldom straightforward in its interpretation to policy ends, and this is likely to result in policymakers wanting to be cautious and so tending towards policies that are based on conclusions they feel most comfortable with.

This last issue is especially prevalent in many areas of social policy services where the relation between user and producer, at least to a certain extent, is still a kind of black-box where it can be difficult from the ‘outside’ to see exactly what is going on, and so to track how policy is being influenced. However, this does not reduce the need for knowing more about what does and does not work in social policy as a way of not only using scarce resources in the best way possible, but also to be able to give users the best possible type of care.

From the perspective of 2013, looking back at a decade of evidence-based policy-making, we should be able to see clear examples of improved policy-making through an increased use of evidence, and a transformation in which research and evaluation drive the policy process, alongside methodological innovation and closer working between researchers and policymakers. Is this actually the case?

The chapters in this book, which were responses to a call asking researchers to specifically address the role of evidence and evaluation in policy-making, suggest a less rosy picture than we might hope.

Chris Deeming's chapter considers the role of randomized controlled trials (RCTs) in shaping public policy, exploring why RCTs have become regarded as the gold standard in many low- and middle-income countries, but not necessarily in the richer, liberal democracies. Deeming also considers a range of issues associated with the use of RCTs in social policy, including ethical and methodological issues, and the use and misuse of evidence and evaluation in social policy. It is a good introduction to the full range of issues covered in the book, as well as making a contribution in its own right through its comparison of the use of RCTs in the global north and developing south.

Katherine Smith's chapter on the ‘tobacco wars’ suggests that, rather than being based on research, policy-making with respect to public health and tobacco is perhaps better explained using the methods of political science, in which coalitions and interest groups work to secure their demands, and that evidence is enrolled to those ends rather than driving the process. Nevertheless, one could argue that without many years of substantive evidence on the link between smoking and health, changes would not have been possible.

From an entirely different context, Emma Partridge presents the case of policy-making from the Northern Territory of Australia where the language of evidence-based policy has been used by both the government and its critics. Similarly to Smith's work, in some respects, Partridge emphasizes the political basis of the relationship between knowledge, evidence and policy-making. More than this, in Partridge's view, using evidence-based policy as a prescriptive ideal can make policymakers lose sight of other possible strategies for improving policy towards indigenous people.

Trude Sunberg and Peter Taylor-Gooby's chapter calls into question the idea that systematic reviews can straightforwardly inform policy-making, suggesting that they require considerable care in their use because of potential biases toward US-based research, and the weaker reporting of book-based research compared to articles. Where secondary research is based on research summary and review, this can be a quick and effective way of generating evidence – but, as the authors demonstrate, it is not without significant problems.

Ray Pawson has been at the centre of methodological innovation with respect to the review and evaluation of social policy since the 1990s. Here, his chapter with Geoff Wong considers many of the concerns of Smith, Sunberg and Taylor-Gooby's work in considering the problems of measuring public support for policy interventions. Pawson and Wong suggest that systematic reviews can be useful, but that we need to take a more considered approach to measure not only snapshots of public opinion, but also how opinion is formed. To illustrate their approach, the authors explore public support for legislation banning smoking in cars carrying children. They show how public opinion data can supplement or even supplant a limited scientific research base through a diachronic assessment of public opinion formation that suggests that public opinion can play a wider role in policy-making than is often assumed to be the case.

Turning to a different set of challenges, Will Bartlett's chapter considers skills policies in EU enlargement areas, but even more saliently, the consequences of the economic crisis on countries in periods of rapid change for policy-making more generally. He suggests that, in these circumstances, state capture and a coercive form of policy transfer occur that are a considerable distance from models of evidence-based policy-making. This apparently depressing story, however, can represent an opportunity for research to play a greater role in policy-making, especially for the evaluation of government programmes, but in a context where advocacy coalitions are a central feature of the political process.

Tina Haux's chapter is an illuminating case study of the relationship between evidence, research and social policy, considering work commissioned by the UK Labour government between 1997 and 2010 to explore the employment barriers for lone parents not in work. Haux finds that research often appears to have been commissioned in areas where it was not taken up by policy, but also that significant gaps occurred where policymakers needed research which was not commissioned. Even though, under the post-2010 Coalition government, policy has ‘intensified’, continuing and expanding many of Labour's ideas, there is still a lack of robust data and research supporting the government's approach, while at the same time the main research vehicle for monitoring changes to the composition and outlook of lone parents has come to an end, giving us incomplete data series. All in all, this is not an encouraging picture.

Lastly, Liz Richardson's chapter presents an innovative perspective in exploring how local government decision-makers can engage more fully with evaluation research – through the use of participative and co-productive methods. The research finds that these methods helped increase local politicians' awareness of their own ideological biases, and even an enhanced capacity for learning. It is important to note, however, the policymakers in the case study struggled to achieve robust research designs and methods, which may in turn have actually increased their appreciation of the value of academic expertise (something which we can enthusiastically endorse). Richardson suggests that politicians' belief in positivism does not have to be a barrier to more participative evaluation, and offers helpful suggestions for how this can be more successfully achieved. Her work offers the special edition an upbeat ending, showing the potential for bridging the gap between research and policy-making.

The work covered in the special edition, then, presents a very mixed picture of the role of evidence and evaluation in social policy in 2013. Politics remains central to decision-making, and in an environment of continued fiscal austerity, the opportunities for conducting funded research for central government are diminished. There is potential for the synthesis of existing research, but such syntheses need to be treated with care and caution rather than simply trying to import methods from systematic reviews and RCTs in scientific and clinical work. However, these kinds of reviews are often also useful in attempting to systematize what we know at least something about and what we do not know anything about, as well as highlighting areas where there is a stronger need for more research.

Naturally, we also need to take greater account of the specific policy and economic context when exploring the role of evidence in policy-making, and trying to assume such variables away does little to increase the persuasiveness or potential to work of our research. Although this is not a counter-argument of using evidence, it is more a word of caution in the way one can use evidence. This also implies a stronger need for students of social policy and administration to know how and to what degree certain types of research are conducted, and how one can substantiate an argument in order to avoid stating that ‘anything goes’. Solid knowledge and information are thus clearly still extremely important in order to ensure the best possible service for the users of the welfare state.

Lastly, Richardson's chapter especially stresses that it is possible for researchers and politicians to work together, but that it will involve compromises on both sides, and that smaller-scale, more localized work might be the way forward. At a time when funded opportunities in Europe appear to be moving towards large-scale, mega-projects, this is a salutary lesson indeed.

References

Davies, H., Nutley, S. and Smith, P. (1999), What Works? The Role of Evidence in Public Sector Policy and Practice, Public Money and Management, 19: 3–5.

Sackett, D., Richardson, W., Rosenberg, W. and Haynes, R. (1997), Evidence-based medicine: how to practice and teach EBM, London: Churchill Livingstone.

1

Trials and Tribulations: The ‘Use’ (and ‘Misuse’) of Evidence in Public Policy

Christopher Deeming

Introduction

We can achieve a sort of control under which the controlled, though they are following a code much more scrupulously than was ever the case under the old system, nevertheless feel free. They are doing what they want to do, not what they are forced to do. That’s the source of the tremendous power of positive reinforcement – there’s no restraint and no revolt. By careful cultural design, we control not the final behaviour, but the inclination to behave – the motives, desires, the wishes. (Skinner 1948: 246–7)

Many observers have commented on the spread of behavioural experiments in global policy. This new and emerging form of public policy often involves conditioning the receipt of welfare – in the form of cash transfers, goods or services – according to specific individual behaviour. Few however, have touched on the contextual differences in terms of the application of research and evidence governing policy, especially social assistance in the developing world and social security within the developed world. Subsequently, the debates on the progress of public policy in the global north and south are not as well connected as they might be.