Understanding and Applying Research Design E-Book

Martin Lee Abbott

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 692

Veröffentlichungsjahr: 2013

Beschreibung

A fresh approach to bridging research design with statistical analysis While good social science requires both research design and statistical analysis, most books treat these two areas separately. Understanding and Applying Research Design introduces an accessible approach to integrating design and statistics, focusing on the processes of posing, testing, and interpreting research questions in the social sciences. The authors analyze real-world data using SPSS software, guiding readers on the overall process of science, focusing on premises, procedures, and designs of social scientific research. Three clearly organized sections move seamlessly from theoretical topics to statistical techniques at the heart of research procedures, and finally, to practical application of research design: * Premises of Research introduces the research process and the capabilities of SPSS, with coverage of ethics, Empirical Generalization, and Chi Square and Contingency Table Analysis * Procedures of Research explores key quantitative methods in research design including measurement, correlation, regression, and causation * Designs of Research outlines various design frameworks, with discussion of survey research, aggregate research, and experiments Throughout the book, SPSS software is used to showcase the discussed techniques, and detailed appendices provide guidance on key statistical procedures and tips for data management. Numerous exercises allow readers to test their comprehension of the presented material, and a related website features additional data sets and SPSS code. Understanding and Applying Research Design is an excellent book for social sciences and education courses on research methods at the upper-undergraduate level. The book is also an insightful reference for professionals who would like to learn how to pose, test, and interpret research questions with confidence.

Leseprobe

Table of Contents

COVER

TITLE PAGE

COPYRIGHT PAGE

DEDICATION

PREFACE

ACKNOWLEDGMENTS

PART I: WHEEL OF SCIENCE: PREMISES OF RESEARCH

1 “DUH” SCIENCE VERSUS “HUH” SCIENCE

HOW DO WE KNOW WHAT WE KNOW?

“DUH” SCIENCE

“HUH SCIENCE”

HOW DOES SOCIAL SCIENCE RESEARCH ACTUALLY WORK?

2 THEORIES AND HYPOTHESES

WHAT ARE THEORIES?

WHAT ARE HYPOTHESES?

OPERATIONALIZING VARIABLES

INDEPENDENT AND DEPENDENT VARIABLES

3 OBSERVATION AND EMPIRICAL GENERALIZATION

QUANTITATIVE DESIGNS

QUALITATIVE DESIGNS

RELIABILITY AND VALIDITY

EMPIRICAL GENERALIZATIONS

CORRELATIONAL VERSUS CAUSAL RELATIONSHIPS

TYPES OF RESEARCH

4 ETHICS

HUMAN SUBJECTS ABUSES

PROTECTION OF HUMANS IN RESEARCH

PROFESSIONAL ETHICAL STANDARDS

PART II: WHEEL OF SCIENCE: PROCEDURES OF RESEARCH

5 MEASUREMENT

VARIABLES AND CONSTANTS

OPERATIONALIZATION

VARIATION

CONSTANTS

LEVELS OF MEASUREMENT

UNITS OF ANALYSIS

RELIABILITY AND VALIDITY OF MEASURES

6 USING SPSS IN RESEARCH

REAL-WORLD DATA

COVERAGE OF STATISTICAL PROCEDURES

SPSS BASICS

GENERAL FEATURES

USING SPSS WITH GENERAL SOCIAL SURVEY DATA

7 CHI-SQUARE AND CONTINGENCY TABLE ANALYSIS

CONTINGENCY TABLES

USING CHI SQUARE TO DETERMINE THE SIGNIFICANCE OF RESEARCH FINDINGS

USING SPSS FOR THE CHI-SQUARE TEST OF INDEPENDENCE

THE CROSSTABS PROCEDURE

EFFECT SIZE: CONTINGENCY COEFFICIENT

EFFECT SIZE: PHI COEFFICIENT

EFFECT SIZE: CRAMER’S V

CREATING AND ANALYZING THE CONTINGENCY TABLE DATA DIRECTLY

CONCLUDING COMMENTS

8 LEARNING FROM POPULATIONS: CENSUSES AND SAMPLES

CENSUSES

SAMPLES

PROBABILITY SAMPLING

TYPES OF PROBABILITY SAMPLES

SAMPLING AND STATISTICS

POTENTIAL BIASES IN PROBABILITY SAMPLES

NONPROBABILITY “SAMPLES”

9 CORRELATION

THE NATURE OF CORRELATION: EXPLORE AND PREDICT

DIFFERENT MEASUREMENT VALUES

CORRELATION MEASURES

INTERPRETING THE PEARSON’S CORRELATION

ASSUMPTIONS FOR CORRELATION

PLOTTING THE CORRELATION: THE SCATTERGRAM

PATTERNS OF CORRELATIONS

STRENGTH OF CORRELATIONS IN SCATTERGRAMS

EVALUATING PEARSON’S r

CORRELATION USING SPSS

INTERPRETING r: EFFECT SIZE

CORRELATION INFLUENCES

CORRELATION IS NOT CAUSATION

AN EXAMPLE OF CORRELATION USING SPSS

NONPARAMETRIC CORRELATION

10 REGRESSION

UNDERSTANDING REGRESSION THROUGH CORRELATION

REGRESSION MODELS

USING SPSS TO UNDERSTAND REGRESSION

INTERPRETING MULTIPLE REGRESSION: THE COMBINED, OMNIBUS FINDINGS

INTERPRETING MULTIPLE REGRESSION: THE INDIVIDUAL PREDICTOR FINDINGS

USING MLR TO ESTABLISH CAUSALITY

USING MLR WITH CATEGORICAL DATA

11 CAUSATION

CRITERIA FOR CAUSATION

REGRESSION ANALYSIS AND TESTING FOR SPURIOUSNESS

PART III: WHEEL OF SCIENCE: DESIGNS OF RESEARCH

12 SURVEY RESEARCH

NATURE OF THE SURVEY

THREE TYPES OF SURVEYS

ONLINE SURVEY METHODS

ONLINE FORUMS

SURVEY ITEM CONSTRUCTION

RELIABILITY AND VALIDITY

BIAS IN SURVEYS

STUDYING CHANGE WITH SURVEYS

USING TIME IN SURVEY STUDIES

13 AGGREGATE RESEARCH

NATURE OF AGGREGATE DATA

14 EXPERIMENTS

EXPERIMENTAL DESIGNS

PRE-EXPERIMENTAL DESIGNS

TRUE EXPERIMENTAL DESIGNS

QUASI-EXPERIMENTAL DESIGNS

FIDELITY OF EXPERIMENTAL DESIGN

EXPERIMENTAL SETTINGS

ETHICS

RELIABILITY AND VALIDITY

15 STATISTICAL METHODS OF DIFFERENCE: T TEST

INDEPENDENT AND DEPENDENT SAMPLES

INDEPENDENT T TEST

INDEPENDENT T TEST: THE PROCEDURE

INDEPENDENT T TEST EXAMPLE

16 ANALYSIS OF VARIANCE

THE NATURE OF THE ANOVA DESIGN

THE COMPONENTS OF VARIANCE

THE PROCESS OF ANOVA

CALCULATING ANOVA

EFFECT SIZE

POST HOC ANALYSES

ASSUMPTIONS OF ANOVA

ADDITIONAL CONSIDERATIONS WITH ANOVA

A REAL-WORLD EXAMPLE OF ANOVA

USING SPSS FOR ANOVA PROCEDURES

SPSS PROCEDURES WITH ONE-WAY ANOVA

SPSS ANOVA RESULTS FOR THE EXAMPLE STUDY

17 FIELD RESEARCH

SELECTING A TOPIC

ENTERING THE FIELD

TAKING DATA IN THE FIELD

RELIABILITY AND VALIDITY

ETHICS

18 CONTENT ANALYSIS

DEFINING THE POPULATION

CENSUS OR SAMPLE?

CODING IN CONTENT ANALYSIS

CODING ROLLING STONE

RELIABILITY AND VALIDITY

PART IV: STATISTICS AND DATA MANAGEMENT

STATISTICAL PROCEDURES UNIT A: WRITING THE STATISTICAL RESEARCH SUMMARY

STATISTICAL PROCEDURES UNIT B: THE NATURE OF INFERENTIAL STATISTICS

PROBABILITY

PROBABILITY, THE NORMAL CURVE, AND P VALUES

POPULATIONS (PARAMETERS) AND SAMPLES (STATISTICS)

THE HYPOTHESIS TEST

STATISTICAL SIGNIFICANCE

PRACTICAL SIGNIFICANCE: EFFECT SIZE

DATA MANAGEMENT UNIT A: USE AND FUNCTIONS OF SPSS

MANAGEMENT FUNCTIONS

ADDITIONAL MANAGEMENT FUNCTIONS

ANALYSIS FUNCTIONS

DATA MANAGEMENT UNIT A: USES AND FUNCTIONS

DATA MANAGEMENT UNIT B: USING SPSS TO RECODE FOR T TEST

USING SPSS TO RECODE QUESTIONNAIRE ITEMS

DATA MANAGEMENT UNIT B: USES AND FUNCTIONS

DATA MANAGEMENT UNIT C: DESCRIPTIVE STATISTICS

DESCRIPTIVE AND INFERENTIAL STATISTICS

DESCRIPTIVE STATISTICS

DESCRIPTIVE PROCEDURES FOR NOMINAL AND ORDINAL DATA

DESCRIPTIVE PROCEDURES FOR INTERVAL DATA

OBTAINING DESCRIPTIVE (NUMERICAL) STATISTICS FROM SPSS

OBTAINING DESCRIPTIVE (VISUAL) STATISTICS FROM SPSS

DATA MANAGEMENT UNIT C: USES AND FUNCTIONS

STATISTICAL PROCEDURES UNIT C: Z SCORES

THE NATURE OF THE NORMAL CURVE

THE STANDARD NORMAL SCORE: Z SCORE

CALCULATING Z SCORES

USING SPSS TO CREATE Z SCORES

STATISTICAL PROCEDURES UNIT C: USES AND FUNCTIONS

GLOSSARY

BIBLIOGRAPHY

INDEX

Cover Image: Courtesy of Dominic Williamson

Copyright © 2013 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Abbott, Martin, 1949-

 Understanding and applying research design / Martin Lee Abbott, Jennifer McKinney.

p. cm.

 Includes bibliographical references.

 ISBN 978-1-118-09648-2 (cloth)

 1. Research–Methodology. 2. Research–Statistical methods. I. McKinney, Jennifer, 1969- II. Title.

 Q180.55.M4A236 2013

 001.4'2–dc23

2012010997

To

Joyce and William McKinney

Hannah Mary and Jacob Hovan

PREFACE

Social scientific research is the systematic and rigorous process of exploring the world around us. Good social science requires good research design and solid analytic skills. Both authors strive to teach students the methods of research design and statistical analysis in order that students learn how to pose research questions, test research questions, and draw conclusions on the research that they have conducted, as well as to critique the research they are exposed to through media, classes, and real-life situations. We have taught research methods and statistics courses at the university level for many years. In addition, we have published articles and books on the subjects and are involved in applied research projects in which we put into practice what we develop in this book.

This book grew from the need to provide a systematic but approachable book for our students. Other research design books often use a stilted approach that masks the vibrancy of research statistics and design (or they focus simply on either statistics or design). In this book, we hope to avoid these issues by providing a creative format and common language that will enable students to understand the content of social research at a more meaningful level.

The layout of the book is a reflection of our approach to teaching, and it targets contemporary student learning styles. We present research design material in approachable language interspersed by content that allows students the opportunity to delve as deeply as they wish in the material. Extended study units in statistical concepts and application exercises are placed strategically throughout the book to enhance the main focus of the book, research design.

We use SPSS®1 screen shots of menus and tables by permission from the IBM® Company. IBM, the IBM logo, ibm.com, and SPSS are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “IBM Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml. We include SPSS screen shots in the following chapters and sections: Chapters 1–3, 6–11, 13, 15, and 16, Statistical Procedures Unit C, and Data Management Units A–C.

In preparing this book, we have distilled the most meaningful content from our class-tested approaches and from our published works. We use current real-world data for our examples and discussions, in particular, the 2010 GSS2 database, a large state (Washington) database3 that compiles school-based data on student achievement, and publicly accessible data from the U.S. Census 2010.4 Much of the content on statistical procedures and using SPSS is adapted from Abbott’s previous work.5 We hope readers enjoy learning about the engaging world of research premises, procedures, and designs.

MARTIN LEE ABBOTTJENNIFER MCKINNEY

Notes

1 SPSS, Inc., an IBM Company. SPSS screen reprints throughout the book are used courtesy of International Business Machines Corporation, © SPSS, Inc., an IBM Company. SPSS was acquired by IBM in October 2009.

2 The GSS data are used by permission. Smith, Tom W, Peter Marsden, Michael Hout, and Jibum Kim. General social surveys, 1972–2010 [machine-readable data file] /Principal Investigator, Tom W. Smith; Co-Principal Investigator, Peter V. Marsden; Co-Principal Investigator, Michael Hout; Sponsored by National Science Foundation. NORC ed. Chicago: National Opinion Research Center [producer]; Storrs, CT: The Roper Center for Public Opinion Research, University of Connecticut [distributor], 2011. (http://www3.norc.org/GSS+Website/)

3 The data are used courtesy of the Office of the Superintendent of Public Instruction, Olympia, Washington. The Web site address is http://www.k12.wa.us/.

4 U.S. Census, 2010.

5 Abbott, Martin Lee, Understanding Educational Statistics using Microsoft Excel® and SPSS®, Wiley, 2011. Also, Abbott, Martin Lee, The Program Evaluation Prism, Wiley, 2010. Both are used by permission of the publisher.

Supplementary material for this book can be found by entering ISBN 9781118096482 at booksupport.wiley.com.

ACKNOWLEDGMENTS

Several people have helped to make this book possible. We would like to thank our friends and colleagues David Diekema, Sara Koenig, Paula Mitchell, Greg Moon, Kevin Neuhouser, Lorraine Shaman, Karen Snedker, Cathy Thwing, Linda Wagner, and Cara Wall-Scheffler. We thank Dominic Williamson for his graphic design that we use in the book (and on the cover) and Roger Finke for allowing us to draw so much from the ARDA. We also thank Jacqueline Palmieri for her continuing support of our efforts to publish accessible social science matter.

Finally, we thank our students who have taught us how to think about teaching statistics and design, and who help us to remember that research methods are fun!

M.L.A.J.M.

PART I

WHEEL OF SCIENCE: PREMISES OF RESEARCH

1

“DUH” SCIENCE VERSUS “HUH” SCIENCE

HOW DO WE KNOW WHAT WE KNOW?

When we go through the education process, we each take several categories of classes, especially if we know we’re headed to college. Often one of these categories is “science” and includes classes in biology, chemistry, or physics. Because of this we come to think of science as particular substantive areas rather than as a particular process. The process of science allows us to follow systematic steps to better understand the world around us. Whether using amino acids, elements along the periodic chart, sound waves, or people’s attitudes, following the process of science allows us to see patterns in our materials. Granted, it’s often harder to think of people as “materials” than it is to think of saltwater solutions as materials. Regardless of what we are looking for, following the scientific process allows us to gauge what is going on in the world.

The process of social science differs from other sciences only in that the social sciences use people to find patterns. While most of us think of people as individuals, each individual lives in a particular social context that has a surprising amount of order to it. For example, Americans drive on the right side of the road; Britons drive on the left. Even though both countries are made up of individuals, they each tend to transfer their cultural order to walking on the same side of the sidewalk. Even though each individual may walk in a unique way (perhaps like Monty Python’s “lumberjack walk”), each tends to gravitate toward the right or left side of a sidewalk depending on country—or cultural order—of origin.

Keeping with a roadway example, have you ever thought about the only thing keeping one vehicle from hitting another in a head-on collision is a measly 6 inches of yellow paint? Think about the 6 inches of white paint that keeps cars traveling in your direction from driving into you. If you consider a large urban area with millions of people trying to travel by car into and out of the area every day, isn’t it amazing how few car accidents there are? In Seattle (even with our perpetually wet weather), there are roughly four million people trying to get into and out of the metropolitan area each weekday. But there are less than a hundred vehicular accidents in a given 24-hour period, illustrating just how effective 6 inches of paint can be in regulating the behavior of millions of people. That people and social patterns have such a high degree of order allows us to study just where these patterns originate and predict when they are going to show up.

Knowing there are social rules and boundaries in place that create a high degree of social order, the task for the social scientist is to measure people’s attitudes, behaviors, and experiences to find common patterns. The question becomes, however, why should you need social science when you live in the same world or social context and experience these things for yourself? Why rely on social science to generalize to a population or group of people or things? How do you know what social science says is true? How do you know what is good information? The only way to truly know about the social patterns around us is to understand the process of science.

Say, for example, your professor distributes a class exercise asking you to evaluate some research finding. You are first asked if the finding is surprising or not, and then you are asked to write down a reason or two why you believe that finding is or is not true. Let’s say that you are given the finding, “Social scientists have found that opposites attract.” Is this finding surprising? How do you evaluate this statement? What evidence do you have that opposites attract? Go ahead and think of or jot down why you believe that opposites attract.

What if your professor is being a bit cagey and secretly handed out two contradictory research findings? Whereas you received “opposites attract,” the other half of the class received the reverse finding that “Birds of a feather flock together.” As the class comes together to discuss the research finding, an interesting thing will happen. When asked how many in the class found “this” finding to be not surprising, most of the class will raise their hands to show how unsurprised they were. That a majority of the class reports their research result is true and not surprising is interesting considering the class had two very different findings. This predicament illustrates the hindsight bias. In hindsight, research results seem like common sense; we take for granted that research findings must be true—after they are given.

As you thought about the finding you were given, you probably searched your experience for one case (person) where “opposites attract” was true. Generally when we hear about research findings after the fact, we think of at least one case of confirming evidence. This means we look to our own experience and try to find one person or situation that fits the finding given. In this case, you probably thought of at least one friend or acquaintance who was in a relationship where opposites attract. Your classmates with the contradictory finding were doing the same thing, trying to find an example of someone they knew in a relationship where birds of a feather flock together. But trying to explain research findings using our own experiences and already being biased by what the result appears to be hurts our ability to see the world as a whole. If you thought of one person who served as an example of each finding, that’s two people. Can two (or even 10 people you may have thought of) represent the whole social spectrum? Even in just an American context, there are well over 310 million people to consider. Do we really want to base our understanding of which adage is more true simply by finding two examples that the finding and to our limited experience? It’s highly unlikely that diametrically opposed research findings like opposites attract or birds of a feather flock together happen exactly randomly and at the same rate in a given social context. So how do we know which is more descriptive of everyday attraction?