Essentials of Research Design and Methodology E-Book

Geoffrey R. Marczyk

• Herausgeber: John Wiley & Sons
• Kategorie: Geisteswissenschaft
• Sprache: Englisch

Seitenzahl: 460

Veröffentlichungsjahr: 2010

Beschreibung

Master the essential skills for designing and conducting a successful research project Essentials of Research Design and Methodology contains practical information on how to design and conduct scientific research in the behavioral and social sciences. This accessible guide covers basic to advanced concepts in a clear, concrete, and readable style. The text offers students and practitioners in the behavioral sciences and related disciplines important insights into identifying research topics, variables, and methodological approaches. Data collection and assessment strategies, interpretation methods, and important ethical considerations also receive significant coverage in this user-friendly guide. Essentials of Research Design and Methodology is the only available resource to condense the wide-ranging topics of the field into a concise, accessible format for handy and quick reference. As part of the Essentials of Behavioral Science series, this book offers a thorough review of the most relevant topics in research design and methodology. Each concise chapter features numerous callout boxes highlighting key concepts, bulleted points, and extensive illustrative material, as well as "Test Yourself" questions that help you gauge and reinforce your grasp of the information covered.

Leseprobe

Table of Contents

Essentials of Behavioral Science Series

Title Page

Copyright Page

Dedication

SERIES PREFACE

Acknowledgements

Essentials of Research Design and Methodology

One - INTRODUCTION AND OVERVIEW

OVERVIEW OF SCIENCE AND THE SCIENTIFIC METHOD

GOALS OF SCIENTIFIC RESEARCH

OVERVIEW OF THE BOOK

Two - PLANNING AND DESIGNING A RESEARCH STUDY

CHOOSING A RESEARCH TOPIC

LITERATURE REVIEW

FORMULATING A RESEARCH PROBLEM

ARTICULATING HYPOTHESES

CHOOSING VARIABLES TO STUDY

RESEARCH PARTICIPANTS

MULTICULTURAL CONSIDERATIONS

SUMMARY

Three - GENERAL APPROACHES FOR CONTROLLING ARTIFACT AND BIAS

A BRIEF INTRODUCTION TO VALIDITY

SOURCES OF ARTIFACT AND BIAS

ACHIEVING CONTROL THROUGH RANDOMIZATION: RANDOM SELECTION AND RANDOM ASSIGNMENT

SUMMARY

Four - DATA COLLECTION, ASSESSMENT METHODS, AND MEASUREMENT STRATEGIES

MEASUREMENT

PSYCHOMETRIC CONSIDERATIONS

MEASUREMENT STRATEGIES FOR DATA COLLECTION

METHODS OF DATA COLLECTION

SUMMARY

Five - GENERAL TYPES OF RESEARCH DESIGNS AND APPROACHES

EXPERIMENTAL DESIGNS

QUASI-EXPERIMENTAL DESIGNS

NONEXPERIMENTAL OR QUALITATIVE DESIGNS

SUMMARY

Six - VALIDITY

INTERNAL VALIDITY

EXTERNAL VALIDITY

CONSTRUCT VALIDITY

STATISTICAL VALIDITY

SUMMARY

Seven - DATA PREPARATION, ANALYSES, AND INTERPRETATION

DATA PREPARATION

DATA ANALYSIS

INTERPRETING DATA AND DRAWING INFERENCES

SUMMARY

Eight - ETHICAL CONSIDERATIONS IN RESEARCH

HISTORICAL BACKGROUND

FUNDAMENTAL ETHICAL PRINCIPLES

INFORMED CONSENT

INSTITUTIONAL REVIEW BOARDS

DATA SAFETY MONITORING

ADVERSE AND SERIOUS ADVERSE EVENTS

SUMMARY

Nine - DISSEMINATING RESEARCH RESULTS AND DISTILLING PRINCIPLES OF RESEARCH ...

DISSEMINATING THE RESULTS OF RESEARCH STUDIES

PRINCIPLES OF RESEARCH DESIGN AND METHODOLOGY

CHECKLIST OF RESEARCH-RELATED CONCEPTS AND CONSIDERATIONS

SUMMARY

References

Index

Essentials of Behavioral Science Series

Founding Editors, Alan S. Kaufman and Nadeen L. Kaufman

Essentials of Statistics for the Social and Behavioral Sciencesby Barry H. Cohen and R. Brooke Lea

Essentials of Psychological Testingby Susana Urbina

Essentials of Research Design and Methodologyby Geoffrey Marczyk, David DeMatteo, and David Festinger

Essentials of Child Psychopathologyby Linda Wilmshurst

Copyright © 2005 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone].

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold with the understanding that the publisher is not engaged in rendering professional services. If legal, accounting, medical, psychological or any other expert assistance is required, the services of a competent professional person should be sought.

Designations used by companies to distinguish their products are often claimed as trademarks. In all instances where John Wiley & Sons, Inc. is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration.

For general information on our other products and services please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. For more information about Wiley products, visit our website at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Marczyk, Geoffrey R., 1964-

Essentials of research design and methodology/Geoffrey Marczyk, David DeMatteo,

David Festinger.

p. cm.—( Essentials of behavioral science series)

Includes bibliographical references and index.

ISBN 0-471-47053-8 (pbk.)

1. Psychology—Research—Methodology. I. DeMatteo, David, 1972- II. Festinger, David.

III. Title. IV. Series.

BF76.5.M317 2005

150’.72—dc22 2004058384

To Helene and my family

G.M.

To Christina and Emma

D.D.

To Tracy, Ashley, and Elijah

D.F.

SERIES PREFACE

In the Essentials of Behavioral Science series, our goal is to provide readers with books that will deliver key practical information in an efficient, accessible style. The series features books on a variety of topics, such as statistics, psychological testing, and research design and methodology, to name just a few. For the experienced professional, books in the series offer a concise yet thorough review of a specific area of expertise, including numerous tips for best practices. Students can turn to series books for a clear and concise overview of the important topics in which they must become proficient to practice skillfully, efficiently, and ethically in their chosen fields.

Wherever feasible, visual cues highlighting key points are utilized alongside systematic, step-by-step guidelines. Chapters are focused and succinct. Topics are organized for an easy understanding of the essential material related to a particular topic. Theory and research are continually woven into the fabric of each book, but always to enhance the practical application of the material, rather than to sidetrack or overwhelm readers. With this series, we aim to challenge and assist readers in the behavioral sciences to aspire to the highest level of competency by arming them with the tools they need for knowledgeable, informed practice.

The purposes of Essentials of Research Design and Methodology are to discuss the various types of research designs that are commonly used, the basic process by which research studies are conducted, the research-related considerations of which researchers should be aware, the manner in which the results of research can be interpreted and disseminated, and the typical pitfalls faced by researchers when designing and conducting a research study. This book is ideal for those readers with minimal knowledge of research as well as for those readers with intermediate knowledge who need a quick refresher regarding particular aspects of research design and methodology. For those readers with an advanced knowledge of research design and methodology, this book can be used as a concise summary of basic research techniques and principles, or as an adjunct to a more advanced research methodology and design textbook. Finally, even for those readers who do not conduct research, this book will become a valuable addition to your bookcase because it will assist you in becoming a more educated consumer of research. Being able to evaluate the appropriateness of a research design or the conclusions drawn from a particular research study will become increasingly more important as research becomes more accessible to nonscientists. In that regard, this book will improve your ability to efficiently and effectively digest and understand the results of a research study.

Alan S. Kaufman, PhD, and Nadeen L. Kaufman, EdD, Founding Editors

Yale University School of Medicine

ACKNOWLEDGMENTS

We would like to thank Karen Dugosh and Audrey Cleary for their helpful comments on earlier drafts of this book. We would also like to thank Susan Matties for her research assistance. Additional thanks go to Dr. Virginia Brabender for introducing us to John Wiley and Sons. Finally we’d like to thank Tracey Belmont, our editor, for her support and sense of humor.

Essentials of Research Design and Methodology

One

INTRODUCTION AND OVERVIEW

Progress in almost every field of science depends on the contributions made by systematic research; thus research is often viewed as the cornerstone of scientific progress. Broadly defined, the purpose of research is to answer questions and acquire new knowledge. Research is the primary tool used in virtually all areas of science to expand the frontiers of knowledge. For example, research is used in such diverse scientific fields as psychology, biology, medicine, physics, and botany, to name just a few of the areas in which research makes valuable contributions to what we know and how we think about things. Among other things, by conducting research, researchers attempt to reduce the complexity of problems, discover the relationship between seemingly unrelated events, and ultimately improve the way we live.

Although research studies are conducted in many diverse fields of science, the general goals and defining characteristics of research are typically the same across disciplines. For example, across all types of science, research is frequently used for describing a thing or event, discovering the relationship between phenomena, or making predictions about future events. In short, research can be used for the purposes of description, explanation, and prediction, all of which make important and valuable contributions to the expansion of what we know and how we live our lives. In addition to sharing similar broad goals, scientific research in virtually all fields of study shares certain defining characteristics, including testing hypotheses, careful observation and measurement, systematic evaluation of data, and drawing valid conclusions.

In recent years, the results of various research studies have taken center stage in the popular media. No longer is research the private domain of research professors and scientists wearing white lab coats. To the contrary, the results of research studies are frequently reported on the local evening news, CNN, the Internet, and various other media outlets that are accessible to both scientists and nonscientists alike. For example, in recent years, we have all become familiar with research regarding the effects of stress on our psychological well-being, the health benefits of a low-cholesterol diet, the effects of exercise in preventing certain forms of cancer, which automobiles are safest to drive, and the deleterious effects of pollution on global warming. We may have even become familiar with research studies regarding the human genome, the Mars Land Rover, the use of stem cells, and genetic cloning. Not too long ago, it was unlikely that the results of such highly scientific research studies would have been shared with the general public to such a great extent.

Despite the accessibility and prevalence of research in today’s society, many people share common misperceptions about exactly what research is, how research can be used, what research can tell us, and the limitations of research. For some people, the term “research” conjures up images of scientists in laboratories watching rats run through mazes or mixing chemicals in test tubes. For other people, the term “research” is associated with telemarketer surveys, or people approaching them at the local shopping mall to “just ask you a few questions about your shopping habits.” In actuality, these stereotypical examples of research are only a small part of what research comprises. It is therefore not surprising that many people are unfamiliar with the various types of research designs, the basics of how research is conducted, what research can be used for, and the limits of using research to answer questions and acquire new knowledge. Rapid Reference 1.1 discusses what we mean by “research” from a scientific perspective.

Before addressing these important issues, however, we should first briefly review what science is and how it goes about telling us what we know.

Rapid Reference 1.1

What Exactly is Research?

Research studies come in many different forms, and we will discuss several of these forms in more detail in Chapter 5. For now, however, we will focus on two of the most common types of research—correlational research and experimental research.

Correlational research: In correlational research, the goal is to determine whether two or more variables are related. (By the way, “variables” is a term with which you should be familiar. A variable is anything that can take on different values, such as weight, time, and height.) For example, a researcher may be interested in determining whether age is related to weight. In this example, a researcher may discover that age is indeed related to weight because as age increases, weight also increases. If a correlation between two variables is strong enough, knowing about one variable allows a researcher to make a prediction about the other variable. There are several different types of correlations, which will be discussed in more detail in Chapter 5. It is important to point out, however, that a correlation—or relationship—between two things does not necessarily mean that one thing caused the other. To draw a cause-and-effect conclusion, researchers must use experimental research. This point will be emphasized throughout this book.

Experimental research: In its simplest form, experimental research involves comparing two groups on one outcome measure to test some hypothesis regarding causation. For example, if a researcher is interested in the effects of a new medication on headaches, the researcher would randomly divide a group of people with headaches into two groups. One of the groups, the experimental group, would receive the new medication being tested. The other group, the control group, would receive a placebo medication (i.e., a medication containing a harmless substance, such as sugar, that has no physiological effects). Besides receiving the different medications, the groups would be treated exactly the same so that the research could isolate the effects of the medications. After receiving the medications, both groups would be compared to see whether people in the experimental group had fewer headaches than people in the control group. Assuming this study was properly designed (and properly designed studies will be discussed in detail in later chapters), if people in the experimental group had fewer headaches than people in the control group, the researcher could conclude that the new medication reduces headaches.

OVERVIEW OF SCIENCE AND THE SCIENTIFIC METHOD

In simple terms, science can be defined as a methodological and systematic approach to the acquisition of new knowledge. This definition of science highlights some of the key differences between how scientists and nonscientists go about acquiring new knowledge. Specifically, rather than relying on mere casual observations and an informal approach to learn about the world, scientists attempt to gain new knowledge by making careful observations and using systematic, controlled, and methodical approaches (Shaughnessy & Zechmeister, 1997). By doing so, scientists are able to draw valid and reliable conclusions about what they are studying. In addition, scientific knowledge is not based on the opinions, feelings, or intuition of the scientist. Instead, scientific knowledge is based on objective data that were reliably obtained in the context of a carefully designed research study. In short, scientific knowledge is based on the accumulation of empirical evidence (Kazdin, 2003a), which will be the topic of a great deal of discussion in later chapters of this book.

The defining characteristic of scientific research is the scientific method (summarized in Rapid Reference 1.2). First described by the English philosopher and scientist Roger Bacon in the 13th century, it is still generally agreed that the scientific method is the basis for all scientific investigation. The scientific method is best thought of as an approach to the acquisition of new knowledge, and this approach effectively distinguishes science from nonscience. To be clear, the scientific method is not actually a single method, as the name would erroneously lead one to believe, but rather an overarching perspective on how scientific investigations should proceed. It is a set of research principles and methods that helps researchers obtain valid results from their research studies. Because the scientific method deals with the general approach to research rather than the content of specific research studies, it is used by researchers in all different scientific disciplines. As will be seen in the following sections, the biggest benefit of the scientific method is that it provides a set of clear and agreed-upon guidelines for gathering, evaluating, and reporting information in the context of a research study (Cozby, 1993).

Rapid Reference 1.2

The Scientific Method

The development of the scientific method is usually credited to Roger Bacon, a philosopher and scientist from 13th-century England, although some argue that the Italian scientist Galileo Galilei played an important role in formulating the scientific method. Later contributions to the scientific method were made by the philosophers Francis Bacon and René Descartes. Although some disagreement exists regarding the exact characteristics of the scientific method, most agree that it is characterized by the following elements:

• Empirical approach

• Observations

• Questions

• Hypotheses

• Experiments

• Analyses

• Conclusions

• Replication

There has been some disagreement among researchers over the years regarding the elements that compose the scientific method. In fact, some researchers have even argued that it is impossible to define a universal approach to scientific investigation. Nevertheless, for over 100 years, the scientific method has been the defining feature of scientific research. Researchers generally agree that the scientific method is composed of the following key elements (which will be the focus of the remainder of this chapter): an empirical approach, observations, questions, hypotheses, experiments, analyses, conclusions, and replication.

Before proceeding any further, one word of caution is necessary. In the brief discussion of the scientific method that follows, we will be introducing several new terms and concepts that are related to research design and methodology. Do not be intimidated if you are unfamiliar with some of the content contained in this discussion. The purpose of the following is simply to set the stage for the chapters that follow, and we will be elaborating on each of the terms and concepts throughout the remainder of the book.

Empirical Approach

The scientific method is firmly based on the empirical approach. The empirical approach is an evidence-based approach that relies on direct observation and experimentation in the acquisition of new knowledge (see Kazdin, 2003a). In the empirical approach, scientific decisions are made based on the data derived from direct observation and experimentation. Contrast this approach to decision making with the way that most nonscientific decisions are made in our daily lives. For example, we have all made decisions based on feelings, hunches, or “gut” instinct. Additionally, we may often reach conclusions or make decisions that are not necessarily based on data, but rather on opinions, speculation, and a hope for the best. The empirical approach, with its emphasis on direct, systematic, and careful observation, is best thought of as the guiding principle behind all research conducted in accordance with the scientific method.

Observations

An important component in any scientific investigation is observation. In this sense, observation refers to two distinct concepts—being aware of the world around us and making careful measurements. Observations of the world around us often give rise to the questions that are addressed through scientific research. For example, the Newtonian observation that apples fall from trees stimulated much research into the effects of gravity. Therefore, a keen eye to your surroundings can often provide you with many ideas for research studies. We will discuss the generation of research ideas in more detail in Chapter 2.

In the context of science, observation means more than just observing the world around us to get ideas for research. Observation also refers to the process of making careful and accurate measurements, which is a distinguishing feature of well-conducted scientific investigations. When making measurements in the context of research, scientists typically take great precautions to avoid making biased observations. For example, if a researcher is observing the amount of time that passes between two events, such as the length of time that elapses between lightning and thunder, it would certainly be advisable for the researcher to use a measurement device that has a high degree of accuracy and reliability. Rather than simply trying to “guesstimate” the amount of time that elapsed between those two events, the researcher would be advised to use a stopwatch or similar measurement device. By doing so, the researcher ensures that the measurement is accurate and not biased by extraneous factors. Most people would likely agree that the observations that we make in our daily lives are rarely made so carefully or systematically.

An important aspect of measurement is an operational definition. Researchers define key concepts and terms in the context of their research studies by using operational definitions. By using operational definitions, researchers ensure that everyone is talking about the same phenomenon. For example, if a researcher wants to study the effects of exercise on stress levels, it would be necessary for the researcher to define what “exercise” is. Does exercise refer to jogging, weight lifting, swimming, jumping rope, or all of the above? By defining “exercise” for the purposes of the study, the researcher makes sure that everyone is referring to the same thing. Clearly, the definition of “exercise” can differ from one study to another, so it is crucial that the researcher define “exercise” in a precise manner in the context of his or her study. Having a clear definition of terms also ensures that the researcher’s study can be replicated by other researchers. The importance of operational definitions will be discussed further in Chapter 2.

Questions

After getting a research idea, perhaps from making observations of the world around us, the next step in the research process involves translating that research idea into an answerable question. The term “answerable” is particularly important in this respect, and it should not be overlooked. It would obviously be a frustrating and ultimately unrewarding endeavor to attempt to answer an unanswerable research question through scientific investigation. An example of an unanswerable research question is the following: “Is there an exact replica of me in another universe?” Although this is certainly an intriguing question that would likely yield important information, the current state of science cannot provide an answer to that question. It is therefore important to formulate a research question that can be answered through available scientific methods and procedures. One might ask, for example, whether exercising (i.e., perhaps operationally defined as running three times per week for 30 minutes each time) reduces cholesterol levels. This question could be researched and answered using established scientific methods.

Hypotheses

The next step in the scientific method is coming up with a hypothesis, which is simply an educated—and testable—guess about the answer to your research question. A hypothesis is often described as an attempt by the researcher to explain the phenomenon of interest. Hypotheses can take various forms, depending on the question being asked and the type of study being conducted (see Rapid Reference 1.3).

A key feature of all hypotheses is that each must make a prediction. Remember that hypotheses are the researcher’s attempt to explain the phenomenon being studied, and that explanation should involve a prediction about the variables being studied. These predictions are then tested by gathering and analyzing data, and the hypotheses can either be supported or refuted (falsified; see Rapid Reference 1.4) on the basis of the data.

In their simplest forms, hypotheses are typically phrased as “if-then” statements. For example, a researcher may hypothesize that “if people exercise for 30 minutes per day at least three days per week, then their cholesterol levels will be reduced.” This hypothesis makes a prediction about the effects of exercising on levels of cholesterol, and the prediction can be tested by gathering and analyzing data.

Two types of hypotheses with which you should be familiar are the null hypothesis and the alternate (or experimental) hypothesis. The null hypothesis always predicts that there will be no differences between the groups being studied. By contrast, the alternate hypothesis predicts that there will be a difference between the groups. In our example, the null hypothesis would predict that the exercise group and the no-exercise group will not differ significantly on levels of cholesterol. The alternate hypothesis would predict that the two groups will differ significantly on cholesterol levels. Hypotheses will be discussed in more detail in Chapter 2.

Rapid Reference 1.3

Relationship Between Hypotheses and Research Design

Hypotheses can take many different forms depending on the type of research design being used. Some hypotheses may simply describe how two things may be related. For example, in correlational research (which will be discussed in Chapter 5), a researcher might hypothesize that alcohol intoxication is related to poor decision making. In other words, the researcher is hypothesizing that there is a relationship between using alcohol and decision making ability (but not necessarily a causal relationship). However, in a study using a randomized controlled design (which will also be discussed in Chapter 5), the researcher might hypothesize that using alcohol causes poor decision making. Therefore, as may be evident, the hypothesis being tested by a researcher is largely dependent on the type of research design being used. The relationship between hypotheses and research design will be discussed in more detail in later chapters.

Rapid Reference 1.4

Falsifiability of Hypotheses

According to the 20th-century philosopher Karl Popper, hypotheses must be falsifiable (Popper, 1963). In other words, the researcher must be able to demonstrate that the hypothesis is wrong. If a hypothesis is not falsifiable, then science cannot be used to test the hypothesis. For example, hypotheses based on religious beliefs are not falsifiable. Therefore, because we can never prove that faith-based hypotheses are wrong, there would be no point in conducting research to test them. Another way of saying this is that the researcher must be able to reject the proposed explanation (i.e., hypothesis) of the phenomenon being studied.

Experiments

After articulating the hypothesis, the next step involves actually conducting the experiment (or research study). For example, if the study involves investigating the effects of exercise on levels of cholesterol, the researcher would design and conduct a study that would attempt to address that question. As previously mentioned, a key aspect of conducting a research study is measuring the phenomenon of interest in an accurate and reliable manner (see Rapid Reference 1.5). In this example, the researcher would collect data on the cholesterol levels of the study participants by using an accurate and reliable measurement device. Then, the researcher would compare the cholesterol levels of the two groups to see if exercise had any effects.

Rapid Reference 1.5

Accuracy vs. Reliability

When talking about measurement in the context of research, there is an important distinction between being accurate and being reliable. Accuracy refers to whether the measurement is correct, whereas reliability refers to whether the measurement is consistent. An example may help to clarify the distinction. When throwing darts at a dart board, “accuracy” refers to whether the darts are hitting the bull’s eye (an accurate dart thrower will throw darts that hit the bull’s eye). “Reliability,” on the other hand, refers to whether the darts are hitting the same spot (a reliable dart thrower will throw darts that hit the same spot). Therefore, an accurate and reliable dart thrower will consistently throw the darts in the bull’s eye. As may be evident, however, it is possible for the dart thrower to be reliable, but not accurate. For example, the dart thrower may throw all of the darts in the same spot (which demonstrates high reliability), but that spot may not be the bull’s eye (which demonstrates low accuracy). In the context of measurement, both accuracy and reliability are equally important.

Analyses

After conducting the study and gathering the data, the next step involves analyzing the data, which generally calls for the use of statistical techniques. The type of statistical techniques used by a researcher depends on the design of the study, the type of data being gathered, and the questions being asked. Although a detailed discussion of statistics is beyond the scope of this text, it is important to be aware of the role of statistics in conducting a research study. In short, statistics help researchers minimize the likelihood of reaching an erroneous conclusion about the relationship between the variables being studied.

A key decision that researchers must make with the assistance of statistics is whether the null hypothesis should be rejected. Remember that the null hypothesis always predicts that there will be no difference between the groups. Therefore, rejecting the null hypothesis means that there is a difference between the groups. In general, most researchers seek to reject the null hypothesis because rejection means the phenomenon being studied (e.g., exercise, medication) had some effect.

It is important to note that there are only two choices with respect to the null hypothesis. Specifically, the null hypothesis can be either rejected or not rejected, but it can never be accepted. If we reject the null hypothesis, we are concluding that there is a significant difference between the groups. If, however, we do not reject the null hypothesis, then we are concluding that we were unable to detect a difference between the groups. To be clear, it does not mean that there is no difference between the two groups. There may in actuality have been a significant difference between the two groups, but we were unable to detect that difference in our study. We will talk more about this important distinction in later chapters.

The decision of whether to reject the null hypothesis is based on the results of statistical analyses, and there are two types of errors that researchers must be careful to avoid when making this decision—type I errors and Type II errors. A Type I error occurs when a researcher concludes that there is a difference between the groups being studied when, in fact, there is no difference. This is sometimes referred to as a “false positive.” By contrast, a Type II error occurs when the researcher concludes that there is not a difference between the two groups being studied when, in fact, there is a difference. This is sometimes referred to as a “false negative.” As previously noted, the conclusion regarding whether there is a difference between the groups is based on the results of statistical analyses. Specifically, with a Type I error, although there is a statistically significant result, it occurred by chance (or error) and there is not actually a difference between the two groups (Wampold, Davis, & Good, 2003). With a Type II error, there is a nonsignificant statistical result when, in fact, there actually is a difference between the two groups (Wampold et al.).

The typical convention in most fields of science allows for a 5% chance of erroneously rejecting the null hypothesis (i.e., of making a Type I error). In other words, a researcher will conclude that there is a significant difference between the groups being studied (i.e., will reject the null hypothesis) only if the chance of being incorrect is less than 5%. For obvious reasons, researchers want to reduce the likelihood of concluding that there is a significant difference between the groups being studied when, in fact, there is not a difference.

The distinction between Type I and Type II errors is very important, although somewhat complicated. An example may help to clarify these terms. In our example, a researcher conducts a study to determine whether a new medication is effective in treating depression. The new medication is given to Group 1, while a placebo medication is given to Group 2. If, at the conclusion of the study, the researcher concludes that there is a significant difference in levels of depression between Groups 1 and 2 when, in fact, there is no difference, the researcher has made a Type I error. In simpler terms, the researcher has detected a difference between the groups that in actuality does not exist; the difference between the groups occurred by chance (or error). By contrast, if the researcher concludes that there is no significant difference in levels of depression between Groups 1 and 2 when, in fact, there is a difference, the researcher has made a Type II error. In simpler terms, the researcher has failed to detect a difference that actually exists between the groups.

Which type of error is more serious—Type I or Type II? The answer to this question often depends on the context in which the errors are made. Let’s use the medical context as an example. If a doctor diagnoses a patient with cancer when, in fact, the patient does not have cancer (i.e., a false positive), the doctor has committed a Type I error. In this situation, it is likely that the erroneous diagnosis will be discovered (perhaps through a second opinion) and the patient will undoubtedly be relieved. If, however, the doctor gives the patient a clean bill of health when, in fact, the patient actually has cancer (i.e., a false negative), the doctor has committed a Type II error. Most people would likely agree that a Type II error would be more serious in this example because it would prevent the patient from getting necessary medical treatment.

You may be wondering why researchers do not simply set up their research studies so that there is even less chance of making a Type I error. For example, wouldn’t it make sense for researchers to set up their research studies so that the chance of making a Type I error is less than 1% or, better yet, 0%? The reason that researchers do not set up their studies in this manner has to do with the relationship between making Type I errors and making Type II errors. Specifically, there is an inverse relationship between Type I errors and Type II errors, which means that by decreasing the probability of making a Type I error, the researcher is increasing the probability of making a Type II error. In other words, if a researcher reduces the probability of making a Type I error from 5% to 1%, there is now an increased probability that the researcher will make a Type II error by failing to detect a difference that actually exists. The 5% level is a standard convention in most fields of research and represents a compromise between making Type I and Type II errors.

CAUTION

Type I Errors vs. Type II Errors

Type I Error (false positive): Concluding there is a difference between the groups being studied when, in fact, there is no difference.

Type II Error (false negative): Concluding there is no difference between the groups being studied when, in fact, there is a difference. Type I and Type II errors can be illustrated using the following table:

Researcher’s ConclusionActual ResultsDifferenceNo DifferenceDifferenceCorrect decisionType I errorNo differenceType II errorCorrect decision

Conclusions

After analyzing the data and determining whether to reject the null hypothesis, the researcher is now in a position to draw some conclusions about the results of the study. For example, if the researcher rejected the null hypothesis, the researcher can conclude that the phenomenon being studied had an effect—a statistically significant effect, to be more precise. If the researcher rejects the null hypothesis in our exercise-cholesterol example, the researcher is concluding that exercise had an effect on levels of cholesterol.

It is important that researchers make only those conclusions that can be supported by the data analyses. Going beyond the data is a cardinal sin that researchers must be careful to avoid. For example, if a researcher conducted a correlational study and the results indicated that the two things being studied were strongly related, the researcher could not conclude that one thing caused the other. An oft-repeated statement that will be explained in later chapters is that correlation (i.e., a relationship between two things) does not equal causation. In other words, the fact that two things are related does not mean that one caused the other.

Replication

One of the most important elements of the scientific method is replication. Replication essentially means conducting the same research study a second time with another group of participants to see whether the same results are obtained (see Kazdin, 1992; Shaughnessy & Zechmeister, 1997). The same researcher may attempt to replicate previously obtained results, or perhaps other researchers may undertake that task. Replication illustrates an important point about scientific research—namely, that researchers should avoid drawing broad conclusions based on the results of a single research study because it is always possible that the results of that particular study were an aberration. In other words, it is possible that the results of the research study were obtained by chance or error and, therefore, that the results may not accurately represent the actual state of things. However, if the results of a research study are obtained a second time (i.e., replicated), the likelihood that the original study’s findings were obtained by chance or error is greatly reduced.

DON’T FORGET

Correlation Does Not Equal Causation

Before looking at an example of why correlation does not equal causation, let’s make sure that we understand what a correlation is. A correlation is simply a relationship between two things. For example, size and weight are often correlated because there is a relationship between the size of something and its weight. Specifically, bigger things tend to weigh more. The results of correlational studies simply provide researchers with information regarding the relationship between two or more variables, which may serve as the basis for future studies. It is important, however, that researchers interpret this relationship cautiously.

For example, if a researcher finds that eating ice cream is correlated with (i.e., related to) higher rates of drowning, the researcher cannot conclude that eating ice cream causes drowning. It may be that another variable is responsible for the higher rates of drowning. For example, most ice cream is eaten in the summer and most swimming occurs in the summer. Therefore, the higher rates of drowning are not caused by eating ice cream, but rather by the increased number of people who swim during the summer.

The importance of replication in research cannot be overstated. Replication serves several integral purposes, including establishing the reliability (i.e., consistency) of the research study’s findings and determining whether the same results can be obtained with a different group of participants. This last point refers to whether the results of the original study are generalizable to other groups of research participants. If the results of a study are replicated, the researchers—and the field in which the researchers work—can have greater confidence in the reliability and generalizability of the original findings.

GOALS OF SCIENTIFIC RESEARCH

As stated previously, the goals of scientific research, in broad terms, are to answer questions and acquire new knowledge. This is typically accomplished by conducting research that permits drawing valid inferences about the relationship between two or more variables (Kazdin, 1992). In later chapters, we discuss the specific techniques that researchers use to ensure that valid inferences can be drawn from their research, and in Rapid References 1.6 and 1.7 we present some research-related terms you should become familiar with. For now, however, our main discussion will focus on the goals of scientific research in more general terms. Most researchers agree that the three general goals of scientific research are description, prediction, and understanding/explanation (Cozby, 1993; Shaughnessy & Zechmeister, 1997).

Description

Perhaps the most basic and easily understood goal of scientific research is description. In short, description refers to the process of defining, classifying, or categorizing phenomena of interest. For example, a researcher may wish to conduct a research study that has the goal of describing the relationship between two things or events, such as the relationship between cardiovascular exercise and levels of cholesterol. Alternatively, a researcher may be interested in describing a single phenomenon, such as the effects of stress on decision making.

Descriptive research is useful because it can provide important information regarding the average member of a group. Specifically, by gathering data on a large enough group of people, a researcher can describe the average member, or the average performance of a member, of the particular group being studied. Perhaps a brief example will help clarify what we mean by this. Let’s say a researcher gathers Scholastic Aptitude Test (SAT) scores from the current freshman class at a prestigious university. By using some simple statistical techniques, the researcher would be able to calculate the average SAT score for the current college freshman at the university. This information would likely be informative for high school students who are considering applying for admittance at the university.

Rapid Reference 1.6

Categories of Research

There are two broad categories of research with which researchers must be familiar.

Quantitative vs. Qualitative

• Quantitative research involves studies that make use of statistical analyses to obtain their findings. Key features include formal and systematic measurement and the use of statistics.

• Qualitative research involves studies that do not attempt to quantify their results through statistical summary or analysis. Qualitative studies typically involve interviews and observations without formal measurement. A case study, which is an in-depth examination of one person, is a form of qualitative research. Qualitative research is often used as a source of hypotheses for later testing in quantitative research.

Nomothetic vs. Idiographic

• The nomothetic approach uses the study of groups to identify general laws that apply to a large group of people. The goal is often to identify the average member of the group being studied or the average performance of a group member.

• The idiographic approach is the study of an individual. An example of the idiographic approach is the aforementioned case study.

The choice of which research approaches to use largely depends on the types of questions being asked in the research study, and different fields of research typically rely on different categories of research to achieve their goals. Social science research, for example, typically relies on quantitative research and the nomothetic approach. In other words, social scientists study large groups of people and rely on statistical analyses to obtain their findings. These two broad categories of research will be the primary focus of this book.

Rapid Reference 1.7

Sample vs. Population

Two key terms that you must be familiar with are “sample” and “population.” The population is all individuals of interest to the researcher. For example, a researcher may be interested in studying anxiety among lawyers; in this example, the population is all lawyers. For obvious reasons, researchers are typically unable to study the entire population. In this case it would be difficult, if not impossible, to study anxiety among all lawyers. Therefore, researchers typically study a subset of the population, and that subset is called a sample.

Because researchers may not be able to study the entire population of interest, it is important that the sample be representative of the population from which it was selected. For example, the sample of lawyers the researcher studies should be similar to the population of lawyers. If the population of lawyers is composed mainly of White men over the age of 35, studying a sample of lawyers composed mainly of Black women under the age of 30 would obviously be problematic because the sample is not representative of the population. Studying a representative sample permits the researcher to draw valid inferences about the population. In other words, when a researcher uses a representative sample, if something is true of the sample, it is likely also true of the population.

One example of descriptive research is correlational research. In correlational research (as mentioned earlier), the researcher attempts to determine whether there is a relationship—that is, a correlation—between two or more variables (see Rapid Reference 1.8 for two types of correlation). For example, a researcher may wish to determine whether there is a relationship between SAT scores and grade-point averages (GPAs) among a sample of college freshmen. The many uses of correlational research will be discussed in later chapters.

Rapid Reference 1.8

Two Types of Correlation

Positive correlation: A positive correlation between two variables means that both variables change in the same direction (either both increase or both decrease). For example, if GPAs increase as SAT scores increase, there is a positive correlation between SAT scores and GPAs.

Negative (inverse) correlation: A negative correlation between two variables means that as one variable increases, the other variable decreases. In other words, the variables change in opposite directions. So, if GPAs decrease as SAT scores increase, there is a negative correlation between SAT scores and GPAs.

Prediction

Another broad goal of research is prediction. Prediction-based research often stems from previously conducted descriptive research. If a researcher finds that there is a relationship (i.e., correlation) between two variables, then it may be possible to predict one variable from knowledge of the other variable. For example, if a researcher found that there is a relationship between SAT scores and GPAs, knowledge of the SAT scores alone would allow the researcher to predict the associated GPAs.

Many important questions in both science and the so-called real world involve predicting one thing based on knowledge of something else. For example, college admissions boards may attempt to predict success in college based on the GPAs and SAT scores of the applicants. Employers may attempt to predict job success based on work samples, test scores, and candidate interviews. Psychologists may attempt to predict whether a traumatic life event leads to depression. Medical doctors may attempt to predict what levels of obesity and high blood pressure are associated with cardiovascular disease and stroke. Meteorologists may attempt to predict the amount of rain based on the temperature, barometric pressure, humidity, and weather patterns. In each of these examples, a prediction is being made based on existing knowledge of something else.

Understanding/Explanation

Being able to describe something and having the ability to predict one thing based on knowledge of another are important goals of scientific research, but they do not provide researchers with a true understanding of a phenomenon. One could argue that true understanding of a phenomenon is achieved only when researchers successfully identify the cause or causes of the phenomenon. For example, being able to predict a student’s GPA in college based on his or her SAT scores is important and very practical, but there is a limit to that knowledge. The most important limitation is that a relationship between two things does not permit an inference of causality. In other words, the fact that two things are related and knowledge of one thing (e.g., SAT scores) leads to an accurate prediction of the other thing (e.g., GPA) does not mean that one thing caused the other. For example, a relationship between SAT scores and freshman GPAs does not mean that the SAT scores caused the freshman-year GPAs. More than likely, the SAT scores are indicative of other things that may be more directly responsible for the GPAs. For example, the students who score high on the SAT may also be the students who spend a lot of time studying, and it is likely the amount of time studying that is the cause of a high GPA.

The ability of researchers to make valid causal inferences is determined by the type of research designs they use. Correlational research, as previously noted, does not permit researchers to make causal inferences regarding the relationship between the two things that are correlated. By contrast, a randomized controlled study, which will be discussed in detail in Chapter 5, permits researchers to make valid cause-and-effect inferences.

There are three prerequisites for drawing an inference of causality between two events (see Shaughnessy & Zechmeister, 1997). First, there must be a relationship (i.e., a correlation) between the two events. In other words, the events must covary—as one changes, the other must also change. If two events do not covary, then a researcher cannot conclude that one event caused the other event. For example, if there is no relationship between television viewing and deterioration of eyesight, then one cannot reasonably conclude that television viewing causes a deterioration of eyesight.

Second, one event (the cause) must precede the other event (the effect). This is sometimes referred to as a time-order relationship. This should make intuitive sense. Obviously, if two events occur simultaneously, it cannot be concluded that one event caused the other. Similarly, if the observed effect comes before the presumed cause, it would make little sense to conclude that the cause caused the effect.

Third, alternative explanations for the observed relationship must be ruled out. This is where it gets tricky. Stated another way, a causal explanation between two events can be accepted only when other possible causes of the observed relationship have been ruled out. An example may help to clarify this last required condition for causality. Let’s say that a researcher is attempting to study the effects of two different psychotherapies on levels of depression. The researcher first obtains a representative sample of people with the same level of depression (as measured by a valid and reliable measure) and then randomly assigns them to one of two groups. Group 1 will get Therapy A and Group 2 will get Therapy B. The obvious goal is to compare levels of depression in both groups after providing the therapy. It would be unwise in this situation for the researcher to assign all of the participants under age 30 to Group 1 and all of the participants over age 30 to Group 2: If, at the conclusion of the study, Group 1 and Group 2 differed significantly in levels of depression, the researcher would be unable to determine which variable—type of therapy or age—was responsible for the reduced depression. We would say that this research has been confounded, which means that two variables (in this case, the type of therapy and age) were allowed to vary (or be different) at the same time. Ideally, only the variable being studied (e.g., the type of therapy) will differ between the two groups.

DON’T FORGET

Prerequisites for Inferences of Causality

• There must be an existing relationship between two events.

• The cause must precede the effect.

• Alternative explanations for the relationship must be ruled out.

OVERVIEW OF THE BOOK

The focus of this book is, obviously, research design and methodology. Although these terms are sometimes incorrectly used interchangeably, they are distinct concepts with well-defined and circumscribed meanings. Therefore, before proceeding any further, it would behoove us to define these terms, at least temporarily. As defined by Kazdin (1992, 2003a), a recognized leader in the field of research, methodology refers to the principles, procedures, and practices that govern research, whereas research design refers to the plan used to examine the question of interest. “Methodology” should be thought of as encompassing the entire process of conducting research (i.e., planning and conducting the research study, drawing conclusions, and disseminating the findings). By contrast, “research design” refers to the many ways in which research can be conducted to answer the question being asked. These concepts will become clearer throughout this book, but it is important that you understand the focus of this book before reading any further.

Essentials of Research Design and Methodology succinctly covers all of the major topic areas within research design and methodology. Each chapter in this book covers a specific research-related topic using easy-to-understand language and illustrative examples. The book is not meant, however, to replace the very extensive and comprehensive coverage of research issues that can be found in other publications. For those readers who would like a more in-depth understanding of the specific topic areas covered in this book, we would suggest looking to the publications included in the reference list at the end of this book. Finally, although each chapter builds upon the knowledge obtained from the previous chapters, each chapter can also be used as a stand-alone summary of the important points within that topic area. For this reason, we occasionally cover some of the same material in more than one chapter.

The chapters in Essentials of Research Design and Methodology are organized in a manner that accurately reflects the logical flow of a research project from development to conclusion. The first three chapters lay the foundation for conducting a research project. This chapter introduced you to some of the key concepts relating to science, research design, and methodology. As will be discussed, at a basic level, the first step in conducting research involves coming up with an idea and translating that idea into a testable question or statement. Chapter 2 discusses these preliminary stages of research, including choosing a research idea, formulating a research problem, choosing appropriate independent and dependent variables, and selecting a sample of participants for your study. As every researcher knows, coming up with a well-designed research study can be a challenging process, but the importance of that task cannot be overstated. Chapter 3 discusses some of the more common pitfalls faced by researchers when thinking about the design of a research study.

After a research question has been formulated, researchers must choose a research design, collect and analyze the data, and draw some conclusions. Chapter 4 will introduce you to the common measurement issues and strategies that must be considered when designing a research study. Chapter 5 will present a concise summary of the most common types of research designs that are available to researchers; as will be discussed, the type of research design chosen for a particular study depends largely on the question being asked. Chapter 6 will focus on one of the most important considerations in all of research—validity. Put simply, validity refers to the soundness of the research design being used, with high validity typically producing more accurate and meaningful results. Validity comes in many forms, and Chapter 6 will discuss each one and how to maximize it in the course of research. Chapter 7 will introduce you to many of the issues faced by researchers when analyzing data and attempting to draw conclusions based on the data.

Most research is subject to oversight by one or more ethical review committees, such as a university-based institutional review board. These committees are charged with the important task of reviewing all proposed research studies to ensure that they comply with applicable regulations governing research, which may be established by the university, the city, the state, or the federal government, depending on the nature of the research being conducted. Knowledge of the commonly encountered ethical issues will assist researchers in avoiding ethical violations and resolving ethical dilemmas. To this end, Chapter 8 will focus on the most commonly encountered ethical issues faced by researchers when designing and conducting a research study. Among other things, Chapter 8 will focus on the important topic of informed consent to research.

Finally, Chapter 9 will present a brief section on the dissemination of research results, including publication in peer-reviewed journals and presentations at professional conferences. Chapter 9 will include a distillation of major principles of research design and methodology that are applicable for those conducting research in a variety of capacities and settings. Chapter 9 will conclude by presenting a checklist of the major research-related concepts and considerations covered throughout this book.

Before concluding this chapter, one word of caution is necessary regarding the focus of this book. As stated previously, research studies come in many different forms, depending on the scientific discipline within which the research is being conducted. For example, most research studies in the field of quantum physics take place in a laboratory and do not involve human participants. Contrast this with the research studies that are conducted by social scientists, which may often take place in real-world settings and involve human participants. For the sake of clarity, consistency, and ease of reading, we thought that it was necessary to narrow the focus of this book to one broad type of research. Therefore, throughout this book, we will focus primarily on empirical research involving human participants, which is most commonly found in the social and behavioral sciences. Focusing on this type of research permits us to explore a wider range of research-related considerations that must be addressed by researchers across many scientific disciplines.

TEST YOURSELF

1. ______________ can be defined as a methodological and systematic approach to the acquisition of new knowledge.

2. The defining characteristic of scientific research is the ______________ ______________.

3. The ______________ approach relies on direct observation and experimentation in the acquisition of new knowledge.

4. Scientists define key concepts and terms in the context of their research studies by using ______________ definitions.

5. What are the three general goals of scientific research?

Answers: 1. Science; 2. scientific method; 3. empirical; 4. operational; 5. description, prediction, and understanding/explaining