Survey Methodology E-Book

Robert M. Groves

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Wiley Series in Survey Methodology
• Sprache: Englisch

Seitenzahl: 958

Veröffentlichungsjahr: 2011

Beschreibung

Praise for the First Edition:

"The book makes a valuable contribution by synthesizing current research and identifying areas for future investigation for each aspect of the survey process."
—Journal of the American Statistical Association

"Overall, the high quality of the text material is matched by the quality of writing . . ."
—Public Opinion Quarterly

". . . it should find an audience everywhere surveys are being conducted."
—Technometrics

This new edition of Survey Methodology continues to provide a state-of-the-science presentation of essential survey methodology topics and techniques. The volume's six world-renowned authors have updated this Second Edition to present newly emerging approaches to survey research and provide more comprehensive coverage of the major considerations in designing and conducting a sample survey.

Key topics in survey methodology are clearly explained in the book's chapters, with coverage including sampling frame evaluation, sample design, development of questionnaires, evaluation of questions, alternative modes of data collection, interviewing, nonresponse, post-collection processing of survey data, and practices for maintaining scientific integrity. Acknowledging the growing advances in research and technology, the Second Edition features:

• Updated explanations of sampling frame issues for mobile telephone and web surveys

• New scientific insight on the relationship between nonresponse rates and nonresponse errors

• Restructured discussion of ethical issues in survey research, emphasizing the growing research results on privacy, informed consent, and confidentiality issues

• The latest research findings on effective questionnaire development techniques

• The addition of 50% more exercises at the end of each chapter, illustrating basic principles of survey design

• An expanded FAQ chapter that addresses the concerns that accompany newly established methods

Providing valuable and informative perspectives on the most modern methods in the field, Survey Methodology, Second Edition is an ideal book for survey research courses at the upper-undergraduate and graduate levels. It is also an indispensable reference for practicing survey methodologists and any professional who employs survey research methods.

Leseprobe

CONTENTS

PREFACE TO THE FIRST EDITION

PREFACE TO THE SECOND EDITION

ACKNOWLEDGMENTS

CHAPTER ONE: AN INTRODUCTION TO SURVEY METHODOLOGY

1.1 INTRODUCTION

1.2 A BRIEF HISTORY OF SURVEY RESEARCH

1.3 SOME EXAMPLES OF ONGOING SURVEYS

1.4 WHAT IS SURVEY METHODOLOGY?

1.5 THE CHALLENGE OF SURVEY METHODOLOGY

1.6 ABOUT THIS BOOK

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER TWO: INFERENCE AND ERROR IN SURVEYS

2.1 INTRODUCTION

2.2 THE LIFE CYCLE OF A SURVEY FROM A DESIGN PERSPECTIVE

2.3 THE LIFE CYCLE OF A SURVEY FROM A QUALITY PERSPECTIVE

2.4 PUTTING IT ALL TOGETHER

2.5 ERROR NOTIONS IN DIFFERENT KINDS OF STATISTICS

2.6 NONSTATISTICAL NOTIONS OF SURVEY QUALITY

2.7 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER THREE: TARGET POPULATIONS, SAMPLING FRAMES, AND COVERAGE ERROR

3.1 INTRODUCTION

3.2 POPULATIONS AND FRAMES

3.3 COVERAGE PROPERTIES OF SAMPLING FRAMES

3.4 ALTERNATIVE FRAMES FOR SURVEYS OF THE TARGET POPULATION OF HOUSEHOLDS OR PERSONS

3.5 FRAME ISSUES FOR OTHER COMMON TARGET POPULATIONS

3.6 COVERAGE ERROR

3.7 REDUCING UNDERCOVERAGE

3.8 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER FOUR: SAMPLE DESIGN AND SAMPLING ERROR

4.1 INTRODUCTION

4.2 SAMPLES AND ESTIMATES

4.3 SIMPLE RANDOM SAMPLING

4.4 CLUSTER SAMPLING

4.5 STRATIFICATION AND STRATIFIED SAMPLING

4.6 SYSTEMATIC SELECTION

4.7 COMPLICATIONS IN PRACTICE

4.8 SAMPLING U.S. TELEPHONE HOUSEHOLDS

4.9 SELECTING PERSONS WITHIN HOUSEHOLDS

4.10 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER FIVE: METHODS OF DATA COLLECTION

5.1 ALTERNATIVE METHODS OF DATA COLLECTION

5.2 CHOOSING THE APPROPRIATE METHOD

5.3 EFFECTS OF DIFFERENT DATA COLLECTION METHODS ON SURVEY ERRORS

5.4 USING MULTIPLE MODES OF DATA COLLECTION

5.5 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER SIX: NONRESPONSE IN SAMPLE SURVEYS

6.1 INTRODUCTION

6.2 RESPONSE RATES

6.3 IMPACT OF NONRESPONSE ON THE QUALITY OF SURVEY ESTIMATES

6.4 THINKING CAUSALLY ABOUT SURVEY NONRESPONSE ERROR

6.5 DISSECTING THE NONRESPONSE PHENOMENON

6.6 DESIGN FEATURES TO REDUCE UNIT NONRESPONSE

6.7 ITEM NONRESPONSE

6.8 ARE NONRESPONSE PROPENSITIES RELATED TO OTHER ERROR SOURCES?

6.9 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER SEVEN: QUESTIONS AND ANSWERS IN SURVEYS

7.1 ALTERNATIVES METHODS OF SURVEY MEASUREMENT

7.2 COGNITIVE PROCESSES IN ANSWERING QUESTIONS

7.3 PROBLEMS IN ANSWERING SURVEY QUESTIONS

7.4 GUIDELINES FOR WRITING GOOD QUESTIONS

7.5 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER EIGHT: EVALUATING SURVEY QUESTIONS

8.1 INTRODUCTION

8.2 EXPERT REVIEWS

8.3 FOCUS GROUPS

8.4 COGNITIVE INTERVIEWS

8.5 FIELD PRETESTS AND BEHAVIOR CODING

8.6 RANDOMIZED OR SPLIT-BALLOT EXPERIMENTS

8.7 APPLYING QUESTION STANDARDS

8.8 SUMMARY OF QUESTION EVALUATION TOOLS

8.9 LINKING CONCEPTS OF MEASUREMENT QUALITY TO STATISTICAL ESTIMATES

8.10 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER NINE: SURVEY INTERVIEWING

9.1 THE ROLE OF THE INTERVIEWER

9.2 INTERVIEWER BIAS

9.3 INTERVIEWER VARIANCE

9.4 STRATEGIES FOR REDUCING INTERVIEWER BIAS

9.5 STRATEGIES FOR REDUCING INTERVIEWER-RELATED VARIANCE

9.6 THE CONTROVERSY ABOUT STANDARDIZED INTERVIEWING

9.7 INTERVIEWER MANAGEMENT

9.8 VALIDATING THE WORK OF INTERVIEWERS

9.9 THE USE OF RECORDED VOICES (AND FACES) IN DATA COLLECTION

9.10 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER TEN: POSTCOLLECTION PROCESSING OF SURVEY DATA

10.1 INTRODUCTION

10.2 CODING

10.3 ENTERING NUMERIC DATA INTO FILES

10.4 EDITING

10.5 WEIGHTING

10.6 IMPUTATION FOR ITEM-MISSING DATA

10.7 SAMPLING VARIANCE ESTIMATION FOR COMPLEX SAMPLES

10.8 SURVEY DATA DOCUMENTATION AND METADATA

10.9 SUMMARY

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER ELEVEN: PRINCIPLES AND PRACTICES RELATED TO ETHICAL RESEARCH

11.1 INTRODUCTION

11.2 STANDARDS FOR THE CONDUCT OF RESEARCH

11.3 STANDARDS FOR DEALING WITH CLIENTS

11.4 STANDARDS FOR DEALING WITH THE PUBLIC

11.5 STANDARDS FOR DEALING WITH RESPONDENTS

11.6 EMERGING ETHICAL ISSUES

11.7 RESEARCH ABOUT ETHICAL ISSUES IN SURVEYS

11.8 ADMINISTRATIVE AND TECHNICAL PROCEDURES FOR SAFEGUARDING CONFIDENTIALITY

11.9 SUMMARY AND CONCLUSIONS

KEYWORDS

FOR MORE IN-DEPTH READING

EXERCISES

CHAPTER TWELVE: FAQS ABOUT SURVEY METHODOLOGY

12.1 INTRODUCTION

12.2 THE QUESTIONS AND THEIR ANSWERS

REFERENCES

INDEX

WILEY SERIES IN SURVEY METHODOLOGY

WILEY SERIES IN SURVEY METHODOLOGY

Established in Part by WALTER A. SHEWHART AND SAMUEL S. WILKS

Editors: Robert M. Groves, Graham Kalton,J.N K. Rao, Norbert Schwarz, Christopher Skinner

A complete list of the titles in this series appears at the end of this volume.

Copyright © 2009 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone] or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the U.S. at [phone], outside the U.S. at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Survey methodology / Robert Groves… [et al.]. — 2nd ed.p. cm.Includes bibliographical references and index.ISBN 978-0-470-46546-2 (paper)1. Surveys—Methodology. 2. Social surveys—Methodology. 3. Social sciences—Research—Statistical methods. I. Groves, Robert M.HA31.2.S873 2009001.4’33--dc222009004196

PREFACE TO THE FIRST EDITION

We wrote this book with a specific purpose in mind. We are all survey methodologists—students of the theories and practices of the various data collection and analysis activities that are called “survey research.” Surveys (in a form that would be recognizable today) are approximately 60–80 years old. Over the past two decades, a set of theories and principles has evolved that offer a unified perspective on the design, conduct, and evaluation of surveys. This perspective is most commonly labeled the “total survey error” paradigm. The framework guides modem research about survey quality and shapes how practicing survey professionals approach their work. The field arising out of this research domain can appropriately be called “survey methodology.”

We increasingly noticed a mismatch, however, between the texts related to surveys and how the science of surveys was evolving. Many survey research texts focused on the application of tools and deemphasized the theories and science underlying those tools. Many texts told students to do things that were no longer or never supported by the methodological research in the area. In short, there were books that emphasized “how to do” surveys but neglected the science underlying the practices that were espoused.

Most harmful we thought was the impression conveyed to those who read the texts that surveys were merely a recipe-like task; if step-by-step instructions were followed, high quality would be guaranteed. In contrast, we saw surveys as requiring the implementation of principles in unique ways to fit a particular substantive purpose for a particular target population.

These issues became particularly important to us when the demand for a one semester graduate level (and senior undergraduate level) course became obvious at the Joint Program in Survey Methodology (JPSM), a consortium graduate program funded by the U.S. Federal statistical agencies in which the authors teach. The students would often have advanced education in another field (e.g., economics, statistics, or psychology) but no formal exposure to the field of survey methodology. We planned a 14-week lecture course with exercises and examinations that began in the Fall of 1998, and we immediately suffered from the absence of a text that could accompany the lectures and motivate the exercises.

We began to envision a text describing the basic principles of survey design discovered in methodological research over the past years and the guidance they offered for decisions that are made in the execution of good quality surveys. We wanted to include exercises that would help integrate an understanding of the field. We wanted to convey that the field is based on experimental and other research findings and that practical survey design was not a mere matter of judgment and opinion but rather the result of a body of research findings.

We drafted this book over several years. After we wrote the first couple of chapters, we hit a dry spell, which was ended when our colleague Nancy Mathiowetz kicked us back in gear. We appreciated her energy in getting us going again.

The manuscript profited greatly from the critique of our student colleagues. The text had a dry run in the Summer of 2003 in a class at the University of Michigan Survey Research Center Summer Institute in Survey Research Techniques, entitled “Introduction to Survey Research Techniques,” taught by Maria Krysan and Sue Ellen Hansen. We thank these instructors for helping improve the manuscript. We learned much from the criticisms and ideas of both Krysan and Hansen and the students in the class: Nike Adebiyi, Jennifer Bowers, Scott Compton, Sanjay Kumar, Dumile Mkhwanazi, Hanne Muller, Vuyelwa Nkambule, Laurel Park, Aaron Russell, Daniel Spiess, Kathleen Stack, Kimiko Tanaka, Dang Viet Phuong, and Christopher Webb.

It is fair to say that this book strongly reflects the lessons taught by many of our own mentors. One deserves special mention. All of the authors were friends and students of Charlie Cannell (some formally; all informally). Charles F. Cannell began his survey career with Rensis Likert at the U.S. Department of Agriculture, Division of Program Surveys. Cannell later joined Likert and others in founding the University of Michigan Survey Research Center in 1946. He was the first director of field operations at the Center and had a long and distinguished career in survey methodology. In memory of Charlie and his work, the Institute for Social Research (the larger institute of which the SRC is part) established the Charles F. Cannell Fund in Survey Methodology. All royalties that result from the sales of this text will be contributed to this fund. The endowment from the fund is designated for support of young scholars developing their research careers in survey methodology. We can think of no better use.

We designed the text to be used in a class where the participants had taken one or more courses in statistics. The key relevant skill is the reading of statistical notation, including summation signs, notation for expected values, and simple algebraic manipulation of summed quantities. Some chapters present quantitative analyses using regression and logistic regression models, and students unfamiliar with linear modeling need some help in understanding these results.

This book has 12 chapters, in the order in which they are presented in the semester-length course on survey methodology called “Fundamentals of Survey Methodology” at the JPSM. We envision that instructors will want to assign additional readings, often from one or more review articles referenced in the chapters.

The first two chapters (“Introduction to Survey Methodology” and “Inference And Error In Surveys”) are conceptual in nature. Chapter 1 presents six example surveys that are used throughout the book to illustrate various principles and practices. The instructor can supplement the text by displaying the Web pages of these surveys in the class and leading class discussions about the key design features and products of the surveys.

The second chapter presents the key components of the total survey error paradigm. Again, at this early stage in the class, we have found that providing students with examples of key error components by referencing the example surveys aids in student understanding. A defining characteristic of surveys as we see them is that they are designed to produce statistical descriptions of populations. Although there are computer programs that will calculate statistics, we think it is critical that a survey methodologist understand the calculations that underlie those statistics. Hence, the book routinely presents statistical notation along with a conceptual discussion of what is being calculated.

The treatment of Chapter 2 would be a good time to devote a class to statistical notation, which, once it is learned, will help the students be more comfortable throughout the rest of the book

Starting with Chapter 3 (“Target Populations, Sampling Frames, and Coverage Error”), each chapter deals with a different component of total survey error and the methodological research discoveries that guide best practices. The focus of these chapters is deliberately the research on which best practices in survey research are based. We have often found that students beginning the study of survey methodology have the perspective that their opinions on a specific design feature are diagnostic of the best practices. The material that is presented in Chapters 3–11 attempts to show that there are scientific studies of survey methods that inform best practice; opinions are of little value unless they are research-based. Some of these studies do not have intuitively obvious findings. Hence, a student of the field must review the past methodological literature and at times do novel research to determine good design. There are two devices in the text that can help convey this perspective of the field. One is the set of embedded references to research in the discussions. The other is the presentation of illustrative boxes that give short descriptions of classic research in the domain covered in the chapter. These are summaries that describe the design, findings, limitations, and impact of the research. The full articles on this research can be used as supplementary readings, which could be discussed in class. There are also suggested supplementary readings at the end of each chapter.

Chapter 4 (“Sample Design and Sampling Error”) uses more statistical notation than most of the other chapters. When many participants in the course need remedial instruction in reading and understanding statistical notation, we have referred them to the small monograph by Kalton, An Introduction to Survey Sampling (Sage, 1983). In some editions of the course, we have spent three weeks on the coverage and sampling chapters.

Each of Chapters 5–10 is normally covered in one week of the course. We have found it useful to emphasize the parallels between equations expressing coverage error and nonresponse error. We have also emphasized how the basic principles of intraclass correlations apply both to sample clustering effects and interviewer variance.

Chapter 11 (“Principles and Practices Related to Scientific Integrity”) is included not just as sensitivity training but because it includes both conceptual frameworks underlying ethical treatment of human subjects and also recent theory and practice regarding disclosure analysis of survey data. Again, we describe how research, as well as judgement, can affect decisions related to ethical issues.

We wrote Chapter 12 (“FAQs About Survey Methodology”) in a very different style. It is a tradition in the course, in a review session prior to the final examination, to have an open question section. At this time, we found students asking the kind of questions that come from attempts to integrate their learning of specific lessons with their larger worldview. Hence, we constructed a “frequently asked questions” format including those global questions and offering our answers to them.

The manuscript was greatly improved by the editorial wisdom of Sarah Dipko and Sonja Ziniel. Adam Kelley assisted in computer-based processing of figures and tables. Lisa Van Horn at Wiley is a production editor with a wonderful sense of when intervention is needed and when it isn’t. We thank them all.

It was great fun writing this book, assembling our views on key research areas, and debating how to convey the excitement of survey methodology as an area of knowledge. We hope you have as much fun as we did.

Ann Arbor, Michigan ROBERT M. GROVESBoston, Massachusetts FLOYD J. FOWLER, JR.Ann Arbor, Michigan MICK P. COUPERAnn Arbor, Michigan JAMES M. LEPKOWSKIAnn Arbor, Michigan ELEANOR SINGERCollege Park, Maryland ROGER TOURANGEAUMarch 2004  

PREFACE TO THE SECOND EDITION

We have been pleased by the acceptance of the first edition of Survey Methodology. It has now been used by instructors around the world and has been translated into several other languages. Some of these instructors and their students have graciously pointed out weaknesses and errors in some sections of the text. Some of them gave us great ideas to improve the text.

In addition, as survey methodologists actively conducting research in the field, we became increasingly aware that some of the lessons in the text were becoming out of date. This was most true of sections of the book that concern the role of survey nonresponse in the quality of survey estimates and the rapidly evolving new modes of data collection.

For those reasons, we assembled the group of coauthors and agreed to update parts of chapters that could most profit from changes. As the reader will see, there is increased discussion of sampling frame issues for mobile telephone and web surveys in Chapter 3. There is an integration of some of the example surveys into the presentation of sample designs in Chapter 4, along with a new section on selection of persons within households. The changes in Chapter 5 update the findings on mobile phone and web surveys. Chapter 6, on survey nonresponse, is radically changed, reflecting new insights into how nonresponse rates and nonresponse errors relate to one another. Chapter 8, on evaluating survey questions, highlights new research findings on effective questionnaire development techniques. Chapter 11, on ethical issues in survey research, is reorganized to emphasize the growing research results on privacy, informed consent, and confidentiality issues. The remaining chapters provide the reader with more recent methodological research findings, especially when they expand our understanding of survey errors. The chapters have about 50% more exercises, following feedback from instructors that such additions would benefit their use of the text.

Two assistants labored over this edition’s manuscript: Michael Guterbock and Kelly Smid. Some Ph.D. students at Michigan read draft chapters (Ashley Bowers, Matthew Jans, Courtney Kennedy, Joe Sakshaug, and Brady West). When we signed the contract with Wiley, we demanded that Lisa Van Hom continue as our production editor. All of the above went beyond the call to make this edition a success. We thank them.

As with the last edition, we want to use the royalties of the text to help persons newly entering the field of survey methodology. They will be given to the Rensis Likert Fund for Research on Survey Methodology, which directly benefits graduate students in survey methodology.

Ann Arbor, Michigan ROBERT M. GROVESBoston, Massachusetts FLOYD J. FOWLER, JR.Ann Arbor, Michigan MICK P. COUPERAnn Arbor, Michigan JAMES M. LEPKOWSKIAnn Arbor, Michigan ELEANOR SINGERCollege Park, Maryland ROGER TOURANGEAUMarch, 2009  

ACKNOWLEDGMENTS

Reprinted tables and figures in Survey Methodology and their copyright holders are listed below. The authors appreciate permission to adapt or reprint them.

Figure 1.5c, from Mokdad, Ford, Bowman, Dietz, Vinicor, Bales, and Marks (2003) with permission of the American Medical Association. Copyright © 2003.

Table 5.1 from Groves (1989) with permission of John Wiley and Sons. Copyright © 1989.

Figure 6.5 from Groves (2006) and text surrounding from Groves and Peytcheva (2008) with permission of the American Association for Public Opinion Research. Copyright © 2006, 2008.

Table in box on page 185, from Merkle and Edelman in Groves, Dillman, Eltinge, and Little (2002) with permission of John Wiley and Sons. Copyright © 2002.

Figure 7.2, from Tourangeau, Rips, and Rasinski (2000) reprinted with permission of Cambridge University Press. Copyright © 2000.

Box on page 236, from Schwarz, Hippler, Deutsch, and Strack (1985) reprinted with permission of the American Association for Public Opinion Research. Copyright © 1985.

Figure 7.3, from Jenkins and Dillman in Lyberg, Biemer, Collins, de Leeuw, Dippo, Schwarz, and Trewin (1997) reprinted with permission of John Wiley and Sons. Copyright © 1997.

Box on p. 268, from Oksenberg, Cannell, and Kalton (1991) reprinted with permission of Statistics Sweden. Copyright © 1991.

Box on p. 293, from Schuman and Converse (1971) reprinted with permission of the American Association for Public Opinion Research. Copyright © 1971.

Box on p. 299, from Kish (1962) reprinted with permission of the American Statistical Association. Copyright © 1962.

Table 9.2, from Fowler and Mangione (1990) reprinted with permission of Sage Publications. Copyright © 1990.

Table 10.5, from Campanelli, Thomson, Moon, and Staples in Lyberg, Biemer, Collins, de Leeuw, Dippo, Schwarz, and Trewin (1997) reprinted with permission of John Wiley and Sons. Copyright © 1997.

CHAPTER ONE

AN INTRODUCTION TO SURVEY METHODOLOGY

A Note to the Reader

You are about to be exposed to a system of principles called “survey methodology” for collecting information about the social and economic world. We have written this book in an attempt to describe the excitement of designing, conducting, analyzing, and evaluating sample surveys. To appreciate this fully, use the devices we have placed in each chapter to enrich your memory of the material. Throughout the book, you will see boxes with illustrations and examples of key principles, terminology notes, and highlights of classic research studies in the field. In the outside margin of each page you will find key terms, at the point where they are defined. At the end of each chapter is a set of exercises that you can use to test your understanding of that chapter’s material. The best strategy is to read the text through, then, at the end of each chapter, go back, read the boxes, and review the key terms.

At 8:30 AM on the day before the first Friday of each month, a group of economists and statisticians enter a soundproof and windowless room in a building at 2 Massachusetts Avenue, NE, in Washington, DC, USA. Once those authorized are present, the room is sealed.

Those in the room are professional staff of the U.S. Bureau of Labor Statistics (BLS), and their task is to review and approve a statistical analysis of key economic data. Indeed, they have spent the week poring over sets of numbers, comparing them, examining indicators of their qualities, looking for anomalies, and writing drafts of a press release describing the numbers. They write the press release in simple language, understandable by those who have no technical knowledge about how the numbers were produced.

At 8:00 AM the next day, a group of journalists assemble in a monitored room in the nearby main Department of Labor building, removed from any contact with the outside world. The BLS staff enter the room and then reveal the results to the journalists. The journalists immediately prepare news stories based on the briefing. At exactly 8:30 AM, they simultaneously electronically transmit their stories to their news organizations and sometimes telephone editors and producers.

The statistics revealed are the unemployment rate of the prior month and the number of jobs created in the prior month. The elaborate protections and security used prior to their release stem from the enormous impact the numbers can have on society. Indeed, in months when the numbers signal important changes in the health of the U.S. economy, thousands of stock market investors around the world make immediate buy and sell decisions. Within 45 minutes of the announcement, trillions of dollars can move in and out of markets around the world based on the two numbers revealed at 8:30 AM.

Both the unemployment rate and the jobs count result from statistical surveys. A household survey produces the unemployment rate; an employer survey, the jobs count. The households and employers surveyed have been carefully selected so that their answers, when summarized, reflect the answers that would be obtained if the entire population were questioned. In the surveys, thousands of individual people answer carefully phrased questions about their own or their company’s attributes. In the household survey, professional interviewers ask the questions and enter the answers onto laptop computers. In the employer survey, the respondents complete a standardized questionnaire either on paper or electronically. Complex data processing steps follow the collection of the data, to assure internal integrity of the numbers.

These two numbers have such an impact because they address an important component of the health of the nation’s economy, and they are credible. Macroeconomic theory and decades of empirical results demonstrate their importance. However, only when decision makers believe the numbers do they gain value. This is a book about the process of generating such numbers through statistical surveys and how survey design can affect the quality of survey statistics. In a real sense, it addresses the question of when numbers from surveys are credible and when they are not.

1.1 INTRODUCTION

This chapter is an introduction to survey methodology as a field of knowledge, as a profession, and as a science. The initial sections of the chapter define the field so that the reader can place it among others. At the end of the chapter, readers will have a sense of what survey methodology is and what survey methodologists do.

A “survey” is a systematic method for gathering information from (a sample of) entities for the purposes of constructing quantitative descriptors of the attributes of the larger population of which the entities are members. The word “systematic” is deliberate and meaningfully distinguishes surveys from other ways of gathering information. The phrase “(a sample of)” appears in the definition because sometimes surveys attempt to measure everyone in a population and sometimes just a sample.

The quantitative descriptors are called “statistics.” Statistics are quantitative summaries of observations on a set of elements. Some are “descriptive statistics,” describing the size and distributions of various attributes in a population (e.g., the mean years of education of persons, the total number of persons in the hospital, the percentage of persons supporting the president). Others are “analytic statistics,” measuring how two or more variables are related (e.g., a regression coefficient describing how much increases in income are associated with increases in years of education; a correlation between education and number of books read in the last year). That goal sets surveys apart from other efforts to describe people or events. The statistics attempt to describe basic characteristics or experiences of large and small populations in our world.

Almost every country in the world uses surveys to estimate their rate of unemployment, basic prevalence of immunization against disease, opinions about the central government, intentions to vote in an upcoming election, and people’s satisfaction with services and products that they buy. Surveys are a key tool in tracking global economic trends, the rate of inflation in prices, and investments in new economic enterprises. Surveys are one of the most commonly used methods in the social sciences to understand the way societies work and to test theories of behavior. In a very real way, surveys are a crucial building block in a modern information-based society.

Although a variety of activities are called surveys, this book focuses on surveys that have the following characteristics:

1) Information is gathered primarily by asking people questions.

2) Information is collected either by having interviewers ask questions and record answers or by having people read or hear questions and record their own answers.

3) Information is collected from only a subset of the population to be described—a sample—rather than from all members.

Since “ology” is Greek for “the study of,” survey methodology is the study of survey methods. It is the study of sources of error in surveys and how to make the numbers produced by surveys as accurate as possible. Here the word “error” refers to deviations or departures from the desired outcome. In the case of surveys, “error” is used to describe deviations from the true values applicable to the population studied. Sometimes, the phrase “statistical error” is used to differentiate this meaning from a reference to simple mistakes.

The way each of the above steps is carried out—which questions are asked, how answers are collected, and which people answer the questions—can affect the quality (or error properties) of survey results. This book will describe how to conduct surveys in the real world and how to evaluate the quality of survey results. It will describe what is known, and not known, about how to minimize error in survey statistics. Most of all, this book will attempt to distill the results of 100 years of scientific studies that have defined the theories and principles, as well as practices, of high-quality survey research.

1.2 A BRIEF HISTORY OF SURVEY RESEARCH

Converse (1987) has produced an important account of the history of survey research in the United States, and we recount some of the highlights here. There are four perspectives on surveys that are worth describing: the purposes to which surveys were put, the development of question design, the development of sampling methods, and the development of data collection methods.

1.2.1 The Purposes of Surveys

Perhaps the earliest type of survey is the census, generally conducted by governments. Censuses are systematic efforts to count an entire population, often for purposes of taxation or political representation. In the United States, the Constitution stipulates that a census must be conducted every ten years, to reapportion the House of Representatives reflecting current population residence patterns. This gives the statistics from a census great political import. Because of this, they are often politically contentious (Anderson, 1990).

A prominent early reason for surveys was to gain understanding of a social problem. Some people trace the origins of modern survey research to Charles Booth, who produced a landmark study titled Life and Labour of the People of London (1889–1903) (http://booth.lse.ac.uk/). As Converse recounts it, Booth spent his own money to collect voluminous data on the poor in London and the reasons why they were poor. He wrote at least 17 volumes based on the data he collected. He did not use methods like the ones we use today—no well-defined sampling techniques, no standardized questions. Indeed, interviewer observation and inference produced much of the information. However, the Booth study used quantitative summaries from systematic measurements to understand a fundamental societal problem.

In contrast to studies of social problems, journalism and market research grew to use surveys to gain a systematic view of “the man on the street.” A particular interest was reactions to political leaders and preferences in upcoming elections. That interest led to the development of modern public opinion polling.

In a related way, market research sought knowledge about reactions of “real” people to existing and planned products or services. As early as the 1930s, there was serious research on what programs and messages delivered via the radio would be most popular. The researchers began to use surveys of broader samples to produce information more useful to commercial decision makers.

Over the early 20th century, public opinion polling and market research, sometimes done by the same companies, evolved to use mail surveys and telephone surveys. They often sampled from available lists, such as telephone, driver’s license, registered voter, or magazine subscriber listings. They collected their data primarily by asking a fixed set of questions; observations by interviewers and proxy reporting of other people’s situations were not part of what they needed. These features were directly tied to the most important difference between what they were doing and what those who had gone before had done; rather than collecting data about facts and objective characteristics of people, the polling and market research surveyors were interested in what people knew, felt, and thought.

Schuman (1997) on “Poll” Versus “Survey”

What is the difference between a poll and a survey? The word “poll” is most often used for private-sector opinion studies, which use many of the same design features as studies that would be called “surveys.” “Poll” is rarely used to describe studies conducted in government or scientific domains. There are, however, no clear distinctions between the meanings of the two terms. Schuman notes that the two terms have different roots: “’Poll’ is a four letter word, generally thought to be from an ancient Germanic term referring to ‘head,’ as in counting heads. The two-syllable word ‘survey,’ on the other hand, comes from the French survee, which in turn derives from Latin super (over) and videre (to look). The first is therefore an expression with appeal to a wider public, the intended consumers of results from Gallup, Harris, and other polls. The second fits the needs of academicians in university institutes who wish to emphasize the scientific or scholarly character of their work.” (page 7)

The measurement of attitudes and opinions is a key foundation of the modern management philosophies that place much weight on customer satisfaction. Customer satisfaction surveys measure expectations of purchasers about the quality of a product or service and how well their expectations were met in specific transactions. Such surveys are ubiquitous tools of management to improve the performance of their organizations.

Politicians and political strategists now believe that opinion polls are critical to good decisions on campaign strategy and messages to the public about important issues. Indeed, a common criticism of modern politicians is that they rely too heavily on polling data to shape their personal opinions, choosing to reflect the public’s views rather than provide leadership to the public about an issue.

1.2.2 The Development of Standardized Questioning

The interest in measuring subjective states (i.e., characteristics that cannot be observed, internalized within a person) also had the effect of focusing attention on question wording and data collection methods. When collecting factual information, researchers had not thought it important to carefully word questions. Often, interviewers were sent out with lists of objectives, such as age, occupation, and education, and the interviewers would decide on how the questions would be worded. Experienced researchers often did the interviewing, with great confidence that they knew how to phrase questions to obtain good answers.

However, the market research and polling organizations were doing large numbers of interviews, using newly hired people with no special background in the social sciences. Of necessity, researchers needed to specify more carefully the information sought by the survey. Further, researchers found that small changes in wording of an attitude question sometimes had unusually large effects on the answers.

Thus, early in the development of opinion surveys, attention began to be paid to giving interviewers carefully worded questions that they were to ask exactly the same way for each interview. Also, as interviewers were used more to ask questions, it was found that how they asked questions and recorded answers could affect the results. This led eventually to researchers training and supervising interviewers more formally than earlier.

Question wording also was influenced as the academics started to pay some attention to what the commercial researchers were doing. Psychometricians, psychologists who quantify psychological states, had been interested in how to put meaningful numbers on subjective states. Measuring intelligence was the first effort in this direction. However, people such as Thurstone also worked on how to assign numbers to attitudes, feelings, and ratings (e.g., Thurstone and Chave, 1929).

For the most part, their approaches were extremely cumbersome and were used primarily when they could get captive college student volunteers to fill out lengthy, highly redundant questionnaires. Such instruments were not going to be useful for most survey interviews with representative samples; they took too long to measure one or a few attitudes. Rensis Likert in his PhD dissertation (Likert, 1932), however, demonstrated that a single, streamlined question, with a scaled set of answers, could accomplish much the same thing as a lengthy series of paired comparisons. Likert applied the work to surveys (and later founded the University of Michigan Survey Research Center in 1946).

1.2.3 The Development of Sampling Methods

Early researchers, such as Booth, essentially tried to collect data on every element of a defined population. Such censuses avoided problems of errors arising from measuring just a subset of the population, but were clearly impractical for large populations. Indeed, the difficulty of analyzing complete census data led to early efforts to summarize a census by taking a sample of returns. Early efforts to sample would study a “typical” town, or they would purposively try to collect individuals to make the samples look like the population—for example, by interviewing about half men and half women, and trying to have them distributed geographically in a way that is similar to the population.

Although the theory of probability was established in the 18th century, its application to practical sample survey work was largely delayed until the 20th century. The first applications were the taking of a “1 in N” systematic selection from census returns. These were “probability samples”; that is, every record had a known nonzero chance of selection into the sample.

A big breakthrough in sampling came from people who did research on agriculture. In order to predict crop yields, statisticians had worked out a strategy they called “area probability sampling.” This is just what it sounds like: they would sample areas or plots of land and find out what farmers were doing with those plots in the spring (for example, if they were planting something on them and, if so, what) in order to project what the fall crops would look like. The same technique was developed to sample households. By drawing samples of geographic blocks in cities or tracts of land in rural areas, listing the housing units on the blocks or rural tracts, then sampling the housing units that were listed, samplers found a way to give all households and, by extension, the people living in them, a chance to be sampled. The attraction of this technique included the elimination of the need for a list of all persons or all households in the population prior to drawing the sample.

The Depression and World War II were major stimuli for survey research. One of the earliest modern probability samples was drawn for the Monthly Survey of Unemployment, starting in December, 1939, led by a 29-year-old statistician, Morris Hansen, who later became a major figure in the field (Hansen, Hurwitz, and Madow, 1953). During the war, the federal government became interested in conducting surveys to measure people’s attitudes and opinions, such as interest in buying war bonds, as well as factual information. Considerable resources were devoted to surveys during the war, and researchers who were recruited to work with the government during the war later came to play critical roles in the development of survey methods. When the war was over, methodologists understood that in order to produce good population-based statistics it was necessary to attend to three aspects of survey methodology: how questions were designed; how the data were collected, including the training of interviewers; and how samples were drawn.

Probability samples are the standard by which other samples are judged. They are routinely used by almost all government statistical agencies when data are used to provide important information for policy makers. They are used for surveys used in litigation. They are used for measurement of media audience sizes, which in turn determine advertising rates. In short, whenever large stakes ride on the value of a sample, probability sampling is generally used.

1.2.4 The Development of Data Collection Methods

The gathering of information in early surveys was only one step more organized than talking to as many people as possible about some topic. The qualitative interviews produced a set of verbal notes, and the task of summarizing them with statistics was huge. Surveys grew to be popular tools because of the evolution of methods to collect systematic data cheaply and quickly.

Mailed paper questionnaires offered very low costs for measuring literate populations. Indeed, by 1960 a formal test of census procedures based on mailed questionnaires succeeded to an extent that the 1970 census was largely a mailed questionnaire survey. Further, mailed questionnaires proved to be much cheaper than sending interviewers to visit sample cases. On the other hand, mailed surveys were subject to the vagaries of the postal service, which, even when it worked perfectly, produced survey periods that lasted months, not weeks.

With the spread of telephone service throughout the country, market researchers first saw two advantages of using the medium as a data collection tool. It was much faster than mail questionnaire surveys, and it was still cheaper than face-to-face surveys. For decades, however, the mode suffered from the clear lack of coverage of telephones among the poor and more transient members of the society. By the 1990s, however, almost all market research had moved away from the face-to-face survey and much scientific research was close behind in the abandonment of that mode. It was largely the federal government that continued to rely on face-to-face household surveys.

Like many fields of human activity, huge leaps in the efficiencies of surveys came from the invention of the computer. One of the first computers made in the United States was used in processing decennial census data. Survey researchers quickly recognized how computers could reduce the large amount of human resources needed to conduct surveys. Survey researchers first used computers to perform the analysis steps of a survey, then they began to use them to assist in checking the raw data for clerical errors, then to assist them in coding text answers, then in the data collection step itself. Now, computers (from handheld devices to networked systems) are used in almost every step of survey design, data collection, and analysis. The fastest growing application is the development of Web surveys.

As these various developments evolved, the field also developed a set of performance guidelines. Empirical studies demonstrated the value of various sample designs on the quality of statistics. Interviewer training guidelines improved the standardization of interviewers. Standards about computing and reporting response rates offered the field measures useful in comparing surveys.

In the 60 years following the advent of surveys, a great deal has been learned about how to design data collection systems to improve the quality of survey statistics. However, as can be seen from this short history, the basic elements of good survey methodology were defined in the first half of the 20th century.

1.3 SOME EXAMPLES OF ONGOING SURVEYS

One way to understand the range of survey methods and the potential of surveys to provide information is to give some examples. The following is a brief description of six surveys. We have chosen them to use as examples throughout the book for several reasons. First, they are all ongoing surveys. They are conducted year after year. By definition, that means that the sponsors think that there is a continuous need for the kind of information that they provide. That also means that someone thinks they are important. These are not particularly typical surveys. They do not include public opinion, political, or market research studies. They do not include any one-time surveys, which are highly prevalent. All of these surveys are funded by government sources.

However, they do differ from one another in numerous ways. One reason we chose this set is they do give a sense of the range of topics that are addressed by surveys and the variety of survey designs that are used. They also were chosen because they provide examples of excellence in survey research. Hence, they will provide opportunities for us to discuss how different methodological problems are addressed and solved.

In the brief summaries and charts provided, we describe some of the basic characteristics of each survey:

1) Their purposes

2) The populations they try to describe

3) The sources from which they draw samples

4) The design of the way they sample people

5) The use of interviewers

6) The mode of data collection

7) The use of computers in the collection of answers

Readers should think about these surveys in two different ways. First, think of them as information sources—what we can learn from them, what questions they answer, what policies and decisions they inform; in short, why they are conducted. Second, compare the design features above in order to see how different survey design features permit the surveys to achieve their different purposes.

1.3.1 The National Crime Victimization Survey

How much crime is there in the United States? Are crimes increasing in frequency or going down in frequency? Who gets victimized by crimes? Every society seeks answers to these questions. In the United States, the answers were sought through quantifying crime in a period of great public concern about organized crime. In the 1930s, the International Association of Chiefs of Police began a collection of administrative record counts. The method rested on the reporting of crimes to police in jurisdictions around the country, based on the administrative records kept by individual sheriffs, transit police, city police, and state police offices. Police chiefs had designed the record systems to have legal documentation of the circumstances of the crime, the victim of the crime, the offender, and any relevant evidence related to the crime. Individual staff members completed the paperwork that produced the administrative records. However, many crimes only come to the attention of the police if a citizen decides to report them. Often, the decision to produce a record or to label an incident was left to a relatively low-level police officer. For years, these records were the key information source on U.S. crime.

Over the years, several weaknesses in the statistics from police records became obvious. Sometimes a new mayor, fulfilling a pledge to reduce crime, created an environment in which more police officers chose not to label an incident as a crime, thus not producing an administrative record. Further, the statistics were tainted by different jurisdictions using different definitions for crime categories. When police believed that the crime would never be solved, they encouraged the resident not to file a formal report. There was growing evidence in some jurisdictions that relations between the public and the police were poor. Fear of the police among the public led to avoidance of reporting criminal incidents. The police officers themselves carried with them attitudes toward subpopulations that led to classifying an incident as a crime for one group but not for another group. It was becoming clear that, whereas major crimes like homicide were well represented in the record systems, the records tended to miss more minor, often unreported, crimes. Some jurisdictions kept very detailed, complete records, whereas others had very shoddy systems.

Thus, over the decades many began to distrust the value of the statistics to address the simplest question: “How much crime is there in the United States?” Further, the simple counts of crimes were not giving policy makers clear information about the characteristics of crimes and their victims, information helpful in considering alternative policies to reducing crime. The President’s Commission on Law Enforcement and the Administration of Justice, established by President Johnson, noted in a task force report that, “If we knew more about the character of both offenders and victims, the nature of their relationships and the circumstances that create a high probability of crime conduct, it seems likely that crime prevention programs could be made much more effective” (President’s Commission, 1967, as cited in Rand and Rennison, 2002).

In the late 1960s, criminologists began exploring the possibilities of using surveys to ask people directly whether they were a victim of a crime. This forced a conceptually different perspective on crime. Instead of a focus on the incident, it focused on one actor in the incident—the victim. This shift of perspective produced clear contrasts with the police reported crime record systems. Most obviously, homicide victims cannot report! Victimizations of young children might not be well reported by their parents (who may not know about incidents at school), and the children may not be good respondents. Crimes against companies present problems of who can report them well. For example, if someone starts a fire that burns down an apartment building, who are the victims—the owner of the property, the renters of the apartments, or the people visiting during the fire? From one perspective, all are victims, but asking all to report the arson as a victimization may complicate the counts of crimes. Victims can sometimes report that an unpleasant event occurred (e.g., someone with no right of entry entered their home). However, they cannot, as do police, gather the information that asserts the intent of the offender (e.g., he was attempting a theft of a television).

On the other hand, using individual reporters, a survey can cover victimizations reported to the police and those not reported to the police. This should provide a more complete picture of crime, if there are crimes not reported to the police. Indeed, self-reported victimizations might be a wonderful addition to statistics on police-reported crimes, as a way to compare the perceived victimization in a society with the officially reported victimization status. Moreover, the survey has the advantage of utilizing standard protocols for the measurement of victimizations across the country.

However, the notion of using a national survey to measure victimization faced other problems. Although all police agencies in the United States could be asked to report statistics from their record systems, it was financially impossible to ask all persons in the United States to report their individual victimizations. If a “representative” sample could be identified, this would make the victimization survey possible. Thus, in contrast to the record data, the survey would be subject to “sampling error” (i.e., errors in statistics because of the omission of some persons in the population).

But could and would people really report their victimizations accurately? Survey methodologists studied the reporting problem in the late 1960s and early 1970s. In methodological studies that sampled police records and then went back to victims of those crimes, they found that, in large part, they provided reports that mirrored the data in the records. However, Gottfredson and Hindelang (1977) among others noted that one problem was a tendency for persons to misdate the time of an incident. Most incidents that were important to them were reported as occurring more recently than they actually did. By and large, however, the pattern of reporting appeared to justify the expense of mounting a completely separate system of tracking crime in the country.

When all the design features were specified for the survey approach, it was clear that there were inevitable differences between the police reports and the victim reports (Rand and Rennison, 2002). The abiding strength of the survey was that it could measure crimes that are not reported to the police or for which police do not make formal documentation. However, the survey would be subject to sampling error. On the other hand, the police-reported crimes include some victims who are not residents of the country, and these would be missed in a survey of the US household population. Further, the police report statistics include homicides and arson, but do not include simple assault. Police reports exclude rapes of males; the survey could include rapes of both sexes. Some crimes have multiple victims, who could report the same incident as occurring to them (e.g., household crimes, group thefts); the survey would count them as multiple crimes, and the police reports as one incident, generally. The police report statistics depend on voluntary partnerships between the federal government and thousands of jurisdictions, but jurisdictions vary in their cooperation; the survey (using traditional methods) would suffer from some omissions of persons who are homeless or transient and from those who choose not to participate. The methodological work suggested that the survey would tend to underreport crimes where the offender is well known to the victim. There was also evidence that the survey would underreport less important crimes, apparently because they were difficult to remember. Finally, both systems have trouble with repeated incidents of the same character. In the survey method, repeated victimizations of the same type were to be counted once as “series incidents” (e.g., repeated beatings of a wife by her husband); the police-reported series would have as many reports as were provided to the police. The National Crime Victimization Survey (NCVS) (Table 1.1) has its roots within the U.S. Department of Justice in 1972. The current Bureau of Justice Statistics has the mission of collecting and disseminating statistical information on crimes, criminal offenders, victims of crimes, and the operations of justice systems at all levels of government. The Bureau of Justice Statistics contracts with the U.S. Census Bureau to collect the data in the NCVS.

The NCVS asks people to report the crimes they have experienced in the 6 months preceding the interview. If they asked people to report for 12 months, the researchers could learn about more events per interview; it would be a more efficient use of interview time. However, early studies showed that there is a marked drop-off in the accuracy of reporting when people are asked to remember events that happened more than 6 months in the past. In fact, there is underreporting of crimes even when the questions ask about the past 6 months. The accuracy of reporting would be higher if the questions asked about only one or two months, or, better yet, only a week or two. However, as the reporting period gets shorter, fewer and fewer people have anything to report, so more and more interviews provide minimal information about victimization. The designers of the survey chose 6 months as a reasonable point at which to balance reporting accuracy and the productivity of interviews.

Table 1.1. Example Survey: National Crime Victimization Survey (NCVS)

The sample for the NCVS is drawn in successive stages, with the goal of giving every person 12 years old and older a known chance of selection and, thereby, producing a way to represent all age eligible persons in the United States. (The jargon for this is a “multistage, stratified clustered area probability sample,” which will be described in Chapter 4.) The sample is restricted to persons who are household members, excluding the homeless, those in institutions, and in group quarters. (The survey methodologists judged that the cost of covering these subpopulations would be prohibitively expensive, detracting from the larger goals for the survey.) The sample is clustered into hundreds of different sample areas (usually counties or groups of counties) and the sample design is repeated samples of households from those same areas over the years of the study. The clustering is introduced to save money by permitting the hiring of a relatively small group of interviewers to train and supervise, who travel out to each sample household to visit the members and conduct interviews. Further, to save money all persons 12 years and over in the household are interviewed; thus, one sample household might produce one interview or many.

A further way to reduce costs of the survey is to repeatedly measure the same address. When the design randomly identifies a household to fall into the NCVS sample, the interviewer requests that, in addition to the first interview, the household be willing to be visited again six months later, and then again and again, for a total of seven interviews over a three-year period. In addition to saving money, this produces higher-quality estimates of change in victimization rates over years. This design is called a “rotating panel design” because each month, different people are being interviewed for the first time, the second time, the third time, the fourth time, the fifth time, the sixth time, and the last (seventh) time. Thus, the sample is changing each month but overlaps with samples taken six months previously.

Each year, the NCVS collects interviews from about 42,000 households containing more than 76,000 persons. About 92% of the sample households provide one or more interviews; overall, about 87% of the persons eligible within the sample households provide an interview. In 2006, the most recent year with published estimates, 91% of households and 86% of persons in interviewed households were interviewed

The interviews contain questions about the frequency, characteristics, and consequences of criminal victimizations the households may have experienced in the previous six months. The interview covers incidents of household victimization and personal victimization: rape, sexual assault, robbery, assault, theft, household burglary, and motor vehicle theft. An interviewer visits those households and asks those who live there about crimes they have experienced over the past six months. One person in the household acts as the informant for all property crimes (like burglary, vandalism, etc.); each person then reports for him- or herself about personal crimes (e.g., assault, theft of personal items). The interview is conducted in person in the first wave; subsequent waves attempt whenever possible to use telephone interviewers calling from two different centralized call centers. Thus, the NCVS statistics are based on a mix of telephone (60%) and face-to-face interviews (40%).

The questionnaire asks the respondent to remember back over the past six months to report any crimes that might have occurred. For example, the questions in the box on page 13 ask about thefts.

I’m going to read some examples that will give you an idea of the kinds of crimes this study covers.

As I go through them, tell me if any of these happened to you in the last 6 months, that is since _ _(MONTH) _ _(DAY), 20_ _.

Was something belonging to YOU stolen, such as:

a) Things that you carry, like luggage, a wallet, purse, briefcase, or book

b) Clothing, jewelry, or calculator

c) Bicycle or sports equipment

d) Things in your home, like a TV, stereo, or tools

e) Things from a vehicle, such as a package, groceries, camera, or cassette tapes

If the respondent answers “yes, someone stole my bicycle,” then the interviewer records that and later in the interview asks questions about details of the incident. Figure 1.1 shows the kind of statistics that can be computed from the NCVS. The percentages of households reporting one or more crimes of three different types (property crimes, vandalism, and violent crime) are displayed for the years 1994–2000. Note that the percentages are declining, showing a reduced frequency of crime over the late 1990s. Policy makers watch these numbers closely as an indirect way to assess the impact of their crime-fighting programs. However, other research notes that when the nation’s economy is strong, with low unemployment, crime rates tend to decline.

“Crime at Lowest Point in 25 Years, Fed Says” reads the headline on CNN.com on December 27, 1998. “Fewer people in the United States were the victims of crimes last year than at any time since 1973, the Justice Department reported Sunday,” reads the first line. Later in the story, “President Bill Clinton applauded the new crime figures Sunday. They ‘again show that our strategy of more police, stricter gun laws, and better crime prevention is working,’ he said in a statement.” These findings resulted from the NCVS. The statement attributing drops in crime to policies promoted by the current administration is common, but generally without strong empirical support. (It is very difficult, given available information, to link changes in crime rates to policies implemented.)

Figure 1.1 Percentage of U.S. households experiencing a crime by type, 1994–2005 National Crime Victimization Survey.

(Source: www.ojp.usdoj.gov/bjs/.)

NCVS data have informed a wide audience concerned with crime and crime prevention. Researchers at academic, government, private, and nonprofit research institutions use NCVS data to prepare reports, policy recommendations, scholarly publications, testimony before Congress, and documentation for use in courts (U.S. Bureau of Justice Statistics, 1994). Community groups and government agencies use the data to develop neighborhood watch and victim assistance and compensation programs. Law enforcement agencies use NCVS findings for training. The data appear in public service announcements on crime prevention and crime documentaries. Finally, print and broadcast media regularly cite NCVS findings when reporting on a host of crime-related topics.

1.3.2 The National Survey on Drug Use and Health