Introduction to Meta-Analysis E-Book

Michael Borenstein

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 787

Veröffentlichungsjahr: 2021

Beschreibung

A clear and thorough introduction to meta-analysis, the process of synthesizing data from a series of separate studies

The first edition of this text was widely acclaimed for the clarity of the presentation, and quickly established itself as the definitive text in this field. The fully updated second edition includes new and expanded content on avoiding common mistakes in meta-analysis, understanding heterogeneity in effects, publication bias, and more. Several brand-new chapters provide a systematic "how to" approach to performing and reporting a meta-analysis from start to finish.

Written by four of the world's foremost authorities on all aspects of meta-analysis, the new edition:

• Outlines the role of meta-analysis in the research process
• Shows how to compute effects sizes and treatment effects
• Explains the fixed-effect and random-effects models for synthesizing data
• Demonstrates how to assess and interpret variation in effect size across studies
• Explains how to avoid common mistakes in meta-analysis
• Discusses controversies in meta-analysis
• Includes access to a companion website containing videos, spreadsheets, data files, free software for prediction intervals, and step-by-step instructions for performing analyses using Comprehensive Meta-Analysis (CMA)

Download videos, class materials, and worked examples at www.Introduction-to-Meta-Analysis.com

"This book offers the reader a unified framework for thinking about meta-analysis, and then discusses all elements of the analysis within that framework. The authors address a series of common mistakes and explain how to avoid them. As the editor-in-chief of the American Psychologist and former editor of Psychological Bulletin, I can say without hesitation that the quality of manuscript submissions reporting meta-analyses would be vastly better if researchers read this book."
—Harris Cooper, Hugo L. Blomquist Distinguished Professor Emeritus of Psychology and Neuroscience, Editor-in-chief of the American Psychologist, former editor of Psychological Bulletin

"A superb combination of lucid prose and informative graphics, the authors provide a refreshing departure from cookbook approaches with their clear explanations of the what and why of meta-analysis. The book is ideal as a course textbook or for self-study. My students raved about the clarity of the explanations and examples."
—David Rindskopf, Distinguished Professor of Educational Psychology, City University of New York, Graduate School and University Center, & Editor of the Journal of Educational and Behavioral Statistics

"The approach taken by Introduction to Meta-analysis is intended to be primarily conceptual, and it is amazingly successful at achieving that goal. The reader can comfortably skip the formulas and still understand their application and underlying motivation. For the more statistically sophisticated reader, the relevant formulas and worked examples provide a superb practical guide to performing a meta-analysis. The book provides an eclectic mix of examples from education, social science, biomedical studies, and even ecology. For anyone considering leading a course in meta-analysis, or pursuing self-directed study, Introduction to Meta-analysis would be a clear first choice."
—Jesse A. Berlin, SCD

Leseprobe

Table of Contents

Cover

Title Page

Copyright

List of Tables

List of Figures

Acknowledgements

Preface

AN ETHICAL IMPERATIVE

FROM NARRATIVE REVIEWS TO SYSTEMATIC REVIEWS

THE SYSTEMATIC REVIEW AND META‐ANALYSIS

META‐ANALYSIS IS USED IN MANY FIELDS OF RESEARCH

META‐ANALYSIS AS PART OF THE RESEARCH PROCESS

THE INTENDED AUDIENCE FOR THIS BOOK

AN OUTLINE OF THIS BOOK’S CONTENTS (UPDATED FOR THE SECOND EDITION)

WHAT THIS BOOK DOES NOT COVER

Further Reading

Preface to the Second Edition

PRACTICAL INFORMATION

LIMITATIONS OF A META‐ANALYSIS

RECENT DEVELOPMENTS

HOW TO EXPLAIN THE RESULTS

NEW WEBSITE AND VIDEOS

Website

PART 1: Introduction

CHAPTER 1: How a Meta‐Analysis Works

INTRODUCTION

INDIVIDUAL STUDIES

THE SUMMARY EFFECT

HETEROGENEITY OF EFFECT SIZES

CHAPTER 2: Why Perform a Meta‐Analysis

INTRODUCTION

THE STREPTOKINASE META‐ANALYSIS

STATISTICAL SIGNIFICANCE

CLINICAL IMPORTANCE OF THE EFFECT

CONSISTENCY OF EFFECTS

PART 2: Effect Size and Precision

CHAPTER 3: Overview

TREATMENT EFFECTS AND EFFECT SIZES

PARAMETERS AND ESTIMATES

OUTLINE OF EFFECT SIZE COMPUTATIONS

CHAPTER 4: Effect Sizes Based on Means

INTRODUCTION

RAW (UNSTANDARDIZED) MEAN DIFFERENCE

D

STANDARDIZED MEAN DIFFERENCE,

d

AND

g

RESPONSE RATIOS

CHAPTER 5: Effect Sizes Based on Binary Data (2 × 2 Tables)

INTRODUCTION

RISK RATIO

ODDS RATIO

RISK DIFFERENCE

CHOOSING AN EFFECT SIZE INDEX

CHAPTER 6: Effect Sizes Based on Correlations

INTRODUCTION

COMPUTING

r

OTHER APPROACHES

CHAPTER 7: Converting Among Effect Sizes

INTRODUCTION

CONVERTING FROM THE LOG ODDS RATIO TO

d

CONVERTING FROM

d

TO THE LOG ODDS RATIO

CONVERTING FROM

r

TO

d

CONVERTING FROM

d

TO

r

CHAPTER 8: Factors that Affect Precision

INTRODUCTION

FACTORS THAT AFFECT PRECISION

SAMPLE SIZE

STUDY DESIGN

CHAPTER 9: Concluding Remarks

Further Reading

Note

PART 3: Fixed-Effect Versus Random-Effects Models

CHAPTER 10: Overview

INTRODUCTION

NOMENCLATURE

CHAPTER 11: Fixed‐Effect Model

INTRODUCTION

THE TRUE EFFECT SIZE

IMPACT OF SAMPLING ERROR

PERFORMING A FIXED‐EFFECT META‐ANALYSIS

CHAPTER 12: Random‐Effects Model

INTRODUCTION

THE TRUE EFFECT SIZES

IMPACT OF SAMPLING ERROR

PERFORMING A RANDOM‐EFFECTS META‐ANALYSIS

CHAPTER 13: Fixed‐Effect Versus Random‐Effects Models

INTRODUCTION

DEFINITION OF A SUMMARY EFFECT

ESTIMATING THE SUMMARY EFFECT

EXTREME EFFECT SIZE IN A LARGE STUDY OR A SMALL STUDY

CONFIDENCE INTERVAL

THE NULL HYPOTHESIS

WHICH MODEL SHOULD WE USE?

MODEL SHOULD NOT BE BASED ON THE TEST FOR HETEROGENEITY

CONCLUDING REMARKS

CHAPTER 14: Worked Examples (Part 1)

INTRODUCTION

WORKED EXAMPLE FOR CONTINUOUS DATA (PART 1)

WORKED EXAMPLE FOR BINARY DATA (PART 1)

WORKED EXAMPLE FOR CORRELATIONAL DATA (PART 1)

PART 4: Heterogeneity

CHAPTER 15: Overview

INTRODUCTION

NOMENCLATURE

WORKED EXAMPLES

CHAPTER 16: Identifying and Quantifying Heterogeneity

INTRODUCTION

ISOLATING THE VARIATION IN TRUE EFFECTS

COMPUTING

Q

ESTIMATING τ

2

THE I

2

STATISTIC

COMPARING THE MEASURES OF HETEROGENEITY

CONFIDENCE INTERVALS FOR τ

2

CONFIDENCE INTERVALS (OR UNCERTAINTY INTERVALS) FOR I

2

CHAPTER 17: Prediction Intervals

INTRODUCTION

PREDICTION INTERVALS IN PRIMARY STUDIES

PREDICTION INTERVALS IN META‐ANALYSIS

CONFIDENCE INTERVALS AND PREDICTION INTERVALS

COMPARING THE CONFIDENCE INTERVAL WITH THE PREDICTION INTERVAL

Further Reading

CHAPTER 18: Worked Examples (Part 2)

INTRODUCTION

WORKED EXAMPLE FOR CONTINUOUS DATA (PART 2)

WORKED EXAMPLE FOR BINARY DATA (PART 2)

WORKED EXAMPLE FOR CORRELATIONAL DATA (PART 2)

CHAPTER 19: An Intuitive Look at Heterogeneity

INTRODUCTION

MOTIVATING EXAMPLE

THE

Q

‐VALUE AND THE

p

‐VALUE DO NOT TELL US HOW MUCH THE EFFECT SIZE VARIES

THE CONFIDENCE INTERVAL DOES NOT TELL US HOW MUCH THE EFFECT SIZE VARIES

THE 2 STATISTIC DOES NOT TELL US HOW MUCH THE EFFECT SIZE VARIES

WHAT

I

2

TELLS US

THE

I

2

INDEX VS. THE PREDICTION INTERVAL

THE PREDICTION INTERVAL

PREDICTION INTERVAL IS CLEAR, CONCISE, AND RELEVANT

COMPUTING THE PREDICTION INTERVAL

HOW TO USE

I

2

HOW TO EXPLAIN HETEROGENEITY

HOW MUCH DOES THE EFFECT SIZE VARY ACROSS STUDIES?

CAVEATS

CONCLUSION

FURTHER READING

THE MEANING OF

I

2

IN FIGURE 19.2

CHAPTER 20: Classifying Heterogeneity as Low, Moderate, or High

INTRODUCTION

INTEREST SHOULD GENERALLY FOCUS ON AN INDEX OF ABSOLUTE HETEROGENEITY

THE CLASSIFICATIONS LEAD THEMSELVES TO MISTAKES OF INTERPRETATION

CLASSIFICATIONS FOCUS ATTENTION IN THE WRONG DIRECTION

PART 5: Explaining Heterogeneity

CHAPTER 21: Subgroup Analyses

INTRODUCTION

FIXED‐EFFECT MODEL WITHIN SUBGROUPS

COMPUTATIONAL MODELS

RANDOM EFFECTS WITH SEPARATE ESTIMATES OF τ

2

RANDOM EFFECTS WITH POOLED ESTIMATE OF τ

2

THE PROPORTION OF VARIANCE EXPLAINED

MIXED‐EFFECTS MODEL

OBTAINING AN OVERALL EFFECT IN THE PRESENCE OF SUBGROUPS

CHAPTER 22: Meta‐Regression

INTRODUCTION

FIXED‐EFFECT MODEL

FIXED OR RANDOM EFFECTS FOR UNEXPLAINED HETEROGENEITY

RANDOM‐EFFECTS MODEL

CHAPTER 23: Notes on Subgroup Analyses and Meta‐Regression

INTRODUCTION

COMPUTATIONAL MODEL

MULTIPLE COMPARISONS

SOFTWARE

ANALYSES OF SUBGROUPS AND REGRESSION ANALYSES ARE OBSERVATIONAL

STATISTICAL POWER FOR SUBGROUP ANALYSES AND META‐REGRESSION

Further Reading

PART 6: Putting it all in Context

CHAPTER 24: Looking at the Whole Picture

INTRODUCTION

METHYLPHENIDATE FOR ADULTS WITH ADHD

IMPACT OF GLP‐1 MIMETICS ON BLOOD PRESSURE

AUGMENTING CLOZAPINE WITH A SECOND ANTIPSYCHOTIC

CONCLUSIONS

CAVEATS

CHAPTER 25: Limitations of the Random‐Effects Model

INTRODUCTION

ASSUMPTIONS OF THE RANDOM‐EFFECTS MODEL

A TEXTBOOK CASE

WHEN STUDIES ARE PULLED FROM THE LITERATURE

A USEFUL FICTION

TRANSPARENCY

A NARROWLY DEFINED UNIVERSE

TWO IMPORTANT CAVEATS

IN CONTEXT

EXTREME CASES

CHAPTER 26: Knapp–Hartung Adjustment

INTRODUCTION

ADJUSTMENT IS RARELY EMPLOYED IN SIMPLE ANALYSES

ADJUSTING THE STANDARD ERROR

THE KNAPP–HARTUNG ADJUSTMENT FOR OTHER EFFECT SIZE INDICES

t

DISTRIBUTION VS. Z DISTRIBUTION

LIMITATIONS OF THE KNAPP–HARTUNG ADJUSTMENT

PART 7: Complex Data Structures

CHAPTER 27: Overview

CHAPTER 28: Independent Subgroups within a Study

INTRODUCTION

COMBINING ACROSS SUBGROUPS

COMPARING SUBGROUPS

CHAPTER 29: Multiple Outcomes or Time‐Points within a Study

INTRODUCTION

COMBINING ACROSS OUTCOMES OR TIME‐POINTS

COMPARING OUTCOMES OR TIME‐POINTS WITHIN A STUDY

Further Reading

CHAPTER 30: Multiple Comparisons within a Study

INTRODUCTION

COMBINING ACROSS MULTIPLE COMPARISONS WITHIN A STUDY

DIFFERENCES BETWEEN TREATMENTS

Further Reading

CHAPTER 31: Notes on Complex Data Structures

INTRODUCTION

SUMMARY EFFECT

DIFFERENCES IN EFFECT

PART 8: Other Issues

CHAPTER 32: Overview

CHAPTER 33: Vote Counting – A New Name for an Old Problem

INTRODUCTION

WHY VOTE COUNTING IS WRONG

VOTE COUNTING IS A PERVASIVE PROBLEM

CHAPTER 34: Power Analysis for Meta‐Analysis

INTRODUCTION

A CONCEPTUAL APPROACH

IN CONTEXT

WHEN TO USE POWER ANALYSIS

PLANNING FOR PRECISION RATHER THAN FOR POWER

POWER ANALYSIS IN PRIMARY STUDIES

POWER ANALYSIS FOR META‐ANALYSIS

POWER ANALYSIS FOR A TEST OF HOMOGENEITY

Further Reading

CHAPTER 35: Publication Bias

INTRODUCTION

THE PROBLEM OF MISSING STUDIES

METHODS FOR ADDRESSING BIAS

ILLUSTRATIVE EXAMPLE

THE MODEL

GETTING A SENSE OF THE DATA

IS THERE EVIDENCE OF ANY BIAS?

HOW MUCH OF AN IMPACT MIGHT THE BIAS HAVE?

SUMMARY OF THE FINDINGS FOR THE ILLUSTRATIVE EXAMPLE

CONFLATING BIAS WITH THE SMALL‐STUDY EFFECT

USING LOGIC TO DISENTANGLE BIAS FROM SMALL‐STUDY EFFECTS

THESE METHODS DO NOT GIVE US THE ‘CORRECT’ EFFECT SIZE

SOME IMPORTANT CAVEATS

PROCEDURES DO NOT APPLY TO STUDIES OF PREVALENCE

THE MODEL FOR PUBLICATION BIAS IS SIMPLISTIC

CONCLUDING REMARKS

PUTTING IT ALL TOGETHER

Further Reading

PART 9: Issues Related to Effect Size

CHAPTER 36: Overview

CHAPTER 37: Effect Sizes Rather than

p

‐Values

INTRODUCTION

RELATIONSHIP BETWEEN ‐VALUES AND EFFECT SIZES

THE DISTINCTION IS IMPORTANT

THE ‐VALUE IS OFTEN MISINTERPRETED

NARRATIVE REVIEWS VS. META‐ANALYSES

CHAPTER 38: Simpson's Paradox

INTRODUCTION

CIRCUMCISION AND RISK OF HIV INFECTION

AN EXAMPLE OF THE PARADOX

Further Reading

CHAPTER 39: Generality of the Basic Inverse‐Variance Method

INTRODUCTION

OTHER EFFECT SIZES

OTHER METHODS FOR ESTIMATING EFFECT SIZES

INDIVIDUAL PARTICIPANT DATA META‐ANALYSES

BAYESIAN APPROACHES

Further Reading

PART 10: Further Methods

CHAPTER 40: Overview

CHAPTER 41: Meta‐Analysis Methods Based on Direction and

p

‐Values

INTRODUCTION

VOTE COUNTING

THE SIGN TEST

COMBINING ‐VALUES

CHAPTER 42: Further Methods for Dichotomous Data

INTRODUCTION

MANTEL–HAENSZEL METHOD

ONE‐STEP (PETO) FORMULA FOR ODDS RATIO

CHAPTER 43: Psychometric Meta‐Analysis

INTRODUCTION

THE ATTENUATING EFFECTS OF ARTIFACTS

META‐ANALYSIS METHODS

EXAMPLE OF PSYCHOMETRIC META‐ANALYSIS

COMPARISON OF ARTIFACT CORRECTION WITH META‐REGRESSION

SOURCES OF INFORMATION ABOUT ARTIFACT VALUES

HOW HETEROGENEITY IS ASSESSED

REPORTING IN PSYCHOMETRIC META‐ANALYSIS

CONCLUDING REMARKS

Further Reading

PART 11: Meta‐Analysis in Context

CHAPTER 44: Overview

CHAPTER 45: When Does it Make Sense to Perform a Meta‐Analysis?

INTRODUCTION

ARE THE STUDIES SIMILAR ENOUGH TO COMBINE?

CAN I COMBINE STUDIES WITH DIFFERENT DESIGNS?

HOW MANY STUDIES ARE ENOUGH TO CARRY OUT A META‐ANALYSIS?

Further Reading

CHAPTER 46: Reporting the Results of a Meta‐Analysis

INTRODUCTION

THE COMPUTATIONAL MODEL

FOREST PLOTS

SENSITIVITY ANALYSIS

Further Reading

CHAPTER 47: Cumulative Meta‐Analysis

INTRODUCTION

WHY PERFORM A CUMULATIVE META‐ANALYSIS?

CHAPTER 48: Criticisms of Meta‐Analysis

INTRODUCTION

ONE NUMBER CANNOT SUMMARIZE A RESEARCH FIELD

THE FILE DRAWER PROBLEM INVALIDATES META‐ANALYSIS

MIXING APPLES AND ORANGES

GARBAGE IN, GARBAGE OUT

IMPORTANT STUDIES ARE IGNORED

META‐ANALYSIS CAN DISAGREE WITH RANDOMIZED TRIALS

META‐ANALYSES ARE PERFORMED POORLY

IS A NARRATIVE REVIEW BETTER?

CONCLUDING REMARKS

Further Reading

CHAPTER 49: Comprehensive Meta‐Analysis Software

INTRODUCTION

FEATURES IN CMA

TEACHING ELEMENTS

DOCUMENTATION

AVAILABILITY

ACKNOWLEDGMENTS

MOTIVATING EXAMPLE

DATA ENTRY

BASIC ANALYSIS

WHAT IS THE AVERAGE EFFECT SIZE?

HOW MUCH DOES THE EFFECT SIZE VARY?

PLOT SHOWING DISTRIBUTION OF EFFECTS

HIGH‐RESOLUTION PLOT

SUBGROUP ANALYSIS

META‐REGRESSION

PUBLICATION BIAS

EXPLAINING RESULTS

CHAPTER 50: How to Explain the Results of an Analysis

INTRODUCTION

THE OVERVIEW

THE MEAN EFFECT SIZE

VARIATION IN EFFECT SIZE

NOTATIONS

IMPACT OF RESISTANCE EXERCISE ON PAIN

CORRELATION BETWEEN LETTER KNOWLEDGE AND WORD RECOGNITION

STATINS FOR PREVENTION OF CARDIOVASCULAR EVENTS

BUPROPION FOR SMOKING CESSATION

MORTALITY FOLLOWING MITRAL‐VALVE PROCEDURES IN ELDERLY PATIENTS

PART 12: Resources

CHAPTER 51: Software for Meta‐Analysis

COMPREHENSIVE META‐ANALYSIS

METAFOR

STATA

REVMAN

CHAPTER 52: Web Sites, Societies, Journals, and Books

WEB SITES

PROFESSIONAL SOCIETIES

JOURNALS

SPECIAL ISSUES DEDICATED TO META‐ANALYSIS

BOOKS ON SYSTEMATIC REVIEW METHODS AND META‐ANALYSIS

References

Index

End User License Agreement

List of Tables

Chapter 3

Table 3.1 Roadmap of formulas in subsequent chapters.

Chapter 5

Table 5.1 Nomenclature for 2 × 2 table of outcome by treatment.

Table 5.2 Fictional data for a 2 × 2 table.

Chapter 8

Table 8.1 Impact of sample size on variance.

Table 8.2 Impact of study design on variance.

Chapter 14

Table 14.1 Dataset 1 – Part A (basic data).

Table 14.2 Dataset 1 – Part B (fixed‐effect computations).

Table 14.3 Dataset 1 – Part C (random‐effects computations).

Table 14.4 Dataset 2 – Part A (basic data).

Table 14.5 Dataset 2 – Part B (fixed‐effect computations).

Table 14.6 Dataset 2 – Part C (random‐effects computations).

Table 14.7 Dataset 3 – Part A (basic data).

Table 14.8 Dataset 3 – Part B (fixed‐effect computations).

Table 14.9 Dataset 3 – Part C (random‐effects computations).

Chapter 16

Table 16.1 Factors affecting measures of dispersion.

Chapter 18

Table 18.1 Dataset 1 – Part D (intermediate computations).

Table 18.2 Dataset 1 – Part E (variance computations).

Table 18.3 Dataset 2 – Part D (intermediate computations).

Table 18.4 Dataset 2 – Part E (variance computations).

Table 18.5 Dataset 3 – Part D (intermediate computations).

Table 18.6 Dataset 3 – Part E (variance computations).

Chapter 19

Table 19.1 Relationship between observed effects and true effects in Figure 1...

Chapter 21

Table 21.1 Fixed effect model – computations.

Table 21.2 Fixed‐effect model – summary statistics.

Table 21.3 Fixed‐effect model – ANOVA table.

Table 21.4 Fixed‐effect model – subgroups as studies.

Table 21.5 Random‐effects model (separate estimates of

τ

2

) – computations...

Table 21.6 Random‐effects model (separate estimates of

τ

2

) – summary stat...

Table 21.7 Random‐effects model (separate estimates of

t

2

) – ANOVA table.

Table 21.8 Random‐effects model (separate estimates of

τ

2

) – subgroups a...

Table 21.9 Statistics for computing a pooled estimate of

τ

2

.

Table 21.10 Random‐effects model (pooled estimate of

τ

2

) – computations....

Table 21.11 Random‐effects model (pooled estimate of

τ

2

) – summary statis...

Table 21.12 Random‐effects model (pooled estimate of

τ

2

) – ANOVA table.

Table 21.13 Random‐effects model (pooled estimate of

τ

2

) – subgroups as s...

Chapter 22

Table 22.1 The BCG dataset.

Table 22.2 Fixed‐effect model – regression results for BCG.

Table 22.3 Fixed‐effect model – ANOVA table for BCG regression.

Table 22.4 Random‐effects model – regression results for BCG.

Table 22.5 Random‐effects model – test of the model.

Table 22.6 Random‐effects model – comparison of model (latitude) versus the n...

Chapter 26

Table 26.1 Knapp–Hartung computations for ADHD analysis.

Table 26.2 Original vs. Knapp–Hartung.

Table 26.3 Impact of using

t

distribution on the confidence interval width.

Chapter 28

Table 28.1 Independent subgroups – five fictional studies.

Table 28.2 Independent subgroups – summary effect.

Table 28.3 Independent subgroups – synthetic effect for study 1.

Table 28.4 Independent subgroups – summary effect across studies.

Chapter 29

Table 29.1 Multiple outcomes – five fictional studies.

Table 29.2 Creating a synthetic variable as the mean of two outcomes.

Table 29.3 Multiple outcomes – summary effect.

Table 29.4 Multiple outcomes – impact of correlation on variance of summary e...

Table 29.5 Creating a synthetic variable as the difference between two outcom...

Table 29.6 Multiple outcomes – difference between outcomes.

Table 29.7 Multiple outcomes – Impact of correlation on the variance of diffe...

Chapter 38

Table 38.1 HIV as function of circumcision (by subgroup).

Table 38.2 HIV as function of circumcision – by study.

Table 38.3 HIV as a function of circumcision – full population.

Table 38.4 HIV as a function of circumcision – by risk group.

Table 38.5 HIV as a function of circumcision/risk group – full population.

Chapter 39

Table 39.1 Simple example of a genetic association study.

Chapter 41

Table 41.1 Streptokinase data – calculations for meta‐analyses of

p

‐values.

Chapter 42

Table 42.1 Nomenclature for 2 × 2 table of events by treatment.

Table 42.2 Mantel–Haenszel – odds ratio.

Table 42.3 Mantel–Haenszel – variance of summary effect.

Table 42.4 One‐step – odds ratio and variance.

Chapter 43

Table 43.1 Fictional data for psychometric meta‐analysis.

Table 43.2 Observed (attenuated) correlations.

Table 43.3 Unattenuated correlations.

List of Illustrations

Chapter 1

Figure 1.1 High dose versus standard dose of statins (adapted from Cannon

et

...

Chapter 2

Figure 2.1 Impact of streptokinase on mortality (adapted from Lau

et al.,

19...

Chapter 4

Figure 4.1 Response ratios are analyzed in log units.

Chapter 5

Figure 5.1 Risk ratios are analyzed in log units.

Figure 5.2 Odds ratios are analyzed in log units.

Chapter 6

Figure 6.1 Correlations are analyzed in Fisher's

z

units.

Chapter 7

Figure 7.1 Converting among effect sizes.

Chapter 8

Figure 8.1 Impact of sample size on variance.

Figure 8.2 Impact of study design on variance.

Chapter 10

Figure 10.1 Symbols for true and observed effects.

Chapter 11

Figure 11.1 Fixed‐effect model – true effects.

Figure 11.2 Fixed‐effect model – true effects and sampling error.

Figure 11.3 Fixed‐effect model – distribution of sampling error.

Chapter 12

Figure 12.1 Random‐effects model – distribution of the true effects.

Figure 12.2 Random‐effects model – true effects.

Figure 12.3 Random‐effects model – true and observed effect in one study.

Figure 12.4 Random‐effects model – between‐study and within‐study variance....

Chapter 13

Figure 13.1 Fixed‐effect model – forest plot showing relative weights.

Figure 13.2 Random‐effects model – forest plot showing relative weights.

Figure 13.3 Very large studies under fixed‐effect model.

Figure 13.4 Very large studies under random‐effects model.

Chapter 14

Figure 14.1 Forest plot of Dataset 1 – fixed‐effect weights.

Figure 14.2 Forest plot of Dataset 1 – random‐effects weights.

Figure 14.3 Forest plot of Dataset 2 – fixed‐effect weights.

Figure 14.4 Forest plot of Dataset 2 – random‐effects weights.

Figure 14.5 Forest plot of Dataset 3 – fixed‐effect weights.

Figure 14.6 Forest plot of Dataset 3 – random‐effects weights.

Chapter 16

Figure 16.1 Dispersion across studies relative to error within studies.

Figure 16.2

Q

in relation to

df

as measure of dispersion.

Figure 16.3 Flowchart showing how

T

2

and

I

2

are derived from

Q

and

df

.

Figure 16.4 Impact of

Q

and number of studies on the

p

‐value.

Figure 16.5 Impact of excess dispersion and absolute dispersion on

T

2

.

Figure 16.6 Impact of excess and absolute dispersion on

T

.

Figure 16.7 Impact of excess dispersion on

I

2

.

Figure 16.8 Factors affecting

T

2

but not

I

2

.

Figure 16.9 Factors affecting

I

2

but not

T

2

.

Chapter 17

Figure 17.1 Prediction interval based on population parameters μ and

τ

2

Figure 17.2 Prediction interval based on sample estimates

M

*

and

T

2

.

Figure 17.3 Simultaneous display of confidence interval and prediction inter...

Figure 17.4 Impact of number of studies on confidence interval and predictio...

Chapter 18

Figure 18.1 Forest plot of Dataset 1 – random‐effects weights with predictio...

Figure 18.2 Forest plot of Dataset 2 – random‐effects weights with predictio...

Figure 18.3 Forest plot of Dataset 3 – random‐effects weights with predictio...

Chapter 19

Figure 19.1 Alcohol use and mortality. Risk ratio < 1 favors drinkers. Three...

Figure 19.2 Alcohol use and mortality. Risk ratio < 1 favors drinkers. Three...

Figure 19.3 Alcohol use and mortality (Forest plot). Risk ratio < 1 favors d...

Figure 19.4 Alcohol use and mortality (true effects). Risk ratio < 1 favors ...

Chapter 20

Figure 20.1 True effects for two meta‐analyses.

Figure 20.2 True effects (inner) and observed effects (outer) for two meta‐a...

Chapter 21

Figure 21.1 Fixed‐effect model – studies and subgroup effects.

Figure 21.2 Fixed‐effect – subgroup effects.

Figure 21.3 Fixed‐effect model – treating subgroups as studies.

Figure 21.4 Flowchart for selecting a computational model.

Figure 21.5 Random‐effects model (separate estimates of

τ

2

) – studies a...

Figure 21.6 Random‐effects model (separate estimates of

τ

2

) – subgroup ...

Figure 21.7 Random‐effects model (separate estimates of

τ

2

) – treating ...

Figure 21.8 Random‐effects model (pooled estimate of

τ

2

) – studies and ...

Figure 21.9 Random‐effects model (pooled estimate of

τ

2

) – subgroup eff...

Figure 21.10 Random‐effects model (pooled estimate of

τ

2

) – treating su...

Figure 21.11 A primary study showing subjects within groups.

Figure 21.12 Random‐effects model – variance within and between subgroups.

Figure 21.13 Proportion of variance explained by subgroup membership.

Chapter 22

Figure 22.1 Fixed‐effect model – forest plot for the BCG data.

Figure 22.2 Fixed‐effect model – regression of log risk ratio on latitude.

Figure 22.3 Fixed‐effect model – population effects as function of covariate...

Figure 22.4 Random‐effects model – population effects as a function of covar...

Figure 22.5 Random‐effects model – forest plot for the BCG data.

Figure 22.6 Random‐effects model – regression of log risk ratio on latitude....

Figure 22.7 Between‐studies variance (

T

2

) with no covariate.

Figure 22.8 Between‐studies variance (

T

2

) with covariate.

Figure 22.9 Proportion of variance explained by latitude.

Chapter 24

Figure 24.1 Three fictional examples where the mean effect is 0.00.

Figure 24.2 Three fictional examples where the mean effect is 0.40.

Figure 24.3 Three fictional examples where the mean effect is 0.80.

Figure 24.4 Methylphenidate for adults with ADHD (Forest plot). Effect size ...

Figure 24.5 Methylphenidate for adults with ADHD (True effects). Effect size...

Figure 24.6 GLP‐1 mimetics and diastolic BP (Forest plot). Mean difference <...

Figure 24.7 GLP‐1 mimetics and diastolic BP (True effects). Mean difference ...

Figure 24.8 Augmenting clozapine (Forest plot). Std mean difference < 0 favo...

Figure 24.9 Augmenting clozapine (True effects). Std mean difference < 0 fav...

Chapter 25

Figure 25.1 Random effects. Confidence interval 60 points wide.

Figure 25.2 Methylphenidate for adults with ADHD. Effect size > 0 favors tre...

Chapter 28

Figure 28.1 Creating a synthetic variable from independent subgroups.

Chapter 33

Figure 33.1 The

p

‐value for each study is > 0.20 but the

p

‐value for the sum...

Chapter 34

Figure 34.1 Power for a primary study as a function of

n

and

δ

.

Figure 34.2 Power for a meta‐analysis as a function of number studies and

δ

...

Figure 34.3 Power for a meta‐analysis as a function of number of studies and...

Chapter 35

Figure 35.1 Passive smoking and lung cancer – forest plot.

Figure 35.2 Passive smoking and lung cancer – funnel plot.

Figure 35.3 Observed studies only.

Figure 35.4 Observed studies and studies imputed by Trim and Fill.

Figure 35.5 Passive smoking and lung cancer – cumulative forest plot.

Chapter 37

Figure 37.1 Estimating the effect size versus testing the null hypothesis.

Figure 37.2 The

p

‐value is a poor surrogate for effect size.

Figure 37.3 Studies where

p

‐values differ but effect sizes is the same.

Figure 37.4 Studies where

p

‐values are the same but effect sizes differ.

Figure 37.5 Studies where the more significant

p

‐value corresponds to weaker...

Chapter 38

Figure 38.1 Circumcision and HIV. Odds Ratio >1 indicates circumcision is as...

Figure 38.2 HIV as function of circumcision – in three sets of studies.

Chapter 41

Figure 41.1 Effect size in four fictional studies.

Chapter 46

Figure 46.1 Forest plot using lines to represent the effect size.

Figure 46.2 Forest plot using boxes to represent the effect size and relativ...

Chapter 47

Figure 47.1 Impact of streptokinase on mortality – forest plot.

Figure 47.2 Impact of streptokinase on mortality – cumulative forest plot.

Chapter 48

Figure 48.1 Forest plot of five fictional studies and a new trial (consisten...

Figure 48.2 Forest plot of five fictional studies and a new trial (heterogen...

Chapter 49

Figure 49.1 Data‐entry screen in CMA.

Figure 49.2 Basic analysis screen in CMA.

Figure 49.3 Average effect size (top), variation in effect size (bottom).

Figure 49.4 Plotting distribution of true effects. ADHD.

Figure 49.5 High‐resolution plot in CMA.

Figure 49.6 Impact of treatment as a function of subgroup: Forest plot.

Figure 49.7 Impact of treatment as a function of subgroup: Statistics.

Figure 49.8 Results for regression, random effects.

Figure 49.9 Regression of effect size on Dose, with SUD held constant.

Figure 49.10 Funnel plot of observed effects.

Figure 49.11 Funnel plot of observed and imputed effects.

Figure 49.12 Regression of effect size (

d

) on Dose and SUD. Plot created in ...

Chapter 50

Figure 50.1 Impact of resistance exercise on pain. Data-entry screen.

Figure 50.2 Impact of resistance exercise on pain.

g

> 0 indicates exercise ...

Figure 50.3 Impact of resistance exercise on pain. Heterogeneity statistics....

Figure 50.4 Impact of resistance exercise on pain. Distribution of true effe...

Figure 50.5 Predicting reading scores. Data-entry screen.

Figure 50.6 Predicting reading scores.

Figure 50.7 Predicting reading scores. Heterogeneity statistics.

Figure 50.8 Predicting reading scores. Distribution of true correlations.

Figure 50.9 Statins for prevention of cardiovascular events. Data-entry scre...

Figure 50.10 Statins for prevention of cardiovascular events. Odds ratio < 1...

Figure 50.11 Statins for prevention of cardiovascular events. Heterogeneity ...

Figure 50.12 Statins for prevention of cardiovascular events. Distribution o...

Figure 50.13 Bupropion for smoking cessation. Data-entry screen.

Figure 50.14 Bupropion for smoking cessation. Risk ratio > 1 shows reduction...

Figure 50.15 Bupropion for smoking cessation. Heterogeneity statistics.

Figure 50.16 Bupropion for smoking cessation. Distribution of true effects....

Figure 50.17 Mortality following mitral‐valve surgery in elderly patients. D...

Figure 50.18 Mortality following mitral‐valve surgery in elderly patients.

Figure 50.19 Mortality following mitral‐valve surgery in elderly patients. H...

Figure 50.20 Mortality following mitral‐valve surgery in elderly patients. D...

Guide

Cover

Table of Contents

Begin Reading

Pages

iii

iv

xv

xvi

xvii

xix

xx

xxi

xxii

xxiii

xxv

xxvii

xxviii

xxix

xxx

xxxi

xxxii

xxxiii

xxxiv

xxxv

xxxvi

xxxvii

1

3

4

5

6

7

9

10

11

12

13

14

15

17

18

19

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

43

44

45

46

47

49

50

51

52

53

55

57

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

119

120

121

122

123

124

125

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

155

156

157

158

159

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

221

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

243

244

245

246

247

248

249

251

253

254

255

256

257

258

259

260

261

263

264

265

266

267

268

269

270

271

272

273

274

275

277

278

279

280

281

282

283

285

287

289

290

291

292

293

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

333

335

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

359

361

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

389

391

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

443

444

445

446

447

448

449

450

451

454

455

456

459

460

461

464

465

466

469

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

491

492

493

494

495

496

497

498

499

500

501

Introduction to Meta‐Analysis

Second Edition

Michael Borenstein

Biostat, Inc, New Jersey, USA.

Larry V. Hedges

Northwestern University, Evanston, USA.

Julian P.T. Higgins

University of Bristol, Bristol, UK

Hannah R. Rothstein

Baruch College, New York, USA.

This edition first published 2021

© 2021 John Wiley & Sons Ltd

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The right of Michael Borenstein, Larry V. Hedges, Julian P.T. Higgins and Hannah R. Rothstein to be identified as the author of this work has been asserted in accordance with law.

Registered Offices

John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

Editorial Office

9600 Garsington Road, Oxford, OX4 2DQ, UK

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print‐on‐demand. Some content that appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

In view of ongoing research, equipment modifications, changes in governmental regulations, and the constant flow of information relating to the use of experimental reagents, equipment, and devices, the reader is urged to review and evaluate the information provided in the package insert or instructions for each chemical, piece of equipment, reagent, or device for, among other things, any changes in the instructions or indication of usage and for added warnings and precautions. While the publisher and author have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and author endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging‐in‐Publication data applied for

HB ISBN: 9781119558354

Cover Design: Wiley

Cover Image: Courtesy of Michael Borenstein

Set in 10.5/13pt TimesLTStd by SPi Global, Chennai, India

List of Tables

Table 3.1 Roadmap of formulas in subsequent chapters

Table 5.1 Nomenclature for 2 × 2 table of outcome by treatment

Table 5.2 Fictional data for a 2 × 2 table

Table 8.1 Impact of sample size on variance

Table 8.2 Impact of study design on variance

Table 14.1 Dataset 1 - Part A (basic data)

Table 14.2 Dataset 1 - Part B (fixed-effect computations)

Table 14.3 Dataset 1 - Part C (random-effects computations)

Table 14.4 Dataset 2 - Part A (basic data)

Table 14.5 Dataset 2 - Part B (fixed-effect computations)

Table 14.6 Dataset 2 - Part C (random-effects computations)

Table 14.7 Dataset 3 - Part A (basic data)

Table 14.8 Dataset 3 - Part B (fixed-effect computations)

Table 14.9 Dataset 3 - Part C (random-effects computations)

Table 16.1 Factors affecting measures of dispersion

Table 18.1 Dataset 1 - Part D (intermediate computations)

Table 18.2 Dataset 1 - Part E (variance computations)

Table 18.3 Dataset 2 - Part D (intermediate computations)

Table 18.4 Dataset 2 - Part E (variance computations)

Table 18.5 Dataset 3 - Part D (intermediate computations)

Table 18.6 Dataset 3 - Part E (variance computations)

Table 19.1 Relationship between observed effects and true effects in Figure 19.2, Panel A

Table 21.1 Fixed effect model - computations

Table 21.2 Fixed-effect model - summary statistics

Table 21.3 Fixed-effect model - ANOVA table

Table 21.4 Fixed-effect model - subgroups as studies

Table 21.5 Random-effects model (separate estimates of τ

2

) - computations

Table 21.6 Random-effects model (separate estimates of τ

2

) - summary statistics

Table 21.7 Random-effects model (separate estimates of t2) - ANOVA table

Table 21.8 Random-effects model (separate estimates of τ

2

) - subgroups as studies

Table 21.9 Statistics for computing a pooled estimate of τ

2

Table 21.10 Random-effects model (pooled estimate of τ

2

) - computations

Table 21.11 Random-effects model (pooled estimate of τ

2

) - summary statistics

Table 21.12 Random-effects model (pooled estimate of τ

2

) - ANOVA table

Table 21.13 Random-effects model (pooled estimate of τ

2

) - subgroups as studies

Table 22.1 The BCG dataset

Table 22.2 Fixed-effect model - Regression results for BCG

Table 22.3 Fixed-effect model - ANOVA table for BCG regression

Table 22.4 Random-effects model - regression results for BCG

Table 22.5 Random-effects model - test of the model

Table 22.6 Random-effects model - comparison of model (latitude) versus the null model

Table 26.1 Knapp.Hartung computations for ADHD analysis

Table 26.2 Original vs. Knapp.Hartung

Table 26.3 Impact of using t distribution on the confidence interval width

Table 28.1 Independent subgroups - five fictional studies

Table 28.2 Independent subgroups - summary effect

Table 28.3 Independent subgroups - synthetic effect for study 1

Table 28.4 Independent subgroups - summary effect across studies

Table 29.1 Multiple outcomes - five fictional studies

Table 29.2 Creating a synthetic variable as the mean of two outcomes

Table 29.3 Multiple outcomes - summary effect

Table 29.4 Multiple outcomes - impact of correlation on variance of summary effect

Table 29.5 Creating a synthetic variable as the difference between two outcomes

Table 29.6 Multiple outcomes - difference between outcomes

Table 29.7 Multiple outcomes - Impact of correlation on the variance of difference

Table 38.1 HIV as function of circumcision (by subgroup)

Table 38.2 HIV as function of circumcision - by study

Table 38.3 HIV as a function of circumcision - full population

Table 38.4 HIV as a function of circumcision - by risk group

Table 38.5 HIV as a function of circumcision/risk group - full population

Table 39.1 Simple example of a genetic association study

Table 41.1 Streptokinase data - calculations for meta-analyses of

p

-values

Table 42.1 Nomenclature for 2 × 2 table of events by treatment

Table 42.2 Mantel.Haenszel - odds ratio

Table 42.3 Mantel-Haenszel - variance of summary effect

Table 42.4 One-step - odds ratio and variance

Table 43.1 Fictional data for psychometric meta-analysis

Table 43.2 Observed (attenuated) correlations

Table 43.3 Unattenuated correlations

List of Figures

Figure 1.1 High-dose versus standard-dose of statins (adapted from Cannon

et al

., 2006)

Figure 2.1 Impact of streptokinase on mortality (adapted from Lau

et al

., 1992)

Figure 4.1 Response ratios are analyzed in log units

Figure 5.1 Risk ratios are analyzed in log units

Figure 5.2 Odds ratios are analyzed in log units

Figure 6.1 Correlations are analyzed in Fisher’s z units

Figure 7.1 Converting among effect sizes

Figure 8.1 Impact of sample size on variance

Figure 8.2 Impact of study design on variance

Figure 10.1 Symbols for true and observed effects

Figure 11.1 Fixed-effect model - true effects

Figure 11.2 Fixed-effect model - true effects and sampling error

Figure 11.3 Fixed-effect model - distribution of sampling error

Figure 12.1 Random-effects model - distribution of the true effects

Figure 12.2 Random-effects model - true effects

Figure 12.3 Random-effects model - true and observed effect in one study

Figure 12.4 Random-effects model - between-study and within-study variance

Figure 13.1 Fixed-effect model - forest plot showing relative weights

Figure 13.2 Random-effects model - forest plot showing relative weights

Figure 13.3 Very large studies under fixed-effect model

Figure 13.4 Very large studies under random-effects model

Figure 14.1 Forest plot of Dataset 1 - fixed-effect weights

Figure 14.2 Forest plot of Dataset 1 - random-effects weights

Figure 14.3 Forest plot of Dataset 2 - fixed-effect weights

Figure 14.4 Forest plot of Dataset 2 - random-effects weights

Figure 14.5 Forest plot of Dataset 3 - fixed-effect weights

Figure 14.6 Forest plot of Dataset 3 - random-effects weights

Figure 16.1 Dispersion across studies relative to error within studies

Figure 16.2

Q

in relation to

df

as measure of dispersion

Figure 16.3 Flowchart showing how

T

2

and

I

2

are derived from

Q

and

df

Figure 16.4 Impact of

Q

and number of studies on the p-value

Figure 16.5 Impact of excess dispersion and absolute dispersion on

T

2

Figure 16.6 Impact of excess and absolute dispersion on

T

Figure 16.7 Impact of excess dispersion on

I

2

Figure 16.8 Factors affecting

T

2 but not

I

2

Figure 16.9 Factors affecting

I

2 but not

T

2

Figure 17.1 Prediction interval based on population parameters

μ

and

τ

2

Figure 17.2 Prediction interval based on sample estimates

M

*

and

T

2

Figure 17.3 Simultaneous display of confidence interval and prediction interval

Figure 17.4 Impact of number of studies on confidence interval and prediction interval

Figure 18.1 Forest plot of Dataset 1 - random-effects weights with prediction interval

Figure 18.2 Forest plot of Dataset 2 - random-effects weights with prediction interval

Figure 18.3 Forest plot of Dataset 3 - random-effects weights with prediction interval

Figure 19.1 Alcohol use and mortality. Risk ratio ≺ 1 favors drinkers. Three possible distributions of true effects

Figure 19.2 Alcohol use and mortality. Risk ratio ≺ 1 favors drinkers. Three possible distributions of true effects (inner) and observed effects (outer)

Figure 19.3 Alcohol use and mortality (Forest plot). Risk ratio ≺ 1 favors drinkers.

Figure 19.4 Alcohol use and mortality (true effects). Risk ratio ≺ 1 favors drinkers.

Figure 20.1 True effects for two meta-analyses

Figure 20.2 True effects (inner) and observed effects (outer) for two meta-analyses

Figure 21.1 Fixed-effect model - studies and subgroup effects

Figure 21.2 Fixed-effect - subgroup effects

Figure 21.3 Fixed-effect model - treating subgroups as studies

Figure 21.4 Flowchart for selecting a computational model

Figure 21.5 Random-effects model (separate estimates of

τ

2) - studies and subgroup effects

Figure 21.6 Random-effects model (separate estimates of

τ

2

) - subgroup effects

Figure 21.7 Random-effects model (separate estimates of

τ

2

) - treating subgroups as studies

Figure 21.8 Random-effects model (pooled estimate of

τ

2

) - studies and subgroup effects

Figure 21.9 Random-effects model (pooled estimate of

τ

2

) - subgroup effects

Figure 21.10 Random-effects model (pooled estimate of

τ

2

) - treating subgroups as studies

Figure 21.11 A primary study showing subjects within groups

Figure 21.12 Random-effects model - variance within and between subgroups

Figure 21.13 Proportion of variance explained by subgroup membership

Figure 22.1 Fixed-effect model - forest plot for the BCG data

Figure 22.2 Fixed-effect model - regression of log risk ratio on latitude

Figure 22.3 Fixed-effect model - population effects as function of covariate

Figure 22.4 Random-effects model - population effects as a function of covariate

Figure 22.5 Random-effects model - forest plot for the BCG data

Figure 22.6 Random-effects model - regression of log risk ratio on latitude

Figure 22.7 Between-studies variance (

T

2) with no covariate

Figure 22.8 Between-studies variance (

T

2) with covariate

Figure 22.9 Proportion of variance explained by latitude

Figure 24.1 Three fictional examples where the mean effect is 0.00

Figure 24.2 Three fictional examples where the mean effect is 0.40

Figure 24.3 Three fictional examples where the mean effect is 0.80

Figure 24.4 Methylphenidate for adults with ADHD (Forest plot). Effect size > 0 favors treatment

Figure 24.5 Methylphenidate for adults with ADHD (True effects). Effect size > 0 favors treatment

Figure 24.6 GLP-1 mimetics and diastolic BP (Forest plot). Mean difference ≺ 0 favors treatment

Figure 24.7 GLP-1 mimetics and diastolic BP (True effects). Mean difference ≺ 0 favors treatment

Figure 24.8 Augmenting clozapine (Forest plot). Std mean difference ≺ 0 favors augmentation

Figure 24.9 Augmenting clozapine (True effects). Std mean difference ≺ 0 favors augmentation

Figure 25.1 Random effects. Confidence interval 60 points wide

Figure 25.2 Methylphenidate for adults with ADHD. Effect size > 0 favors treatment

Figure 28.1 Creating a synthetic variable from independent subgroups

Figure 33.1 The p-value for each study is > 0.20 but the p-value for the summary effect is ≺ 0.02

Figure 34.1 Power for a primary study as a function of n and

τ

Figure 34.2 Power for a meta-analysis as a function of number studies and

τ

Figure 34.3 Power for a meta-analysis as a function of number studies and heterogeneity

Figure 35.1 Passive smoking and lung cancer - forest plot

Figure 35.2 Passive smoking and lung cancer - funnel plot

Figure 35.3 Observed studies only

Figure 35.4 Observed studies and studies imputed by Trim and Fill

Figure 35.5 Passive smoking and lung cancer - cumulative forest plot

Figure 37.1 Estimating the effect size versus testing the null hypothesis

Figure 37.2 The

p

-value is a poor surrogate for effect size

Figure 37.3 Studies where

p

-values differ but effect sizes is the same

Figure 37.4 Studies where

p

-values are the same but effect sizes differ

Figure 37.5 Studies where the more significant p-value corresponds to weaker effect size

Figure 38.1 Circumcision and HIV. Odds Ratio > 1 indicates circumcision is associated with lower risk of HIV.

Figure 38.2 HIV as function of circumcision - in three sets of studies

Figure 41.1 Effect size in four fictional studies

Figure 46.1 Forest plot using lines to represent the effect size

Figure 46.2 Forest plot using boxes to represent the effect size and relative weight

Figure 47.1 Impact of streptokinase on mortality - forest plot

Figure 47.2 Impact of streptokinase on mortality - cumulative forest plot

Figure 48.1 Forest plot of five fictional studies and a new trail (consistent effects)

Figure 48.2 Forest plot of five fictional studies and a new trial (heterogeneous effects)

Figure 49.1 Data-entry screen in CMA.

Figure 49.2 Basic analysis screen in CMA

Figure 49.3 Average effect size (top), Variation in effect size (bottom)

Figure 49.4 Plotting distribution of true effects. ADHD

Figure 49.5 High-resolution plot in CMA

Figure 49.6 Impact of treatment as a function of subgroup: Forest plot

Figure 49.7 Impact of treatment as a function of subgroup: Statistics

Figure 49.8 Results for regression, random effects

Figure 49.9 Regression of effect size on Dose, with SUD held constant

Figure 49.10 Funnel plot of observed effects

Figure 49.11 Funnel plot of observed and imputed effects

Figure 49.12 Regression of effect size (

d

) on Dose and SUD. Plot created in Excel (TM)

Figure 50.1 Impact of resistance exercise on pain. Data-entry screen

Figure 50.2 Impact of resistance exercise on pain.

g

> 0 indicates exercise reduced pain

Figure 50.3 Impact of resistance exercise on pain. Heterogeneity statistics

Figure 50.4 Impact of resistance exercise on pain. Distribution of true effects

Figure 50.5 Predicting reading scores. Data-entry screen

Figure 50.6 Predicting reading scores

Figure 50.7 Predicting reading scores. Heterogeneity statistics

Figure 50.8 Predicting reading scores. Distribution of true correlations

Figure 50.9 Statins for prevention of cardiovascular events. Data-entry screen

Figure 50.10 Statins for prevention of cardiovascular events. Odds ratio ≺ 1 shows reduction in events

Figure 50.11 Statins for prevention of cardiovascular events. Heterogeneity statistics

Figure 50.12 Statins for prevention of cardiovascular events. Distribution of true effects

Figure 50.13 Bupropion for smoking cessation. Data-entry screen

Figure 50.14 Bupropion for smoking cessation. Risk ratio > 1 shows reduction in smoking

Figure 50.15 Bupropion for smoking cessation. Heterogeneity statistics

Figure 50.16 Bupropion for smoking cessation. Distribution of true effects

Figure 50.17 Mortality following mitral-valve surgery in elderly patients. Data-entry screen

Figure 50.18 Mortality following mitral-valve surgery in elderly patients

Figure 50.19 Mortality following mitral-valve surgery in elderly patients. Heterogeneity statistics

Figure 50.20 Mortality following mitral-valve surgery in elderly patients. Distribution of true risks

Acknowledgements

This book was funded by the following grants from the National Institutes of Health: Combining data types in meta‐analysis (AG021360), Publication bias in meta‐analysis (AG20052), Software for meta‐regression (AG024771), from the National Institute on Aging, under the direction of Dr. Sidney Stahl; and Forest plots for meta‐analysis (DA019280), from the National Institute on Drug Abuse, under the direction of Dr. Thomas Hilton.

These grants allowed us to convene a series of workshops on meta‐analysis, and parts of this volume reflect ideas developed as part of these workshops. We would like to acknowledge and thank Doug Altman, Betsy Becker, Jesse Berlin, Michael Brannick, Harris Cooper, Kay Dickersin, Sue Duval, Roger Harbord, Despina Contopoulos‐Ioannidis, John Ioannidis, Spyros Konstantopoulos, Mark Lipsey, Mike McDaniel, Ingram Olkin, Fred Oswald, Terri Pigott, Simcha Pollack, David Rindskopf, Stephen Senn, Will Shadish, Jonathan Sterne, Alex Sutton, Thomas Trikalinos, Jeff Valentine, Jack Vevea, Vish Viswesvaran, and David Wilson.

Steven Tarlow helped to edit this book and to ensure the accuracy of all formulas and examples. We would like to acknowledge and thank our editors at Wiley, including Kathryn Sharples, Ashley Alliano, Alison Oliver, Sarah Keegan, Kimberly Monroe‐Hill and Viktoria Hartl‐Vida. We would especially like to thank Adalfin Jayasingh who served as the production editor for this volume. His attention to detail and his patience in working through revisions are very much appreciated.

Preface

In his best‐selling book Baby and Child Care, Dr. Benjamin Spock wrote ‘I think it is preferable to accustom a baby to sleeping on his stomach from the beginning if he is willing’. This statement was included in most editions of the book, and in most of the 50 million copies sold from the 1950s into the 1990s. The advice was not unusual, in that many pediatricians made similar recommendations at the time.

During this same period, from the 1950s into the 1990s, more than 100,000 babies died of sudden infant death syndrome (SIDS), also called crib death in the United States and cot death in the United Kingdom, where a seemingly healthy baby goes to sleep and never wakes up.

In the early 1990s, researchers became aware that the risk of SIDS decreased by at least 50% when babies were put to sleep on their backs rather than face down. Governments in various countries launched educational initiatives such as the Back to sleep campaigns in the United Kingdom and the United States, which led to an immediate and dramatic drop in the number of SIDS deaths.

While the loss of more than 100,000 children would be unspeakably sad in any event, the real tragedy lies in the fact that many of these deaths could have been prevented. Gilbert et al. (2005) write

Advice to put infants to sleep on the front for nearly half a century was contrary to evidence available from 1970 that this was likely to be harmful. Systematic review of preventable risk factors for SIDS from 1970 would have led to earlier recognition of the risks of sleeping on the front and might have prevented over 10,000 infant deaths in the UK and at least 50,000 in Europe, the USA and Australasia.

AN ETHICAL IMPERATIVE

This example is one of several cited by Sir Iain Chalmers in a talk entitled The scandalous failure of scientists to cumulate scientifically (Chalmers, 2006). The theme of this talk was that we live in a world where the utility of almost any intervention will be tested repeatedly, and that rather than looking at any study in isolation, we need to look at the body of evidence. While not all systematic reviews carry the urgency of SIDS, the logic of looking at the body of evidence, rather than trying to understand studies in isolation, is always compelling.

Meta‐analysis refers to the statistical synthesis of results from a series of studies. While the statistical procedures used in a meta‐analysis can be applied to any set of data, the synthesis will be meaningful only if the studies have been collected systematically. This could be in the context of a systematic review, the process of systematically locating, appraising, and then synthesizing data from a large number of sources. Or, it could be in the context of synthesizing data from a select group of studies, such as those conducted by a pharmaceutical company to assess the efficacy of a new drug.

If a treatment effect (or effect size) is consistent across the series of studies, these procedures enable us to report that the effect is robust across the kinds of populations sampled, and also to estimate the magnitude of the effect more precisely than we could with any of the studies alone. If the treatment effect varies across the series of studies, these procedures enable us to report on the range of effects, and may enable us to identify factors associated with the magnitude of the effect size.

FROM NARRATIVE REVIEWS TO SYSTEMATIC REVIEWS

Prior to the 1990s, the task of combining data from multiple studies had been primarily the purview of the narrative review. An expert in a given field would read the studies that addressed a question, summarize the findings, and then arrive at a conclusion – for example, that the treatment in question was, or was not, effective. However, this approach suffers from some important limitations.

One limitation is the subjectivity inherent in this approach, coupled with the lack of transparency. For example, different reviewers might use different criteria for deciding which studies to include in the review. Once a set of studies has been selected, one reviewer might give more credence to larger studies, while another gives more credence to ‘quality’ studies and yet another assigns a comparable weight to all studies. One reviewer may require a substantial body of evidence before concluding that a treatment is effective, while another uses a lower threshold. In fact, there are examples in the literature where two narrative reviews come to opposite conclusions, with one reporting that a treatment is effective while the other reports that it is not. As a rule, the narrative reviewer will not articulate (and may not even be fully aware of) the decision‐making process used to synthesize the data and arrive at a conclusion.

A second limitation of narrative reviews is that they become less useful as more information becomes available. The thought process required for a synthesis requires the reviewer to capture the finding reported in each study, to assign an appropriate weight to that finding, and then to synthesize these findings across all studies in the synthesis. While a reviewer may be able to synthesize data from a few studies in their head, the process becomes difficult and eventually untenable as the number of studies increases. This is true even when the treatment effect (or effect size) is consistent from study to study. Often, however, the treatment effect will vary as a function of study level covariates, such as the patient population, the dose of medication, the outcome variable, and other factors. In these cases, a proper synthesis requires that the researcher be able to understand how the treatment effect varies as a function of these variables, and the narrative review is poorly equipped to address these kinds of issues.

THE SYSTEMATIC REVIEW AND META‐ANALYSIS

For these reasons, beginning in the mid‐1980s and taking root in the 1990s, researchers in many fields have been moving away from the narrative review, and adopting systematic reviews and meta‐analysis.

For systematic reviews, a clear set of rules is used to search for studies, and then to determine which studies will be included in or excluded from the analysis. Since there is an element of subjectivity in setting these criteria, as well as in the conclusions drawn from the meta‐analysis, we cannot say that the systematic review is entirely objective. However, because all of the decisions are specified clearly, the mechanisms are transparent.

A key element in most systematic reviews is the statistical synthesis of the data, or the meta‐analysis. Unlike the narrative review, where reviewers implicitly assign some level of importance to each study, in meta‐analysis the weights assigned to each study are based on mathematical criteria that are specified in advance. While the reviewers and readers may still differ on the substantive meaning of the results (as they might for a primary study), the statistical analysis provides a transparent, objective, and replicable framework for this discussion.

The formulas used in meta‐analysis are extensions of formulas used in primary studies, and are used to address similar kinds of questions to those addressed in primary studies. In primary studies we would typically report a mean and standard deviation for the subjects. If appropriate, we might also use analysis of variance or multiple regression to determine if (and how) subject scores were related to various factors. Similarly, in a meta‐analysis, we might report a mean and standard deviation for the treatment effect. And, if appropriate, we would also use procedures analogous to analysis of variance or multiple regression to assess the relationship between the effect and study‐level covariates.

Meta‐analyses are conducted for a variety of reasons, not only to synthesize evidence on the effects of interventions or to support evidence‐based policy or practice. The purpose of the meta‐analysis, or more generally, the purpose of any research synthesis, has implications for when it should be performed, what model should be used to analyze the data, what sensitivity analyses should be undertaken, and how the results should be interpreted. Losing sight of the fact that meta‐analysis is a tool with multiple applications causes confusion and leads to pointless discussions about what is the right way to perform a research synthesis, when there is no single right way. It all depends on the purpose of the synthesis, and the data that are available. Much of this book will expand on this idea.

META‐ANALYSIS IS USED IN MANY FIELDS OF RESEARCH

In medicine, systematic reviews and meta‐analysis form the core of a movement to ensure that medical treatments are based on the best available empirical data. For example, The Cochrane Collaboration has published the results of over 3700 meta‐analyses (as of January 2009) which synthesize data on treatments in all areas of health care