Measuring Agreement E-Book

Pankaj K. Choudhary

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Veröffentlichungsjahr: 2018

Beschreibung

Presents statistical methodologies for analyzing common types of data from method comparison experiments and illustrates their applications through detailed case studies

Measuring Agreement: Models, Methods, and Applications features statistical evaluation of agreement between two or more methods of measurement of a variable with a primary focus on continuous data. The authors view the analysis of method comparison data as a two-step procedure where an adequate model for the data is found, and then inferential techniques are applied for appropriate functions of parameters of the model. The presentation is accessible to a wide audience and provides the necessary technical details and references. In addition, the authors present chapter-length explorations of data from paired measurements designs, repeated measurements designs, and multiple methods; data with covariates; and heteroscedastic, longitudinal, and categorical data. The book also:

• Strikes a balance between theory and applications

• Presents parametric as well as nonparametric methodologies

• Provides a concise introduction to Cohen’s kappa coefficient and other measures of agreement for binary and categorical data

• Discusses sample size determination for trials on measuring agreement

• Contains real-world case studies and exercises throughout

• Provides a supplemental website containing the related datasets and R code

Measuring Agreement: Models, Methods, and Applications is a resource for statisticians and biostatisticians engaged in data analysis, consultancy, and methodological research. It is a reference for clinical chemists, ecologists, and biomedical and other scientists who deal with development and validation of measurement methods. This book can also serve as a graduate-level text for students in statistics and biostatistics.

Leseprobe

WILEY SERIES IN PROBABILITY AND STATISTICS

Established by Walter A. Shewhart and Samuel S. Wilks

Editors: David J. Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice, Geof H. Givens, Harvey Goldstein, Geert Molenberghs, David W. Scott, Adrian F. M. Smith, Ruey S. Tsay

Editors Emeriti: J. Stuart Hunter, Iain M. Johnstone, Joseph B. Kadane, Jozef L. Teugels

The Wiley Series in Probability and Statistics is well established and authoritative. It covers many topics of current research interest in both pure and applied statistics and probability theory. Written by leading statisticians and institutions, the titles span both state-of-the-art developments in the field and classical methods.

Reflecting the wide range of current research in statistics, the series encompasses applied, methodological and theoretical statistics, ranging from applications and new techniques made possible by advances in computerized practice to rigorous treatment of theoretical approaches. This series provides essential and invaluable reading for all statisticians, whether in academia, industry, government, or research.

A complete list of titles in this series can be found at http://www.wiley.com/go/wsps

MEASURING AGREEMENT

Models, Methods, and Applications

Pankaj K. Choudhary

The University of Texas at Dallas

Haikady N. Nagaraja

The Ohio State University

This edition first published 2017 © 2017 John Wiley & Sons, Inc.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The rights of Pankaj K. Choudhary and Haikady N. Nagaraja to be identified as the authors of this work has been asserted in accordance with law.

Registered OfficeJohn Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

Editorial Office111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of WarrantyWhile the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging-in-Publication Data Names: Choudhary, Pankaj K. (Pankaj Kumar), 1975-author. | Nagaraja, H. N. (Haikady Navada), 1954-author.  Title: Measuring agreement : models, methods, and applications / by Pankaj K. Choudhary, Haikady N. Nagaraja.  Description: Hoboken, NJ : John Wiley & Sons, 2017. | Series: Wiley series in probability and statistics | Includes bibliographical references and index. | Description based on print version record and CIP data provided by publisher; resource not viewed.  Identifiers: LCCN 2017022085 (print) | LCCN 2017037255 (ebook) | ISBN 9781118553145 (pdf) | ISBN 9781118553244 (epub) | ISBN 9781118078587 (cloth) Subjects: LCSH: Statistics--Methodology.  Classification: LCC QA276.A2 (ebook) | LCC QA276.A2 C46 2017 (print) | DDC 001.4/22--dc23 LC record available at https://lccn.loc.gov/2017022085

Cover images: (Background) © BlackJack3D/Gettyimages; (Graph)Courtesy of Pankaj K. Choudhary and Haikady N. Nagaraja Cover design by Wiley

To: My parents, and Swati, Aalo, and Arushi—PK C Jyothi—HNN

To: My parents, and Swati, Aalo, and Arushi—PKC Jyothi—HNN

Contents

PREFACE

CHAPTER 1 INTRODUCTION

1.1 PREVIEW

1.2 NOTATIONAL CONVENTIONS

1.3 BASIC CHARACTERISTICS OF A MEASUREMENT METHOD

1.4 METHOD COMPARISON STUDIES

1.5 MEANING OF AGREEMENT

1.6 A MEASUREMENT ERROR MODEL

1.7 SIMILARITY VERSUS AGREEMENT

1.8 A TOY EXAMPLE

1.9 CONTROVERSIES AND OUR VIEW

1.10 CONCEPTS RELATED TO AGREEMENT

1.11 ROLE OF CONFIDENCE INTERVALS AND HYPOTHESES TESTING

1.12 COMMON MODELS FOR PAIRED MEASUREMENTS DATA

1.13 THE BLAND-ALTMAN PLOT

1.14 COMMON REGRESSION APPROACHES

1.15 INAPPROPRIATE USE OF COMMON TESTS IN METHOD COMPARISON STUDIES

1.16 KEY STEPS IN THE ANALYSIS OF METHOD COMPARISON DATA

1.17 CHAPTER SUMMARY

1.18 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 2 COMMON APPROACHES FOR MEASURING AGREEMENT

2.1 PREVIEW

2.2 INTRODUCTION

2.3 MEAN SQUARED DEVIATION

2.4 CONCORDANCE CORRELATION COEFFICIENT

2.5 A DIGRESSION: TOLERANCE AND PREDICTION INTERVALS

2.6 LIN’S PROBABILITY CRITERION AND BLAND-ALTMAN CRITERION

2.7 LIMITS OF AGREEMENT

2.8 TOTAL DEVIATION INDEX AND COVERAGE PROBABILITY

2.9 INFERENCE ON AGREEMENT MEASURES

2.10 CHAPTER SUMMARY

2.11 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 3 A GENERAL APPROACH FOR MODELING AND INFERENCE

3.1 PREVIEW

3.2 MIXED-EFFECTS MODELS

3.3 A LARGE-SAMPLE APPROACH TO INFERENCE

3.4 MODELING AND ANALYSIS OF METHOD COMPARISON DATA

3.5 CHAPTER SUMMARY

3.6 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 4 PAIRED MEASUREMENTS DATA

4.1 PREVIEW

4.2 MODELING OF DATA

4.3 EVALUATION OF SIMILARITY AND AGREEMENT

4.4 CASE STUDIES

4.5 CHAPTER SUMMARY

4.6 TECHNICAL DETAILS

4.7 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 5 REPEATED MEASUREMENTS DATA

5.1 PREVIEW

5.2 INTRODUCTION

5.3 DISPLAYING DATA

5.4 MODELING OF DATA

5.6 EVALUATION OF REPEATABILITY

5.7 CASE STUDIES

5.8 CHAPTER SUMMARY

5.9 TECHNICAL DETAILS

5.10 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 6 HETEROSCEDASTIC DATA

6.1 PREVIEW

6.2 INTRODUCTION

6.3 VARIANCE FUNCTION MODELS

6.4 REPEATED MEASUREMENTS DATA

6.5 PAIRED MEASUREMENTS DATA

6.6 CHAPTER SUMMARY

6.7 TECHNICAL DETAILS

6.8 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 7 DATA FROM MULTIPLE METHODS

7.1 PREVIEW

7.2 INTRODUCTION

7.3 DISPLAYING DATA

7.4 EXAMPLE DATASETS

7.5 MODELING UNREPLICATED DATA

7.6 MODELING REPEATED MEASUREMENTS DATA

7.7 MODEL FITTING AND EVALUATION

7.8 EVALUATION OF SIMILARITY AND AGREEMENT

7.9 EVALUATION OF REPEATABILITY

7.10 CASE STUDIES

7.11 CHAPTER SUMMARY

7.12 TECHNICAL DETAILS

7.13 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 8 DATA WITH COVARIATES

8.1 PREVIEW

8.2 INTRODUCTION

8.3 MODELING OF DATA

8.4 EVALUATION OF SIMILARITY, AGREEMENT, AND REPEATABILITY

8.5 CASE STUDY

8.6 CHAPTER SUMMARY

8.7 TECHNICAL DETAILS

8.8 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 9 LONGITUDINAL DATA

9.1 PREVIEW

9.2 INTRODUCTION

9.3 MODELING OF DATA

9.4 EVALUATION OF SIMILARITY AND AGREEMENT

9.5 CASE STUDY

9.6 CHAPTER SUMMARY

9.7 TECHNICAL DETAILS

9.8 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 10 A NONPARAMETRIC APPROACH

10.1 PREVIEW

10.2 INTRODUCTION

10.3 THE STATISTICAL FUNCTIONAL APPROACH

10.4 EVALUATION OF SIMILARITY AND AGREEMENT

10.5 CASE STUDIES

10.6 CHAPTER SUMMARY

10.7 TECHNICAL DETAILS

10.8 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 11 SAMPLE SIZE DETERMINATION

11.1 PREVIEW

11.2 INTRODUCTION

11.3 THE SAMPLE SIZE METHODOLOGY

11.4 CASE STUDY

11.5 CHAPTER SUMMARY

11.6 BIBLIOGRAPHIC NOTE

EXERCISES

CHAPTER 12 CATEGORICAL DATA

12.1 PREVIEW

12.2 INTRODUCTION

12.3 EXPERIMENTAL SETUPS AND EXAMPLES

12.4 COHEN’S KAPPA COEFFICIENT FOR DICHOTOMOUS DATA

12.5 KAPPA TYPE MEASURES FOR MORE THAN TWO CATEGORIES

12.6 CASE STUDIES

12.7 MODELS FOR EXPLORING AGREEMENT

12.8 DISCUSSION

12.9 CHAPTER SUMMARY

12.10 BIBLIOGRAPHIC NOTE

EXERCISES

REFERENCES

DATASET LIST

INDEX

EULA

Guide

Cover

Title page

Copyright

Pages

ii

iii

iv

v

xv

xvi

xvii

1

2

3

4

5

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

89

90

91

93

94

95

96

97

98

99

100

103

105

106

107

108

109

110

111

112

113

114

115

116

118

119

121

122

125

126

127

128

130

131

132

133

134

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

Preface

This book presents statistical models and methods for analyzing common types of data collected in method comparison experiments and illustrates their application through detailed case studies. The main aim of these trials is to evaluate agreement between two or more methods of measurement. Although such studies are particularly abundant in health-related fields, they are also conducted in other disciplines, including metrology, ecology, and social and behavioral sciences.

Currently, at least six books cover the topic of agreement evaluation, including von Eye and Mun (2004), Carstensen (2010), Dunn (2004), Shoukri (2010), Broemeling (2009), and Lin et al. (2011). Of these, the first focuses exclusively on categorical data, and the second on continuous data. Others consider both types of data with varying levels of depth and choice of topics. Our book also considers both but with a primary focus on continuous data and one chapter devoted to categorical data. By providing chapter-length treatments of the common types of continuous data, it offers a comprehensive coverage of the topic, and its scope is broader than any other book currently available. It, however, by no means offers a complete survey of the literature. For example, measurement error models, Bayesian methods, and approaches based on generalized estimating equations are not included.

Essentially two principles guided us while writing this book. The first was to view the analysis of method comparison data as a two-step procedure where, in step 1, an adequate model for the data is found, and in step 2, inferential techniques are applied for appropriate functions of the parameters of the model found in step 1. For modeling of data, we primarily rely on mixed-effects models because they capture dependence in a subject’s measurements in an intuitively appealing manner by means of random subject effects; and they also offer a unified framework for dealing with a variety of data types. Besides, they can be fit by the maximum likelihood method using any commonly available statistical software package. For inference, we use the standard large-sample theory and invoke a bootstrap approach whenever the sample size seems too small for the asymptotic methods to be accurate. The second principle was to strive to make the presentation accessible to a wide audience while at the same time making the book theoretically rigorous and self-contained with necessary technical details and references. We have attempted to strike this balance by separating the technical details from the methodological descriptions, forgoing the references in favor of a bibliographic note at the end of each chapter, and by presenting detailed analyses of several real datasets.

The book is organized into twelve chapters. The first eleven are concerned with continuous data while the last covers categorical data. Chapter 1 provides a general introduction to studies comparing two measurement methods and discusses key concepts and statistical issues and tools involved in their analysis. Chapter 2 introduces various measures of agreement for continuous data. Chapter 3 describes mixed-effects models in general and presents the large-sample approach for inference. It provides the technical foundation for the rest of the book and can be skipped by a reader interested in applications. Chapters 4 through 9 consider continuous data collected from various types of experiments, with study designs becoming increasingly more complex. In order, these chapters are devoted to designs with paired measurements, repeated measurements, heteroscedastic measurements, more than two methods, covariates, and longitudinal data. Chapter 10 presents a nonparametric approach for data that do not satisfy assumptions of a mixed-effects model. Chapter 11 considers sample size determination for designing a method comparison study with continuous data. Chapter 12 takes up the question of agreement with categorical data.

Even though the presentation is self-contained, some statistical background is expected from the readers. Familiarity with basic statistical concepts such as maximum likelihood estimation, hypothesis testing, confidence intervals, correlation, and linear regression is necessary. A prior introduction to mixed-effects models and linear algebra will enhance the understanding of the technical details.

The free statistical software R (R Core Team, 2015) has been used to perform all the computations and to generate all the graphics presented in this book. However, the R code is not presented. Much of the code and many of the datasets used here are publicly available at the companion website: http://www.utdallas.edu/~pankaj/agreement_book/

Some familiarity with R programming is assumed for following the code and understanding the output produced. In addition to the base and graphics packages of R, the the following packages and their dependencies have been used in preparing this book: lattice (Sarkar, 2008), latticeExtra (Sarkar and Andrews, 2013), Matrix (Bates and Maechler, 2015), mvtnorm (Genz et al., 2015), multcomp (Hothorn et al., 2008), nlme (Pinheiro et al., 2015), numDeriv (Gilbert and Varadhan, 2015), tikzDevice (Sharpsteen and Bracken, 2015), and xtable (Dahl, 2016).

The book is targeted primarily towards two groups of researchers. The first consists of biomedical and social and behavioral scientists interested in the development and validation of measurement methods. The second includes statisticians engaged in the design and analysis of method comparison studies and in the development of associated statistical methodologies. It can also serve as a textbook for a semester-long special topics course at the graduate level. With that purpose, we have incorporated numerous theoretical and data-centric exercises at the end of the chapters that expand on the material covered in the main body. These exercises provide practice for mastering methodological details and applying the results.

We appreciate the support from our institutions as we marched through this project and for their outstanding library and computing facilities. We thank all those scientists whose dedicated research we were able to highlight in this work. We thank our long-time friends and colleagues for their advice and encouragement, including Professors Babis Papachristou (Rowan University), Michael Baron (American University), Vladimir Dragovic and Vish Ramakrishna (UT Dallas), and Tom Santner and Doug Wolfe (Ohio State University). We thank Professor Phill Cassey (University of Adelaide) for introducing us to applications in ecology and providing datasets, and Professor Chaitra Nagaraja (Fordham University) for producing the plots in Chapter 12. We also thank Professors Huiman Barnhart (Duke University), Douglas Hawkins (University of Minnesota), Vernon Chinchilli (Pennsylvania State University), and Michael Haber (Emory University) for sharing their datasets.

We are grateful to Professors Mohamed Shoukri (King Faisal Specialist Hospital and Research Centre) and Tony Ng (Southern Methodist University) for reading an earlier draft of the manuscript and providing valuable comments. We thank Susanne Steitz-Filler, Allison McGinniss, and Melissa Yanuzzi from John Wiley for guiding the project from start to finish and for their patience and perseverance. We invite the input of our readers on the coverage and presentation here as well as on the companion website as there is always room for improvement.

This book would not have been possible without the support of our family members. They gracefully sacrificed their time with us to allow us to work on a project that seemed to take forever. We take this opportunity to thank them all from the bottom of our hearts.

P. K. Choudhary & H. N. Nagaraja

Richardson, Texas

Columbus, Ohio

July, 2017