Introduction to Mixed Modelling E-Book

N. W. Galwey

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 799

Veröffentlichungsjahr: 2014

Beschreibung

Mixed modelling is very useful, and easier than you think!

Mixed modelling is now well established as a powerful approach to statistical data analysis. It is based on the recognition of random-effect terms in statistical models, leading to inferences and estimates that have much wider applicability and are more realistic than those otherwise obtained.

Introduction to Mixed Modelling leads the reader into mixed modelling as a natural extension of two more familiar methods, regression analysis and analysis of variance. It provides practical guidance combined with a clear explanation of the underlying concepts.

Like the first edition, this new edition shows diverse applications of mixed models, provides guidance on the identification of random-effect terms, and explains how to obtain and interpret best linear unbiased predictors (BLUPs).   It also introduces several important new topics, including the following:

• Use of the software SAS, in addition to GenStat and R.
• Meta-analysis and the multiple testing problem.
• The Bayesian interpretation of mixed models.

Including numerous practical exercises with solutions, this book provides an ideal introduction to mixed modelling for final year undergraduate students, postgraduate students and professional researchers. It will appeal to readers from a wide range of scientific disciplines including statistics, biology, bioinformatics, medicine, agriculture, engineering, economics, archaeology and geography.

Praise for the first edition:

“One of the main strengths of the text is the bridge it provides between traditional analysis of variance and regression models and the more recently developed class of mixed models...Each chapter is well-motivated by at least one carefully chosen example...demonstrating the broad applicability of mixed models in many different disciplines...most readers will likely learn something new, and those previously unfamiliar with mixed models will obtain a solid foundation on this topic.”—Kerrie Nelson University of South Carolina, in American Statistician, 2007

Leseprobe

Table of Contents

Cover

Title Page

Copyright

Preface

References

Chapter 1: The need for more than one random-effect term when fitting a regression line

1.1 A data set with several observations of variable

Y

at each value of variable

X

1.2 Simple regression analysis: Use of the software GenStat to perform the analysis

1.3 Regression analysis on the group means

1.4 A regression model with a term for the groups

1.5 Construction of the appropriate

F

test for the significance of the explanatory variable when groups are present

1.6 The decision to specify a model term as random: A mixed model

1.7 Comparison of the tests in a mixed model with a test of lack of fit

1.8 The use of REsidual Maximum Likelihood (REML) to fit the mixed model

1.9 Equivalence of the different analyses when the number of observations per group is constant

1.10 Testing the assumptions of the analyses: Inspection of the residual values

1.11 Use of the software R to perform the analyses

1.12 Use of the software SAS to perform the analyses

1.13 Fitting a mixed model using GenStat's Graphical User Interface (GUI)

1.14 Summary

1.15 Exercises

References

Chapter 2: The need for more than one random-effect term in a designed experiment

2.1 The split plot design: A design with more than one random-effect term

2.2 The analysis of variance of the split plot design: A random-effect term for the main plots

2.3 Consequences of failure to recognize the main plots when analysing the split plot design

2.4 The use of mixed modelling to analyse the split plot design

2.5 A more conservative alternative to the

F

and Wald statistics

2.6 Justification for regarding block effects as random

2.7 Testing the assumptions of the analyses: Inspection of the residual values

2.8 Use of R to perform the analyses

2.9 Use of SAS to perform the analyses

2.10 Summary

2.11 Exercises

References

Chapter 3: Estimation of the variances of random-effect terms

3.1 The need to estimate variance components

3.2 A hierarchical random-effects model for a three-stage assay process

3.3 The relationship between variance components and stratum mean squares

3.4 Estimation of the variance components in the hierarchical random-effects model

3.5 Design of an optimum strategy for future sampling

3.6 Use of R to analyse the hierarchical three-stage assay process

3.7 Use of SAS to analyse the hierarchical three-stage assay process

3.8 Genetic variation: A crop field trial with an unbalanced design

3.9 Production of a balanced experimental design by ‘padding’ with missing values

3.10 Specification of a treatment term as a random-effect term: The use of mixed-model analysis to analyse an unbalanced data set

3.11 Comparison of a variance component estimate with its standard error

3.12 An alternative significance test for variance components

3.13 Comparison among significance tests for variance components

3.14 Inspection of the residual values

3.15 Heritability: The prediction of genetic advance under selection

3.16 Use of R to analyse the unbalanced field trial

3.17 Use of SAS to analyse the unbalanced field trial

3.18 Estimation of variance components in the regression analysis on grouped data

3.19 Estimation of variance components for block effects in the split-plot experimental design

3.20 Summary

3.21 Exercises

References

Chapter 4: Interval estimates for fixed-effect terms in mixed models

4.1 The concept of an interval estimate

4.2 Standard errors for regression coefficients in a mixed-model analysis

4.3 Standard errors for differences between treatment means in the split-plot design

4.4 A significance test for the difference between treatment means

4.5 The least significant difference (LSD) between treatment means

4.6 Standard errors for treatment means in designed experiments: A difference in approach between analysis of variance and mixed-model analysis

4.7 Use of R to obtain SEs of means in a designed experiment

4.8 Use of SAS to obtain SEs of means in a designed experiment

4.9 Summary

4.10 Exercises

References

Chapter 5: Estimation of random effects in mixed models: Best Linear Unbiased Predictors (BLUPs)

5.1 The difference between the estimates of fixed and random effects

5.2 The method for estimation of random effects: The best linear unbiased predictor (BLUP) or ‘shrunk estimate’

5.3 The relationship between the shrinkage of BLUPs and regression towards the mean

5.4 Use of R for the estimation of fixed and random effects

5.5 Use of SAS for the estimation of random effects

5.6 The Bayesian interpretation of BLUPs: Justification of a random-effect term without invoking an underlying infinite population

5.7 Summary

5.8 Exercises

References

Chapter 6: More advanced mixed models for more elaborate data sets

6.1 Features of the models introduced so far: A review

6.2 Further combinations of model features

6.3 The choice of model terms to be specified as random

6.4 Disagreement concerning the appropriate significance test when fixed- and random-effect terms interact: ‘The great mixed-model muddle’

6.5 Arguments for specifying block effects as random

6.6 Examples of the choice of fixed- and random-effect specification of terms

6.7 Summary

6.8 Exercises

References

Chapter 7: Three case studies

7.1 Further development of mixed modelling concepts through the analysis of specific data sets

7.2 A fixed-effects model with several variates and factors

7.3 Use of R to fit the fixed-effects model with several variates and factors

7.4 Use of SAS to fit the fixed-effects model with several variates and factors

7.5 A random coefficient regression model

7.6 Use of R to fit the random coefficients model

7.7 Use of SAS to fit the random coefficients model

7.8 A random-effects model with several factors

7.9 Use of R to fit the random-effects model with several factors

7.10 Use of SAS to fit the random-effects model with several factors

7.11 Summary

7.12 Exercises

References

Chapter 8: Meta-analysis and the multiple testing problem

8.1 Meta-analysis: Combined analysis of a set of studies

8.2 Fixed-effect meta-analysis with estimation only of the main effect of treatment

8.3 Random-effects meta-analysis with estimation of study × treatment interaction effects

8.4 A random-effect interaction between two fixed-effect terms

8.5 Meta-analysis of individual-subject data using R

8.6 Meta-analysis of individual-subject data using SAS

8.7 Meta-analysis when only summary data are available

8.8 The multiple testing problem: Shrinkage of BLUPs as a defence against the Winner's Curse

8.9 Fitting of multiple models using R

8.10 Fitting of multiple models using SAS

8.11 Summary

8.12 Exercises

References

Chapter 9: The use of mixed models for the analysis of unbalanced experimental designs

9.1 A balanced incomplete block design

9.2 Imbalance due to a missing block: Mixed-model analysis of the incomplete block design

9.3 Use of R to analyse the incomplete block design

9.4 Use of SAS to analyse the incomplete block design

9.5 Relaxation of the requirement for balance: Alpha designs

9.6 Approximate balance in two directions: The alphalpha design

9.7 Use of R to analyse the alphalpha design

9.8 Use of SAS to analyse the alphalpha design

9.9 Summary

9.10 Exercises

References

Chapter 10: Beyond mixed modelling

10.1 Review of the uses of mixed models

10.2 The generalized linear mixed model (GLMM): Fitting a logistic (sigmoidal) curve to proportions of observations

10.3 Use of R to fit the logistic curve

10.4 Use of SAS to fit the logistic curve

10.5 Fitting a GLMM to a contingency table: Trouble-shooting when the mixed modelling process fails

10.6 The hierarchical generalized linear model (HGLM)

10.7 Use of R to fit a GLMM and a HGLM to a contingency table

10.8 Use of SAS to fit a GLMM to a contingency table

10.9 The role of the covariance matrix in the specification of a mixed model

10.10 A more general pattern in the covariance matrix: Analysis of pedigrees and genetic data

10.11 Estimation of parameters in the covariance matrix: Analysis of temporal and spatial variation

10.12 Use of R to model spatial variation

10.13 Use of SAS to model spatial variation

10.14 Summary

10.15 Exercises

References

Chapter 11: Why is the criterion for fitting mixed models called REsidual Maximum Likelihood?

11.1 Maximum likelihood and residual maximum likelihood

11.2 Estimation of the variance

σ

2

from a single observation using the maximum-likelihood criterion

11.3 Estimation of

σ

2

from more than one observation

11.4 The

μ

-effect axis as a dimension within the sample space

11.5 Simultaneous estimation of

μ

and

σ

2

using the maximum-likelihood criterion

11.6 An alternative estimate of

σ

2

using the REML criterion

11.7 Bayesian justification of the REML criterion

11.8 Extension to the general linear model: The fixed-effect axes as a sub-space of the sample space

11.9 Application of the REML criterion to the general linear model

11.10 Extension to models with more than one random-effect term

11.11 Summary

11.12 Exercises

References

Index

End User License Agreement

Pages

xi

xii

xiii

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

168

167

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

Guide

Cover

Table of Contents

Preface

Begin Reading

List of Illustrations

Figure 1.1

Figure 1.2

Figure 1.3

Figure 1.4

Figure 1.5

Figure 1.6

Figure 1.7

Figure 1.8

Figure 1.9

Figure 1.10

Figure 1.11

Figure 1.12

Figure 1.13

Figure 1.14

Figure 1.15

Figure 1.16

Figure 1.17

Figure 1.18

Figure 2.1

Figure 2.2

Figure 2.3

Figure 3.1

Figure 3.2

Figure 3.3

Figure 3.4

Figure 3.5

Figure 3.6

Figure 4.1

Figure 4.2

Figure 4.3

Figure 4.4

Figure 4.5

Figure 5.1

Figure 5.2

Figure 5.3

Figure 5.4.

Figure 5.5.

Figure 6.1

Figure 6.2

Figure 6.3

Figure 7.1

Figure 7.2

Figure 7.3

Figure 7.4

Figure 7.5

Figure 7.6

Figure 7.7

Figure 8.1

Figure 8.2

Figure 8.3

Figure 8.4

Figure 8.5

Figure 8.6

Figure 9.1

Figure 10.1

Figure 10.2

Figure 10.3

Figure 10.4

Figure 10.5

Figure 10.6

Figure 10.7

Figure 10.8

Figure 10.9

Figure 10.10

Figure 10.11

Figure 10.12

Figure 11.1

Figure 11.2

Figure 11.3

Figure 11.4

Figure 11.5

Figure 11.6

Figure 11.7

Figure 11.8

Figure 11.9

Figure 11.10

Figure 11.11

Figure 11.12

Figure 11.13

List of Tables

Table 1.1

Table 1.2

Table 1.3

Table 1.4

Table 1.5

Table 1.6

Table 1.7

Table 1.8

Table 1.9

Table 2.1

Table 2.2

Table 2.3

Table 2.4

Table 2.5

Table 3.1

Table 3.2

Table 3.3

Table 3.4

Table 3.5

Table 3.6

Table 3.7

Table 3.8

Table 3.9

Table 3.10

Table 3.11

Table 3.12

Table 3.13

Table 4.1

Table 4.2

Table 4.3

Table 4.4

Table 4.5

Table 4.6

Table 4.7

Table 5.1

Table 5.2

Table 6.1

Table 6.2

Table 6.3

Table 6.4

Table 6.5

Table 6.6

Table 6.7

Table 6.8

Table 6.9.

Table 7.1

Table 7.2

Table 7.3

Table 7.4

Table 7.5

Table 7.6

Table 7.7

Table 7.8

Table 7.9

Table 7.10

Table 7.11

Table 7.12

Table 7.13

Table 7.14

Table 7.15

Table 8.1

Table 8.2

Table 8.3

Table 8.4

Table 8.5

Table 8.6

Table 8.7

Table 8.8

Table 9.1

Table 9.2

Table 9.3

Table 9.4

Table 9.5

Table 10.1

Table 10.2

Table 10.3

Table 10.4

Table 10.5

Table 10.6

Table 10.7

Table 10.8

Table 10.9

Table 10.10

Table 10.11

Table 10.12

Table 11.1

Table 11.2

Table 11.3

Table 11.4

Table 11.5

Table 11.6

Introduction to Mixed Modelling

Beyond Regression and Analysis of Variance

Second Edition

N. W. Galwey

Statistical Consulting Group, GlaxoSmithKline, UK

This edition first published 2014

© 2014 John Wiley & Sons, Ltd

Registered office

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. It is sold on the understanding that the publisher is not engaged in rendering professional services and neither the publisher nor the author shall be liable for damages arising herefrom. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data

Galwey, Nicholas

Introduction to mixed modelling : beyond regression and analysis of variance / N. W. Galwey. – Second edition.

pages cm

Includes bibliographical references and index.

ISBN 978-1-119-94549-9 (cloth)

1. Multilevel models (Statistics) 2. Experimental design. 3. Regression analysis. 4. Analysis of variance. I. Title.

QA276.G33 2014

519.5–dc23

2014021670

A catalogue record for this book is available from the British Library.

ISBN: 978-1-119-94549-9

Preface

This book is intended for research workers and students who have made some use of the statistical techniques of regression analysis and analysis of variance (anova), but who are unfamiliar with mixed models and the criterion for fitting them called REsidual Maximum Likelihood (REML, also known as REstricted Maximum Likelihood). Such readers will know that, broadly speaking, regression analysis seeks to account for the variation in a response variable by relating it to one or more explanatory variables, whereas anova seeks to detect variation among the mean values of groups of observations. In regression analysis, the statistical significance of each explanatory variable is tested using the same estimate of residual variance, namely the residual mean square, and this estimate is also used to calculate the standard error of the effect of each explanatory variable. However, this choice is not always appropriate. Sometimes, one or more of the terms in the regression model (in addition to the residual term) represents random variation, and such a term will contribute to the observed variation in other terms. It should therefore contribute to the significance tests and standard errors of these terms: but in an ordinary regression analysis, it does not do so. Anova, on the other hand, does allow the construction of models with additional random-effect terms, known as block terms. However, it does so only in the limited context of balanced experimental designs.

The capabilities of regression analysis can be combined with those of anova by fitting to the data a mixed model, so called because it contains both fixed-effect and random-effect terms. A mixed model allows the presence of additional random-effects terms to be recognized in the full range of regression models, not only in balanced designs. Any statistical analysis that can be specified by a general linear model (the broadest form of linear regression model) or by anova can also be specified by a mixed model. However, the specification of a mixed model requires an additional step. The researcher must decide, for each term in the model, whether effects of that term (e.g. the deviations of group means from the grand mean) can be regarded as values of a random variable—usually taken to mean that they are a random sample from some much larger population—or whether they are a fixed set. In some cases, this decision is straightforward: in others, the distinction is subtle and the decision difficult. However, provided that an appropriate decision is made (see Section 6.3), the mixed model specifies a statistical analysis which is of broader validity than regression analysis or anova, and which is nearly or fully equivalent to those methods in the special cases where they are applicable.

It is fairly straightforward to specify the calculations required for regression analysis and anova, and this is done in many standard textbooks. For example, Draper and Smith (1998) give a clear, thorough and extensive account of the methods of regression analysis, and Mead (1988) does the same for the analysis of variance. To solve the equations that specify a mixed model is much less straightforward. The model is fitted—that is, the best estimates of its parameters are obtained—using the REML criterion, but the fitting process requires recursive numerical methods. It is largely because of this burden of calculation that mixed models are less familiar than regression analysis and anova: it is only in about the past three decades that the development of computer power and user-friendly statistical software has allowed them to be used routinely in research. This book aims to provide a guide to the use of mixed models that is accessible to the broad community of research scientists. It focuses not on the details of calculation, but on the specification of mixed models and the interpretation of the results.

The numerical examples in this book are presented and analysed using three statistical software systems, namely

GenStat, distributed by VSN International Ltd, Hemel Hempstead, via the website

https://www.vsni.co.uk/

.

R, from The R Project for Statistical Computing. This software can be downloaded free of charge from the website

http://www.r-project.org/

.

SAS, available from the SAS Institute via the website

http://www.sas.com/technologies/analytics/statistics/stat/

.

GenStat is a natural choice of software to illustrate the concepts and methods employed in mixed modelling because its facilities for this purpose are straightforward to use, extensive and well integrated with the rest of the system and because their output is clearly laid out and easy to interpret. Above all, the recognition of random terms in statistical models lies at the heart of GenStat. GenStat's method of specifying anova models requires the distinction of random-effect (block) and fixed-effect (treatment) terms, which makes the interpretation of designed experiments uniquely reliable and straightforward. This approach extends naturally to mixed models and provides a firm framework within which the researcher can think and plan. Despite these merits, GenStat is not among the most widely used statistical software systems, and the numerical examples are therefore also analysed using the increasingly popular software R, as well as SAS, which is long-established and widely used in the clinical and agricultural research communities.

The book's website, http://www.wiley.com/go/beyond_regression, provides solutions to the end-of-chapter exercises, as well as data files, and programs in GenStat, R and SAS, for many of the examples in this book.

This second edition incorporates many additions and changes, as well as some corrections. The most substantial are

the addition of SAS to the software systems used;

a new chapter on meta-analysis and the multiple testing problem;

recognition of situations in which it is appropriate to specify the interaction between two factors as a random-effect term, even though both of the corresponding main effects are fixed-effect terms;

an account of the Bayesian interpretation of mixed models, an alternative to the random-sample (frequentist) interpretation mentioned above;

a fuller account of the ‘great mixed model muddle’;

the random coefficient regression model.

I am grateful to the following individuals for their valuable comments and suggestions on the manuscript of this book, and/or for introducing me to mixed-modelling concepts and techniques in the three software systems: David Balding, Aruna Bansal, Caroline Galwey, Toby Johnson, Peter Lane, Roger Payne, James Roger and David Willé. I am also grateful to the participants in the GenStat Discussion List for their helpful responses to many enquiries. (Access to this lively forum can be obtained via the website https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=GENSTAT.) Any errors or omissions of fact or interpretation that remain are my sole responsibility. I would also like to express my gratitude to the many individuals and organizations who have given permission for the reproduction of data in the numerical examples presented. They are acknowledged individually in their respective places, but the high level of support that they have given me deserves to be recognized here.

References

Draper, N.R. and Smith, H. (1998)

Applied Regression Analysis

, 3rd edn, John Wiley & Sons, Inc., New York, 706 pp.

Mead, R. (1988)

The Design of Experiments: Statistical Principles for Practical Application

, Cambridge University Press, Cambridge, 620 pp.

Chapter 1The need for more than one random-effect term when fitting a regression line

1.1 A data set with several observations of variable Y at each value of variable X

One of the commonest, and simplest, uses of statistical analysis is the fitting of a straight line, known for historical reasons as a regression line, to describe the relationship between an explanatory variable, X and a response variable, Y. The departure of the values of Y from this line is called the residual variation, and is regarded as random. It is natural to ask whether the part of the variation in Y that is explained by the relationship with X is more than could reasonably be expected by chance: or more formally, whether it is significant relative to the residual variation. This is a simple regression analysis, and for many data sets it is all that is required. However, in some cases, several observations of Y are taken at each value of X. The data then form natural groups, and it may no longer be appropriate to analyse them as though every observation were independent: observations of Y at the same value of X may lie at a similar distance from the line. We may then be able to recognize two sources of random variation, namely

variation among groups

variation among observations within each group.

This is one of the simplest situations in which it is necessary to consider the possibility that there may be more than a single of random variation—or, in the language of mixed modelling, that a model with more than one may be required. In this chapter, we will examine a data set of this type and explore how the usual regression analysis is modified by the fact that the data form natural groups.