Generalized, Linear, and Mixed Models E-Book

Charles E. McCulloch

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Wiley Series in Probability and Statistics
• Sprache: Englisch

Seitenzahl: 499

Veröffentlichungsjahr: 2011

Beschreibung

An accessible and self-contained introduction to statistical models-now in a modernized new edition Generalized, Linear, and Mixed Models, Second Edition provides an up-to-date treatment of the essential techniques for developing and applying a wide variety of statistical models. The book presents thorough and unified coverage of the theory behind generalized, linear, and mixed models and highlights their similarities and differences in various construction, application, and computational aspects. A clear introduction to the basic ideas of fixed effects models, random effects models, and mixed models is maintained throughout, and each chapter illustrates how these models are applicable in a wide array of contexts. In addition, a discussion of general methods for the analysis of such models is presented with an emphasis on the method of maximum likelihood for the estimation of parameters. The authors also provide comprehensive coverage of the latest statistical models for correlated, non-normally distributed data. Thoroughly updated to reflect the latest developments in the field, the Second Edition features: * A new chapter that covers omitted covariates, incorrect random effects distribution, correlation of covariates and random effects, and robust variance estimation * A new chapter that treats shared random effects models, latent class models, and properties of models * A revised chapter on longitudinal data, which now includes a discussion of generalized linear models, modern advances in longitudinal data analysis, and the use between and within covariate decompositions * Expanded coverage of marginal versus conditional models * Numerous new and updated examples With its accessible style and wealth of illustrative exercises, Generalized, Linear, and Mixed Models, Second Edition is an ideal book for courses on generalized linear and mixed models at the upper-undergraduate and beginning-graduate levels. It also serves as a valuable reference for applied statisticians, industrial practitioners, and researchers.

Leseprobe

Contents

Preface

Preface First Edition

CHAPTER 1: INTRODUCTION

1.1 MODELS

1.2 FACTORS, LEVELS, CELLS, EFFECTS AND DATA

1.3 FIXED EFFECTS MODELS

1.4 RANDOM EFFECTS MODELS

1.5 LINEAR MIXED MODELS (LMMs)

1.6 FIXED OR RANDOM?

1.7 INFERENCE

1.8 COMPUTER SOFTWARE

1.9 EXERCISES

CHAPTER 2: ONE-WAY CLASSIFICATIONS

2.1 NORMALITY AND FIXED EFFECTS

2.2 NORMALITY, RANDOM EFFECTS AND MLE

2.3 NORMALITY, RANDOM EFFECTS AND REML

2.4 MORE ON RANDOM EFFECTS AND NORMALITY

2.5 BINARY DATA: FIXED EFFECTS

2.6 BINARY DATA: RANDOM EFFECTS

2.7 COMPUTING

2.8 EXERCISES

CHAPTER 3: SINGLE-PREDICTOR REGRESSION

3.1 INTRODUCTION

3.2 NORMALITY: SIMPLE LINEAR REGRESSION

3.3 NORMALITY: A NONLINEAR MODEL

3.4 TRANSFORMING VERSUS LINKING

3.5 RANDOM INTERCEPTS: BALANCED DATA

3.6 RANDOM INTERCEPTS: UNBALANCED DATA

3.7 BERNOULLI - LOGISTIC REGRESSION

3.8 BERNOULLI - LOGISTIC WITH RANDOM INTERCEPTS

3.9 EXERCISES

CHAPTER 4: LINEAR MODELS (LMs)

4.1 A GENERAL MODEL

4.2 A LINEAR MODEL FOR FIXED EFFECTS

4.3 MAXIMUM LIKELIHOOD UNDER NORMALITY

4.4 SUFFICIENT STATISTICS

4.5 MANY APPARENT ESTIMATORS

4.6 ESTIMABLE FUNCTIONS

4.7 A NUMERICAL EXAMPLE

4.8 ESTIMATING RESIDUAL VARIANCE

4.9 THE ONE- AND TWO-WAY CLASSIFICATIONS

4.10 TESTING LINEAR HYPOTHESES

4.11 t-TESTS AND CONFIDENCE INTERVALS

4.12 UNIQUE ESTIMATION USING RESTRICTIONS

4.13 EXERCISES

CHAPTER 5: GENERALIZED LINEAR MODELS (GLMs)

5.1 INTRODUCTION

5.2 STRUCTURE OF THE MODEL

5.3 TRANSFORMING VERSUS LINKING

5.4 ESTIMATION BY MAXIMUM LIKELIHOOD

5.5 TESTS OF HYPOTHESES

5.6 MAXIMUM QUASI-LIKELIHOOD

5.7 EXERCISES

CHAPTER 6: LINEAR MIXED MODELS (LMMs)

6.1 A GENERAL MODEL

6.2 ATTRIBUTING STRUCTURE TO VAR(y)

6.3 ESTIMATING FIXED EFFECTS FOR V KNOWN

6.4 ESTIMATING FIXED EFFECTS FOR V UNKNOWN

6.5 PREDICTING RANDOM EFFECTS FOR V KNOWN

6.6 PREDICTING RANDOM EFFECTS FOR V UNKNOWN

6.8 MAXIMUM LIKELIHOOD (ML) ESTIMATION

6.9 RESTRICTED MAXIMUM LIKELIHOOD (REML)

6.10 NOTES AND EXTENSIONS

6.11 APPENDIX FOR CHAPTER 6

6.12 EXERCISES

CHAPTER 7: GENERALIZED LINEAR MIXED MODELS (GLMMs)

7.1 INTRODUCTION

7.2 STRUCTURE OF THE MODEL

7.3 CONSEQUENCES OF HAVING RANDOM EFFECTS

7.4 ESTIMATION BY MAXIMUM LIKELIHOOD

7.5 OTHER METHODS OF ESTIMATION

7.6 TESTS OF HYPOTHESES

7.7 ILLUSTRATION: CHESTNUT LEAF BLIGHT

7.8 EXERCISES

CHAPTER 8: MODELS FOR LONGITUDINAL DATA

8.1 INTRODUCTION

8.2 A MODEL FOR BALANCED DATA

8.3 A MIXED MODEL APPROACH

8.4 RANDOM INTERCEPT AND SLOPE MODELS

8.5 PREDICTING RANDOM EFFECTS

8.7 UNBALANCED DATA

8.8 MODELS FOR NON-NORMAL RESPONSES

8.9 A SUMMARY OF RESULTS

8.10 APPENDIX

8.11 EXERCISES

CHAPTER 9: MARGINAL MODELS

9.1 INTRODUCTION

9.2 EXAMPLES OF MARGINAL REGRESSION MODELS

9.3 GENERALIZED ESTIMATING EQUATIONS

9.4 CONTRASTING MARGINAL AND CONDITIONAL MODELS

9.5 EXERCISES

CHAPTER 10: MULTIVARIATE MODELS

10.1 INTRODUCTION

10.2 MULTIVARIATE NORMAL OUTCOMES

10.3 NON-NORMALLY DISTRIBUTED OUTCOMES

10.4 CORRELATED RANDOM EFFECTS

10.5 LIKELIHOOD-BASED ANALYSIS

10.6 EXAMPLE: OSTEOARTHRITIS INITIATIVE

10.7 NOTES AND EXTENSIONS

CHAPTER 11: NONLINEAR MODELS

11.1 INTRODUCTION

11.2 EXAMPLE: CORN PHOTOSYNTHESIS

11.3 PHARMACOKINETIC MODELS

11.4 COMPUTATIONS FOR NONLINEAR MIXED MODELS

11.5 EXERCISES

CHAPTER 12: DEPARTURES FROM ASSUMPTIONS

12.1 INTRODUCTION

12.2 INCORRECT MODEL FOR RESPONSE

12.3 INCORRECT RANDOM EFFECTS DISTRIBUTION

12.4 DIAGNOSING MISSPECIFICATION

12.5 A SUMMARY OF RESULTS

12.6 EXERCISES

CHAPTER 13: PREDICTION

13.1 INTRODUCTION

13.2 BEST PREDICTION (BP)

13.3 BEST LINEAR PREDICTION (BLP)

13.4 LINEAR MIXED MODEL PREDICTION (BLUP)

13.5 REQUIRED ASSUMPTIONS

13.6 ESTIMATED BEST PREDICTION

13.7 HENDERSON’S MIXED MODEL EQUATIONS

b. Solutions

13.8 APPENDIX

13.9 EXERCISES

CHAPTER 14: COMPUTING

14.1 INTRODUCTION

14.3 COMPUTING ML ESTIMATES FOR GLMMs

14.4 PENALIZED QUASI–LIKELIHOOD AND LAPLACE

14.5 ITERATIVE BOOTSTRAP BIAS CORRECTION

14.6 EXERCISES

Appendix 1

M.1 VECTORS AND MATRICES OF ONES

M.2 KRONECKER (OR DIRECT) PRODUCTS

M.3 A MATRIX NOTATION IN TERMS OF ELEMENTS

M.4 GENERALIZED INVERSES

M.5 DIFFERENTIAL CALCULUS

Appendix 2

S.1 MOMENTS

S.2 NORMAL DISTRIBUTIONS

S.3 EXPONENTIAL FAMILIES

S.4 MAXIMUM LIKELIHOOD

S.5 LIKELIHOOD RATIO TESTS

S.6 MLE UNDER NORMALITY

References

Index

WILEY SERIES IN PROBABILITY AND STATISTICS

Established by WALTER A. SHEWHART and SAMUEL S. WILKS

Editors: David J. Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice, Iain M. Johnstone, Geert Molenberghs, David W. Scott, Adrian F. M. Smith, Ruey S. Tsay, Sanford Weisberg

Editors Emeriti: Vic Barnett, J. Stuart Hunter, Jozef L. Teugels

The Wiley Series in Probability and Statistics is well established and authoritative. It covers many topics of current research interest in both pure and applied statistics and probability theory. Written by leading statisticians and institutions, the titles span both state-of-the-art developments in the field and classical methods.

Reflecting the wide range of current research in statistics, the series encompasses applied, methodological and theoretical statistics, ranging from applications and new techniques made possible by advances in computerized practice to rigorous treatment of theoretical approaches.

This series provides essential and invaluable reading for all statisticians, whether in aca-demia, industry, government, or research.

† ABRAHAM and LEDOLTER · Statistical Methods for Forecasting

AGRESTI · Analysis of Ordinal Categorical Data

AGRESTI · An Introduction to Categorical Data Analysis, Second Edition

AGRESTI · Categorical Data Analysis, Second Edition

ALTMAN, GILL, and McDONALD · Numerical Issues in Statistical Computing for the Social Scientist

AMARATUNGA and CABRERA · Exploration and Analysis of DNA Microarray and Protein Array Data

ANDEL · Mathematics of Chance

ANDERSON · An Introduction to Multivariate Statistical Analysis, Third Edition

* ANDERSON · The Statistical Analysis of Time Series

ANDERSON, AUQUIER, HAUCK, OAKES, VANDAELE, and WEISBERG · Statistical Methods for Comparative Studies

ANDERSON and LOYNES · The Teaching of Practical Statistics

ARMITAGE and DAVID (editors) · Advances in Biometry

ARNOLD, BALAKRISHNAN, andNAGARAJA · Records

* ARTHANARI and DODGE · Mathematical Programming in Statistics

* BAILEY · The Elements of Stochastic Processes with Applications to the Natural Sciences

BALAKRISHNAN and KOUTRAS · Runs and Scans with Applications

BALAKRISHNAN and NG · Precedence-Type Tests and Applications

BARNETT · Comparative Statistical Inference, Third Edition

BARNETT · Environmental Statistics

BARNETT and LEWIS · Outliers in Statistical Data, Third Edition

BARTOSZYNSKI and NIEWIADOMSKA-BUGAJ · Probability and Statistical Inference

BASILEVSKY · Statistical Factor Analysis and Related Methods: Theory and Applications

BASU and RIGDON · Statistical Methods for the Reliability of Repairable Systems

BATES and WATTS · Nonlinear Regression Analysis and Its Applications

BECHHOFER, SANTNER, and GOLDSMAN · Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons

* Now available in a lower priced paperback edition in the Wiley Classics Library.

† Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

BELSLEY · Conditioning Diagnostics: Collinearity and Weak Data in Regression

† BELSLEY, KUH, and WELSCH · Regression Diagnostics: Identifying Influential Data and Sources of Collinearity

BENDAT and PIERSOL · Random Data: Analysis and Measurement Procedures, Third Edition

BERRY, CHALONER, and GEWEKE · Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner

BERNARDO and SMITH · Bayesian Theory

BHAT and MILLER · Elements of Applied Stochastic Processes, Third Edition

BHATTACHARYA and WAYMIRE · Stochastic Processes with Applications

BILLINGSLEY · Convergence of Probability Measures, Second Edition

BILLINGSLEY · Probability and Measure, Third Edition

BIRKES and DODGE · Alternative Methods of Regression

BISWAS, DATTA, FINE, and SEGAL · Statistical Advances in the Biomedical Sciences:

Clinical Trials, Epidemiology, Survival Analysis, and Bioinformatics

BLISCHKE AND MURTHY (editors) · Case Studies in Reliability and Maintenance

BLISCHKE AND MURTHY · Reliability: Modeling, Prediction, and Optimization

BLOOMFIELD · Fourier Analysis of Time Series: An Introduction, Second Edition

BOLLEN · Structural Equations with Latent Variables

BOLLEN and CURRAN · Latent Curve Models: A Structural Equation Perspective

BOROVKOV · Ergodicity and Stability of Stochastic Processes

BOULEAU · Numerical Methods for Stochastic Processes

BOX · Bayesian Inference in Statistical Analysis

BOX · R. A. Fisher, the Life of a Scientist

BOX and DRAPER · Response Surfaces, Mixtures, and Ridge Analyses, Second Edition

* BOX and DRAPER · Evolutionary Operation: A Statistical Method for Process Improvement

BOX and FRIENDS · Improving Almost Anything, Revised Edition

BOX, HUNTER, and HUNTER · Statistics for Experimenters: Design, Innovation, and Discovery, Second Editon

BOX and LUCENO · Statistical Control by Monitoring and Feedback Adjustment

BRANDIMARTE · Numerical Methods in Finance: A MATLAB-Based Introduction

† BROWN and HOLLANDER · Statistics: A Biomedical Introduction

BRUNNER, DOMHOF, and LANGER · Nonparametric Analysis of Longitudinal Data in Factorial Experiments

BUCKLEW · Large Deviation Techniques in Decision, Simulation, and Estimation

CAIROLI and DALANG · Sequential Stochastic Optimization

CASTILLO, HADI, BALAKRISHNAN, and SARABIA · Extreme Value and Related Models with Applications in Engineering and Science

CHAN · Time Series: Applications to Finance

CHARALAMBIDES · Combinatorial Methods in Discrete Distributions

CHATTERJEE and HADI · Regression Analysis by Example, Fourth Edition CHATTERJEE and HADI · Sensitivity Analysis in Linear Regression CHERNICK · Bootstrap Methods: A Guide for Practitioners and Researchers, Second Edition

CHERNICK and FRIIS · Introductory Biostatistics for the Health Sciences

CHILES and DELFINER · Geostatistics: Modeling Spatial Uncertainty

CHOW and LIU · Design and Analysis of Clinical Trials: Concepts and Methodologies, Second Edition

CLARKE and DISNEY · Probability and Random Processes: A First Course with Applications, Second Edition

* COCHRAN and COX · Experimental Designs, Second Edition

CONGDON · Applied Bayesian Modelling

CONGDON · Bayesian Models for Categorical Data

*Now available in a lower priced paperback edition in the Wiley Classics Library.

† Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

CONGDON · Bayesian Statistical Modelling

CONOVER · Practical Nonparametric Statistics, Third Edition

COOK · Regression Graphics

COOK and WEISBERG · Applied Regression Including Computing and Graphics

COOK and WEISBERG · An Introduction to Regression Graphics

CORNELL · Experiments with Mixtures, Designs, Models, and the Analysis of Mixture Data, Third Edition

COVER and THOMAS · Elements of Information Theory

COX · A Handbook of Introductory Statistical Methods

* COX · Planning of Experiments

CRESSIE · Statistics for Spatial Data, Revised Edition

CSORGO and HORVATH · Limit Theorems in Change Point Analysis

DANIEL · Applications of Statistics to Industrial Experimentation

DANIEL · Biostatistics: A Foundation for Analysis in the Health Sciences, Eighth Edition

* DANIEL · Fitting Equations to Data: Computer Analysis of Multifactor Data, Second Edition

DASU and JOHNSON · Exploratory Data Mining and Data Cleaning

DAVID and NAGARAJA · Order Statistics, Third Edition

* DEGROOT, FIENBERG, and KADANE · Statistics and the Law

DEL CASTILLO · Statistical Process Adjustment for Quality Control

DEMARIS · Regression with Social Data: Modeling Continuous and Limited Response Variables

DEMIDENKO · Mixed Models: Theory and Applications

DENISON, HOLMES, MALLICK and SMITH · Bayesian Methods for Nonlinear Classification and Regression

DETTE and STUDDEN · The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis

DEY and MUKERJEE · Fractional Factorial Plans

DILLON and GOLDSTEIN · Multivariate Analysis: Methods and Applications

DODGE · Alternative Methods of Regression

* DODGE and ROMIG · Sampling Inspection Tables, Second Edition

* DOOB · Stochastic Processes

DOWDY, WEARDEN, and CHILKO · Statistics for Research, Third Edition

DRAPER and SMITH · Applied Regression Analysis, Third Edition

DRYDEN and MARDIA · Statistical Shape Analysis

DUDEWICZ and MISHRA · Modern Mathematical Statistics

DUNN and CLARK · Basic Statistics: A Primer for the Biomedical Sciences, Third Edition

DUPUIS and ELLIS · A Weak Convergence Approach to the Theory of Large

Deviations EDLER and KITSOS · Recent Advances in Quantitative Methods in Cancer and Human Health Risk Assessment

* ELANDT-JOHNSON and JOHNSON · Survival Models and Data Analysis

ENDERS · Applied Econometric Time Series

† ETHIER and KURTZ · Markov Processes: Characterization and Convergence

EVANS, HASTINGS, and PEACOCK · Statistical Distributions, Third Edition

FELLER · An Introduction to Probability Theory and Its Applications, Volume I, Third Edition, Revised; Volume II, Second Edition

FISHER and VAN BELLE · Biostatistics: A Methodology for the Health Sciences

FITZMAURICE, LAIRD, and WARE · Applied Longitudinal Analysis

* FLEISS · The Design and Analysis of Clinical Experiments

FLEISS · Statistical Methods for Rates and Proportions, Third Edition

† FLEMING and HARRINGTON · Counting Processes and Survival Analysis

FULLER · Introduction to Statistical Time Series, Second Edition

*Now available in a lower priced paperback edition in the Wiley Classics Library.

† Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

† FULLER · Measurement Error Models GALLANT · Nonlinear Statistical Models

GEISSER · Modes of Parametric Statistical Inference

GELMAN and MENG · Applied Bayesian Modeling and Causal Inference from Incomplete-Data Perspectives

GEWEKE · Contemporary Bayesian Econometrics and Statistics

GHOSH, MUKHOPADHYAY, and SEN · Sequential Estimation

GIESBRECHT and GUMPERTZ · Planning, Construction, and Statistical Analysis of Comparative Experiments

GIFI · Nonlinear Multivariate Analysis

GIVENS and HOETING · Computational Statistics

GLASSERMAN and YAO · Monotone Structure in Discrete-Event Systems

GNANADESIKAN · Methods for Statistical Data Analysis of Multivariate Observations, Second Edition

GOLDSTEIN and LEWIS · Assessment: Problems, Development, and Statistical Issues

GREENWOOD and NIKULIN · A Guide to Chi-Squared Testing

GROSS and HARRIS · Fundamentals of Queueing Theory, Third Edition

* HAHN and SHAPIRO · Statistical Models in Engineering

HAHN and MEEKER · Statistical Intervals: A Guide for Practitioners

HALD · A History of Probability and Statistics and their Applications Before 1750

HALD · A History of Mathematical Statistics from 1750 to 1930

† HAMPEL · Robust Statistics: The Approach Based on Influence Functions

HANNAN and DEISTLER · The Statistical Theory of Linear Systems

HEIBERGER · Computation for the Analysis of Designed Experiments

HEDAYAT and SINHA · Design and Inference in Finite Population Sampling

HEDEKER and GIBBONS · Longitudinal Data Analysis

HELLER · MACSYMA for Statisticians

HINKELMANN and KEMPTHORNE · Design and Analysis of Experiments, Volume 1: Introduction to Experimental Design, Second Edition

HINKELMANN and KEMPTHORNE · Design and Analysis of Experiments, Volume 2: Advanced Experimental Design

HOAGLIN, MOSTELLER, and TUKEY · Exploratory Approach to Analysis of Variance

* HOAGLIN, MOSTELLER, and TUKEY · Exploring Data Tables, Trends and Shapes

* HOAGLIN, MOSTELLER, and TUKEY · Understanding Robust and Exploratory Data Analysis

HOCHBERG and TAMHANE · Multiple Comparison Procedures

HOCKING · Methods and Applications of Linear Models: Regression and the Analysis of Variance, Second Edition

HOEL · Introduction to Mathematical Statistics, Fifth Edition

HOGG and KLUGMAN · Loss Distributions

HOLLANDER and WOLFE · Nonparametric Statistical Methods, Second Edition

HOSMER and LEMESHOW · Applied Logistic Regression, Second Edition

HOSMER, LEMESHOW, and MAY · Applied Survival Analysis: Regression Modeling of Time-to-Event Data, Second Edition

† HUBER · Robust Statistics

HUBERTY · Applied Discriminant Analysis

HUBERTY and OLEJNIK · Applied MANOVA and Discriminant Analysis, Second Edition

HUNT and KENNEDY · Financial Derivatives in Theory and Practice, Revised Edition

HURD and MIAMEE · Periodically Correlated Random Sequences: Spectral Theory and Practice

* Now available in a lower priced paperback edition in the Wiley Classics Library.

† Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

HUSKOVA, BERAN, and DUPAC · Collected Works of Jaroslav Hajek—with Commentary

HUZURBAZAR · Flowgraph Models for Multistate Time-to-Event Data

IMAN and CONOVER · A Modern Approach to Statistics

† JACKSON · A User’s Guide to Principle Components

JOHN · Statistical Methods in Engineering and Quality Assurance

JOHNSON · Multivariate Statistical Simulation

JOHNSON and BALAKRISHNAN · Advances in the Theory and Practice of Statistics: A Volume in Honor of Samuel Kotz

JOHNSON and BHATTACHARYYA · Statistics: Principles and Methods, Fifth Edition

JOHNSON and KOTZ · Distributions in Statistics

JOHNSON and KOTZ (editors) · Leading Personalities in Statistical Sciences: From the Seventeenth Century to the Present

JOHNSON, KOTZ, and BALAKRISHNAN · Continuous Univariate Distributions, Volume 1, Second Edition

JOHNSON, KOTZ, and BALAKRISHNAN · Continuous Univariate Distributions, Volume 2, Second Edition

JOHNSON, KOTZ, and BALAKRISHNAN · Discrete Multivariate Distributions

JOHNSON, KEMP, and KOTZ · Univariate Discrete Distributions, Third Edition

JUDGE, GRIFFITHS, HILL, LUTKEPOHL, and LEE · The Theory and Practice of Econometrics, Second Edition

JURECKOVA and SEN · Robust Statistical Procedures: Aymptotics and Interrelations

JUREK and MASON · Operator-Limit Distributions in Probability Theory

KADANE · Bayesian Methods and Ethics in a Clinical Trial Design

KADANE AND SCHUM · A Probabilistic Analysis of the Sacco and Vanzetti Evidence

KALBFLEISCH and PRENTICE · The Statistical Analysis of Failure Time Data, Second Edition

KARIYA and KURATA · Generalized Least Squares

KASS and VOS · Geometrical Foundations of Asymptotic Inference

† KAUFMAN and ROUSSEEUW · Finding Groups in Data: An Introduction to Cluster Analysis

KEDEM and FOKIANOS · Regression Models for Time Series Analysis

KENDALL, BARDEN, CARNE, and LE · Shape and Shape Theory

KHURI · Advanced Calculus with Applications in Statistics, Second Edition

KHURI, MATHEW, and SINHA · Statistical Tests for Mixed Linear Models

KLEIBER and KOTZ · Statistical Size Distributions in Economics and Actuarial Sciences

KLUGMAN, PANJER, and WILLMOT · Loss Models: From Data to Decisions, Second Edition

KLUGMAN, PANJER, and WILLMOT · Solutions Manual to Accompany Loss Models: From Data to Decisions, Second Edition

KOTZ, BALAKRISHNAN, and JOHNSON · Continuous Multivariate Distributions, Volume 1, Second Edition

KOVALENKO, KUZNETZOV, and PEGG · Mathematical Theory of Reliability of Time-Dependent Systems with Practical Applications

KOWALSKI and TU · Modern Applied U-Statistics

KROONENBERG · Applied Multiway Data Analysis

KVAM and VIDAKOVIC · Nonparametric Statistics with Applications to Science and Engineering

LACHIN · Biostatistical Methods: The Assessment of Relative Risks

LAD · Operational Subjective Statistical Methods: A Mathematical, Philosophical, and Historical Introduction

LAMPERTI · Probability: A Survey of the Mathematical Theory, Second Edition

LANGE, RYAN, BILLARD, BRILLINGER, CONQUEST, and GREENHOUSE · Case Studies in Biometry

*Now available in a lower priced paperback edition in the Wiley Classics Library.

† Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

LARSON · Introduction to Probability Theory and Statistical Inference, Third Edition

LAWLESS · Statistical Models and Methods for Lifetime Data, Second Edition

LAWSON · Statistical Methods in Spatial Epidemiology

LE · Applied Categorical Data Analysis

LE · Applied Survival Analysis

LEE and WANG · Statistical Methods for Survival Data Analysis, Third Edition

LEPAGE and BILLARD · Exploring the Limits of Bootstrap

LEYLAND and GOLDSTEIN (editors) · Multilevel Modelling of Health Statistics

LIAO · Statistical Group Comparison

LINDVALL · Lectures on the Coupling Method

LIN · Introductory Stochastic Analysis for Finance and Insurance

LINHART and ZUCCHINI · Model Selection

LITTLE and RUBIN · Statistical Analysis with Missing Data, Second Edition

LLOYD · The Statistical Analysis of Categorical Data

LOWEN and TEICH · Fractal-Based Point Processes

MAGNUS and NEUDECKER · Matrix Differential Calculus with Applications in Statistics and Econometrics, Revised Edition

MALLER and ZHOU · Survival Analysis with Long Term Survivors

MALLOWS · Design, Data, and Analysis by Some Friends of Cuthbert Daniel

MANN, SCHAFER, and SINGPURWALLA · Methods for Statistical Analysis of Reliability and Life Data

MANTON, WOODBURY, and TOLLEY · Statistical Applications Using Fuzzy Sets

MARCHETTE · Random Graphs for Statistical Pattern Recognition

MARDIA and JUPP · Directional Statistics

MASON, GUNST, and HESS · Statistical Design and Analysis of Experiments with Applications to Engineering and Science, Second Edition

McCULLOCH, SEARLE, and NEUHAUS · Generalized, Linear, and Mixed Models, Second Edition

McFADDEN · Management of Data in Clinical Trials, Second Edition

* McLACHLAN · Discriminant Analysis and Statistical Pattern Recognition

McLACHLAN, DO, and AMBROISE · Analyzing Microarray Gene Expression Data

McLACHLAN and KRISHNAN · The EM Algorithm and Extensions, Second Edition

McLACHLAN and PEEL · Finite Mixture Models

McNEIL · Epidemiological Research Methods

MEEKER and ESCOBAR · Statistical Methods for Reliability Data

MEERSCHAERT and SCHEFFLER · Limit Distributions for Sums of Independent Random Vectors: Heavy Tails in Theory and Practice

MICKEY, DUNN, and CLARK · Applied Statistics: Analysis of Variance and Regression, Third Edition

* MILLER · Survival Analysis, Second Edition

MONTGOMERY, JENNINGS, and KULAHCI · Introduction to Time Series Analysis and Forecasting

MONTGOMERY, PECK, and VINING · Introduction to Linear Regression Analysis, Fourth Edition

MORGENTHALER and TUKEY · Configural Polysampling: A Route to Practical Robustness

MUIRHEAD · Aspects of Multivariate Statistical Theory

MULLER and STOYAN · Comparison Methods for Stochastic

Models and Risks MURRAY · X-STAT 2.0 Statistical Experimentation, Design Data Analysis, and Nonlinear Optimization

MURTHY, XIE, and JIANG · Weibull Models

MYERS and MONTGOMERY · Response Surface Methodology: Process and Product Optimization Using Designed Experiments, Second Edition

*Now available in a lower priced paperback edition in the Wiley Classics Library.

†Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

MYERS, MONTGOMERY, and VINING · Generalized Linear Models. With Applications in Engineering and the Sciences

† NELSON · Accelerated Testing, Statistical Models, Test Plans, and Data Analyses

† NELSON · Applied Life Data Analysis

NEWMAN · Biostatistical Methods in Epidemiology

OCHI · Applied Probability and Stochastic Processes in Engineering and Physical Sciences

OKABE, BOOTS, SUGIHARA, and CHIU · Spatial Tesselations: Concepts and

Applications of Voronoi Diagrams, Second Edition

OLIVER and SMITH · Influence Diagrams, Belief Nets and Decision Analysis

PALTA · Quantitative Methods in Population Health: Extensions of Ordinary Regressions

PANJER · Operational Risk: Modeling and Analytics

PANKRATZ · Forecasting with Dynamic Regression Models

PANKRATZ · Forecasting with Univariate Box-Jenkins Models: Concepts and Cases

* PARZEN · Modern Probability Theory and Its Applications

PENA, TIAO, and TSAY · A Course in Time Series Analysis

PIANTADOSI · Clinical Trials: A Methodologic Perspective

PORT · Theoretical Probability for Applications

POURAHMADI · Foundations of Time Series Analysis and Prediction Theory

POWELL · Approximate Dynamic Programming: Solving the Curses of Dimensionality

PRESS · Bayesian Statistics: Principles, Models, and Applications

PRESS · Subjective and Objective Bayesian Statistics, Second Edition

PRESS and TANUR · The Subjectivity of Scientists and the Bayesian Approach

PUKELSHEIM · Optimal Experimental Design

PURI, VILAPLANA, and WERTZ · New Perspectives in Theoretical and Applied Statistics

† PUTERMAN · Markov Decision Processes: Discrete Stochastic Dynamic Programming

QIU · Image Processing and Jump Regression Analysis

* RAO · Linear Statistical Inference and Its Applications, Second Edition

RAUSAND and H0YLAND · System Reliability Theory: Models, Statistical Methods, and Applications, Second Edition

RENCHER · Linear Models in Statistics

RENCHER · Methods of Multivariate Analysis, Second Edition

RENCHER · Multivariate Statistical Inference with Applications

* RIPLEY · Spatial Statistics

* RIPLEY · Stochastic Simulation

ROBINSON · Practical Strategies for Experimenting

ROHATGI and SALEH · An Introduction to Probability and Statistics, Second Edition

ROLSKI, SCHMIDLI, SCHMIDT, and TEUGELS · Stochastic Processes for Insurance and Finance

ROSENBERGER and LACHIN · Randomization in Clinical Trials: Theory and Practice

ROSS · Introduction to Probability and Statistics for Engineers and Scientists

ROSSI, ALLENBY, and McCULLOCH · Bayesian Statistics and Marketing

† ROUSSEEUW and LEROY · Robust Regression and Outlier Detection

* RUBIN · Multiple Imputation for Nonresponse in Surveys

RUBINSTEIN and KROESE · Simulation and the Monte Carlo Method, Second Edition

RUBINSTEIN and MELAMED · Modern Simulation and Modeling

RYAN · Modern Engineering Statistics

RYAN · Modern Experimental Design

RYAN · Modern Regression Methods

RYAN · Statistical Methods for Quality Improvement, Second Edition

SALEH · Theory of Preliminary Test and Stein-Type Estimation with Applications

* SCHEFFE · The Analysis of Variance

SCHIMEK · Smoothing and Regression: Approaches, Computation, and Application

*Now available in a lower priced paperback edition in the Wiley Classics Library.

† Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

SCHOTT · Matrix Analysis for Statistics, Second Edition

SCHOUTENS · Levy Processes in Finance: Pricing Financial Derivatives

SCHUSS · Theory and Applications of Stochastic Differential Equations

SCOTT · Multivariate Density Estimation: Theory, Practice, and Visualization

† SEARLE · Linear Models for Unbalanced Data

† SEARLE · Matrix Algebra Useful for Statistics

† SEARLE, CASELLA, and McCULLOCH · Variance Components

SEARLE and WILLETT · Matrix Algebra for Applied Economics

SEBER · A Matrix Handbook For Statisticians

† SEBER · Multivariate Observations

SEBER and LEE · Linear Regression Analysis, Second Edition

† SEBER and WILD · Nonlinear Regression

SENNOTT · Stochastic Dynamic Programming and the Control of Queueing

Systems · SERFLING · Approximation Theorems of Mathematical Statistics

SHAFER and VOVK · Probability and Finance: It’s Only a Game!

SILVAPULLE and SEN · Constrained Statistical Inference: Inequality, Order, and Shape Restrictions

SMALL and McLEISH · Hilbert Space Methods in Probability and Statistical Inference

SRIVASTAVA · Methods of Multivariate Statistics

STAPLETON · Linear Statistical Models

STAPLETON · Models for Probability and Statistical Inference: Theory and Applications

STAUDTE and SHEATHER · Robust Estimation and Testing

STOYAN, KENDALL, and MECKE · Stochastic Geometry and Its Applications, Second Edition

STOYAN and STOYAN · Fractals, Random Shapes and Point Fields: Methods of Geometrical Statistics

STREET and BURGESS · The Construction of Optimal Stated Choice Experiments: Theory and Methods

STY AN · The Collected Papers of T. W. Anderson: 1943-1985

SUTTON, ABRAMS, JONES, SHELDON, and SONG · Methods for Meta-Analysis in Medical Research

TAKEZAWA · Introduction to Nonparametric Regression

TANAKA · Time Series Analysis: Nonstationary and Noninvertible Distribution Theory

THOMPSON · Empirical Model Building

THOMPSON · Sampling, Second Edition

THOMPSON · Simulation: A Modeler’s Approach

THOMPSON and SEBER · Adaptive Sampling

THOMPSON, WILLIAMS, and FINDLAY · Models for Investors in Real World Markets

TIAO, BISGAARD, HILL, PENA, and STIGLER (editors) · Box on Quality and Discovery: with Design, Control, and Robustness

TIERNEY · LISP-STAT: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics

TSAY · Analysis of Financial Time Series, Second Edition

UPTON and FINGLETON · Spatial Data Analysis by Example, Volume II: Categorical and Directional Data

† VAN BELLE · Statistical Rules of Thumb, Second Edition

VAN BELLE, FISHER, HEAGERTY, and LUMLEY · Biostatistics: A Methodology for the Health Sciences, Second Edition

VESTRUP · The Theory of Measures and Integration

VIDAKOVIC · Statistical Modeling by Wavelets

VINOD and REAGLE · Preparing for the Worst: Incorporating Downside Risk in Stock Market Investments

WALLER and GOTWAY · Applied Spatial Statistics for Public Health Data

*Now available in a lower priced paperback edition in the Wiley Classics Library.

† Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

WEERAHANDI · Generalized Inference in Repeated Measures: Exact Methods in MANOVA and Mixed Models

WEISBERG · Applied Linear Regression, Third Edition

WELSH · Aspects of Statistical Inference

WESTFALL and YOUNG · Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment

WHITTAKER · Graphical Models in Applied Multivariate Statistics

WINKER · Optimization Heuristics in Economics: Applications of Threshold Accepting

WONNACOTT and WONNACOTT · Econometrics, Second Edition

WOODING · Planning Pharmaceutical Clinical Trials: Basic Statistical Principles

WOODWORTH · Biostatistics: A Bayesian Introduction

WOOLSON and CLARKE · Statistical Methods for the Analysis of Biomedical Data, Second Edition

WU and HAM AD A · Experiments: Planning, Analysis, and Parameter Design Optimization

WU and ZHANG · Nonparametric Regression Methods for Longitudinal Data Analysis

YANG · The Construction Theory of Denumerable Markov Processes

YOUNG, VALERO-MORA, and FRIENDLY · Visual Statistics: Seeing Data with Dynamic Interactive Graphics

ZELTERMAN · Discrete Distributions—Applications in the Health Sciences

* ZELLNER · An Introduction to Bayesian Inference in Econometrics

ZHOU, OBUCHOWSKI, and McCLISH · Statistical Methods in Diagnostic Medicine

*Now available in a lower priced paperback edition in the Wiley Classics Library.

†Now available in a lower priced paperback edition in the Wiley–Interscience Paperback Series.

Copyright © 2008 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

McCulloch, Charles E.

Generalized, linear, and mixed models / Charles E. McCulloch, Shayle R. Searle, John M. Neuhaus. — 2nd ed.

p. cm.

Includes bibliographical references and index.

ISBN 978-0-470-07371-1 (cloth)

1. Linear models (Statistics) I. Searle, S. R. (Shayle R.), 1928– II. Neuhaus, John M. III. Title.

QA279.M3847 2008

519.5′35—dc22

2008002724

Preface

For the Second Edition of Generalized, Linear and Mixed Models we incorporated advances of the intervening years since the First Edition. Chapter 8, the former Chapter 7, includes more extensive development of longitudinal data models, especially for non-normally distributed outcomes. There are three new chapters: Chapter 9 extracts and expands on the First Edition’s treatment of marginal models, Chapter 10 describes methods of incorporating multiple outcomes (especially of different distributional types), and Chapter 12 explores the consequences of departures from assumptions and suggests remedies when assumptions are violated. Models that incorporate all the features of the book, e.g., generalized, linear, mixed models are assumption laden and it is important to delineate which assumptions are crucial, ways to diagnose them, and possible remedies. In addition, the rest of the book has been lightly revised.

Although this edition is expanded, the text can still be used as originally intended. The additional chapters serve to expand the basic offerings and show a wider variety of ways in which mixed models can be used and some of the practical issues in implementation. Chapters 1 through 3 are the introductory chapters. Chapter 1 lays out the basic terminology and gives a of number examples. Chapters 2 and 3 cover the central ideas of the entire book in simple scenarios (one-way classification and single predictor regression). Chapters 4 through 7 form the “meat” of the book and describe the main classes of models. The rest of the Chapters cover more specialized topics and can be read or taught in a relatively independent manner, as interest allows, following Chapters 4 through 7. Chapters 8 through 11 cover variations on the main models (longitudinal data models, models incorporating multivariate responses, and nonlinear models) and Chapters 12 through 14 cover issues (departures from assumptions, prediction and computing) that largely cross-cut the topics of previous chapters.

As before, the emphasis is on the applications of these models and the assumptions necessary for valid statistical inference. The focus is not on the details of data analysis nor on the use of statistical software, though we do briefly mention some examples.

Charles E. McCullochShayle R. SearleJohn M. Neuhaus

San Francisco, CAIthaca, NYMay 2008

Preface to the First Edition

The last thirty or so years have been a time of enormous development of analytic results for the linear model (LM). This has generated extensive publication of books and papers on the subject. Much of this activity has focused on the normal distribution and homoscedasticity. Even for unbalanced data, many useful, analytically tractable results have become available. Those results center largely around analysis of variance (ANOVA) procedures, and there is abundant computing software which will, with wide reliability, compute those results from submitted data.

Also within the realm of normal distributions, but permitting heterogeneity of variance, there has been considerable work on linear mixed models (LMMs) wherein the variance structure is based on random effects and their variance components. Algebraic results in this context are much more limited and complicated than with LMs. However, with the advent of readily available computing power and the development of broadly applicable computing procedures (e.g., the EM algorithm) we are now at a point where models such as the LMM are available to the practitioner. Furthermore, models that are nonlinear and incorporate non-normal distributions are now feasible. It is to understanding these models and appreciating the available computing procedures that this book is directed.

We begin by reviewing the basics of LMs and LMMs, to serve as a starting point for proceeding to generalized linear models (GLMs), generalized linear mixed models (GLMMs) and some nonlinear models. All of these are encompassed within the title “Generalized, Linear, and Mixed Models.”

The progress from easy to difficult models (e.g. from LMs to GLMMs) necessitates a certain repetition of basic analysis methods, but this is appropriate because the book deals with a variety of models and the application to them of standard statistical methods. For example, maximum likelihood (ML) is used in almost every chapter, on models that get progressively more difficult as the book progresses. There is, indeed, purposeful concentration on ML and, very noticeably, an (almost complete) absence of analysis of variance (ANOVA) tables.

Although analysis of variance methods are quite natural for fixed effects linear models with normal distributions, even in the case of linear mixed models with normal distributions they have much less appeal. For example, with unbalanced data from mixed models, it is not clear what the “appropriate” ANOVA table should be. Furthermore, from a theoretical viewpoint, any such table represents an over-summarization of data: except in special cases, it does not contain sufficient statistics and therefore engenders a loss of information and efficiency. And these deficiencies are aggravated if one tries to generalize analysis of variance to models based on non-normal distributions such as, for example, the Poisson or binomial. To deal with these we therefore concentrate on ML procedures.

Although ML estimation under non-normality is limited in yielding analytic results, we feel that its generality and efficiency (at least with large samples) make it a natural method to use in today’s world. Today’s computing environment compensates for the analytic intractability of ML and helps makes ML more palatable.

As prelude to the application of ML to non-normal models we often show details of using it on models where it yields easily interpreted analytic results. The details are lengthy, but studying them engenders a confidence in the ML method that hopefully carries over to non-normal models. For these, the details are often not lengthy, because there are so few of them (as a consequence of the model’s inherent intractability) and they yield few analytic results. The brevity of describing them should not be taken as a lack of emphasis or importance, but merely as a lack of neat, tidy results. It is a fact of modern statistical practice that computing procedures are used to gain numerical information about the underlying nature of algebraically intractable results. Our aim in this book is to illuminate this situation.

The book is intended for graduate students and practicing statisticians. We begin with a chapter in which we introduce the basic ideas of fixed and random factors and mixed models and briefly discuss general methods for the analysis of such models. Chapters 2 and 3 introduce all the main ideas of the remainder of the book in two simple contexts (one-way classifications and linear regression) with a minimum of emphasis on generality of results and notation. These three chapters could form the core of a quarter course or, with supplementation, the basis of a semester-long course for Master’s students. Alternatively, they could be used to introduce generalized mixed models towards the end of a linear models class.

Chapters 4, 5, 6 and 8 cover the main classes of models (linear, generalized linear, linear mixed, and generalized linear mixed) in more generality and breadth. Chapter 7 discusses some of the special features of longitudinal data and shows how they can be accommodated within LMMs. Chapter 9 presents the idea of prediction of realized values of random effects. This is an important distinction introduced by considering models containing random effects. Chapter 10 covers computing issues, one of the main barriers to adoption of mixed models in practice. Lest the reader think that everything can be accommodated under the rubric of the generalized linear mixed model, Chapter 11 briefly mentions nonlinear mixed models. And the book ends with two short appendices, M and S, containing some pertinent results in matrices and statistics.

For students with some training in linear models, the first 10 chapters, with light emphasis on Chapters 1 through 4 and 6, could form a “second” course extending their linear model knowledge to generalized linear models. Of course, the book could also be used for a semester long course on generalized mixed models, although in-depth coverage of all of the topics would clearly be difficult.

Our emphasis throughout is on modeling and model development. Thus we provide important information about the consequences of model assumptions, techniques of model fitting and methods of inference which will be required for data analysis, as opposed to data analysis itself. However, to illustrate the concepts we do also include analysis or illustration of the techniques for a variety of real data sets.

The chapters are quite variable in length, but all of them have sections, subsections and sub-subsections, each with its own title, as shown in the Table of Contents. At times we have sacrificed the flow of the narrative to make the book more accessible as a reference. For example, Section 2.Id is basically a catalogue of results with titles that make retrieval more straightforward, particularly because those titles are all listed in the table of contents.

Charles E. McCullochShayle R. Searle

Ithaca, NYSeptember 2000