Bayesian Modeling Using WinBUGS E-Book

Ioannis Ntzoufras

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Wiley Series in Computational Statistics
• Sprache: Englisch

Seitenzahl: 744

Veröffentlichungsjahr: 2011

Beschreibung

A hands-on introduction to the principles of Bayesian modeling using WinBUGS Bayesian Modeling Using WinBUGS provides an easily accessible introduction to the use of WinBUGS programming techniques in a variety of Bayesian modeling settings. The author provides an accessible treatment of the topic, offering readers a smooth introduction to the principles of Bayesian modeling with detailed guidance on the practical implementation of key principles. The book begins with a basic introduction to Bayesian inference and the WinBUGS software and goes on to cover key topics, including: * Markov Chain Monte Carlo algorithms in Bayesian inference * Generalized linear models * Bayesian hierarchical models * Predictive distribution and model checking * Bayesian model and variable evaluation Computational notes and screen captures illustrate the use of both WinBUGS as well as R software to apply the discussed techniques. Exercises at the end of each chapter allow readers to test their understanding of the presented concepts and all data sets and code are available on the book's related Web site. Requiring only a working knowledge of probability theory and statistics, Bayesian Modeling Using WinBUGS serves as an excellent book for courses on Bayesian statistics at the upper-undergraduate and graduate levels. It is also a valuable reference for researchers and practitioners in the fields of statistics, actuarial science, medicine, and the social sciences who use WinBUGS in their everyday work.

Leseprobe

CONTENTS

PREFACE

ACKNOWLEDGMENTS

ACRONYMS

CHAPTER 1: INTRODUCTION TO BAYESIAN INFERENCE

1.1 INTRODUCTION: BAYESIAN MODELING IN THE 21 ST CENTURY

1.2 DEFINITION OF STATISTICAL MODELS

1.3 BAYES THEOREM

1.4 MODEL-BASED BAYESIAN INFERENCE

1.5 INFERENCE USING CONJUGATE PRIOR DISTRIBUTIONS

1.6 NONCONJUGATE ANALYSIS

Problems

CHAPTER 2: MARKOV CHAIN MONTE CARLO ALGORITHMS IN BAYESIAN INFERENCE

2.1 SIMULATION, MONTE CARLO INTEGRATION, AND THEIR IMPLEMENTATION IN BAYESIAN INFERENCE

2.2 MARKOV CHAIN MONTE CARLO METHODS

2.3 POPULAR MCMC ALGORITHMS

2.4 SUMMARY AND CLOSING REMARKS

Problems

CHAPTER 3: WinBUGS SOFTWARE: INTRODUCTION, SETUP, AND BASIC ANALYSIS

3.1 INTRODUCTION AND HISTORICAL BACKGROUND

3.2 THE WinBUGS ENVIRONMENT

3.3 PRELIMINARIES ON USING WinBUGS

3.4 BUILDING BAYESIAN MODELS IN WinBUGS

3.5 COMPILING THE MODEL AND SIMULATING VALUES

3.6 BASIC OUTPUT ANALYSIS USING THE SAMPLE MONITOR TOOL

3.7 SUMMARIZING THE PROCEDURE

3.8 CHAPTER SUMMARY AND CONCLUDING COMMENTS

Problems

CHAPTER 4: WinBUGS SOFTWARE: ILLUSTRATION, RESULTS, AND FURTHER ANALYSIS

4.1 A COMPLETE EXAMPLE OF RUNNING MCMC IN WinBUGS FOR A SIMPLE MODEL

4.2 FURTHER OUTPUT ANALYSIS USING THE INFERENCE MENU

4.3 MULTIPLE CHAINS

4.4 CHANGING THE PROPERTIES OF A FIGURE

4.5 OTHER TOOLS AND MENUS

4.6 SUMMARY AND CONCLUDING REMARKS

Problems

CHAPTER 5: INTRODUCTION TO BAYESIAN MODELS: NORMAL MODELS

5.1 GENERAL MODELING PRINCIPLES

5.2 MODEL SPECIFICATION IN NORMAL REGRESSION MODELS

5.3 USING VECTORS AND MULTIVARIATE PRIORS IN NORMAL REGRESSION MODELS

5.4 ANALYSIS OF VARIANCE MODELS

Problems

CHAPTER 6: INCORPORATING CATEGORICAL VARIABLES IN NORMAL MODELS AND FURTHER MODELING ISSUES

6.1 ANALYSIS OF VARIANCE MODELS USING DUMMY VARIABLES

6.2 ANALYSIS OF COVARIANCE MODELS

6.3 A BIOASSAY EXAMPLE

6.4 FURTHER MODELING ISSUES

6.5 CLOSING REMARKS

Problems

CHAPTER 7: INTRODUCTION TO GENERALIZED LINEAR MODELS: BINOMIAL AND POISSON DATA

7.1 INTRODUCTION

7.2 PRIOR DISTRIBUTIONS

7.3 POSTERIOR INFERENCE

7.4 POISSON REGRESSION MODELS

7.5 BINOMIAL RESPONSE MODELS

7.6 MODELS FOR CONTINGENCY TABLES

Problems

CHAPTER 8: MODELS FOR POSITIVE CONTINUOUS DATA, COUNT DATA, AND OTHER GLM-BASED EXTENSIONS

8.1 MODELS WITH NONSTANDARD DISTRIBUTIONS

8.2 MODELS FOR POSITIVE CONTINUOUS RESPONSE VARIABLES

8.3 ADDITIONAL MODELS FOR COUNT DATA

8.4 FURTHER GLM-BASED MODELS AND EXTENSIONS

Problems

CHAPTER 9: BAYESIAN HIERARCHICAL MODELS

9.1 INTRODUCTION

9.2 SOME SIMPLE EXAMPLES

9.3 THE GENERALIZED LINEAR MIXED MODEL FORMULATION

9.4 DISCUSSION, CLOSING REMARKS, AND FURTHER READING

Problems

CHAPTER 10: THE PREDICTIVE DISTRIBUTION AND MODEL CHECKING

10.1 INTRODUCTION

10.2 ESTIMATING THE PREDICTIVE DISTRIBUTION FOR FUTURE OR MISSING OBSERVATIONS USING MCMC

10.3 USING THE PREDICTIVE DISTRIBUTION FOR MODEL CHECKING

10.4 USING CROSS-VALIDATION PREDICTIVE DENSITIES FOR MODEL CHECKING, EVALUATION, AND COMPARISON

10.5 ILLUSTRATION OF A COMPLETE PREDICTIVE ANALYSIS: NORMAL REGRESSION MODELS

10.6 DISCUSSION

Problems

CHAPTER 11: BAYESIAN MODEL AND VARIABLE EVALUATION

11.1 PRIOR PREDICTIVE DISTRIBUTIONS AS MEASURES OF MODEL COMPARISON: POSTERIOR MODEL ODDS AND BAYES FACTORS

11.2 SENSITIVITY OF THE POSTERIOR MODEL PROBABILITIES: THE LINDLEY-BARTLETT PARADOX

11.3 COMPUTATION OF THE MARGINAL LIKELIHOOD

11.4 COMPUTATION OF THE MARGINAL LIKELIHOOD USING WinBUGS

11.5 BAYESIAN VARIABLE SELECTION USING GIBBS-BASED METHODS

11.6 POSTERIOR INFERENCE USING THE OUTPUT OF BAYESIAN VARIABLE SELECTION SAMPLERS

11.7 IMPLEMENTATION OF GIBBS VARIABLE SELECTION IN WinBUGS USING AN ILLUSTRATIVE EXAMPLE

11.8 THE CARLIN–CHIB METHOD

11.9 REVERSIBLE JUMP MCMC (RJMCMC)

11.10 USING POSTERIOR PREDICTIVE DENSITIES FOR MODEL EVALUATION

11.11 INFORMATION CRITERIA

11.12 DISCUSSION AND FURTHER READING

Problems

APPENDIX A: MODEL SPECIFICATION VIA DIRECTED ACYCLIC GRAPHS: THE DOODLE MENU

A.1 INTRODUCTION: STARTING WITH DOODLE

A.2 NODES

A.3 EDGES

A.4 PANELS

A.5 A SIMPLE EXAMPLE

APPENDIX B: THE BATCH MODE: RUNNING A MODEL IN THE BACKGROUND USING SCRIPTS

B.1 INTRODUCTION

B.2 BASIC COMMANDS: COMPILING AND RUNNING THE MODEL

APPENDIX C: CHECKING CONVERGENCE USING CODA/BOA

C.1 INTRODUCTION

C.2 A SHORT HISTORICAL REVIEW

C.3 DIAGNOSTICS IMPLEMENTED BY CODA/BOA

C.4 A FIRST LOOK AT CODA/BOA

C.5 A SIMPLE EXAMPLE

APPENDIX D: NOTATION SUMMARY

D.1 MCMC

D.2 SUBSCRIPTS AND INDICES

D.3 PARAMETERS

D.4 RANDOM VARIABLES AND DATA

D.5 SAMPLE ESTIMATES

D.6 SPECIAL FUNCTIONS, VECTORS, AND MATRICES

D.7 DISTRIBUTIONS

D.8 DISTRIBUTION-RELATED NOTATION

D.9 NOTATION USED IN ANOVA AND ANCOVA

D.10 VARIABLE AND MODEL SPECIFICATION

D.11 DEVIANCE INFORMATION CRITERION (DIC)

D.12 PREDICTIVE MEASURES

REFERENCES

INDEX

WILEY SERIES IN COMPUTATIONAL STATISTICS

Consulting Editors:

Paolo GiudiciUniversity of Pavia, Italy

Geof H. GivensColorado State University, USA

Bani K. MallickTexas A&M University, USA

Wiley Series in Computational Statistics is comprised of practical guides and cutting edge research books on new developments in computational statistics. It features quality authors with a strong applications focus. The texts in the series provide detailed coverage of statistical concepts, methods and case studies in areas at the interface of statistics, computing, and numerics.

With sound motivation and a wealth of practical examples, the books show in concrete terms how to select and to use appropriate ranges of statistical computing techniques in particular fields of study. Readers are assumed to have a basic understanding of introductory terminology.

The series concentrates on applications of computational methods in statistics to fields of bioinformatics, genomics, epidemiology, business, engineering, finance and applied statistics.

A complete list of titles in this series appears at the end of the volume.

Copyright © 2009 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data is available.

Ntzoufras, Ioannis, 1973-Bayesian modeling using WinBUGS / Ioannis Ntzoufras.p. cm.Includes bibliographical references and index.ISBN 978-0-470-14114-4 (pbk.)1. Bayesian statistical decision theory. 2. WinBUGS. I. Title.QA279.5.N89 2009519.5′42—dc22    2008033316

To Ioanna and our baby daughter

PREFACE

Since the mid-1980s, the development of widely accessible powerful computers and the implementation of Markov chain Monte Carlo (MCMC) methods have led to an explosion of interest in Bayesian statistics and modeling. This was followed by an extensive research for new Bayesian methodologies generating the practical application of complicated models used over a wide range of sciences. During the late 1990s, BUGS emerged in the foreground. BUGS was a free software that could fit complicated models in a relatively easy manner, using standard MCMC methods. Since 1998 or so, WinBUGS, the Windows version of BUGS, has earned great popularity among researchers of diverse scientific fields. Therefore, an increased need for an introductory book related to Bayesian models and their implementation via WinBUGS has been realized.

The objective of the present book is to offer an introduction to the principles of Bayesian modeling, with emphasis on model building and model implementation using WinBUGS. Detailed examples are provided, ranging from very simple to more advanced and realistic ones. Generalized linear models (GLMs), which are familiar to most students and researchers, are discussed. Details concerning model building, prior specification, writing the WinBUGS code and the analysis and interpretation of the WinBUGS output are also provided. Because of the introductory character of the book, I focused on elementary models, starting from the normal regression models and moving to generalized linear models. Even more advanced readers, familiar with such models, may benefit from the Bayesian implementation using WinBUGS.

Basic knowledge of probability theory and statistics is assumed. Computations that could not be performed in WinBUGS are illustrated using R. Therefore, a minimum knowledge of R is also required.

This manuscript can be used as the main textbook in a second-level course of Bayesian statistics focusing on modeling and/or computation. Alternatively, it can serve as a companion (to a main textbook) in an introductory course of a Bayesian statistics. Finally, because of its structure, postgraduate students and other researchers can complete a self-taught tutorial course on Bayesian modeling by following the material of this book.

All datasets and code used in the book are available in the book’s Webpage: www.stat-athens.aueb.gr/~jbn/winbugs_book.

IOANNIS NTZOUFRAS

Athens, GreeceJune 29, 2008

ACKNOWLEDGMENTS

I am indebted to the people at Wiley publications for their understanding and assistance during the preparation of the manuscript. Acknowledgments are due to the anonymous referees. Their suggestions and comments led to a substantial improvement of the present book. I would particularly like to thank Dimitris Fouskakis, colleague and good friend, for his valuable comments on an early version of chapters 1-6 and 10-11. I am also grateful to Professor Brani Vidakovic for proposing and motivating this book. Last but not least, I wish to thank my wife Ioanna for her love, support, and patience during the writing of this book as well as for her suggestions on the manuscript.

I. N.

ACRONYMS

ACFAutocorrelationAICAkaike information criterionANOVAAnalysis of varianceANCOVAAnalysis of covarianceARAttributable riskBFBayes factorBICBayes information criterionBOABayesian output analysis (R package)BPBivariate PoissonBODBiological oxygen demand (data variable in example 6.3)BUGSBayesian inference using Gibbs (software)CDFCumulative distribution functionCODChemical oxygen demand (data variable in example 6.3)CODAConvergence diagnostics and output analysis software for Gibbs sampling analysis (R package)CPOConditional Predictive OrdinateCRcorner (constraint)CVCross-validationCV-1Leave-one-out cross-validationDAGDirected acyclic graphDIDispersion indexDIBPDiagonal inflated bivariate Poisson distributionDICDeviance information criterionGLMGeneralized linear modelGPGeneralized PoissonGVSGibbs variable selectionICPOInverse conditional predictive ordinatei.i.d.Independent identically distributedLSLogarithmic scoreMAPMaximum a posterioriMP modelMedian probabilityMCMCMarkov chain Monte CarloMCEMonte Carlo errorMLMaximum likelihoodMLEMaximum-likelihood estimate/estimatorNBNegative binomialOROdds ratioPBFPosterior Bayes factorPDPoisson differencep.d.f.Probability density functionPOPosterior model oddsPPOPosterior predictive ordinateRJMCMCReversible jump Markov chain Monte CarloRRRelative riskSDStandard deviationSEStandard errorSSVSStochastic search variable selectionSTZsum-to-zero (constraint)TSTotal solids(data variable in example 6.3)TVSTotal volatile solids (data variable in example 6.3)WinBUGSWindows version of BUGS (software)ZIZero inflatedZIDZero inflated distributionZIPZero inflated Poisson distributionZINBZero inflated negative binomial distributionZIGPZero inflated generalized Poisson distributionZIBPZero inflated bivariate Poisson distribution

CHAPTER 1

INTRODUCTION TO BAYESIAN INFERENCE

1.1 INTRODUCTION: BAYESIAN MODELING IN THE 21 ST CENTURY

The beginning of the 21 st century found Bayesian statistics to be fashionable in science. But until the late 1980s, Bayesian statistics were considered only as an interesting alternative to the “classical” theory. The main difference between the classical statistical theory and the Bayesian approach is that the latter considers parameters as random variables that are characterized by a prior distribution. This prior distribution is combined with the traditional likelihood to obtain the posterior distribution of the parameter of interest on which the statistical inference is based. Although the main tool of Bayesian theory is probability theory, for many years Bayesians were considered as a heretic minority for several reasons. The main objection of “classical” statisticians was the subjective view point of the Bayesian approach introduced in the analysis via the prior distribution. However, as history had proved, the main reason why Bayesian theory was unable to establish a foothold as a well accepted quantitative approach for data analysis was the intractabilities involved in the calculation of the posterior distribution. Asymptotic methods had provided solutions to specific problems, but no generalization was possible. Until the early 1990s two groups of statisticians had (re)discovered Markov chain Monte Carlo (MCMC) methods (Gelfand and Smith, 1990; Gelfand et al., 1990). Physicists were familiar with MCMC methodology from the 1950s. Nick Metropolis and his associates had developed one of the first electronic supercomputers (for those days) and had been testing their theories in physics using Monte Carlo techniques. Implementation of the MCMC methods in combination with the rapid evolution of personal computers made the new computational tool popular within a few years. Bayesian statistics suddenly became fashionable, opening new highways for statistical research. Using MCMC, we can now set up and estimate complicated models that describe and solve problems that could not be solved with traditional methods.

Since 1990, when MCMC first appeared in statistical science, many important related papers have appeared in the literature. During 1990–1995, MCMC-related research focused on the implementation of new methods in various popular models [see, e.g., Gelman and Rubin (1992), Gelfand, Smith and Lee (1992), Gilks and Wild (1992), Dellaportas and Smith (1993)]. The development of MCMC methodology had also promoted the implementation of random effects and hierarchical models.

Green’s (1995) publication on reversible jump Markov chain Monte Carlo (RJMCMC) algorithm boosted research on model averaging, selection and model exploration algorithms [see, e.g., Dellaportas and Forster (1999), Dellaportas et al. (2002), Sisson (2005), Hans et al. (2007)]. During the same period, the early versions of BUGS software appeared. BUGS was computing-language-oriented software in which the user only needed to specify the structure of the model. Then, BUGS was using MCMC methods to generate samples from the posterior distribution of the specified model. The most popular version of BUGS (v.05) was available via the Internet in 1996 [manual date August 14, 1996; see, Spiegelhalter et al. (1996a)]. Currently WinBUGS version 1.4.31 is available via the WinBUGS project Webpage (Spiegelhalter et al., 2003d). Many add-ons, utilities, and variations of the package are also available. The development of WinBUGS had proved valuable for the implementation of Bayesian models in a wide variety of scientific disciplines. In parallel, many workshops and courses have been organized on Bayesian inference, data analysis, and modeling using WinBUGS software. WinBUGS is a key factor in the growing popularity of Bayesian methods in science.

Development, extensions, and improvement of MCMC methods have also been considered in statistical research since the mid-1990s. Automatic samplers, which will be directly applicable in any set of data, are within this frame of research and have led to the slice sampler (Higdon, 1998; Damien et al., 1999). Various samplers designed for model and variable evaluation have been also produced; for a comprehensive review, see Sisson (2005). Perfect sampling (Propp and Wilson, 1996; Møller, 1999) and population-based MCMC methods (Laskey and Myers, 2003; Jasra et al., 2007) can also be considered as interesting examples of the more recent development of MCMC algorithms.

Finally, more recent advancements in genetics have given new impetus to Bayesian theory. The generally large amount of data (in terms of both sample size and variable size) have rendered the more traditional methods inapplicable. Hence Bayesian methods, with the help MCMC methodology, are appropriate for exploration of large model and parameter spaces and tracing the most important associations; see, e.g., Yi (2004).

This book focuses on building statistical models using WinBUGS. It aims to assist students and practitioners in using WinBUGS for fitting models starting from the simpler generalized linear-type models and progressing to more realistic ones by incorporating more complicated structures in the model.