Computational Statistics E-Book

Geof H. Givens

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Wiley Series in Computational Statistics
• Sprache: Englisch

Seitenzahl: 916

Veröffentlichungsjahr: 2012

Beschreibung

This new edition continues to serve as a comprehensive guide to modern and classical methods of statistical computing. The book is comprised of four main parts spanning the field: * Optimization * Integration and Simulation * Bootstrapping * Density Estimation and Smoothing Within these sections,each chapter includes a comprehensive introduction and step-by-step implementation summaries to accompany the explanations of key methods. The new edition includes updated coverage and existing topics as well as new topics such as adaptive MCMC and bootstrapping for correlated data. The book website now includes comprehensive R code for the entire book. There are extensive exercises, real examples, and helpful insights about how to use the methods in practice.

Leseprobe

Contents

Cover

Wiley Series in Computational Statistics

Title Page

Copyright

Dedication

Preface

Acknowledgements

Chapter 1: Review

1.1 Mathematical Notation

1.2 Taylor's Theorem and Mathematical Limit Theory

1.3 Statistical Notation and Probability Distributions

1.4 Likelihood Inference

1.5 Bayesian Inference

1.6 Statistical Limit Theory

1.7 Markov Chains

1.8 Computing

Part I: Optimization

Chapter 2: Optimization and Solving Nonlinear Equations

2.1 Univariate Problems

2.2 Multivariate Problems

Chapter 3: Combinatorial Optimization

3.1 Hard Problems and NP-Completeness

3.2 Local Search

3.3 Simulated Annealing

3.4 Genetic Algorithms

3.5 Tabu Algorithms

Chapter 4: Em Optimization Methods

4.1 Missing Data, Marginalization, and Notation

4.2 The EM Algorithm

4.3 EM Variants

Part II: Integration and Simulation

Chapter 5: Numerical Integration

5.1 Newton–Côtes Quadrature

5.2 Romberg Integration

5.3 Gaussian Quadrature

5.4 Frequently Encountered Problems

Chapter 6: Simulation and Monte Carlo Integration

6.1 Introduction to the Monte Carlo Method

6.2 Exact Simulation

6.3 Approximate Simulation

6.4 Variance Reduction Techniques

Chapter 7: Markov Chain Monte Carlo

7.1 Metropolis–Hastings Algorithm

7.2 Gibbs Sampling

7.3 Implementation

Chapter 8: Advanced Topics in MCMC

8.1 Adaptive MCMC

8.2 Reversible Jump MCMC

8.3 Auxiliary Variable Methods

8.4 Other Metropolis–Hastings Algorithms

8.5 Perfect Sampling

8.6 Markov Chain Maximum Likelihood

8.7 Example: MCMC for Markov Random Fields

Part III: Bootstrapping

Chapter 9: Bootstrapping

9.1 The Bootstrap Principle

9.2 Basic Methods

9.3 Bootstrap Inference

9.4 Reducing Monte Carlo Error

9.5 Bootstrapping Dependent Data

9.6 Bootstrap Performance

9.7 Other Uses of the Bootstrap

9.8 Permutation Tests

Part IV: Density Estimation and Smoothing

Chapter 10: Nonparametric Density Estimation

10.1 Measures of Performance

10.2 Kernel Density Estimation

10.3 Nonkernel Methods

10.4 Multivariate Methods

Chapter 11: Bivariate Smoothing

11.1 Predictor–Response Data

11.2 Linear Smoothers

11.3 Comparison of Linear Smoothers

11.4 Nonlinear Smoothers

11.5 Confidence Bands

11.6 General Bivariate Data

Chapter 12: Multivariate Smoothing

12.1 Predictor–Response Data

12.2 General Multivariate Data

Data Acknowledgments

References

Index

Wiley Series in Computational Statistics

Wiley Series in Computational Statistics

Consulting Editors:

Paolo GiudiciUniversity of Pavia, Italy

Geof H. GivensColorado State University, USA

Bani K. MallickTexas &M University, USA

Wiley Series in Computational Statistics is comprised of practicalguides and cutting edge research books on new developments in computationalstatistics. It features quality authors with a strong applications focus. Thetexts in the series provide detailed coverage of statistical concepts, methods, and case studies in areas at the interface of statistics, computing, andnumerics.

With sound motivation and a wealth of practical examples, the books showinconcrete terms how to select and to use appropriate ranges of statisticalcomputing techniques in particular fields of study. Readers are assumed tohave a basic understanding of introductory terminology.

The series concentrates on applications of computational methods instatistics to fieldsof bioformatics, genomics, epidemiology, business, engineering, finance, andapplied statistics.

Copyright © 2013 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Givens, Geof H.

Computational statistics / Geof H. Givens, Jennifer A. Hoeting. –2nd ed.

p. cm.

Includes index.

ISBN 978-0-470-53331-4 (cloth)

1. Mathematical statistics–Data processing. I. Hoeting, Jennifer A. (Jennifer Ann), 1966– II. Title.

QA276.4.G58 2013

519.5–dc23

2012017381

To Natalie and Neil

Preface

This book covers most topics needed to develop a broad and thorough working knowledge of modern computational statistics. We seek to develop a practical understanding of how and why existing methods work, enabling readers to use modern statistical methods effectively. Since many new methods are built from components of existing techniques, our ultimate goal is to provide scientists with the tools they need to contribute new ideas to the field.

A growing challenge in science is that there is so much of it. While the pursuit of important new methods and the honing of existing approaches is a worthy goal, there is also a need to organize and distill the teeming jungle of ideas. We attempt to do that here. Our choice of topics reflects our view of what constitutes the core of the evolving field of computational statistics, and what will be interesting and useful for our readers.

Our use of the adjective modern in the first sentence of this preface is potentially troublesome: There is no way that this book can cover all the latest, greatest techniques. We have not even tried. We have instead aimed to provide a reasonably up-to-date survey of a broad portion of the field, while leaving room for diversions and esoterica.

The foundations of optimization and numerical integration are covered in this book. We include these venerable topics because (i) they are cornerstones of frequentist and Bayesian inference; (ii) routine application of available software often fails for hard problems; and (iii) the methods themselves are often secondary components of other statistical computing algorithms. Some topics we have omitted represent important areas of past and present research in the field, but their priority here is lowered by the availability of high-quality software. For example, the generation of pseudo-random numbers is a classic topic, but one that we prefer to address by giving students reliable software. Finally, some topics (e.g., principal curves and tabu search) are included simply because they are interesting and provide very different perspectives on familiar problems. Perhaps a future researcher may draw ideas from such topics to design a creative and effective new algorithm.

In this second edition, we have both updated and broadened our coverage, and we now provide computer code. For example, we have added new MCMC topics to reflect continued activity in that popular area. A notable increase in breadth is our inclusion of more methods relevant for problems where statistical dependency is important, such as block bootstrapping and sequential importance sampling. This second edition provides extensive new support in R. Specifically, code for the examples in this book is available from the book website www.stat.colostate.edu/computationalstatistics.

Our target audience includes graduate students in statistics and related fields, statisticians, and quantitative empirical scientists in other fields. We hope such readers may use the book when applying standard methods and developing new methods.

The level of mathematics expected of the reader does not extend much beyond Taylor series and linear algebra. Breadth of mathematical training is more helpful than depth. Essential review is provided in Chapter 1. More advanced readers will find greater mathematical detail in the wide variety of high-quality books available on specific topics, many of which are referenced in the text. Other readers caring less about analytical details may prefer to focus on our descriptions of algorithms and examples.

The expected level of statistics is equivalent to that obtained by a graduate student in his or her first year of study of the theory of statistics and probability. An understanding of maximum likelihood methods, Bayesian methods, elementary asymptotic theory, Markov chains, and linear models is most important. Many of these topics are reviewed in Chapter 1.

With respect to computer programming, we find that good students can learn as they go. However, a working knowledge of a suitable language allows implementation of the ideas covered in this book to progress much more quickly. We have chosen to forgo any language-specific examples, algorithms, or coding in the text. For those wishing to learn a language while they study this book, we recommend that you choose a high-level, interactive package that permits the flexible design of graphical displays and includes supporting statistics and probability functions, such as R and MATLAB.1 These are the sort of languages often used by researchers during the development of new statistical computing techniques, and they are suitable for implementing all the methods we describe, except in some cases for problems of vast scope or complexity. We use R and recommend it. Although lower-level languages such as C++ could also be used, they are more appropriate for professional-grade implementation of algorithms after researchers have refined the methodology.

The book is organized into four major parts: optimization (Chapters 2, 3, and 4), integration and simulation (Chapters 5, 6, 7, and 8), bootstrapping (Chapter 9) and density estimation and smoothing (Chapters 10, 11, and 12). The chapters are written to stand independently, so a course can be built by selecting the topics one wishes to teach. For a one-semester course, our selection typically weights most heavily topics from Chapters 2, 3, 6, 7, 9, 10, and 11. With a leisurely pace or more thorough coverage, a shorter list of topics could still easily fill a semester course. There is sufficient material here to provide a thorough one-year course of study, notwithstanding any supplemental topics one might wish to teach.

A variety of homework problems are included at the end of each chapter. Some are straightforward, while others require the student to develop a thorough understanding of the model/method being used, to carefully (and perhaps cleverly) code a suitable technique, and to devote considerable attention to the interpretation of results. A few exercises invite open-ended exploration of methods and ideas. We are sometimes asked for solutions to the exercises, but we prefer to sequester them to preserve the challenge for future students and readers.

The datasets discussed in the examples and exercises are available from the book website, www.stat.colostate.edu/computationalstatistics. The R code is also provided there. Finally, the website includes an errata. Responsibility for all errors lies with us.

Note

1.R is available for free from www.r-project.org. Information about MATLAB can be found at www.mathworks.com.

Acknowledgments

The course upon which this book is based was developed and taught by us at Colorado State University from 1994 onwards. Thanks are due to our many students who have been semiwilling guinea pigs over the years. We also thank our colleagues in the Statistics Department for their continued support. The late Richard Tweedie merits particular acknowledgment for his mentoring during the early years of our careers.

We owe a great deal of intellectual debt to Adrian Raftery, who deserves special thanks not only for his teaching and advising, but also for his unwavering support and his seemingly inexhaustible supply of good ideas. In addition, we thank our influential advisors and teachers at the University of Washington Statistics Department, including David Madigan, Werner Stuetzle, and Judy Zeh. Of course, each of our chapters could be expanded into a full-length book, and great scholars have already done so. We owe much to their efforts, upon which we relied when developing our course and our manuscript.

Portions of the first edition were written at the Department of Mathematics and Statistics, University of Otago, in Dunedin, New Zealand, whose faculty we thank for graciously hosting us during our sabbatical in 2003. Much of our work on the second edition was undertaken during our sabbatical visit to the Australia Commonwealth Scientific and Research Organization in 2009–2010, sponsored by CSIRO Mathematics, Informatics and Statistics, and hosted at the Longpocket Laboratory in Indooroopilly, Australia. We thank our hosts and colleagues there for their support.

Our manuscript has been greatly improved through the constructive reviews of John Bickham, Ben Bird, Kate Cowles, Jan Hannig, Alan Herlihy, David Hunter, Devin Johnson, Michael Newton, Doug Nychka, Steve Sain, David W. Scott, N. Scott Urquhart, Haonan Wang, Darrell Whitley, and eight anonymous referees. We also thank the sharp-eyed readers listed in the errata for their suggestions and corrections. Our editor Steve Quigley and the folks at Wiley were supportive and helpful during the publication process. We thank Nélida Pohl for permission to adapt her photograph in the cover design of the first edition. We thank Melinda Stelzer for permission to use her painting “Champagne Circuit,” 2001, for the cover of the second edition. More about her art can be found at www.facebook.com/geekchicart. We also owe special note of thanks to Zube (a.k.a. John Dzubera), who kept our own computers running despite our best efforts to the contrary.

Funding from National Science Foundation (NSF) CAREER grant #SBR-9875508 was a significant source of support for the first author during the preparation of the first edition. He also thanks his colleagues and friends in the North Slope Borough, Alaska, Department of Wildlife xvii Management for their longtime research support. The second author gratefully acknowledges the support of STAR Research Assistance Agreement CR-829095 awarded to Colorado State University by the U.S. Environmental Protection Agency (EPA). The views expressed here are solely those of the authors. NSF and EPA do not endorse any products or commercial services mentioned herein.

Finally, we thank our parents for enabling and supporting our educations and for providing us with the “stubbornness gene” necessary for graduate school, the tenure track, or book publication—take your pick! The second edition is dedicated to our kids, Natalie and Neil, for continuing to show us what is important and what is not.

Geof H. GivensJennifer A. Hoeting