Introduction to Stochastic Processes with R E-Book

Robert P. Dobrow

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 571

Veröffentlichungsjahr: 2016

Beschreibung

An introduction to stochastic processes through the use of R

Introduction to Stochastic Processes with R is an accessible and well-balanced presentation of the theory of stochastic processes, with an emphasis on real-world applications of probability theory in the natural and social sciences. The use of simulation, by means of the popular statistical software R, makes theoretical results come alive with practical, hands-on demonstrations.

Written by a highly-qualified expert in the field, the author presents numerous examples from a wide array of disciplines, which are used to illustrate concepts and highlight computational and theoretical results. Developing readers’ problem-solving skills and mathematical maturity, Introduction to Stochastic Processes with R features:

• More than 200 examples and 600 end-of-chapter exercises
• A tutorial for getting started with R, and appendices that contain review material in probability and matrix algebra
• Discussions of many timely and stimulating topics including Markov chain Monte Carlo, random walk on graphs, card shuffling, Black–Scholes options pricing, applications in biology and genetics, cryptography, martingales, and stochastic calculus
• Introductions to mathematics as needed in order to suit readers at many mathematical levels
• A companion web site that includes relevant data files as well as all R code and scripts used throughout the book

Introduction to Stochastic Processes with R is an ideal textbook for an introductory course in stochastic processes. The book is aimed at undergraduate and beginning graduate-level students in the science, technology, engineering, and mathematics disciplines. The book is also an excellent reference for applied mathematicians and statisticians who are interested in a review of the topic.

Leseprobe

TABLE OF CONTENTS

COVER

TITLE PAGE

COPYRIGHT

DEDICATION

PREFACE

ACKNOWLEDGMENTS

LIST OF SYMBOLS AND NOTATION

NOTATION CONVENTIONS

ABBREVIATIONS

ABOUT THE COMPANION WEBSITE

CHAPTER 1: INTRODUCTION AND REVIEW

1.1 DETERMINISTIC AND STOCHASTIC MODELS

1.2 WHAT IS A STOCHASTIC PROCESS?

1.3 MONTE CARLO SIMULATION

1.4 CONDITIONAL PROBABILITY

1.5 CONDITIONAL EXPECTATION

CHAPTER 2: MARKOV CHAINS: FIRST STEPS

2.1 INTRODUCTION

2.2 MARKOV CHAIN CORNUCOPIA

2.3 BASIC COMPUTATIONS

2.4 LONG-TERM BEHAVIOR—THE NUMERICAL EVIDENCE

2.5 SIMULATION

2.6 MATHEMATICAL INDUCTION*

CHAPTER 3: MARKOV CHAINS FOR THE LONG TERM

3.1 LIMITING DISTRIBUTION

3.2 STATIONARY DISTRIBUTION

3.3 CAN YOU FIND THE WAY TO STATE

a

?

3.4 IRREDUCIBLE MARKOV CHAINS

3.5 PERIODICITY

3.6 ERGODIC MARKOV CHAINS

3.7 TIME REVERSIBILITY

3.8 ABSORBING CHAINS

3.9 REGENERATION AND THE STRONG MARKOV PROPERTY*

3.10 PROOFS OF LIMIT THEOREMS*

CHAPTER 4: BRANCHING PROCESSES

4.1 INTRODUCTION

4.2 MEAN GENERATION SIZE

4.3 PROBABILITY GENERATING FUNCTIONS

4.4 EXTINCTION IS FOREVER

CHAPTER 5: MARKOV CHAIN MONTE CARLO

5.1 INTRODUCTION

5.2 METROPOLIS–HASTINGS ALGORITHM

5.3 GIBBS SAMPLER

5.4 PERFECT SAMPLING*

5.5 RATE OF CONVERGENCE: THE EIGENVALUE CONNECTION*

5.6 CARD SHUFFLING AND TOTAL VARIATION DISTANCE*

CHAPTER 6: POISSON PROCESS

6.1 INTRODUCTION

6.2 ARRIVAL, INTERARRIVAL TIMES

6.3 INFINITESIMAL PROBABILITIES

6.4 THINNING, SUPERPOSITION

6.5 UNIFORM DISTRIBUTION

6.6 SPATIAL POISSON PROCESS

6.7 NONHOMOGENEOUS POISSON PROCESS

6.8 PARTING PARADOX

CHAPTER 7: CONTINUOUS-TIME MARKOV CHAINS

7.1 INTRODUCTION

7.2 ALARM CLOCKS AND TRANSITION RATES

7.3 INFINITESIMAL GENERATOR

7.4 LONG-TERM BEHAVIOR

7.5 TIME REVERSIBILITY

7.6 QUEUEING THEORY

7.7 POISSON SUBORDINATION

CHAPTER 8: BROWNIAN MOTION

8.1 INTRODUCTION

8.2 BROWNIAN MOTION AND RANDOM WALK

8.3 GAUSSIAN PROCESS

8.4 TRANSFORMATIONS AND PROPERTIES

8.5 VARIATIONS AND APPLICATIONS

8.6 MARTINGALES

CHAPTER 9: A GENTLE INTRODUCTION TO STOCHASTIC CALCULUS*

9.1 INTRODUCTION

9.2 ITO INTEGRAL

9.3 STOCHASTIC DIFFERENTIAL EQUATIONS

APPENDIX A: GETTING STARTED WITH R

APPENDIX B: PROBABILITY REVIEW

B.1 DISCRETE RANDOM VARIABLES

B.2 JOINT DISTRIBUTION

B.3 CONTINUOUS RANDOM VARIABLES

B.4 COMMON PROBABILITY DISTRIBUTIONS

B.5 LIMIT THEOREMS

B.6 MOMENT-GENERATING FUNCTIONS

APPENDIX C: SUMMARY OF COMMON PROBABILITY DISTRIBUTIONS

APPENDIX D: MATRIX ALGEBRA REVIEW

D.1 BASIC OPERATIONS

D.2 LINEAR SYSTEM

D.3 MATRIX MULTIPLICATION

D.4 DIAGONAL, IDENTITY MATRIX, POLYNOMIALS

D.5 TRANSPOSE

D.6 INVERTIBILITY

D.7 BLOCK MATRICES

D.8 LINEAR INDEPENDENCE AND SPAN

D.9 BASIS

D.10 VECTOR LENGTH

D.11 ORTHOGONALITY

D.12 EIGENVALUE, EIGENVECTOR

D.13 DIAGONALIZATION

ANSWERS TO SELECTED ODD-NUMBERED EXERCISES

REFERENCES

INDEX

End User License Agreement

Pages

xi

xii

xiii

xv

xvii

xviii

xix

xxi

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

475

476

477

478

479

Guide

Cover

Table of Contents

Preface

Begin Reading

List of Illustrations

CHAPTER 1: INTRODUCTION AND REVIEW

Figure 1.1 Growth of a bacteria population. The deterministic exponential growth curve (dark line) is plotted against six realizations of the stochastic process.

Figure 1.2 Five-page webgraph. Vertex labels show long-term probabilities of reaching each page.

Figure 1.3 Four outcomes of the Reed–Frost epidemic model.

Figure 1.4 Random walk and gambler's ruin.

Figure 1.5 Simulations of two-dimensional Brownian motion.

CHAPTER 2: MARKOV CHAINS: FIRST STEPS

Figure 2.1 Graph on six vertices.

Figure 2.2 (a) Cycle graph on nine vertices. (b) Complete graph on five vertices.

Figure 2.3 The -hypercube graphs for and .

Figure 2.4 Weighted graph with loops.

Figure 2.5 Markov transition graphs.

Figure 2.6 Lung cancer network as a weighted, directed graph (weights not shown).

Source:

Newton et al. (2012).

Figure 2.7 Frog starts hopping from vertex 1 of the 25-cycle graph.

Figure 2.8 Transition graph of the graduation Markov chain.

Figure 2.10

Figure 2.11

CHAPTER 3: MARKOV CHAINS FOR THE LONG TERM

Figure 3.1 Lollipop graph.

Figure 3.2 The general two-state Markov chain expressed as a weighted graph.

Figure 3.3 Transition graphs.

Figure 3.4 Transition graph for an irreducible Markov chain.

Figure 3.5

Figure 3.6

Figure 3.7 Periodic Markov chains.

Figure 3.8

Figure 3.9

Figure 3.10

Figure 3.11 Weighted graph.

Figure 3.12 Random walk with partially reflecting boundaries.

Figure 3.13 Children's Snakes and Ladders game. Board drawn using TiZ TeX package.

Source

: http://tex.stackexchange.com/questions/85411/chutes-and-ladders/. Reproduced with permission of Serge Ballif.

Figure 3.14

Figure 3.15

Figure 3.16

Figure 3.17

Figure 3.18

Figure 3.19 Mouse in a maze.

Figure 3.20

CHAPTER 4: BRANCHING PROCESSES

Figure 4.1 Branching process.

Figure 4.2 Graph of .

CHAPTER 5: MARKOV CHAIN MONTE CARLO

Figure 5.1 Weighted graph on sequences with no adjacent 1s.

Figure 5.2 Co-occurrence matrix for Darwin's finches.

Figure 5.3 Transition graph for co-occurrence matrices.

Figure 5.4 Simulated distribution of the number of checkerboards in a uniformly random co-occurrence matrix. The probability of at least 333 checkerboards is 0.0004.

Figure 5.5 Simulation of standard normal distribution with MCMC.

Figure 5.6 Bivariate standard normal simulation.

Figure 5.7 Joint and marginal distributions for trivariate distribution.

Figure 5.8 Sample configuration on grid.

Figure 5.9 Ising simulation. Parameter values are : , : , : , : .

Figure 5.10 Perfect sampling on a 4-state chain. Coalescence occurs for the chain started at time . The algorithm outputs state as a sample from the stationary distribution.

Figure 5.11 Monotone chain. All paths are sandwiched between the maximal and minimal paths.

Figure 5.12 An exact sample of the Ising model on a grid at the critical temperature .

Source:

Propp and Wilson (1996). Reproduced with permission of John Wiley and Sons, Inc.

Figure 5.13 Cutoff phenomenon for top-to-random shuffle.

Figure 5.14 Total variation distance for riffle shuffling.

CHAPTER 6: POISSON PROCESS

Figure 6.1 Counting process.

Figure 6.2 Arrival times , and interarrival times

Figure 6.3 Realizations of a Poisson process with .

Figure 6.4 Waiting time distributions for Amy and Zach. Zach arrives 10 minutes after Amy. By memorylessness, the distributions are the same.

Figure 6.5 A partition of where each subinterval contains 0 or 1 point.

Figure 6.6 With , there are no arrivals between .

Figure 6.7 The process is the superposition of , , and .

Figure 6.8 Embedding the birthday problem in a superposition of Poisson processes.

Figure 6.9 Samples of a spatial Poisson process.

Figure 6.10 Plot of tree locations in a hardwood swamp in South Carolina. Squares are locations of cypress trees and dots locations of any other species.

Source

: Dixon (2012).

Figure 6.11 Intensity function.

Figure 6.12 Pick a number from 0 to 200. Is your number in a short or long interval?

CHAPTER 7: CONTINUOUS-TIME MARKOV CHAINS

Figure 7.1 Realization of a continuous-time weather chain.

Figure 7.2 Two-state process.

Figure 7.3 Transition rates for three-state chain.

Figure 7.4 Transition rate graph for registration line Markov chain.

Figure 7.5 Realization of the size of the line at the registrar's office.

Figure 7.6 Computing the matrix exponential with

WolframAlpha. Source:

See Wolfram Alpha LLC 2015.

Figure 7.7 Estimated transition probabilities for stages of liver cirrhosis.

Source

: Bartolomeo et al. 2011.

Figure 7.8 Trees.

Figure 7.9 Birth-and-death process.

Figure 7.10 Little's formula is obtained by finding the area of the shaded region two ways.

Figure 7.11 Both graphs describe the same Markov chain. Nodes are labeled with holding time parameters, edges are labeled with transition probabilities. The graph on the right shows the chain subordinated to a Poisson process. Holding time parameters are constant, and transitions to the same state are allowed.

Figure 7.12

CHAPTER 8: BROWNIAN MOTION

Figure 8.1 Brownian motion path. Superimposed on the graph are normal density curves with mean 0 and variance .

Figure 8.2 Sample paths of Brownian motion on .

Figure 8.3 (a) Realization of a simple random walk. (b) Walk is extended to a continuous path by linear interpolation.

Figure 8.4 Top graph is a simple symmetric random walk for 100 steps. Bottom graphs show the scaled process , for .

Figure 8.5 Transformations of Brownian motion. (a) Standard Brownian motion. (b) Reflection across -axis. (c) Brownian motion started at . (d) Translation.

Figure 8.6 is the first time Brownian motion hits level .

Figure 8.7 Reflection principle.

Figure 8.8 Arcsine distribution.

Figure 8.9 Last time players are tied in 10,000 coin flips.

Figure 8.10 Brownian bridge sample paths.

Figure 8.11 Empirical distribution function for sample data.

Figure 8.12 Fifty sample paths of a stock's price over 90 days modeled as geometric Brownian motion. Dotted lines are drawn at the mean function, and the mean plus or minus two standard deviations.

Figure 8.13 First hitting time that Brownian motion hits level or .

Figure 8.14 First time Brownian motion hits the line .

Figure 8.15 First time to hit for Brownian motion with drift and variance .

CHAPTER 9: A GENTLE INTRODUCTION TO STOCHASTIC CALCULUS*

Figure 9.1 Realizations of the stochastic integral . The integral is normally distributed with mean 0 and variance .

Figure 9.2 Simulations of . The light gray curve is the underlying standard Brownian motion.

Figure 9.3 Simulation of white noise signal.

Figure 9.4 Sample paths for the logistic SDE, with , , and . Smooth curve is the deterministic logistic function.

Figure 9.5 Realizations of the Ornstein–Uhlenbeck process with and . (a) . (b) .

Figure 9.6 Sample paths for the solution of the random genetic drift SDE.

Figure 9.7 Simulating the distribution of in the random genetic drift model, for (top-left to bottom-right).

Figure 9.8

Figure 9.9

Figure 9.10

APPENDIX A: GETTING STARTED WITH R

Figure 1.1

Figure 1.2

Figure 1.3

Figure 1.4

Figure 1.5

List of Tables

CHAPTER 1: INTRODUCTION AND REVIEW

Table 1.1 Probability of Claim for Flood Damage

Table 1.2 Probability of Claim for Those Within Five Miles of Atlantic Coast

CHAPTER 2: MARKOV CHAINS: FIRST STEPS

Table 2.1 Probabilities, After Steps, of the Frog's Position Near the Top and Bottom of the 25-Cycle Graph

CHAPTER 4: BRANCHING PROCESSES

Table 4.1 Simulations of a Branching Process for Three Choices of

Table 4.2 Simulation of a Supercritical Branching Process, with . Four of the 12 runs go extinct by the 10th generation

CHAPTER 5: MARKOV CHAIN MONTE CARLO

Table 5.1 Comparison of Markov chain Monte Carlo Estimates with Exact Probabilities for Power-Law Distribution

Table 5.2 Total Variation Distance for Shuffles of 52 Cards

CHAPTER 6: POISSON PROCESS

Table 6.1 Number of Points in a Circle of Radius for a Spatial Poisson Process with

CHAPTER 7: CONTINUOUS-TIME MARKOV CHAINS

Table 7.1 Types of Birth-and-Death Processes

CHAPTER 8: BROWNIAN MOTION

Table 8.1 Random Walk and Brownian Motion Probabilities for the Last Zero ()

Table 8.2 Table for Basketball Data Probabilities that the Home Team Wins the Game Given that they are in the Lead by Points After a Fraction of the Game is Completed

Table 8.3 Results by Quarter of 493 NBA Games

APPENDIX A: GETTING STARTED WITH R

Table A.1 Logical Connectives

Table A.2 Probability Distributions in R

INTRODUCTION TO STOCHASTIC PROCESSES WITH R

ROBERT P. DOBROW

Copyright © 2016 by John Wiley & Sons, Inc. All rights reserved

Published by John Wiley & Sons, Inc., Hoboken, New Jersey

Published simultaneously in Canada

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data applied for:

Dobrow, Robert P., author.

Introduction to stochastic processes with R / Robert P. Dobrow.

pages cm

Includes bibliographical references and index.

ISBN 978-1-118-74065-1 (cloth)

1. Stochastic processes. 2. R (Computer program language) I. Title.

QC20.7.S8D63 2016

519.2′302855133–dc23

2015032706

To my family

PREFACE

The last thing one discovers in composing a work is what to put first.

—Blaise Pascal

The intended audience for this book are students who like probability. With that prerequisite, I am confident that you will love stochastic processes.

Stochastic, or random, processes is the dynamic side of probability. What differential equations is to calculus, stochastic processes is to probability. The material appeals to those who like applications and to those who like theory. It is both excellent preparation for future study, as well as a terminal course, in the sense that we do not have to tell students to wait until the next class or the next year before seeing the good stuff. This is the good stuff! Stochastic processes, as a branch of probability, speaks in the language of rolling dice, flipping coins, and gambling games, but in the service of applications as varied as the spread of infectious diseases, the evolution of genetic sequences, models for climate change, and the growth of the World Wide Web.

The book assumes that the reader has taken a calculus-based probability course and is familiar with matrix algebra. Conditional probability and conditional expectation, which are essential tools, are offered in the introductory chapter, but may be skimmed over depending upon students' background. Some topics assume a greater knowledge of linear algebra than basic matrices (such as eigenvalues and eigenvectors) but these are optional, and relevant sections are starred. The book does not assume background in combinatorics, differential equations, or real analysis. Necessary mathematics is introduced as needed.

A focus of this book is the use of simulation. I have chosen to use the popular statistical freeware R, which is an accessible interactive computing environment. The use of simulation, important in its own right for applied work and mathematical research, is a powerful pedagogical tool for making theoretical concepts come alive with practical, hands-on demonstrations. It is not necessary to use R in order to use this book; code and script files are supplemental. However, the software is easy—andfun—to learn, and there is a tutorial and exercises in an appendix for bringing students up to speed.

The book contains more than enough material for a standard one-semester course. Several topics may lend themselves to individual or group projects, such as card shuffling, perfect sampling (coupling from the past), queueing theory, stochastic calculus, martingales, and stochastic differential equations. Such specialized material is contained in starred sections.

An undergraduate textbook poses many challenges. I have struggled with trying to find the right balance between theory and application, between conceptual understanding and formal proof. There are, of course, some things that cannot be said. Continuous-time processes, in particular, require advanced mathematics based on measure theory to be made precise. Where these subjects are presented I have emphasized intuition over rigor.

Following is a synopsis of the book's nine chapters.

Chapter 1 introduces stochastic and deterministic models, the generic features of stochastic processes, and simulation. This is essential material. The second part of the chapter treats conditional probability and conditional expectation, which can be reviewed at a fast pace.

The main features of discrete-time Markov chains are covered in Chapters 2 and 3. Many examples of Markov chains are introduced, and some of them are referenced throughout the book. Numerical and simulation-based methods motivate the discussion of limiting behavior. In addition to basic computations, topics include stationary distributions, ergodic and absorbing chains, time reversibility, and the strong Markov property. Several important limit theorems are discussed in detail, with proofs given at the end of the chapter. Instructors may choose to limit how much time is spent on proofs.

Branching processes are the topic of Chapter 4. Although branching processes are Markov chains, the methods of analysis are different enough to warrant a separate chapter. Probability-generating functions are introduced, and do not assume prior exposure.

The focus of Chapter 5 is Markov chain Monte Carlo, a relatively new topic but one with exponentially growing application. Instructors will find many subjects to pick and choose. Several case studies make for excellent classroom material, in particular (i) a randomized method for decoding text, from Diaconis (2009), and (ii) an application that combines ecology and counting matrices with fixed row and column totals, based on Cobb and Chen (2003). Other topics include coupling from the past, card shuffling, and rates of convergence of Markov chains.

Chapter 6 is devoted to the Poisson process. The approach emphasizes three alternate definitions and characterizations, based on the (i) counting process, (ii) arrival process, and (iii) infinitesimal description. Additional topics are spatial processes, nonhomogeneous Poisson processes, embedding, and arrival time paradoxes.

Continuous-time Markov chains are discussed in Chapter 7. For continuous-time stochastic processes, here and in Chapter 8, there is an emphasis on intuition, examples, and applications. In addition to basic material, there are sections onqueueing theory (with Little's formula), absorbing processes, and Poisson subordination.

Brownian motion is the topic of Chapter 8. The material is more challenging. Topics include the invariance principle, transformations, Gaussian processes, martingales, and the optional stopping theorem. Examples include scoring in basketball and animal tracking. The Black–Scholes options pricing formula is derived.

Chapter 9 is a gentle introduction to stochastic calculus. Gentle means no measure theory, sigma fields, or filtrations, but an emphasis on examples and applications. I decided to include this material because of its growing popularity and application. Stochastic differential equations are introduced. Simulation and numerical methods help make the topic accessible.

Book appendices include (i) getting started with R, with exercises, (ii) probability review, with short sections on the main discrete and continuous probability distributions, (iii) summary table of common probability distributions, and (iv) matrix algebra review. Resources for students include a suite of R functions and script files for generating many of the processes from the book.

The book contains more than 200 examples, and about 600 end-of-chapter exercises. Short solutions to most odd-numbered exercises are given at the end of the book. A web site www.people.carleton.edu/rdobrow/stochbook is established. It contains errata and relevant files. All the R code and script files used in the book are available at this site. A solutions manual with detailed solutions to all exercises is available for instructors.

Much of this book is a reflection of my experience teaching the course over the past 10 years. Here is a suggested one-semester syllabus, which I have used.

1.

Introduction and review—1.1, 1.2, 1.3 (quickly skim 1.4 and 1.5)

2.

One-day introduction to

R

—Appendix A

3.

Markov chains—All of

chapters 2

and

3

4.

Branching processes—

Chapter 4

5.

MCMC—5.1, 5.2

6.

Poisson process—6.1, 6.2, 6.4, 6.5, 6.8

7.

Continuous-time Markov chains—7.1, 7.2, 7.3, 7.4

8.

Brownian motion—8.1, 8.2, 8.3, 8.4, 8.5, 8.7

If instructors have questions on syllabus, homework assignments, exams, or projects, I am happy to share resources and experiences teaching this most rewarding course.

Stochastic Processes is a great mathematical adventure. Bon voyage!

ACKNOWLEDGMENTS

I am indebted to family, friends, students, and colleagues who encouraged, supported, and inspired this project. My past students are all responsible, in some measure, for this work. The students of my Spring 2015 Introduction to Stochastic Processes class at Carleton College field-tested the first draft. Those who found errors in the text, and thus earned extra credit points (some quite ample), were Taeyoung Choi, Elsa Cristofaro, Graham Earley, Michelle Marinello, Il Shan Ng, Risako Owan, John Pedtke, and Maggie Sauer. Also, Edward Heo, Jack Hessle, Alex Trautman, and Zach Wood-Doughty contributed to case studies and examples, R code, and aesthetic design.

Professors Bill Peterson at Middlebury College, and Michele Joynor at East Tennessee State University, gave helpful feedback based on field-testing the manuscript in their classes. Others who offered suggestions and encouragement include David Aldous, Laura Chihara, Hosam Mahmoud, Jack O'Brien, Jacob Spear, Jeff Rosenthal, and John Wierman.

Carleton College and the Department of Mathematics and Statistics were enormously supportive. Thanks to Sue Jandro, the Department's Administrative Assistant, and Mike Tie, the Department's Technical Director, for help throughout the past 2 years.

The staff at Wiley, including Steve Quigley, Amy Hendrickson, Jon Gurstelle, and Sari Friedman, provided encouragement and assistance in preparing this book.

I am ever grateful to my family, who gave loving support. My wife Angel was always enthusiastic and gave me that final nudge to “go for it” when I was still in the procrastination stage. My sons Tom and Joe read many sections, including the preface and introduction, with an eagle's eye to style and readability. Special thanks to my son, Danny, and family friend Kellen Kirchberg, for their beautiful design of the front cover.

LIST OF SYMBOLS AND NOTATION

Such is the advantage of a well constructed language that its simplified notation often becomes the source of profound theories.

—Pierre-Simon Laplace

empty set

union of sets

and

intersection of sets

and

is a subset of

is an element of set

binomial coefficient

floor of

ceiling of

length of vector

matrix inverse of

matrix transpose of

matrix exponential of

standard Brownian motion

correlation of

and

covariance of