Probabilistic Forecasts and Optimal Decisions E-Book

Roman Krzysztofowicz

• Herausgeber: John Wiley & Sons
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 1062

Veröffentlichungsjahr: 2024

Beschreibung

Account for uncertainties and optimize decision-making with this thorough exposition

Decision theory is a body of thought and research seeking to apply a mathematical-logical framework to assessing probability and optimizing decision-making. It has developed robust tools for addressing all major challenges to decision making. Yet the number of variables and uncertainties affecting each decision outcome, many of them beyond the decider’s control, mean that decision-making is far from a ‘solved problem’. The tools created by decision theory remain to be refined and applied to decisions in which uncertainties are prominent.

Probabilistic Forecasts and Optimal Decisions introduces a theoretically-grounded methodology for optimizing decision-making under conditions of uncertainty. Beginning with an overview of the basic elements of probability theory and methods for modeling continuous variates, it proceeds to survey the mathematics of both continuous and discrete models, supporting each with key examples. The result is a crucial window into the complex but enormously rewarding world of decision theory.

Readers of Probablistic Forecasts and Optimal Decisions will also find:

• Extended case studies supported with real-world data
• Mini-projects running through multiple chapters to illustrate different stages of the decision-making process
• End of chapter exercises designed to facilitate student learning

Probabilistic Forecasts and Optimal Decisions is ideal for advanced undergraduate and graduate students in the sciences and engineering, as well as predictive analytics and decision analytics professionals.

Leseprobe

Table of Contents

Cover

Table of Contents

Title Page

Copyright

Dedication

Preface

About the Companion Website

1 Forecast–Decision Theory

1.1 Decision Problem

1.2 Forecast–Decision System

1.3 Rational Deciding

1.4 Mathematical Modeling

1.5 Notes on Using the Book

Bibliographical Notes

Part I: Elements of Probability

2 Basic Elements

2.1 Sets and Functions

2.2 Variates and Sample Spaces

2.3 Distributions

2.4 Moments

2.5 The Uniform Distribution

2.6 The Gaussian Distributions

2.7 The Gamma Function

2.8 The Incomplete Gamma Function

Exercises

3 Distribution Modeling

3.1 Distribution Modeling Methodology

3.2 Constructing Empirical Distribution

3.3 Specifying the Sample Space

3.4 Hypothesizing Parametric Models

3.5 Estimating Parameters

3.6 Evaluating Goodness of Fit

3.7 Illustration of Modeling Methodology

3.8 Derived Distribution Theory

Exercises

Mini-Projects

Part II: Discrete Models

4 Judgmental Forecasting

4.1 A Perspective on Probability

4.2 Judgmental Probability

4.3 Forecasting Fraction of Events

4.4 Revising Probability Sequentially

4.5 Analysis of Judgmental Task

Historical Notes

Bibliographical Notes

Exercises

Mini-Projects

5 Statistical Forecasting

5.1 Bayesian Forecaster

5.2 Samples and Examples

5.3 Modeling and Estimation

5.4 An Application

5.5 Informativeness of Predictor

Bibliographical Notes

Exercises

Mini-Projects

6 Verification of Forecasts

6.1 Data and Inputs

6.2 Calibration

6.3 Informativeness

6.4 Verification Scores

6.5 Forecast Attributes and Mental Processes

6.6 Concepts and Proofs

Bibliographical Notes

Exercises

Mini-Projects

7 Detection-Decision Theory

7.1 Prototypical Decision Problems

7.2 Basic Decision Model

7.3 Decision with Perfect Forecast

7.4 Decision Model with Forecasts

7.5 Informativeness of Forecaster

7.6 Concepts and Proofs

Bibliographical Notes

Exercises

Mini-Projects

8 Various Discrete Models

8.1 Search Planning Model

8.2 Flash-Flood Warning Model

Exercises

Mini-Projects

Part III: Continuous Models

9 Judgmental Forecasting

9.1 A Perspective on Forecasting

9.2 Judgmental Distribution Function

9.3 Parametric Distribution Function

9.4 Group Forecasting

9.5 Adjusting Distribution Function

9.6 Applications

9.7 Judgment, Data, Analytics

9.8 Concepts and Proofs

Bibliographical Notes

Exercises

Mini–Projects

10 Statistical Forecasting

10.1 Bayesian Forecaster

10.2 Bayesian Gaussian Forecaster

10.3 Estimation and Validation

10.4 Informativeness of Predictor

10.5 Communication of Probabilistic Forecast

10.6 Application

10.7 Forecaster of the Sum of Two Variates

10.8 Prior and Posterior Sums

10.9 Concepts and Proofs

Bibliographical Notes

Exercises

Mini-Projects

11 Verification of Forecasts

11.1 Data and Inputs

11.2 Calibration

11.3 Informativeness

11.4 Verification of Bayesian Forecaster

11.5 Analysis of Judgmental Task

11.6 Applications

11.7 Concepts and Proofs

Bibliographical Notes

Exercises

Mini-Projects

12 Target-Decision Theory

12.1 Target-Setting Problem

12.2 Two-Piece Linear Opportunity Loss

12.3 Incomplete Expectations

12.4 Quadratic Difference Opportunity Loss

12.5 Impulse Utility

12.6 Implications for Analysts

12.7 Weapon-Aiming Model

12.8 Weapon-Aiming-with-Friend Model

12.9 General Modeling Methodology

12.10 General Forecast–Decision System

Bibliographical Notes

Exercises

13 Inventory and Capacity Models

13.1 Inventory Systems

13.2 Basic Inventory Model

13.3 Model with Initial Stock Level

13.4 Capacity Planning Model

13.5 Inventory and Macroeconomy

13.6 Concepts and Proofs

Exercises

Mini-Projects

14 Investment Models

14.1 Investment Choice Problem

14.2 Stochastic Dominance Relation

14.3 Utility Function

14.4 Investment Choice Model

14.5 Capital Allocation Model

14.6 Portfolio Design Model

14.7 Concepts and Proofs

Bibliographical Notes

Exercises

Mini-Projects

15 Various Continuous Models

15.1 Asking Price Model

15.2 Yield Control Model: Airline Reservations

15.3 Yield Control Model: College Admissions

Exercises

Mini-Projects

A Rationality Postulates

A.1 Postulates

A.2 Implications

A.3 Training for Rationality

Bibliographical Notes

B Parameter Estimation Methods

B.1 Method of Least Squares

B.2 Method of Uniform Distance

B.3 Method of Moments

B.4 Other Methods

C Special Univariate Distributions

C.1 Usage Guide

C.2 Distributions on the Unbounded Interval

C.3 Distributions on a Bounded-Below Interval

C.4 Distributions on a Bounded Interval

C.5 Distributions on a Bounded-Above Interval

The Greek Alphabet

References

Index

End User License Agreement

List of Tables

Chapter 2

Table 2.1 The uniform distribution.

Table 2.2 Rational function approximation to the probability of the stan...

Table 2.3 Rational function approximation to the standard normal quantile

Table 2.4 The normal distribution.

Table 2.5 The log-normal distribution.

Table 2.6 The log-ratio normal distribution.

Table 2.7 The reflected log-normal distribution.

Table 2.8 Polynomial approximation to the gamma function .

Table 2.9 Asymptotic approximation to the complementary incomplete gamma fun...

Table 2.10 A guide to the magnitude of : for a given , there are two thres...

Chapter 3

Table 3.1 Plotting positions calculated from three formulae for a sample o...

Table 3.2 Critical values for the Kolmogorov-Smirnov goodness-of-fit test....

Table 3.3 Calculation of the absolute differences for the Kolmogorov-Smirnov...

Table 3.4 Total procedure for modeling distribution function of variate ,...

Table 3.5 Estimation, via the least squares method, of the scale parameter

Table 3.6 Evaluation of the goodness of fit of the

reflected

exponential dis...

Table 3.7 Scores from four final examinations received by students in the co...

Table 3.8 Fuel mileage [mpg] of passenger cars developing between 220 and 33...

Table 3.9 The diameter [cm] of a log.

Table 3.10 Flood crests recorded by a river gauge on the Tygart Valley River...

Table 3.11 Return rate from a stock, as the percentage of the return rate fr...

Table 3.12 Regional average yield of soybeans in the state of Mato Grosso, B...

Chapter 4

Table 4.1 A subset of the London Olympics data published by

The Wall Street

...

Table 4.2 The sign rules of interaction between the information factor and...

Table 4.3 Results for Example 4.10 of sequential revision of the prior proba...

Table 4.4 Statistics of the judgmental probabilities assessed in Task 4.1, r...

Table 4.5 General knowledge hypotheses.

Table 4.6 General knowledge hypotheses.

Table 4.7 Probabilistic forecast for a track meet by expert : college who...

Chapter 5

Table 5.1 Structure of the data: a long prior sample of predictand ; a shor...

Table 5.2 Example of forecasts on three days.

Table 5.3 Sample sizes and estimates of the prior probability of precipita...

Table 5.4 Randomly generated joint sample of the coordinates , in meters,...

Table 5.5 Record of the joint temperature () in degrees Fahrenheit, and the...

Table 5.6 Record of scores from two tests of spoken English taken by 65 fore...

Table 5.7 A prior sample at Buffalo from January–February 1998–1999; first ...

Table 5.8 A joint sample at Buffalo from January–February 2000–2001; first c...

Table 5.9 Joint sample I of size from Pima Indians Diabetes Database: , a...

Table 5.10 Joint sample II of size from Pima Indians Diabetes Database: ,...

Chapter 6

Table 6.1 Format of the contingency table for verification of probabilistic ...

Table 6.2 Example of the contingency table for verification of probabilistic...

Table 6.3 Estimates of the prior probability obtained from different prior...

Table 6.4 Example of the conditional probability functions , of the proba...

Table 6.5 Example of the expected probability function of the probability ...

Table 6.6 Example of the expected probability function of the probability ...

Table 6.7 Consequences of the detection decision, conditional on event.

Table 6.8 ROC tableau: calculation of the probabilities of detection and t...

Table 6.9 ROC tableau for a perfect forecaster and an uninformative forecast...

Table 6.10 ROC tableau illustrating three special cases: a nonmonotone seque...

Table 6.11 Joint sample I of forecast probability and actual snowfall fr...

Table 6.12 Joint sample II of forecast probability and actual snowfall f...

Chapter 7

Table 7.1 Disutilities of outcomes resulting from each pair of decision an...

Table 7.2 Labels attached to decision , conditional on event : quiet state...

Chapter 8

Table 8.1 Revision of the prior probability function into the posterior pr...

Table 8.2 Three-stage revision of the prior probability function into the ...

Table 8.3 One-stage revision of the prior probability function into the po...

Table 8.4 Observable events and their counts that characterize the perfo...

Table 8.5 Consequences comprising the outcome of each decision–event pair ...

Table 8.6 Performance states of a flash-flood warning system: detection (),...

Chapter 9

Table 9.1 Estimation of the normal distribution from quantiles.

Table 9.2 Estimation of the log-normal distribution from quantiles.

Table 9.3 Estimation of the log-ratio normal distribution from quantiles.

Table 9.4 Estimation of the reflected log-normal distribution from quantiles...

Table 9.5 Modeling the judgmental distribution function of predictand ; t...

Chapter 10

Table 10.1 Prior sample of the predictand and joint sample of the predicto...

Table 10.2 Estimates of parameters of the Bayesian Gaussian forecaster obtai...

Table 10.3 Validation of the estimated distribution functions for the Bayesi...

Table 10.4 Three probabilistic forecasts for the general public in the simpl...

Table 10.5 National average price of regular grade gasoline [$/gallon] and m...

Table 10.6 Measurements of the seasonal (April–July) snowmelt runoff volume ...

Table 10.7 Estimates of the seasonal (April–July) snowmelt runoff volume in ...

Table 10.8 Measurements of the seasonal (April–July) snowmelt runoff volume ...

Table 10.9 Estimates of the seasonal (April–July) snowmelt runoff volume in ...

Chapter 11

Table 11.1 Summary of the calibration analysis for the minimal (just three q...

Table 11.2 Output from the uniform calibration algorithm applied to the join...

Table 11.3 Samples of ...

Table 11.4 Empirical distribution functions: ...

Table 11.5 Output from the in-the-margin calibration algorithm applied to th...

Table 11.6 Calibration of four groups of forecasters: each forecaster assess...

Table 11.7 Calibration of group VE of forecasters (see Table 11.6) with resp...

Table 11.8 Calibration of two groups of auditors with respect to a predictan...

Table 11.9 Calibration of a group of meteorologists with respect to each pre...

Table 11.10 Calibration reports from an experiment on probabilistic forecast...

Table 11.11 Record of — the forecast quantiles, and of — the weekly chan...

Chapter 12

Table 12.1 Under the two-piece linear opportunity loss function, the ratio

Table 12.2 Analytic solutions to estimation, or target-setting, problems wit...

Table 12.3 The optimal decision in an estimation, or target-setting, probl...

Chapter 13

Table 13.1 Optimal reliability of supply , and optimal probability of short...

Chapter 14

Table 14.1 Solution to an investment choice problem with four normal distrib...

Table 14.2 Solution to a capital allocation problem with four normal distrib...

Table 14.3 Solution to a capital allocation problem when the capital is cons...

Table 14.4 Impact of correlation between the return rates on the optimal s...

Table 14.5 Two cases of portfolio choice: the alternatives come from Table...

Table 14.6 Parameter values of the distribution NM of outcome from an inv...

Table 14.7 Simplified example of a generally suggested interval for allocati...

Table 14.8 Mean annual return rates [%], as defined in Section 14.5.1, for...

Table 14.9 Estimates of utility parameter for potential young investors ob...

Chapter 15

Table 15.1 Record of fall admissions of the first-year undergraduate applica...

A

Table A.1 Demonstration that incoherence implies irrationality. The conseque...

Table A.2 Percentage of subjects in each group who chose the particular two ...

C

Table C.1 Special parametric distributions of continuous variates.

List of Illustrations

Chapter 1

Figure 1.1 Forecast–decision system — a cascade coupling of the forecast sys...

Figure 1.2 The main connections between chapters. Parts II and III are indep...

Chapter 2

Figure 2.1 The normal distribution NM. With parameter values and , it is...

Figure 2.2 The log-normal distribution LN. The density value at the mode

Figure 2.3 The log-ratio normal distribution LR1-NM . The density function...

Figure 2.4 The log-ratio normal distribution LR1-NM. The density function

Figure 2.5 The reflected log-normal distribution LN. The density value at...

Figure 2.6 The gamma function .

Chapter 3

Figure 3.1 Empirical distribution function of a continuous variate constru...

Figure 3.2 A hypothesized distribution function of a continuous variate ...

Figure 3.3 Elements of the Kolmogorov-Smirnov test of a hypothesized distrib...

Figure 3.4 Linearized quantile function of the hypothesized distribution, ...

Figure 3.5 Empirical distribution function of travel time constructed from...

Figure 3.6 A simple system with input ...

Figure 3.7 A transformation ...

Figure 3.8 The reflected exponential distribution function of variate EX

Chapter 4

Figure 4.1 Judgmental task of assessing probability.

Figure 4.2 Validation of internal coherence: if events and are judged eq...

Figure 4.3 Sequential Bayesian revision of the prior probability of event

Figure 4.4 Stochastic dependence (SD) structures in the sequential Bayesian ...

Figure 4.5 Stochastic dependence (SD) structures between event (or hypothe...

Figure 4.6 Schemata of two-step revision of the prior probability into the...

Chapter 5

Figure 5.1 A schema of the Bayesian forecaster of binary predictand using ...

Figure 5.2 Empirical distribution functions of the relative vorticity , c...

Figure 5.3 Parametric distribution functions of the relative vorticity , ...

Figure 5.4 Parametric conditional distribution functions from Figure 5.3...

Figure 5.5 Conditional density functions corresponding to the conditiona...

Figure 5.6 Likelihood ratio against the occurrence of precipitation as a f...

Figure 5.7 Posterior probability of precipitation occurrence as a function...

Figure 5.8 Receiver operating characteristic (ROC) of the relative vorticity...

Figure 5.9 Receiver operating characteristics (ROCs) of three predictors

Chapter 6

Figure 6.1 Discretization of the continuous probability variate : dot diagr...

Figure 6.2 Conditional probability functions and of the probability vari...

Figure 6.3 Expected probability function of the probability variate the ...

Figure 6.4 The probability calibration function (PCF) of the forecaster wh...

Figure 6.5 Sketches of generic probability calibration functions.

Figure 6.6 Conceptual explanation of overconfidence in judgmental probabilit...

Figure 6.7 The receiver operating characteristic (ROC) of the forecaster who...

Figure 6.8 The ROC outlines of three forecasters: A, B, C.

Figure 6.9 The ROC calculated in Table 6.10; it consists of three operating ...

Chapter 7

Figure 7.1 Decision tree of a detection problem.

Figure 7.2 The disutility of decision , the minimum disutility of decisio...

Figure 7.3 The scale for subjective assessment of the disutilities of outc...

Figure 7.4 Decision tree of a detection problem with the clairvoyant.

Figure 7.5 The value of a perfect forecaster as a function of the probabil...

Figure 7.6 The detection set when the posterior probabilities form (a) a...

Chapter 8

Figure 8.1 The search area is the union of areas , which are bounded and ...

Figure 8.2 Evolution of the probability function on the set of areas fro...

Figure 8.3 Structure of the flash-flood warning system.

Figure 8.4 Elements of the flash-flood hydrograph: observed (heavy line up t...

Figure 8.5 Decision tree of the flash-flood warning problem, and four perfor...

Figure 8.6 Conditional receiver operating characteristic (ROC) of the foreca...

Figure 8.7 Performance trade-off characteristic (PTC) of the monitor–forecas...

Chapter 9

Figure 9.1 Idealized distribution function of a continuous variate .

Figure 9.2 Pointwise representation of the distribution function of a cont...

Figure 9.3 Influence diagram of a conceptual model for judgmental forecastin...

Figure 9.4 Schema for assessing (a) the median , (b) the -quantile , and ...

Figure 9.5 Schema for validating the coherence of (a) the 50% central credib...

Figure 9.6 The assessed distribution function ...

Chapter 10

Figure 10.1 A schema of the Bayesian forecaster of continuous predictand u...

Figure 10.2 Examples of the three limiting cases of the likelihood regressio...

Figure 10.3 Prior distribution function of predictand : the estimated par...

Figure 10.4 Regression of predictor on predictand : the estimated linear ...

Figure 10.5 Distribution function of residual variate from the regression ...

Figure 10.6 Validation of homoscedasticity of residual variate from the re...

Figure 10.7 Posterior mean of predictand, , as a function of predictor real...

Figure 10.8 Two posterior distribution functions of predictand (solid li...

Figure 10.9 Two posterior density functions of predictand (solid lines),...

Figure 10.10 Stochastic dependence (SD) structures between predictands and...

Chapter 11

Figure 11.1 The distribution calibration function (DCF) of the forecaster ...

Figure 11.2 The

uniform calibration function

of the forecaster: a graph of t...

Figure 11.3 The

marginal calibration function

of the forecaster: a graph of ...

Chapter 12

Figure 12.1 Special criterion functions for an estimation, or a target-setti...

Figure 12.2 Optimal solution to a target-setting problem: (a) the density fu...

Figure 12.3 A trimodal density function of the target position on a line...

Figure 12.4 Special criterion function for weapon-aiming- with-friend model:...

Figure 12.5 Elements of the weapon-aiming-with-friend model: example with pi...

Figure 12.6 Elements of the weapon-aiming-with-friend model: example with lo...

Figure 12.7 Mathematical structure of the decision system: (a) with single-o...

Figure 12.8 Mathematical structure of the general forecast–decision system

Figure 12.9 Two generators of a realization of predictor , conditional on p...

Chapter 13

Figure 13.1 Basic inventory system.

Figure 13.2 Optimal solution to the basic inventory problem: the distributio...

Figure 13.3 Graph of the value of a perfect forecast, , versus the ratio of...

Figure 13.4 Two-piece linear opportunity loss function of the demand , wh...

Figure 13.5 Inventory model with initial stock level: (a) the decision tree;...

Chapter 14

Figure 14.1 Distribution functions offered in Task 14.1; stochastically do...

Figure 14.2 Distribution functions offered in Task 14.2; and cross each ...

Figure 14.3 Judgmental task of assessing utility.

Figure 14.4 Inference of the global shape of the utility function based on...

Figure 14.5 Normal distribution functions ...

Chapter 15

Figure 15.1 Elements of the asking price model: (a) the density function o...

Figure 15.2 Elements of the yield control model: (a) the density function ...

A

Figure A.1 Illustration of Axiom 4 (substitution principle) in the degenerat...

Figure A.2 A test of coherence of two simple decisions. Task A.1: the choice...

C

Figure C.2.1 The logistic distribution LG.

Figure C.2.2 The Laplace distribution LP.

Figure C.2.3 The Gumbel distribution GB.

Figure C.2.4 The reflected Gumbel distribution RG.

Figure C.3.1 The exponential distribution EX.

Figure C.3.2 The Weibull distribution WB.

Figure C.3.3 The inverted Weibull distribution IW.

Figure C.3.4 The log-Weibull distribution LW.

Figure C.3.5 The log-logistic distribution LL.

Figure C.4.1 The power type I distribution P1.

Figure C.4.2 The power type II distribution P2.

Figure C.4.3.1 The log-ratio transformation.

Figure C.4.3.2 Left panel: the log-ratio logistic distribution LR1-LG; wit...

Figure C.4.3.3 Left panel: the log-ratio logistic distribution LR1-LG; wit...

Figure C.4.3.4 Left panel: the log-ratio logistic distribution LR1-LG; wit...

Guide

Cover

Table of Contents

Title Page

Copyright

Dedication

Preface

About the Companion Website

Begin Reading

A Rationality Postulates

B Parameter Estimation Methods

C Special Univariate Distributions

The Greek Alphabet

References

Index

End User License Agreement

Pages

iii

iv

v

xxi

xxii

xxiii

1

2

3

4

5

6

7

8

9

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

527

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

Probabilistic Forecasts and Optimal Decisions

Roman Krzysztofowicz

University of Virginia

Charlottesville, Virginia

USA

This edition first published 2025© 2025 by John Wiley & Sons Ltd. All rights reserved.

All rights reserved, including rights for text and data mining and training of artificial intelligence technologies or similar technologies. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The right of Roman Krzysztofowicz to be identified as the author of this work has been asserted in accordance with law.

Registered Office(s)John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USAJohn Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard print versions of this book may not be available in other formats.

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of WarrantyWhile the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging-in-Publication Data Applied for

Hardback ISBN: 9781394221868

Cover Design: WileyCover Image: Courtesy of Roman Krzysztofowicz; © Kingline/Shutterstock

Those who fall in love with practice

without science

are like a sailor who enters a ship

without a helm or compass

and who never can be certain

whither he is going.

Leonardo da Vinci(1452–1519)

Preface

This book has been written for upper-level undergraduate students and first-year graduate students of sciences (mathematical, statistical, decision, data, economic, management) and engineering (systems, industrial, operations research). The prerequisite knowledge includes the elements of calculus and probability — each at the level of an undergraduate course.

The book aims to be introductory yet specialized. Its first objective is to present the fundamentals of probabilistic forecasting and optimal decision making under uncertainty, which form the building blocks of forecast–decision systems for the real world. Its second objective is to teach mathematical modeling, probabilistic reasoning, statistical estimation, judgmental assessment, rational decision making, and numerical calculations — the skills which an aspiring modeler needs to develop and then to integrate into a systemic methodology for solving problems. Its third objective is to show the student with a quantitative bent the usefulness and the beauty of mathematics as a means of describing, understanding, and solving forecast–decision problems, ranging from personal to industrial.

Within the realm of established subjects, the book may be categorized as an introduction to probabilistic forecasting and statistical decision theory from a Bayesian perspective. But unlike many introductory texts, this one is unique and specialized in that (i) it integrates the two subjects into one, under the Bayesian principles of coherence, and (ii) it covers only selected models and procedures so that they can be treated in the depth necessary for fundamental understanding and rigorous application.

Following the definition of a forecast–decision system as the subject of study (Chapter 1), whose mathematical foundation is the postulates of rationality (Appendix A), Part I reviews the basic elements of probability theory (Chapter 2) and presents a methodology for modeling distribution functions of continuous variates (Chapter 3), together with parameter estimation methods (Appendix B) and a catalogue of 20 parametric families of special univariate distributions (Appendix C); this material is used throughout the book.

The main body of the book has two parts, which are independent yet parallel: Part II (Chapters 4–8) is devoted to discrete models; Part III (Chapters 9–15) is devoted to continuous models. Each part covers identical topics; thereby, it reinforces the terminology and the concepts learned in the other part, while it contrasts the mathematics of continuous models against the mathematics of discrete models. The topics covered are: judgmental forecasting (Chapters 4, 9), statistical forecasting (Chapters 5, 10), verification of forecasts (Chapters 6, 11), decision making (Chapters 7, 8, 12, 13, 14, 15).

The problems at the end of each chapter are grouped under two headings. Exercises are short and have a narrow scope. Mini-projects are long and have a broad scope intended to mimic, at least in a rudimentary way, real-world problems; many include real data, require substantial calculations, and give the teacher several options regarding the data or the models to be used. Some mini-projects continue through two or three chapters as they require forecasting, verification of forecasts, and decision making — thereby providing a vehicle for the student to develop the skill of integrative analysis and to acquire mental stamina, a trait essential for an aspiring analyst.

The overall scope of this book has been dictated by the feasibility of teaching from it at the senior–graduate level in one semester (while allowing a choice of advanced topics and of applications). Consequently, what is presented constitutes but a subset of a vastly larger body of knowledge, which I call the Bayesian forecast–decision theory. While its elements have been emerging since the 1940s through the works of many scholars, and applications of these elements have penetrated successfully into many fields, from signal detection theory of electrical engineers to flood forecast–response systems of hydrologists, its exposition as a comprehensive and coherent theory is still wanting.

Acknowledgments. Thanks are due to the teaching assistants who supported my pedagogy; to the students who took my course and then served as assistants — testing and grading the exercises and mini-projects; to Christopher M. Myers who made figures for Appendix C, and to those who made figures for the chapters; to Michael Gurlitz who coded the pilot version of the DFit software for Appendix B, and to Wray Mills who designed, coded, and maintained the current version.

I have been privileged to work with doctoral students who embraced the Bayesian theory and enthusiastically researched with me various aspects of forecast–decision systems: Dou Long, Karen S. Kelly, Ashley A. Sigrest, Coire J. Maranzano, Henry D. Herr, and Jie Liu. Our research was supported by the National Science Foundation and the National Weather Service.

Sherry Crane started typing the manuscript, then Margaret Heritage continued it through many versions; their expert and devoted work has been invaluable.

I dedicate the book to my wife Liana, and to our children Arman and Nayiri.

September 2023Charlottesville, Virginia

Roman Krzysztofowicz

About the Companion Website

This book is accompanied by a companion website:

www.wiley.com/go/ProbabilisticForecastsandOptimalDecisions1e

This website includes:Tables and Solutions Manuals

1Forecast–Decision Theory

1.1 Decision Problem

1.1.1 Decision

Nothing is more difficult, and therefore more valuable and admirable, than to be able to decide.

(Napoléon Bonaparte)

What makes deciding difficult? Whereas a precise answer depends on the problem at hand, major sources of the decisional difficulty, and the accompanying decisional stress, may be grouped under four headings. (i) Complexity of the situation in the context of which one must decide. (ii) Multiplicity of objectives one wants to achieve. (iii) Multiplicity of alternative courses of action one can pursue. (iv) Uncertainty about the outcome, when it depends not only upon one’s decision, but also upon inputs beyond one’s control.

Since its origin in the eighteenth century, decision theory has developed a coherent logical-mathematical framework and effective analytical tools for dealing with all major sources of the decisional difficulty. This book focuses on mathematical models for solving basic decision problems in which uncertainty constitutes the major difficulty.

1.1.2 Uncertainty

Uncertainty is ubiquitous. Its existence is increasingly recognized. And the advantages of taking it into account in decision making, rather than ignoring it and relying on deterministic models (mental or mathematical), are progressively winning the argument among professionals of many disciplines. Here is a handful of examples.

Example 1.1 (Accreditation Board for Engineering and Technology (ABET))

In the 1990s, the ABET, which every 6 years reviews and accredits each undergraduate engineering degree program in the USA, began to require at least one probability and statistics course in each engineering curriculum, regardless of the discipline (e.g., computer, electrical, civil, mechanical). The premise was that every contemporary engineer must acquire at least a rudimentary appreciation of uncertainty and its quantification.

Example 1.2 (American Meteorological Society (AMS))

In 2002, the AMS issued a statement that endorsed making and disseminating probabilistic forecasts of weather elements (e.g., precipitation amount, temperature), in lieu of, or in addition to, traditional deterministic forecasts, which give only a single number (a so-called best estimate). The statement argued that forecasts in probabilistic terms “would allow the user to make decisions based on quantified uncertainty with resulting economic and social benefits”.

Example 1.3 (Secretary of the US Department of the Treasury)

In his 1999 commencement address at the University of Pennsylvania, Robert E. Rubin, former Secretary of the Treasury, recalled from his early career on Wall Street an incident in which he lost a lot of money on a stock. But another security trader, who had believed with “absolute certainty” that particular events would occur and had purchased a large volume of the same stock, “lost an amount way beyond reason — and his job”, when his belief turned out to be wrong. Rubin’s advice to the young graduates: “Reject absolute answers and recognize uncertainty … then all decisions become matters of judging the probability of different outcomes, and the costs and benefits of each.”

1.2 Forecast–Decision System

1.2.1 Structure

To conquer the difficulty of making a rational decision under uncertainty, decision theory offers a way of structuring the problem as follows. The two major activities, quantifying uncertainty and making decisions, can be conceptualized as being performed by a forecast–decision system (F–D system) — a cascade coupling of two components (Figure 1.1): the forecast system (in short, the forecaster), and the decision system (in short, the decider). The coupling can be analyzed in each phase of the system’s life: the design, the operation, the evaluation.

1.2.2 Design: Requirements

The design phase involves (i) specification of requirements for each system component, and (ii) formulation of models and procedures for the forecaster and the decider.

Requirements for decision system

The design begins with the decision system, which should meet the requirements of a client who wants to make rational decisions. To identify the requirements, four basic questions should be asked: (i) What is the purpose of the system? (ii) What is the decision to be made? (iii) What is the outcome of concern to the decider? (iv) What is the future input which is beyond the control of the decider and which, together with the decision, determines the outcome? When this input is uncertain at the decision time, it constitutes a random variable (in short, a variate), and it must be forecasted.

Requirements for forecast system

The design of the forecast system should meet the requirements of the decision system (Figure 1.1) with respect to (i) the predictand — the variate to be forecasted; (ii) the lead time of the forecast — the time that elapses from the moment the forecast is made to the moment the input occurs or can be observed; (iii) the forecast frequency — when the decision is to be made repeatedly (e.g., every day, week, month). Additional requirements may be specified, for instance, regarding information which should be used to prepare the forecast.

Inasmuch as many books are devoted to the general problem of system design, this book does not address any further the first phase of the design process. Instead, system requirements are either specified, or implied through assumptions or problem descriptions, and the text concentrates on the second phase, which is modeling.

1.2.3 Design: Models

Given system requirements, models and procedures must be developed. This is accomplished in two steps. (i) A verbal description of the forecast problem or the decision problem is transformed into a mathematical formulation, which consists of symbols defining variables, sets, and functions (basically the notation which is given meaning in the context of the problem at hand). A well-defined mathematical formulation has two advantages. First, it provides a structure which bestows the clarity and precision on the modeler’s thought process. Second, it prescribes a decomposition which facilitates the next step. (ii) Detailed modeling of functions, estimation of parameters, and writing of procedures are undertaken, separately for each system component.

Figure 1.1 Forecast–decision system — a cascade coupling of the forecast system with the decision system in three phases of the system’s life: (i) design (coupling through system requirements), (ii) operation (coupling through input–output), and (iii) evaluation (coupling through forecast verification). The topics covered in the book are: judgmental forecasting (Chapters 4, 9), statistical forecasting (Chapters 5, 10), verification of forecasts (Chapters 6, 11), decision making (Chapters 7, 8, 12, 13, 14, 15).

Forecaster

The major task is to model (to quantify) uncertainty about the predictand in terms of a distribution function. This probabilistic forecaster may be (i) a human expert preparing forecasts judgmentally based on the available information (quantitative and qualitative), or (ii) a statistical model calculating a forecast based on the realization of a quantitative predictor (a variate which is stochastically dependent on the predictand).

Decider

There are three major tasks. (i) To model (to assess) the preferences of a rational decider over possible outcomes in terms of a criterion function (a utility function, a profit function, an opportunity loss function). (ii) To model the outcome function that maps the decision and the input into an outcome. (iii) To formulate a decision procedure that integrates the probabilistic forecast, the criterion function, and the output function; and to find an optimal decision — one that optimizes (maximizes or minimizes) the expected value of the criterion function.

1.2.4 Operation

The forecaster prepares a probabilistic forecast of the predictand. Next, the decider makes the optimal decision under uncertainty as quantified by the forecast (Figure 1.1). The forecast and the realization of the predictand (once it becomes known) should be archived for the purpose of verification.

1.2.5 Evaluation

Periodically, a set of forecasts should be verified against actual realizations of the predictand. The purpose is to evaluate, and to track over time, the performance of the forecaster with respect to the needs of the decider (Figure 1.1). The Bayesian verification measures quantify two attributes of probabilistic forecasts (calibration and informativeness) and provide (i) feedback to the forecaster, and (ii) statistics to the decider regarding the calibration, the informativeness, and the economic value of forecasts.

1.2.6 Coupling

The above overview of the design, operation, and evaluation phases of the F–D system reveals the intrinsic nature of the coupling between the forecast system and the decision system (Figure 1.1). We shall study this coupling, especially its economic implications, because it is illustrative of a more general interrelationship between information and decision, which is omnipresent.

1.3 Rational Deciding

1.3.1 The Procedure

“Jerky thinking is liable to give rise to jerky behavior.” Coined by Good (1961), this is a witty antonym to the link between “methodical thinking” and “rational behavior”.

Definition Rational deciding under uncertainty is a methodical procedure for (i) analyzing all elements of a decision problem and then (ii) synthesizing them to reach a decision in a manner that adheres to the four postulates of rationality stated in Appendix A.

Depending on the complexity of the decision problem, this methodical procedure (i) may be as simple as thinking and doing arithmetic on a yellow pad, as Rubin (2023) recounts doing in his 50-year long career on Wall Street and in the Federal government, and as we shall learn to do in Chapters 7–8; or (ii) may require calculus, as we shall learn in Chapters 12–15; or (iii) may require complex Monte Carlo simulations and numerical calculations as, for example, a corporation must do to manage a global supply chain system — an extension of the topic of Chapter 13.

In the approach of this book: the analysis leads to a mathematical formulation of the decision model; the synthesis leads to the rational decision procedure. Its mathematically oriented synonym is the optimal decision procedure because it maximizes, or minimizes, the expected value of the criterion function in order to find the optimal decision. But this optimality is usually personal, not absolute, because, when faced with identical decision problems, different deciders may employ different criterion functions, and hence may reach different optimal decisions. This nature of optimality is explicit in the synonym the most preferred decision, which is apt for personal decisions, such as investment decisions — the topic of Chapter 14.

It follows that the term rational decision should be understood as a decision made via a rational decision procedure, not as any particular decision.

1.3.2 The Mind-Set

To become a wise decider, who copes well with uncertainty, one should adopt the mind-set of a Roman stoic (Seneca, 2004). For no matter how carefully the rational decision procedure is implemented, the decision is made ex ante — based on an imperfect forecast of the predictand. Ergo, one should be mindful of the logical implications.

The rational decision procedure does not guarantee a “good” outcome on any particular occasion. It only guarantees that, if it were applied consistently on a large number of occasions, then the resultant sequence of outcomes would be preferred.

On any particular occasion, the realized outcome may be “good” or “bad” because it is generated by a chance mechanism, which is beyond the decider’s control. Although the rational decision procedure optimizes the trade-off between the probability of a good outcome and the probability of a bad outcome (as we shall learn in

Chapters 7

and

14)

, there always remains the risk of a bad outcome.

One should not, therefore, judge the decision

ex post

based on the realized outcome. When the outcome is good, do not say the decision was good, and do not feel smart — you were lucky. When the outcome is bad, do not say the decision was bad, and do not feel depressed — you were unlucky. More philosophically: do not let the outcome touch your emotions. Just accept the outcome, like a Roman stoic would.

The record of past forecasts, decisions, and outcomes does provide feedback, which can be used constructively. Realizations of the predictand can be used to recalibrate the forecaster, if necessary, as we shall learn in

Chapters 6

and

11

. Realizations of the outcomes may trigger a change in the criterion function (e.g., by reassessing the utility function, and the implied risk attitude of the decider, as we shall learn in

Chapter 14)

. The forecast model and the decision model may be refined. Ultimately, it is the quality of the models and procedures, which comprise the F–D system, that determines the degree to which the decisions are actually optimal (and,

ipso facto

, the degree to which the potential advantage of adherence to the rationality postulates of

Appendix A

is actually realized).

1.4 Mathematical Modeling

1.4.1 The Model

Toward our grand objective — to study models and procedures for the F–D system — let us picture the modeling task.

Pause for a moment, and imagine listening to beautiful, your favorite, music. A renowned Italian maestro, Riccardo Muti, once confided:

“The most difficult thing in making music is to express the complex in the simplest possible way.”

Now exchange “making music” for “developing a model”. This states one of our challenges. Ergo, while learning the modeling we may cheer ourselves up with the thought that mathematics and music are twins.

Definition A model is an intellectual construct used to represent, in well-defined terms, a fragment of reality that contains the problem at hand (e.g., a forecast problem, a decision problem).

Four remarks are in order. (i) The means of modeling are not prescribed; they may include linguistic, logical, and mathematical expressions. (ii) It goes without saying that the reality is complex. (iii) The “well-defined terms” are what distinguishes an engineered model from a mental model. (iv) According to one school, the human mind does not analyze reality directly, only a model of reality created through perception. But this mental model is not explicitly stated and is often fuzzy; hence it is not communicable, not comparable, and not analyzable in any objective sense. To make it so is the intellectual challenge of mathematical modeling.

1.4.2 The Guideposts

From theories and experiences, systems engineers, decision scientists, and applied mathematicians deduced eclectic methodologies for modeling problems in various domains. An aspiring modeler should consult this vast literature. Here is a synopsis — seven modeling guideposts apt for the subject of this book.

The model should have a

purpose

and a

domain

(of its applicability), which should be negotiated with the client and then stated precisely.

Effective modeling requires a

trade-off

between the veracity (or the level of detail) of the model and the cost (or the duration) of the development.

Every modeling situation requires one or more

assumptions

. This is the nature of reductionism. Moreover, different modelers may choose different assumptions for different reasons. Hence,“modeling appears almost as a studio art” (Wymore,

1976

, p. 53).

The model should satisfy its purpose in the simplest possible way. The challenge is “not creating complexity but retaining

informative simplicity

”(Howard,

1980

, p. 8).

There may be more than one model suitable for the stated purpose. No perfect model may exist, but only different, more or less plausible,

approximations

to reality (Marschak,

1979

, p. 172).

The theoretic principles plus experience … “combined with often laborious

trial and error

will yield suitable formulations” of models (Bellman,

1957

, p. 82).

The value of a mathematical model derives not only (and not always) from the numbers it outputs, but also (and often foremost) from the

explanation

and the

understanding

(of the problem) that it affords us.

Being written for a first course on the subject, this book presents mathematical formulations of the basic, univariate, analytic models for selected types of ever-important forecast problems and decision problems.

1.5 Notes on Using the Book

These notes expand on the preface and explain the intentions behind some features of the book.

On organization

The organization of the book is diagrammed in Figure 1.2. Together with introductory remarks to the chapters, it should facilitate designing a course and tracking its progress. There is enough material to design several versions of the course: first, by deciding the format (lecture-based, project-based, seminar); second, by deciding the balance between breadth and depth of coverage; and third, by selecting the exercises and mini-projects. For example, in an undergraduate one-semester, lecture-based course, I always covered Chapters 3, 4–7, 9–12, and each year added one of Chapters 8, 13, 14, 15; Chapter 2 and Appendices B, C were assigned for self-study; Appendix A was assigned for reading and discussion. In a graduate course, I also covered advanced concepts, derivations, and proofs; these are relegated to the terminal sections of the chapters. (Most of the proofs are provided, either in the finished form within the text, or in the form of hints to the exercises.)

On “warm-up” tasks

Chapters 4, 9, 14, 15, and Appendix A include tasks to be performed by the student; they require judgments or intuitive decisions. I administer the task via a work sheet at the beginning of a lecture, and discuss the answers after the relevant material has been covered. The class answers may be juxtaposed with the normative answers, if they exist, and compared with answers by other groups, if the chapter reports them.

On applications

Only a few examples of applications appear within the text, but many are framed as exercises and mini-projects. These should be read, therefore, even if not performed, to get a sense of the possibilities.

On options

Many exercises and mini-projects include several options regarding the data or the models to be used. They multiply the number of assignments with distinct solutions to the same problem. The teacher needs to specify the option.

Figure 1.2 The main connections between chapters. Parts II and III are independent of each other. Appendix A supports mainly Chapter 14. Appendices B and C support mainly Chapter 3. Chapter 3 supports mainly Chapters 5, 9, 10.

On modeling

Exercises and mini-projects are stated mostly in words, purposely avoiding notation. The objective: to teach the art of mathematical modeling, which begins with mapping the words into the well-defined notation.

On numerical calculations and graphs

Almost all formulae and algorithms necessary for numerical calculations are provided, so that the student can implement them (by encoding in a spreadsheet or by writing computer code). The objective: to educate a versatile mathematical modeler, not just a software user. I make three exceptions. (i) Graphs of functions, which should look professional and be drawn to scale, can be created with any software. (ii) Numerical integrations, required in some mini-projects, can be done with any software. (iii) Modeling and estimation of parametric distributions, after the students have mastered the simple methods, can be performed with DFit software (see Appendix B).

On communication

Many exercises and all mini-projects end with directives such as: explain the graph, interpret the results, draw conclusions, make recommendations. The objectives: (i) to reinforce the learning of proper mathematical terms, (ii) to build intuitive understanding, and (iii) to practice the skill of technical writing, which is critical for success in a professional career and graduate study. A good answer to the above directives has the property: it is correct, complete, clear, concise.

Bibliographical Notes

The forecast–decision system is a paraphrase of the information-and-decision system, which was conceptualized by Marschak (1974). The original text attributed to Napoléon Bonaparte (1769–1821), the emperor of France (1804–1815) and the exquisite decider on the battlefields of Europe, reads: “Rien n’est plus difficile, et donc plus précieux et merveilleux, que de pouvoir décider.” The decisional stress, its symptoms, its sources, and the patterns that people follow while coping with it, have been researched by Janis and Mann (1977). A transcript of Robert E. Rubin’s speech appeared under the heading “A Healthy Respect for Uncertainty” in Decision Analysis Newsletter, 18(2), 9–10, Institute for Operations Research and the Management Sciences (INFORMS), August 1999.

The notion of rationality has been debated in philosophy, statistics, economics, and decision science. A collection of articles by Good (1983) conveys a flavor of many debates. I.J. Good was a statistician working on cryptanalysis in the British Foreign Office during World War II and later University Distinguished Professor of Statistics at Virginia Polytechnique Institute and State University.

The modeling guideposts are culled mainly from the writings of Wayne Wymore (1976), the founder of the first systems engineering department in the world, in 1960 at the University of Arizona; Ronald A. Howard (1980), the intellectual leader of the “modeling school” of decision analysis at Stanford University; and Richard Bellman (1957), professor of mathematics at the University of Southern California, the inventor of dynamic programming, a theory for modeling and solving sequential decision problems, and a prolific modeler in several disciplines.

The quotation from Riccardo Muti appeared in The Wall Street Journal (17 September 2010) when he became music director of the Chicago Symphony Orchestra, after he had led Milan’s La Scala, Vienna Philharmonic, and the Philadelphia Orchestra.

Part IElements of Probability

2Basic Elements

This chapter defines basic terminology and notation, basic concepts of mathematical analysis, and basic constructs of probability and statistics that are used freely throughout the remainder of the book. The exposition takes the form of a review, assuming that the reader is familiar with set theory and probability theory at the level of a good introductory text.

2.1 Sets and Functions

2.1.1 Sets

A set is a collection of objects. Depending on the context, a set may be called a class, or family, and an object may be called an element, member, or point of , written . The symbolic definition of a set is

which reads: “ is the set of elements such that …”. For instance, if is the set of real numbers, and an element is selected, then is the set of real numbers such that each is less than ; it is an uncountable set. A countable set is defined by listing all of its elements, when is finite,

or by listing a few initial elements, when is countably infinite (or denumerable),

The union of two sets and , denoted , is the set of elements that belong to either or . The intersection of two sets and , denoted , is the set of elements that belong to both and . Set is a subset of set , denoted or , if every member of is also a member of .

An interval is a set which is a subset of the real numbers, , and such that (i) contains at least two points, and (ii) if and , then Let and . There exist four types of bounded intervals:

the

open interval

,

the

closed interval

,

the

semi-closed interval

,

the

semi-closed interval

;

the interval may be described as left-open, right-closed; the interval may be described as left-closed, right-open. There exist five types of unbounded intervals:

,

,

,

,

.

The Cartesian product of and is the set , which consists of all ordered pairs with in and in :

In the Cartesian coordinate system, represents a point whose abscissa is and ordinate is .

2.1.2 Functions

A function is a rule which associates with each element in (the domain of definition of ) an element in (the range of ). A function may be also referred to as a mapping, transform, transformation, or operator, and is usually defined by writing

which reads: “function from into ” or “ maps into ”. The graph of function is a set

which is a subset of the product set .

When , the set of values which takes on is called the image ofunder and is denoted by

Of course, . If the image of the domain