Probability Concepts and Theory for Engineers E-Book

Harry Schwarzlander

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 1126

Veröffentlichungsjahr: 2011

Beschreibung

A thorough introduction to the fundamentals of probability theory

This book offers a detailed explanation of the basic models and mathematical principles used in applying probability theory to practical problems. It gives the reader a solid foundation for formulating and solving many kinds of probability problems for deriving additional results that may be needed in order to address more challenging questions, as well as for proceeding with the study of a wide variety of more advanced topics.

Great care is devoted to a clear and detailed development of the ‘conceptual model' which serves as the bridge between any real-world situation and its analysis by means of the mathematics of probability. Throughout the book, this conceptual model is not lost sight of. Random variables in one and several dimensions are treated in detail, including singular random variables, transformations, characteristic functions, and sequences. Also included are special topics not covered in many probability texts, such as fuzziness, entropy, spherically symmetric random variables, and copulas.

Some special features of the book are:

• a unique step-by-step presentation organized into 86 topical Sections, which are grouped into six Parts
• over 200 diagrams augment and illustrate the text, which help speed the reader's comprehension of the material
• short answer review questions following each Section, with an answer table provided, strengthen the reader's detailed grasp of the material contained in the Section
• problems associated with each Section provide practice in applying the principles discussed, and in some cases extend the scope of that material
• an online separate solutions manual is available for course tutors.

The various features of this textbook make it possible for engineering students to become well versed in the ‘machinery' of probability theory. They also make the book a useful resource for self-study by practicing engineers and researchers who need a more thorough grasp of particular topics.

Leseprobe

Contents

Cover

Title Page

Copyright

Dedication

Preface

Introduction

Motivation and General Approach

Organization of Material

Notation and Terminology

Representative Syllabi

Part I: The Basic Model

Part I Introduction

Section 1: Dealing with ‘Real-World’ Problems

Section 2: The Probabilistic Experiment

Section 3: Outcome

Section 4: Events

Section 5: The Connection to the Mathematical World

Section 6: Elements and Sets

Section 7: Classes of Sets

Section 8: Elementary Set Operations

Section 9: Additional Set Operations

Section 10: Functions

Section 11: The Size of a Set

Section 12: Multiple and Infinite Set Operations

Section 13: More About Additive Classes

Section 14: Additive Set Functions

Section 15: More about Probabilistic Experiments

Section 16: The Probability Function

Section 17: Probability Space

Section 18: Simple Probability Arithmetic

Part I Summary

Part II: The Approach to Elementary Probability Problems

Part II Introduction

Section 19: About Probability Problems

Section 20: Equally Likely Possible Outcomes

Section 21: Conditional Probability

Section 22: Conditional Probability Distributions

Section 23: Independent Events

Section 24: Classes of Independent Events

Section 25: Possible Outcomes Represented as Ordered k-Tuples

Section 26: Product Experiments and Product Spaces

Section 27: Product Probability Spaces

Section 28: Dependence Between the Components in an Ordered k-Tuple

Section 29: Multiple Observations Without Regard to Order

Section 30: Unordered Sampling with Replacement

Section 31: More Complicated Discrete Probability Problems

Section 32: Uncertainty and Randomness

Section 33: Fuzziness

Part II Summary

Part III: Introduction to Random Variables

Part III Introduction

Section 34: Numerical-Valued Outcomes

Section 35: The Binomial Distribution

Section 36: The Real Numbers

Section 37: General Definition of a Random Variable

Section 38: The Cumulative Distribution Function

Section 39: The Probability Density Function

Section 40: The Gaussian Distribution

Section 41: Two Discrete Random Variables

Section 42: Two Arbitrary Random Variables

Section 43: Two-Dimensional Distribution Functions

Section 44: Two-Dimensional Density Functions

Section 45: Two Statistically Independent Random Variables

Section 46: Two Statistically Independent Random Variables—Absolutely Continuous Case

Part III Summary

Part IV: Transformations and Multiple Random Variables

Part IV Introduction

Section 47: Transformation of a Random Variable

a) Transformation of a discrete random variable

b) Transformation of an arbitrary random variable

c) Transformation of an absolutely continuous random variable

Section 48: Transformation of a Two-Dimensional Random Variable

Section 49: The Sum of Two Discrete Random Variables

Section 50: The Sum of Two Arbitrary Random Variables

Section 51: n-Dimensional Random Variables

Section 52: Absolutely Continuous n-Dimensional R.V.'s

Section 53: Coordinate Transformations

Section 54: Rotations and the Bivariate Gaussian Distribution

Section 55: Several Statistically Independent Random Variables

Section 56: Singular Distributions in One Dimension

Section 57: Conditional Induced Distribution, Given an Event

Section 58: Resolving a Distribution into Components of Pure Type

Section 59: Conditional Distribution Given the Value of a Random Variable

Section 60: Random Occurrences in Time

Part IV Summary

Part V: Parameters for Describing Random Variables and Induced Distributions

Part V Introduction

Section 61: Some Properties of a Random Variable

Section 62: Higher Moments

Section 63: Expectation of a Function of a Random Variable

a) Scale change and shift of origin

b) General formulation

c) Sum of random variables

d) Powers of a random variable

e) Product of random variables

Section 64: The Variance of a Function of a Random Variable

Section 65: Bounds on the Induced Distribution

Section 66: Test Sampling

a) A simple random sample

b) Unbiased estimators

c) Variance of the sample average

d) Estimating the population variance

e) Sampling with replacement

Section 67: Conditional Expectation with Respect to an Event

Section 68: Covariance and Correlation Coefficient

Section 69: The Correlation Coefficient as Parameter in a Joint Distribution

Section 70: More General Kinds of Dependence Between Random Variables

Section 71: The Covariance Matrix

Section 72: Random Variables as the Elements of a Vector Space

Section 73: Estimation

a) The concept of estimating a random variable

b) Optimum constant estimates

c) Mean-square estimation using random variables

d) Linear mean-square estimation

Section 74: The Stieltjes Integral

Part V Summary

Part VI: Further Topics in Random Variables

Part VI Introduction

Section 75: Complex Random Variables

Section 76: The Characteristic Function

Section 77: Characteristic Function of a Transformed Random Variable

Section 78: Characteristic Function of a Multidimensional Random Variable

Section 79: The Generating Function

Section 80: Several Jointly Gaussian Random Variables

Section 81: Spherically Symmetric Vector Random Variables

Section 82: Entropy Associated with Random Variables

a) Discrete random variables

b) Absolutely continuous random variables

Section 83: Copulas

Section 84: Sequences of Random Variables

a) Preliminaries

b) Simple gambling schemes

c) Operations on sequences

Section 85: Convergent Sequences and Laws of Large Numbers

a) Convergence of Sequences

b) Laws of Large Numbers

c) Connection with Statistical Regularity

Section 86: Convergence of Probability Distributions and the Central Limit Theorem

Part VI Summary

Appendices

Answers to Queries

Table of the Gaussian Integral

Part I Problems

Part II Problems

Part III Problems

Part IV Problems

Part V Problems

Part VI Problems

Notation and Abbreviations

Plain Lower Case Text Letters

Plain Upper Case Text Letters

Bold Numerals and Upper Case Text Letters

Italic Lower Case Text Letters

Bold Italic Lower Case Text Letters

Italic Upper Case Text Letters

Bold Italic Upper Case Text Letters

Plain Upper Case Block Letters

Plain Upper Case Enhanced Block Letters

Upper Case Script Letters

Upper Case Gothic Letters

Plain and Italic Lower Case Greek Letters

Bold Lower Case Greek Letters

Plain and italic Upper Case Greek Letters

Symbols and markings

References

Subject Index

This edition first published 2011

© 2011 John Wiley & Sons, Ltd

Registered office

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about howto apply for permission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data

Schwarzlander, Harry.

Probability concepts and theory for engineers / Harry Schwarzlander.

p. cm.

Includes bibliographical references and index.

ISBN 978-0-470-74855-8 (hardback)

1. Probabilities. 2. Electrical engineering–Mathematics. I. Title.

TK7864.S39 2011

519.202'462–dc22

2010033582

Print ISBN: 9780470748558 (hb)

ePDF ISBN: 9780470976470

oBook ISBN: 9781119990895

ePub ISBN: 9780470976463

A catalogue record for this book is available from the British Library.

This book is dedicated to students young and old whose thinking will help shape the future.

Preface

This book had its earliest beginnings as a gradually expanding set of supplementary class notes while I was in the Department of Electrical Engineering (later renamed Department of Electrical and Computer Engineering, and now Department of Electrical Engineering and Computer Science) at Syracuse University. Our graduate course in ‘Probabilistic Methods’ was one of the courses I taught quite a few times—in the day and evening programs on campus as well as at the University's off-campus Graduate Centers and via satellite transmission.

Early on I found that existing textbooks, while providing a good coverage of probability mathematics, seemed to lack a consistent and systematic approach for applying the mathematics to problems in the ‘real world.’ My notes therefore focused initially on that aspect. I began to understand that mathematics cannot be ‘applied to the real world.’ One can only apply the mathematics to one's thoughts about the external world. This makes it important that these thoughts are appropriately structured—that a real-world problem gets conceptualized in a manner that allows the correct and consistent application of mathematical principles. This conceptualizing is of little concern in most applications of mathematics since it occurs rather automatically. But the application of Probability Theory calls for much more attention to be devoted to formulating a suitable conceptual model.

Each time I taught the course I tried to improve and add to my notes. After my retirement from the Department I happened to look at these notes again, and they struck me as sufficiently interesting to make it worthwhile to expand and rework them into a textbook. Now, many years later, after incorporation of much more material and a great deal of editing and revising, as well as the creation of a large number of problems and review exercises, here is that book.

Naturally, many individuals and sources have helped me to move forward with this project and to bring it to completion. First, I am indebted to Profs. C. Goffman, H. Teicher, and M. Golomb, among others, in the Department of Mathematics at Purdue University, for helping me strengthen my mathematical thinking. Of course, I derived inspiration and deepened my understanding through the Probability textbooks I used in my course at different times—books by Gnedenko, Meyer, Papoulis, Parzen, Pfeiffer, and Reza—as well as through interactions with my students. Discussions with colleagues at Syracuse University have also been helpful. Of these, I want to specifically acknowledge Prof. D. D. Weiner and Dr. M. Rangaswamy, who introduced me to spherically symmetric random variables; as well as Prof. F. Schlereth, who directed my attention to copulas.

However, the writing of this book could not have come to completion without the loving care and encouragement I received over all these years from my wife, Patricia Carey Schwarzlander. I also appreciate the understanding and support of my children, as well as my grandchildren, over the stressful period of nearly two years during which I converted the manuscript into publishable form and worked my way through the page proofs. I am also grateful for the assistance and encouragement provided by the staff of Wiley, Chichester. And I am indebted to the Department of Electrical Engineering and Computer Science at Syracuse University for having continued to provide me with an office, without which this project would have been very difficult to carry out.

Harry Schwarzlander

Syracuse, NY

November 2010

Introduction

Motivation and General Approach

The intent of this book is to give readers with an engineering perspective a good grounding in the basic machinery of Probability Theory and its application. The level of presentation and the organization of the material make the book suitable as the text for a one-semester course at the beginning graduate level, for students who are likely to have had an introductory undergraduate course in Probability. Nevertheless, the basic approach to applying probability mathematics to practical problems is not short-changed. Many years of teaching graduate students in Electrical Engineering have made it clear to me that it is very helpful to begin with a thorough treatment of the basic model, rather than gloss over this important material in favor of rushing on to a greater variety of advanced topics. This is the strategy pursued here.

Although the material is developed from first principles, a greater mathematical maturity is called for than is usually expected from most undergraduates, and intuitive motivational arguments are minimized. Nevertheless, portions of the book can be used for an undergraduate course, although it would differ significantly from a typical undergraduate ‘Probability and Statistics’ course. Furthermore, the material is presented in sufficient detail and organized in such a way that the book is also a very suitable text for self-study.

The book is unusual in a number of respects. It is intended for the student who applies probability theory. Yet there is not much discussion of specific applications, which would require much space to be devoted to establishing the specific problem contexts. Instead, it has been my intent to provide a thorough understanding (1) of probability theory as a mathematical tool, emphasizing its structure, and (2) of the underlying conceptual model that is needed for the correct application of that tool to practical problems. The manner in which a connection is made between a real-world problem and its mathematical representation is stressed from the beginning. On the other hand, not much emphasis is placed on combinatorics and on the properties of special distributions.

The degree of sophistication in probability expected of engineering students at the graduate level has changed over the years. In preparation for further study and research in stochastic processes, information theory, automata theory, detection theory, radar, and many other areas, familiarity with some of the more abstract topics, such as transformations of random variables, singular distributions, and sequences of random variables, is desirable. Such material is introduced with care at appropriate points in the book.

Throughout, my aim has been to help the reader achieve a solid understanding and feel comfortable and confident in applying probability theory in engineering research as well as in practical problems. Since most graduate students in engineering do not have the opportunity to go through two or three theoretical mathematics courses before starting into probability, this has meant placing more emphasis on introducing or reviewing various mathematical ideas needed in the development of the theory.

Organization of Material

The basic unit of presentation is the Section. Sections are numbered consecutively but are grouped into six major subdivisions—Parts I through VI—which could also be thought of as chapters. Each Section introduces and develops one or several related new concepts. There are problems associated with each Section and these are presented in the Appendix. Sections are somewhat self-contained, and the numbering of equations, definitions, theorems and problems begins a new in each Section, with the Section number as prefix.

To help the student reflect on the various concepts and become more familiar with them, each Section ends with one or more ‘queries,’ which are short review exercises. These also facilitate self-study. An answer table to the queries appears at the beginning of the Appendix.

Part I covers the development of the basic conceptual and mathematical models: the probabilistic experiment and the probability space. Devoting all of Part I to this introductory development was motivated by (1) my experience indicating that even students who do have an undergraduate probability background tend nevertheless to have a weak grasp of this material, and (2) the fact that it is an essential foundation on which to build a good understanding of more advanced concepts. For further emphasis, the first few Sections have intentionally been kept short.

In Part II there is brought together a mixture of topics pertaining to ‘elementary probability problems’— that is, to problems formulated in terms of a basic probability space without requiring the introduction of random variables. This portion of the book provides a bridge between Part I and the introduction of random variables in Part III. Some of the material is relatively simple, such as sampling with and without replacement. It is included for the sake of completeness, and some of this might be skipped in a graduate-level course. On the other hand conditional probability, independence, and product spaces are of course essential.

Part III serves to introduce the basics of random variables and an induced probability space. Discrete and absolutely continuous random variables are clearly distinguished. Discussion of multidimensional random variables has been divided into two steps. First, only two-dimensional random variables are introduced, whereas higher-dimensional random variables are discussed in Part IV. The reasons for this are that (1) it allows the student to first get a good grasp of the two-dimensional case, which is more easily visualized, and (2) it allows pursuit of a somewhat simplified syllabus from which higher-dimensional random variables are omitted. From Part III onward, some special attention is devoted to Gaussian distributions and Gaussian random variables because of their mathematical importance as well as their significance in engineering applications.

Transformations of random variables are addressed in considerable detail in Part IV. In addition to higher-dimensional random variables, Part IV also covers a number of other topics that do not require reference to expectations and moments, which are dealt with extensively in Part V. Delaying this important material to a later portion of the book is a choice I have made for pedagogic reasons. I believe it is helpful to gain a thorough grasp of the basic mechanics of random variables before getting involved in the various descriptors of random variables. Naturally, an instructor using this book can bring in some of that material earlier, if desired. Part V concludes with an introduction to the Riemann–Stieltjes integral, which allows some of the definitions in Part V to be extended to singular random variables. This has made it possible to bypass a discussion of the Lebesgue integral.

Part VI begins with a Section on complex random variables and then introduces the characteristic function and the generating function. There is a discussion of multidimensional Gaussian and Gaussian-like (‘spherically symmetric’) random variables. And following up on the Section on ‘entropy’ in Part II, this measure of randomness is now applied to random variables. Also introduced is the relatively new topic of ‘copulas.’ Then, sequences of random variables are treated in some detail, leading to the laws of large numbers and the Central Limit Theorem.

Notation and Terminology

Specialized notation and abbreviations have been avoided as much as possible. Frequent reference to a ‘probabilistic experiment’ has made the abbreviation ‘p.e.’ convenient. For intervals I have found Feller's [Fe2] overbar notation useful in order to avoid a proliferation of parentheses and brackets (see Section 36). It has been my experience that students readily accept this. A table summarizing all abbreviations and notation is provided in the Appendix.

Representative Syllabi

For a one-semester graduate course, the following is one way of selecting material from this book and grouping it into twelve weekly assignments:

Week:Sections:Subject matter:11–8The probabilistic experiment and set theory concepts.29–16More set theory, functions, statistical regularity, probability.317–22Probability space, simple probability problems, conditional probability.423–27Independence, product spaces.534–39Discrete and absolutely continuous random variables.641–46Two-dimensional random variables, statistical independence.740, 47–50Gaussian random variables, transformations.851–55, 57n-dimensional random variables, coordinate transformations, conditional distributions.959–63Conditional distributions, random occurrences in time, expectations.1064, 67–71Variance of a function of a random variable, conditional expectations, correlation coefficient, covariance matrix.1175–80Characteristic functions, n-dimensional Gaussian random variables.1284–86Sequences, laws of large numbers, Central Limit Theorem.

For an undergraduate course, the book would be used in a different way, with greater emphasis on the development of the basic conceptual and mathematical ideas and on discrete probability. In this case, the following path into the book might be found useful:

Week:Sections:Subject matter:11–7The probabilistic experiment and set theory concepts.28–13Set operations, functions.314–18Statistical regularity, the probability function and probability space419–22Simple probability problems, conditional probability.523–26Independence, product experiments.627–30Product probability spaces, sampling experiments.734–37Random variable defined.838, 39, 41, 42C.d.f. and p.d.f., two random variables.943, 44, 462-dimensional c.d.f. and p.d.f., statistical independence.1061–63, 66Expectation and variance, sampling.1167–70Conditional expectations, correlation coefficient.12Selections from the remainder of the book.

A flow diagram depicting the prerequisite relationships among Sections is shown in Figure F.1. A reader who is interested primarily in studying a particular Section can see from this diagram with what background material he or she should be acquainted.

Figure F.1 Prerequisite relations among Sections

Part I

The Basic Model

Part I Introduction

This First Part is devoted to the development of the basic conceptual and mathematical formulation of a probability problem. The conceptual formulation serves to build up a clear and consistent way of thinking about a problem situation to which probability analysis is to be applied. The mathematical formulation is then constructed, step by step, by connecting set-theoretic concepts to the conceptual building blocks, and the relevant notation is introduced. Also, those principles of Set Theory that are needed for the mathematical formulation are presented in detail. The result is the ‘basic model’, which consists of a conceptual part (the ‘probabilistic experiment’) and a mathematical part (the ‘probability space’). Part I ends with some preliminary exercising of this model, and the establishment of several simple rules of probability arithmetic.

For the sake of emphasis, the individual Sections of Part I have intentionally been kept short. This is also intended to help ease the reader into a somewhat faster pace in the remaining Parts. Readers who have some familiarity with Probability principles should not skip Part I, but may find that they can assimilate the material presented here more quickly than those without that background.

About Queries:

Queries appearing at the end of a Section are short-answer review exercises that are intended to be worked out, pencil in hand, after studying that Section. A table of answers to queries appears in the Appendix. The number in brackets at the end of a query is the key for locating the answer in the answer table.

About Problems:

There are problems associated with each Section. These are located in the Appendix, arranged by Section.

Section 1

Dealing with ‘Real-World’ Problems

Probability theory is a branch of mathematics that has been developed for more effective mathematical treatment of those situations in the real world that involve uncertainty, or randomness, in some sense. Most nontrivial real-world problems incorporate just such situations, so that the practical importance of probability theory hardly needs to be stressed. It turns out, however, that intensive study of the theory alone can still leave the would-be practitioner peculiarly inept in its application. Why should this be so?

It is not inappropriate to ask this question right at the outset. After all, when commencing the exploration of new territory, it helps to be alerted about the nature of the obstacles that lie ahead. A few reflections can lead to an answer to our question.

Example 1.1

A young man working as a sales clerk in a furniture store is asked to count all the chairs that are on display in the store. These chairs may differ from each other in design and material. Nevertheless, the sales clerk automatically adds to his count each time his gaze comes to rest on an object that, in fact, anyone else would also regard as an instance of ‘chair’. This is because the concept ‘chair’ is firmly established in his mind. Possession of the concept ‘chair’ permits the clerk in this situation to carry out the simple mathematical operation of counting. Furthermore, the widely shared agreement regarding the concept ‘chair’ assures that the count will be the same as that obtained by anyone else, barring oversights or counting errors. We may say that with the aid of the concept ‘chair’, the clerk avoids conceptual errors in his task, although not necessarily mathematical errors, or procedural errors.

We can imagine how the clerk in this Example has his store environment modeled in terms of a large variety of concepts, some as simple and common as ‘chair’. These concepts serve to classify the environment, and act as discriminators in simple tasks such as discussed in the Example. On the other hand, it is important to realize that the applicability of a concept is not always clear-cut; concepts have a certain ‘fuzziness’. Surely we can devise an object that neither obviously is a chair, nor obviously isn't a chair.

In Example, the sales clerk is performing a ‘real-world’ task. He is concerned with the immediate experience of his store environment, to which he is applying his counting ability. This is the sense in which we will use ‘real world’. A person who picks up two dice and throws them acts in the real world, in contrast to another person who merely talks about throwing two dice.

Whenever mathematics is applied to a real-world problem, an important step is the establishment of a suitable model, or idealization, of the physical problem under consideration. This makes possible the application of the logical rules of mathematics to the real world. When we have a very simple problem, the modeling seems to take place rather automatically, so that we may not even be aware of it, as is surely the case with the sales clerk in Example 1.1. In more sophisticated situations, more effort generally goes into the modeling process. Because this modeling process always involves ‘conceptualization’—fitting the real-world problem at hand into our way of thinking about things—we will call such models conceptual models.

Example 1.2

An electronic engineer is handed a small cylindrical capsule with a short wire protruding from each end. He tries to ‘identify’ this object as a particular kind of electronic component, such as a resistor, or a capacitor. Actually, ‘resistor’, ‘capacitor’, etc., are conceptual models for real-world objects—the engineer's way of viewing the great variety of electronic components he encounters. His task is, therefore, to determine which of these models is most appropriate for the object at hand.

The model called ‘resistor’, for instance, pertains to real-world objects having two (or more) distinct places for electrical connections. Furthermore, the applicability of this model depends on the effect produced by the component in question when incorporated into an electric circuit. This effect depends on the internal construction of the component.

We noted before that a conceptual model makes possible the application of mathematics to the real world. In the situation just described it is the mathematics of electrical network theory. For instance, the conceptual model ‘resistor’ implies to the engineer that under suitable conditions the relationship

(1.1)

adequately characterizes the current i through the device in question, at an instant when the voltage across the terminals is e, where k is a proportionality constant. The conceptual model ‘resistor’ can therefore be considered as a bridge between a real-world object and the mathematical model that is expressed by the relation (1.1).

Of course, for a given real-world problem, the choice of a suitable model need not be unique, and the mathematical result obtained may depend on the particular model used. The appropriateness of any one model can be judged by how well the results obtained with it agree with, or predict, the actual physical nature or behavior of the situation being considered.

‘Networks’ are the conceptual models to which Network Theory is applied. Network Theory is not used in building the model, when some real-world electrical circuit is analyzed. But, once a model has been decided on, then Network Theory allows various conclusions to be drawn because it provides rules for applying abstract mathematical techniques to the model. The same process can be discerned whenever mathematics is applied to real-world problems. The application of Probability Theory to situations of randomness is also made through suitable conceptual models. However, as we will see in Section 2, the conceptual models required for probability problems are considerably more complicated than those needed in connection with most other branches of applied mathematics.

Just as Network Theory cannot be applied where there are no network models, so is Probability Theory useless without appropriate models. The establishment of an adequate model is therefore an essential ingredient in tackling any real-world problem by means of Probability Theory. Neglect of this important point easily results in confused and erroneous applications of the theory. Here lies the answer to the question posed at the beginning of this Section1.

As we pursue our study of Probability in this book, we will try to pay attention to the connection between real-world problems and their mathematical formulation from the start. This will be possible with the help of a standard scheme for building conceptual models—a scheme that can be applied to all probability problems.

1. For further reading on the role of models in Applied Mathematics and the Sciences, see for example [Fr1], [Ri1], [MM1].

Section 2

The Probabilistic Experiment

To begin with, a name should be given to situations of uncertainty in the real world to which we will apply Probability Theory. The word ‘experiment’ suggests itself when thinking of various real-world situations that involve uncertainty:

the throwing of dice;

the measuring of a physical parameter, such as length, temperature,

or magnetic field strength;

sampling a batch of manufactured items.

We want to be able to express quantitatively the likelihood with which particular results will arise in situations such as these. But before we can even begin to apply any mathematics to such problems, we must have a clear picture of what it is we are actually dealing with. Now, ‘experiment’ is a very widely used word, and therefore imprecise in meaning. We will qualify it and will refer to any real-world ‘experiment’ as a ‘probabilistic experiment’ if it has been modeled in a manner that allows probability mathematics to be applied. It is also worth noting that, when trying to envision real-world situations such as referred to in the above list, we find that in each case there appears an observer—the experimenter—as an explicit or implicit part of the setting. This should not be surprising, since ‘uncertainty’ can only exist as the result of some sort of contemplation, and thus is really a state of mind—the state of mind of the experimenter.

Thus, the first question to be addressed is: what are the essential features of a suitable conceptual model—a probabilistic experiment? These are stated in the form of four distinct requirements in the Definition below, and are then illustrated by means of an Example. These requirements will assure that the model brings into focus those aspects of real-world ‘experiments’ that are essential to the correct application of the mathematical theory. In other words, we shall ‘talk probability’ (ask probability questions, and compute probabilities) only in those real-world contexts that we are able to model in the manner specified below. When this is not possible we do not have a legitimate probability problem. (We note that definitions within the conceptual realm lack the precision of a mathematical definition. As a reminder of this, the word ‘Definition’ appears in parentheses.)

(Definition) 2.1: Probabilistic experiment.

A probabilistic experiment is a conceptual model that consists of four distinct parts, as follows:

1.Statement of a purpose. By this is meant an expression of the intent to make specific real-world observations. It must specify a particular real-world context, configuration, or environment, which is pertinent to these observations. It must also include a listing or precise delineation of the various distinct ‘possibilities’ (properties, alternatives, facts, values, or observable conditions) that are considered to be of interest and that are to be watched for when the intended real-world observations are actually carried out.

2.Description of an experimental procedure. By experimental procedure is meant the specification of an unambiguous sequence of actions to be carried out by the experimenter, leaving no choice to the experimenter, and leading to the observations that are called for in the purpose.

3.Execution of the experimental procedure. This may only take place after the purpose has been stated and the description of the experimental procedure is completed.

4.Noting of the results. By this is meant the identification of those ‘possibilities’, listed as part of the purpose, which are actually observed to exist upon complete execution of the experimental procedure.

The following Example provides a first step toward understanding the significance of this Definition. Here, we model the throwing of a die as a probabilistic experiment.

Example 2.1: A die-throwing experiment.

1.The purpose of the experiment is to determine one of the following six possibilities that are observable upon throwing a given die, whose faces are marked in the standard way, onto a firm and reasonably smooth horizontal surface of adequate size to allow the die to roll freely and come to rest:

one dot faces up

two dots face up

......

......

six dots face up

2.The procedure is: The experimenter is to throw the die in the customary manner onto a surface such as specified in the purpose, with enough force so as to cause the die to roll. When the die comes to rest, the experimenter is to observe the number of dots facing up.

3.This procedure is carried out: An appropriate surface and a die are available. The experimenter throws the die in the specified manner. It comes to rest at a certain spot on the surface with various numbers of dots facing in various directions and, in particular, five dots facing upward.

4. Of all that is perceived by the experimenter upon carrying out the procedure—the path of the die, the final location or orientation of the die, the lighting conditions, the sound accompanying the throw of the die, etc.—only the property ‘five dots face up’ is one of the possibilities that was initially stated to be of interest. This property is noted.

This example is, of course, rather trivial. Nevertheless, it serves to illustrate the importance of each of the four steps making up the probabilistic experiment; for, knowledge of only one (or even two or three) of the four parts making up the model does not guarantee a unique description of the complete experiment. Thus, suppose that item 1 in the description of the die-throwing experiment were missing. From a knowledge of items 2, 3, and 4 it is not certain that 1 should read as it actually appears above. For instance, the purpose might actually have been to observe one of the properties

five dots face up

five dots face sideways

neither of the above.

It turns out that if this were the purpose, the probability problem arising from the experiment would be quite different.

Now suppose the description of the experimental procedure were missing. Even if item 3 is known to us—that is, we have seen the experimental procedure being carried out—we cannot usually be sure what the procedure really was. Just consider the possibility of the following sentence added to item 2 in the above Example: ‘If on the first throw the number of dots facing up is six, the die is thrown again.’ There would be no way of inferring this from a knowledge of items 3 and 4 alone.

Without knowledge of item 3, on the other hand, it is not clear whether properties noted under item 4 are observed as a result of carrying out the procedure as specified in item 2, or some other sequence of actions. It might also be proposed that items 2 and 3 be combined; however, there is a definite reason for maintaining them as separate items: The description of the experimental procedure must not change during the course of the experiment—that is, while the procedure is being carried out. Another way of stating this is to say that item 2 is strictly ‘deterministic’, whereas item 3 brings into evidence occurrences and properties about which there is initial uncertainty.

Finally, it is clear that an essential part of the experiment is missing if knowledge about the observed properties of interest, as specified in item 4, is not available. This situation will be considered again in Section 3.

Thus, we have seen that even as innocent an activity as throwing a die calls for a careful description if it is to be modeled as a probabilistic experiment. Even more care will be needed when dealing with more complicated experiments, but this can only be appreciated after our study has progressed.

Particular attention has to be given to the purpose of a probabilistic experiment. The purpose cannot be in the form of a question, for instance. Consider: ‘Will hydrochloric acid produce a precipitate in the unknown liquid sample?’ This sentence does not conform with item 1 of our Definition and therefore cannot be the statement of purpose of a probabilistic experiment; it gives no clear account of the possible alternatives that are to be looked for in this experiment. Also, a probabilistic experiment cannot have as its purpose the determination of a probability. At this point we have not yet assigned a technical meaning to the word ‘probability’. We are not yet concerned with probability, only with experiments. But if we stick to the common language sense of ‘probability’, which we might paraphrase as ‘likelihood’, or ‘chance’, this surely is not a property that is observable in the real world as a result of performing an experiment. Similarly, the purpose of a probabilistic experiment cannot be to decide something. Decisions might be made on the basis of the results of the experiment, but this circumstance does not enter into our model.

Probability Theory is actually not concerned with completed probabilistic experiments—only with probabilistic experiments prior to their execution. Nevertheless, a picture of the complete probabilistic experiment must be in our mind, as will become clearer as we proceed. We will sometimes refer to a probabilistic experiment prior to execution—i.e., to parts 1 and 2 of the Definition—as the ‘experimental plan’.

The discussion so far leads us to view our involvement in a real-world problem as taking place in three different domains, as illustrated in Figure 2.1. In the external world or ‘Real World’, a person experiences a profusion of perceptions. The ‘Conceptual World’ provides the opportunity to clarify, organize and prioritize these perceptions. Furthermore, it forms the bridge between real-world experience and mathematical analysis. In the Conceptual World, as mentioned earlier, ‘definitions’ are of a different kind from those in the Mathematical World. They generally are rules for relating the real world to the conceptual world, and thus are susceptible to interpretation. There is usually a fuzziness to these definitions, that is, their application to any given problem is not always entirely clear-cut. It is then necessary to use judgment based on experience, or to proceed with caution and be prepared to modify one's approach.

Figure 2.1 Applying mathematics to real-world problems

The notion of probability evolved in the conceptual world. It does not exist in the real world. Designating our conceptual experimental model a ‘probabilistic experiment’ therefore seems fitting. The definition of a probabilistic experiment is our first encounter with a ‘definition’ in the Conceptual World. We may think of it also as a rule for constructing the right kind of conceptual model for any given real-world experiment—a rule telling us how to organize our thinking about the experiment. A probabilistic experiment will always serve as our conceptual model when we try to apply Probability Theory to real-world problems, as indicated in Figure 2.1. Probability Theory may not lead to meaningful results when dealing with a real-world problem that cannot be modeled in accordance with Definition 1 or some other suitable conceptual model.1

Real-world problems that can be modeled as probabilistic experiments have a peculiar feature that is not shared by most other types of problems to which Mathematics can be applied. It is the fact that a human being, an observer—the experimenter—is an essential part of the model. The experimenter's mind harbors the uncertainty (or certainty) about the properties that will be observed in an experiment. Thus, Probability Theory is unique, since it permits the application of mathematical techniques to a class of problems that in a certain sense include the interaction of a human being with his/her environment.

Queries

Note: Queries appearing at the end of a section are to be considered part of the material of that section. The reader should answer them, pencil and paper at hand, in order to assure a correct understanding of the ideas presented. The number in brackets following each Query identifies the answer in the Answer Table (see Appendix).

2.1 Which of the following are correct statements, which incorrect, based on a reasonable interpretation of the definition of a probabilistic experiment? If incorrect, which part(s) of Definition 2.1 are violated?

a. A person playing a game of checkers is conducting a probabilistic experiment.

b. A person, prior to playing a game of checkers, is deciding on a strategy. This constitutes a probabilistic experiment.

c. A person decides to take a walk along a certain stretch of seashore to see if she might find something interesting, such as driftwood, shells, etc. She is embarking on a probabilistic experiment.

d. A person is waiting at Kennedy airport for a friend who is to arrive on a particular flight. She wonders whether or not the friend will arrive safely. She is engaged in a probabilistic experiment.

e. A person with high blood pressure is about to fly from New York to London. He wonders whether he will get there or whether he will die from a heart attack or perish in a crash along the way. He is engaging in a probabilistic experiment.

f. A mail clerk puts a letter on a letter scale to see whether it weighs more than an ounce or not. He is performing a probabilistic experiment. [184]

1. The Definition of this Section, and the additional requirements put on the conceptual model in Section 15, follow closely the specifications presented in [AR1]. Other ‘systems’ of probability exist, which are not built on the particular kind of conceptual model introduced here. This fact can be a source of confusion, because nearly the same terminology and mathematics appears in all of them. Our approach here is along those lines that are now most widely accepted, especially in the physical and natural sciences and in engineering.

Section 3

Outcome

The ‘probabilistic experiment’ (henceforth abbreviated p.e.) is the conceptual model through which we will view any real world problem to which we want to apply Probability Theory. As we proceed, we will become quite accustomed to working with p.e.'s. We will also explore some of the difficulties that can arise when the conceptual model is bypassed. But before establishing connections to mathematics, a little more needs to be said about the model.

(Definition) 3.1

Of the distinct possibilities of interest in a particular p.e., all those that are actually observed after the procedure has been carried out are collectively called the actual outcome (or simply, outcome) of the p.e.

For instance, the die-throwing experiment in Example has as the actual outcome the single property ‘five dots face up‘. But an outcome can also consist of several properties, all of which are observed upon executing the procedure. On the other hand it follows from the above Definition that an experiment cannot have more than one outcome. An experimental plan leads to one actual outcome from among a variety of candidates, upon completion of the experiment. These various candidates are referred to as the possible outcomes of the p.e., or of the experimental plan. Thus, the die-throwing experiment of Section 2 has six possible outcomes, and each happens to be associated with exactly one of the six properties that are of interest.

It is often convenient to incorporate the notion of ‘possible outcomes' in the description of a p.e.: In the statement of purpose of a p.e., we can replace the list of ‘possibilities' or ‘properties', etc., by a listing of all the possible outcomes. In our die-throwing experiment this happens to cause no change, since the possible outcomes coincide with the various properties of interest. This is not always the case.

Example 3.1: Throw of two dice.

An experiment similar to the one modeled in Example is to be performed with two dice—one red and one white die. There are now 12 properties of interest:

one dot faces up on red die

. . .

. . .

six dots face up on red die

one dot faces up on white die

. . .

. . .

six dots face up on white die

On the other hand, this p.e. has 6 × 6 = 36 possible outcomes, covering all possible combinations of results on the red and the white die. Thus, each possible outcome is made up of two ‘properties of interest'. In the statement of purpose for this p.e., it is simpler to list all properties of interest than all possible outcomes.

Now let us suppose the die-throwing experiment is performed on a sidewalk and the die falls down a drain. How does this situation fit into the schema of Definition 2.1? None of the properties of interest in the experiment can be observed in this case, so that there is no outcome! Although this experiment may have been ‘properly' performed, in the sense that the experimenter carried out all the required actions, it nevertheless turned out in an undesirable manner. We say that such an experiment, which has no outcome, was unsatisfactorily performed.

Henceforth, unsatisfactorily performed experiments will be excluded from our considerations. The theory to be developed, which will allow us to treat mathematically those situations in which observables arise with some degree of uncertainty, presupposes experiments which are modeled according to Definition 2.1—i.e., as probabilistic experiments—and furthermore, these experiments must be satisfactorily performed. This should not be regarded as restricting the applicability of the theory; it merely refines our conceptual model. For, we have in fact two ways of accommodating the situation described above, where a die falls down a drain:

a. As an unsatisfactorily performed experiment it does not exist in our conceptual model world—we simply do not think of the actions that lead to the loss of the die as belonging to a p.e. Or:

b. As an experiment whose possible outcomes include ‘die is lost' or some such property, it does have an acceptable representation in our conceptual model world. In this way it can be arranged that an otherwise unsatisfactorily performed experiment gets treated as a satisfactorily performed one.

In practice, any situation involving uncertainties can be framed within some kind of hypothetical experiment to which the scheme of Definition 2.1 can be applied. Often, several different possible experiments suggest themselves. Each of these different formulations may result in a different mathematical treatment of the problem at hand.

Queries

3.1 Consider various p.e.'s whose procedure calls for one throw of an ordinary die onto a table top. For each of the following lists, decide whether it can be a listing of the possible outcomes of such an experiment.

a.‘one dot faces up'

‘two dots face up'

‘three dots face up'

‘four dots face up'

‘five dots face up'

b. ‘three or fewer dots face up'

‘an even number of dots faces up'

‘four or more dots face up'

c. ‘one dot faces up'

d. ‘an even number of dots faces up'

‘one dot faces up'

e.‘die comes to rest on table top'

‘die falls off table'

f.‘the experiment is satisfactorily performed'

‘the experiment is unsatisfactorily performed'. [42]

3.2 In the statement of purpose of a certain p.e. there appears a list of exactly three observable properties, or features, which are to be looked for upon executing the procedure. Allowing only satisfactorily performed experiments, what is the maximum number of possible outcomes that might exist for this experiment? The minimum number? [226]

3.3 The purpose of a particular p.e. includes a listing of various observable features, or properties, that are of interest in this experiment and that are to be looked for upon executing the procedure. What is the least number of observable features that must be in this listing if it is known that the p.e. has five possible outcomes? [195]

3.4 An experimenter who is about to perform a particular p.e. claims that she knows which one of the possible outcomes will be the actual outcome, prior to performing the experiment. This implies which of these:

a. She will see to it that the experiment will be unsatisfactorily performed

b. The experiment is ‘rigged'

c. The experiment has only one possible outcome

d. She has performed the experiment previously. [213]

Section 4

Events

Often, we will be especially interested in some group or collection of possible outcomes from among all the various possible outcomes of a p.e. We may then wish to express the fact that the actual outcome, when the experiment is performed, belongs to this particular collection of possible outcomes. Consider again the die-throwing experiment with possible outcomes ‘one dot faces up’, ‘two dots face up’, … ‘six dots face up’. We might be particularly interested to see whether the actual outcome of this experiment is characterized by the description