Time Series Analysis and Forecasting by Example E-Book

Søren Bisgaard

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Wiley Series in Probability and Statistics
• Sprache: Englisch

Seitenzahl: 428

Veröffentlichungsjahr: 2011

Beschreibung

An intuition-based approach enables you to master time series analysis with ease Time Series Analysis and Forecasting by Example provides the fundamental techniques in time series analysis using various examples. By introducing necessary theory through examples that showcase the discussed topics, the authors successfully help readers develop an intuitive understanding of seemingly complicated time series models and their implications. The book presents methodologies for time series analysis in a simplified, example-based approach. Using graphics, the authors discuss each presented example in detail and explain the relevant theory while also focusing on the interpretation of results in data analysis. Following a discussion of why autocorrelation is often observed when data is collected in time, subsequent chapters explore related topics, including: * Graphical tools in time series analysis * Procedures for developing stationary, non-stationary, and seasonal models * How to choose the best time series model * Constant term and cancellation of terms in ARIMA models * Forecasting using transfer function-noise models The final chapter is dedicated to key topics such as spurious relationships, autocorrelation in regression, and multiple time series. Throughout the book, real-world examples illustrate step-by-step procedures and instructions using statistical software packages such as SAS, JMP, Minitab, SCA, and R. A related Web site features PowerPoint slides to accompany each chapter as well as the book's data sets. With its extensive use of graphics and examples to explain key concepts, Time Series Analysis and Forecasting by Example is an excellent book for courses on time series analysis at the upper-undergraduate and graduate levels. it also serves as a valuable resource for practitioners and researchers who carry out data and time series analysis in the fields of engineering, business, and economics.

Leseprobe

Table of Contents

Series Page

Title Page

Copyright

Dedication Page

Preface

Chapter 1: Time Series Data: Examples and Basic Concepts

1.1 Introduction

1.2 Examples of Time Series Data

1.3 Understanding Autocorrelation

1.4 The Wold Decomposition

1.5 The Impulse Response Function

1.6 Superposition Principle

1.7 Parsimonious Models

Chapter 2: Visualizing Time Series Data Structures: Graphical Tools

2.1 Introduction

2.2 Graphical Analysis of Time Series

2.3 Graph Terminology

2.4 Graphical Perception

2.5 Principles of Graph Construction

2.6 Aspect Ratio

2.7 Time Series Plots

2.8 Bad Graphics

Chapter 3: Stationary Models

3.1 Basics of Stationary Time Series Models

3.2 Autoregressive Moving Average (ARMA) Models

3.3 Stationarity and Invertibility of ARMA Models

3.4 Checking for Stationarity Using Variogram

3.5 Transformation of Data

Chapter 4: Nonstationary Models

4.1 Introduction

4.2 Detecting Nonstationarity

4.3 Autoregressive Integrated Moving Average (ARIMA) Models

4.4 Forecasting Using Arima Models

4.5 Example 2: Concentration Measurements from a Chemical Process

4.6 The EWMA Forecast

Chapter 5: Seasonal Models

5.1 Seasonal Data

5.2 Seasonal Arima Models

5.3 Forecasting Using Seasonal Arima Models

5.4 Example 2: Company X's Sales Data

Chapter 6: Time Series Model Selection

6.1 Introduction

6.2 Finding the “BEST” Model

6.3 Example: Internet Users Data

6.4 Model Selection Criteria

6.5 Impulse Response Function to Study the Differences in Models

6.6 Comparing Impulse Response Functions for Competing Models

6.7 Arima Models as Rational Approximations

6.8 Ar Versus Arma Controversy

6.9 Final Thoughts on Model Selection

6.10 Appendix 6.1: How to Compute Impulse Response Functions with a Spreadsheet

Chapter 7: Additional Issues in Arima Models

7.1 Introduction

7.2 Linear Difference Equations

7.3 Eventual Forecast Function

7.4 Deterministic Trend Models

7.5 Yet Another Argument for Differencing

7.6 Constant Term in Arima Models

7.7 Cancellation of Terms in Arima Models

7.8 Stochastic Trend: Unit Root Nonstationary Processes

7.9 Overdifferencing and Underdifferencing

7.10 Missing Values in Time Series Data

Chapter 8: Transfer Function Models

8.1 Introduction

8.2 Studying Input–Output Relationships

8.3 Example 1: The Box–Jenkins' Gas Furnace

8.4 Spurious Cross Correlations

8.5 Prewhitening

8.6 Identification of the Transfer Function

8.7 Modeling the Noise

8.8 The General Methodology for Transfer Function Models

8.9 Forecasting Using Transfer Function–Noise Models

8.10 Intervention Analysis

Chapter 9: Additional Topics

9.1 Spurious Relationships

9.2 Autocorrelation in Regression

9.3 Process Regime Changes

9.4 Analysis of Multiple Time Series

9.5 Structural Analysis of Multiple Time Series

Appendix A: Datasets used in the Examples

Appendix B: Datasets used in the Exercises

Bibliography

Wiley Series

Index

Copyright © 2010 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone]. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Bisgaard, Soren, 1938-

Time series analysis and forecasting by example / Soren Bisgaard, Murat Kulahci.

p. cm. - (Wiley series in probability and statistics)

Includes bibliographical references and index.

ISBN 978-0-470-54064-0 (cloth)

1. Time-series analysis. 2. Forecasting. I. Kulahci, Murat. II. Title.

QA280.B575 2011

519.55–dc22

2010048281

To the memory of

Soren Bisgaard

Preface

Data collected in time often shows serial dependence. This, however, violates one of the most fundamental assumptions in our elementary statistics courses where data is usually assumed to be independent. Instead, such data should be treated as a time series and analyzed accordingly. It has, unfortunately, been our experience that many practitioners found time series analysis techniques and their applications complicated and subsequently were left frustrated. Recent advances in computer technology offer some help. Nowadays, most statistical software packages can be used to apply many techniques we cover in this book. These often user-friendly software packages help the spreading of the use of time series analysis and forecasting tools. Although we wholeheartedly welcome this progress, we also believe that statistics welcomes and even requires the input from the analyst who possesses the knowledge of the system being analyzed as well as the shortfalls of the statistical techniques being used in this analysis. This input can only enhance the learning experience and improve the final analysis.

Another important characteristic of time series analysis is that it is best learned by applications (as George Box used to say for statistical methods in general) akin to learning how to swim. One can read all the theoretical background on the mechanics of swimming, yet the real learning and joy can only begin when one is in the water struggling to stay afloat and move forward. The real joy of statistics comes out with the discovery of the hidden information in the data during the application. Time series analysis is no different.

It is with all these ideas/concerns in mind that Søren and I wrote our first Quality Quandaries in Quality Engineering in 2005. It was about how the stability of processes can be checked using the variogram. This led to a series of Quality Quandaries on various topics in time series analysis. The main focus has always been to explain a seemingly complicated issue in time series analysis by providing the simple intuition behind it with the help of a numerical example. These articles were quite well received and we decided to write a book. The challenge was to make a stand-alone book with just enough theory to make the reader grasp the explanations provided with the example from the Quality Quandaries. Therefore, we added the necessary amount of theory to the book as the foundation while focusing on explaining the topics through examples. In that sense, some readers may find the general presentation approach of this book somewhat unorthodox. We believe, however, that this informal and intuition-based approach will help the readers see the time series analysis for what it really is—a fantastic tool of discovery and learning for real-life applications.

As mentioned earlier, throughout this book, we try to keep the theory to an absolute minimum and whenever more theory is needed, we refer to the seminal books by Box et al. (2008) and Brockwell and Davis (2002). We start with an introductory chapter where we discuss why we observe autocorrelation when data is collected in time with the help of the simple pendulum example by Yule (1927). In the same chapter we also discuss why we should prefer parsimonious models and always seek the simpler model when all else is the same. Chapter 2 is somewhat unique for a time series analysis book. In this chapter, we discuss the fundamentals of graphical tools. We are strong believers of these tools and always recommend using them before attempting to do any rigorous statistical analysis. This chapter is inspired by the works of Tufte and particularly Cleveland with particular focus on the use of graphical tools in time series analysis. In Chapter 3, we discuss fundamental concepts such as stationarity, autocorrelation, and partial autocorrelation functions to lay down the foundation for the rest of the book. With the help of an example, we discuss the autoregressive moving average (ARMA) model building procedure. Also, in this chapter we introduce the variogram, an important tool that provides insight about certain characteristics of the process. In real life, we cannot expect systems to remain around a constant mean and variance as implied by stationarity. For that, we discuss autoregressive integrated moving average (ARIMA) models in Chapter 4. With the help of two examples, we go through the modeling procedure. In this chapter, we also introduce the basic principles of forecasting using ARIMA models. At the end of the chapter, we discuss the close connection between EWMA, a popular smoothing and forecasting technique, and ARIMA models. Some time series, such as weather patterns, sales and inventory data, and so on, exhibit cyclic behavior that can be analyzed using seasonal ARIMA models. We discuss these models in Chapter 5 with the help of two classic examples from the literature. In our modeling efforts, we always keep in mind the famous quote by George Box “All models are wrong, some are useful.” In time series analysis, sometimes more than one model can fit the data equally well. Under those circumstances, system knowledge can help to choose the more relevant model. We can also make use of some numerical criteria such as AIC and BIC, which are introduced in Chapter 6 where we discuss the model identification issues in ARIMA models. Chapter 7 consists of many sections on additional issues in ARIMA models such as constant term and cancellation of terms in ARIMA models, overdifferencing and underdifferencing, and missing values in the data. In Chapter 8, we introduce an input variable and discuss ways to improve our forecasts with the help of this input variable through the transfer function–noise models. We use two examples to illustrate in detail the steps of the procedure for developing transfer function–noise models. In this chapter, we also discuss the intervention models with the help of two examples. In the last chapter, we discuss additional topics such as spurious relationships, autocorrelation in regression, multiple time series, and structural analysis of multiple time series using principal component analysis and canonical analysis.

This book would not have been possible without the help of many friends and colleagues. I would particularly like to thank John Sølve Tyssedal and Erik Vanhatalo who provided a comprehensive review of an earlier draft. I would also like to thank Johannes Ledolter for providing a detailed review of Chapters 3 and 7. I have tried to incorporate their comments and suggestions into the final version of the manuscript.

Data sets and additional material related to this book can be found at ftp://ftp.wiley.com/public/sci_tech_med/times_series_example.

I would also like to extend special thanks to my wife, Stina, and our children Minna and Emil for their continuing love, support, and patience throughout this project.

I am indebted to the editors of Quality Engineering as well as Taylor and Francis and the American Society for Quality (ASQ), copublishers of Quality Engineering for allowing us to use the Quality Quandaries that Søren and I wrote over the last few years as the basis of this book.

In the examples presented in this book, the analyses are performed using R, SCA, SAS JMP version 7 and Minitab version 16. SCA software is a registered trademark of Scientific Computing Associates Corp. SAS JMP is a registered trademark of SAS Institute Inc., Cary, NC, USA. Portions of the output contained in this book are printed with permission of Minitab Inc. All material remains the exclusive property and copyright of Minitab Inc. All rights reserved.

While we were writing this book, Søren got seriously ill. However, he somehow managed to keep on working on the book up until his untimely passing last year. While finishing the manuscript, I tried to stay as close as I possibly can to our original vision of writing an easy-to-understand-and-use book on time series analysis and forecasting. Along the way, I have definitely missed his invaluable input and remarkable ability to explain in simple terms even the most complicated topics. But more than that, I have missed our lively discussions on the topic and on statistics in general. This book is dedicated to the memory of my mentor and dear friend Søren Bisgaard.

Murat Kulahci

Lyngby, Denmark

Chapter 1

Time Series Data: Examples and Basic Concepts

1.1 Introduction

In many fields of study, data is collected from a system (or as we would also like to call it a process) over time. This sequence of observations generates a time series such as the closing prices of the stock market, a country's unemployment rate, temperature readings of an industrial furnace, sea level changes in coastal regions, number of flu cases in a region, inventory levels at a production site, and so on. These are only a few examples of a myriad of cases where time series data is used to better understand the dynamics of a system and to make sensible forecasts about its future behavior.

Most physical processes exhibit inertia and do not change that quickly. This, combined with the sampling frequency, often makes consecutive observations correlated. Such correlation between consecutive observations is called autocorrelation. When the data is autocorrelated, most of the standard modeling methods based on the assumption of independent observations may become misleading or sometimes even useless. We therefore need to consider alternative methods that take into account the serial dependence in the data. This can be fairly easily achieved by employing time series models such as autoregressive integrated moving average (ARIMA) models. However, such models are usually difficult to understand from a practical point of view. What exactly do they mean? What are the practical implications of a given model and a specific set of parameters? In this book, our goal is to provide intuitive understanding of seemingly complicated time series models and their implications. We employ only the necessary amount of theory and attempt to present major concepts in time series analysis via numerous examples, some of which are quite well known in the literature.

1.2 Examples of Time Series Data

Examples of time series can be found in many different fields such as finance, economics, engineering, healthcare, and operations management, to name a few. Consider, for example, the gross national product (GNP) of the United States from 1947 to 2010 in Figure 1.1 where GNP shows a steady exponential increase over the years. However, there seems to be a “hiccup” toward the end of the period starting with the third quarter of 2008, which corresponds to the financial crisis that originated from the problems in the real estate market. Studying such macroeconomic indices, which are presented as time series, is crucial in identifying, for example, general trends in the national economy, impact of public policies, or influence of global economy.

Figure 1.1 GNP (nominal) of the United States from 1947 to 2010 (in billion dollars).

Speaking of problems with the real estate market, Figure 1.2 shows the median sales prices of houses in the United States from 1988 to the second quarter of 2010. One can argue that the signs of the upcoming crisis could be noticed as early as in 2007. However, the more crucial issue now is to find out what is going to happen next. Homeowners would like to know whether the value of their properties will fall further and similarly the buyers would like to know whether the market has hit the bottom yet. These forecasts may be possible with the use of appropriate models for this and many other macroeconomic time series data.

Figure 1.2 Median sales prices of houses in the United States.

Businesses are also interested in time series as in inventory and sales data. Figure 1.3 shows the well-known number of airline passengers data from 1949 to 1960, which will be discussed in greater detail in Chapter 5. On the basis of the cyclical travel patterns, we can see that the data exhibits a seasonal behavior. But we can also see an upward trend, suggesting that air travel is becoming more and more popular. Resource allocation and investment efforts in a company can greatly benefit from proper analysis of such data.