Book:George E.P. Box/Time Series Analysis: Forecasting and Control/Third Edition

George E.P. Box, Gwilym M. Jenkins and Gregory C. Reinsel: Time Series Analysis: Forecasting and Control (3rd Edition)

Published $\text {1994}$, Prentice-Hall International, Inc.

ISBN 0-13-060774-6

Subject Matter

• Probability Theory
• Statistics

Contents

PREFACE

$\textbf 1$ INTRODUCTION

$1.1$ Four Important Practical Problems

$\textit {1.1.1}$ Forecasting Time Series

$\textit {1.1.2}$ Estimation of Transfer Functions

$\textit {1.1.3}$ Analysis of Effects of Unusual Intervention Events

$\textit {1.1.4}$ Discrete Control Systems

$1.2$ Stochastic and Deterministic Dynamic Mathematical Models

$\textit {1.2.1}$ Stationary and Nonstationary Stochastic Models for Forecasting and Control

$\textit {1.2.2}$ Transfer Function Models

$\textit {1.2.3}$ Models for Discrete Control Systems

$1.3$ Basic Ideas in Model Building

$\textit {1.3.1}$ Parsimony

$\textit {1.3.2}$ Iterative Stages in the Selection of a Model

Part $\textbf {I}$ Stochastic Models and Their Forecasting

$\textbf 2$ AUTOCORRELATION FUNCTION AND SPECTRUM OF STATIONARY PROCESSES

$2.1$ Autocorrelation Properties of Stationary Models

$\textit {2.1.1}$ Time Series and Stochastic Processes

$\textit {2.1.2}$ Stationary Stochastic Processes

$\textit {2.1.3}$ Positive Definiteness and the Autocovariance Matrix

$\textit {2.1.4}$ Autocovariance and Autocorrelation Functions

$\textit {2.1.5}$ Estimation of Autocovariance and Autocorrelation Functions

$\textit {2.1.6}$ Standard Error of Autocorrelation Estimates

$2.2$ Spectral Properties of Stationary Models

$\textit {2.2.1}$ Periodogram of a Time Series

$\textit {2.2.2}$ Analysis of Variance

$\textit {2.2.3}$ Spectrum and Spectral Density Function

$\textit {2.2.4}$ Simple Examples of Autocorrelation and Spectral Density Functions

$\textit {2.2.5}$ Advantages and Disadvantages of the Autocorrelation and Spectral Density Functions

$\text A 2.1$ Link Between the Sample Spectrum and Autocovariance Function Estimate

$\textbf 3$ LINEAR STATIONARY MODELS

$3.1$ General Linear Process

$\textit {3.1.1}$ Two Equivalent Forms for the Linear Process

$\textit {3.1.2}$ Autocovariance Generating Function of a Linear Process

$\textit {3.1.3}$ Stationarity and Invertibility Conditions for a Linear Process

$\textit {3.1.4}$ Autoregressive and Moving Average Processes

$3.2$ Autoregressive Processes

$\textit {3.2.1}$ Stationarity Condituons for Autoregressive Processes

$\textit {3.2.2}$ Autocorrelation Function and Spectrum of Autoregressive Processes

$\textit {3.2.3}$ First-Order Autoregressive (Markov) Process

$\textit {3.2.4}$ Second-Order Autoregressive Process

$\textit {3.2.5}$ Partial Autocorrelation Function

$\textit {3.2.6}$ Estimation of the Partial Autocorrelation Function

$\textit {3.2.7}$ Standard Errors of Partial Autocorrelation Estimates

$3.3$ Moving Average Processes

$\textit {3.3.1}$ Invertibility Conditions for Moving Average Processes

$\textit {3.3.2}$ Autocorrelation Function and Spectrum of Moving Average Process

$\textit {3.3.3}$ First-Order Moving Average Process

$\textit {3.3.4}$ Second-Order Moving Average Process

$\textit {3.3.5}$ Duality Between Autoregressive and Moving Average Processes

$3.4$ Mixed Autoregressive--Moving Average Processes

$\textit {3.4.1}$ Stationarity and Invertibility Properties

$\textit {3.4.2}$ Autocorrelation Function and Spectrum of Mixed Processes

$\textit {3.4.3}$ First-Order Autoregressive--First-Order Moving Average Process

$\textit {3.4.4}$ Summary

$\text A 3.1$ Autocovariances, Autocovariance Generating Function, and Stationary Conditions for a General Linear Process

$\text A 3.2$ Recursive Method for Calculating Estimates of Autoregressive Parameters

$\textbf 4$ LINEAR NONSTATIONARY MODELS

$4.1$ Autoregressive Integrated Moving Average Processes

$\textit {4.1.1}$ Nonstationary First-Order Autoregressive Process

$\textit {4.1.2}$ General Model for a Nonstationary Process Exhibiting Homogeneity

$\textit {4.1.3}$ General Form of the Autoregressive Integrated Moving Average Process

$4.2$ Three Explicit Forms for the Autoregressive Integrated Moving Average Model

$\textit {4.2.1}$ Difference Equation Form of the Model

$\textit {4.2.2}$ Random Shock Form of the Model

$\textit {4.2.3}$ Inverted Form of the Model

$4.3$ Integrated Moving Average Processes

$\textit {4.3.1}$ Integrated Moving Average Process of Order $\tuple {0, 1, 1}$

$\textit {4.3.2}$ Integrated Moving Average Process of Order $\tuple {0, 2, 2}$

$\textit {4.3.3}$ General Integrated Moving Average Process or Order $\tuple {0, d, q}$

$\text A 4.1$ Linear Difference Equations

$\text A 4.2$ IMA $\tuple {0, 1, 1}$ Process with Deterministic Drift

$\text A 4.3$ ARIMA Processes with Added Noise

$\textit {A 4.3.1}$ Sum of Two Independent Moving Average Processes

$\textit {A 4.3.2}$ Effect of Added Noise on the General Model

$\textit {A 4.3.3}$ Example for an IMA $\tuple {0, 1, 1}$ Process with Added White Noise

$\textit {A 4.3.4}$ Relation Between the IMA $\tuple {0, 1, 1}$ Process and a Random Walk

$\textit {A 4.3.5}$ Autocovariance Function of the General Model with Added Correlated Noise

$\textbf 5$ FORECASTING

$5.1$ Minimum Mean Square Error Forecasts and Their Properties

$\textit {5.1.1}$ Derivation of the Minimum Mean Square Error Forecasts

$\textit {5.1.2}$ Three Basic Forms for the Forecast

$5.2$ Calculating and Updating Forecasts

$\textit {5.2.1}$ Convenient Format for the Forecasts

$\textit {5.2.2}$ Calculation of the $\phi$ Weights

$\textit {5.2.3}$ Use of the $\phi$ Weights in Updating the Forecasts

$\textit {5.2.4}$ Calculation of the Probability Limits of the Forecasts at Any Lead Time

$5.3$ Forecast Function and Forecast Weights

$\textit {5.3.1}$ Eventual Forecast Function Determined by the Autoregressive Operator

$\textit {5.3.2}$ Rule of the Moving Average Operator in Fixing the Initial Values

$\textit {5.3.3}$ Lead $1$ Forecast Weights

$5.4$ Examples of Forecast Functions and Their Updating

$\textit {5.4.1}$ Forecasting an IMA $\tuple {0, 1, 1}$ Process

$\textit {5.4.2}$ Forecasting an IMA $\tuple {0, 2, 2}$ Process

$\textit {5.4.3}$ Forecasting a General IMA $\tuple {0, d, q}$ Process

$\textit {5.4.4}$ Forecasting Autoregressive Processes

$\textit {5.4.5}$ Forecasting a $\tuple {1, 0, 1}$ Process

$\textit {5.4.6}$ Forecasting a $\tuple {1, 1, 1}$ Process

$5.5$ Use of State Model Formulation for Exact Forecasting

$\textit {5.5.1}$ State Space Model Representation for the ARIMA Process

$\textit {5.5.2}$ Kalman Filtering Relations for Use in Prediction

$5.6$ Summary

$\text A 5.1$ Correlations Between Forecast Errors

$\textit {A 5.1.1}$ Autocorrelation Function of Forecast Errors at Different Origins

$\textit {A 5.1.2}$ Correlation Between Forecast Errors at the Same Origin with Different Lead Times

$\text A 5.2$ Forecast Weights for Any Lead Time

$\text A 5.3$ Forecasting in Terms of the General Integrated Form

$\textit {A 5.3.1}$ General Method of Obtaining the Integrated Form

$\textit {A 5.3.2}$ Updating the General Integrated Form

$\textit {A 5.3.3}$ Comparison with the Discounted Least Squares Method

Part $\textbf {II}$ Stochastic Model Building

$\textbf 6$ MODEL IDENTIFICATION

$6.1$ Objectives of Identification

$\textit {6.1.1}$ Stages in the Identification Procedure

$6.2$ Identification Techniques

$\textit {6.2.1}$ Use of the Autocorrelation and Partial Autocorrelation Functions in Identification

$\textit {6.2.2}$ Standard Errors for Estimated Autocorrelations and Partial Autocorrelations

$\textit {6.2.3}$ Identification of Some Actual Time Series

$\textit {6.2.4}$ Some Additional Model Identification Tools

$6.3$ Initial Estimates for the Parameters

$\textit {6.3.1}$ Uniqueness of Estimates Obtained from the Autocovariance Function

$\textit {6.3.2}$ Initial Estimates for Moving Average Processes

$\textit {6.3.3}$ Initial Estimates for Autoregressive Processes

$\textit {6.3.4}$ Initial Estimates for Mixed Autoregressive--Moving Average Processes

$\textit {6.3.5}$ Choice Between Stationary and Nonstationary Models in Doubtful Cases

$\textit {6.3.6}$ More Formal Tests for Unit Roots in ARIMA Models

$\textit {6.3.7}$ Initial Estimate of Residual Variance

$\textit {6.3.8}$ Approximate Standard Error for $\overline w$

$6.4$ Model Multiplicity

$\textit {6.4.1}$ Multiplicity of Autoregressive--Moving Average Models

$\textit {6.4.2}$ Multiple Moment Solutions for Moving Average Parameters

$\textit {6.4.3}$ Use of the Backward Process to Determine Starting Values

$\text A 6.1$ Expected Behavior of the Estimated Autocorrelation Function for a Nonstationary Process

$\text A 6.2$ General Method for Obtaining Initial Estimates of the Parameters of a Mixed Autoregressive--Moving Average Process

$\textbf 7$ MODEL ESTIMATION

$7.1$ Study of the Likelihood and Sum of Squares Functions

$\textit {7.1.1}$ Likelihood Function

$\textit {7.1.2}$ Conditional Likelihood for an ARIMA Process

$\textit {7.1.3}$ Choice of Starting Values for Conditional Calculation

$\textit {7.1.4}$ Unconditional Likelihood; Sum of Squares Function; Least Squares Estimate

$\textit {7.1.5}$ General Procedure for Calculating the Unconditional Sum of Squares

$\textit {7.1.6}$ Graphical Study of the Sum of Squares Function

$\textit {7.1.7}$ Description of "Well-Behaved" Estimation Simulations; Confidence Regions

$7.2$ Nonlinear Estimation

$\textit {7.2.1}$ General Method of Approach

$\textit {7.2.2}$ Numerical Estimates of the Derivatives

$\textit {7.2.3}$ Direct Evaluation of the Derivatives

$\textit {7.2.4}$ General Least Squares Algorithm for the Conditional Matrix

$\textit {7.2.5}$ Summary of Models Fitted to Series A to F

$\textit {7.2.6}$ Large-Sample Information Matrices and Covariance Estimates

$7.3$ Some Estimation Results for Specific Models

$\textit {7.3.1}$ Autoregressive Processes

$\textit {7.3.2}$ Moving Average Processes

$\textit {7.3.3}$ Mixed Processes

$\textit {7.3.4}$ Separation of Linear and Nonlinear Components in Estimation

$\textit {7.3.5}$ Parameter Redundancy

$7.4$ Estimation using Bayes' Theorem

$\textit {7.4.1}$ Bayes' Theorem

$\textit {7.4.2}$ Bayesian Estimation of Parameters

$\textit {7.4.3}$ Autoregressive Processes

$\textit {7.4.4}$ Moving Average Processes

$\textit {7.4.5}$ Mixed processes

$7.5$ Likelihood Function Based on The State Space Model

$\text A 7.1$ Review of Normal Distribution Theory

$\textit {A 7.1.1}$ Partitioning of a Positive-Definite Quadratic FOrm

$\textit {A 7.1.2}$ Two Useful Integrals

$\textit {A 7.1.3}$ Normal Distribution

$\textit {A 7.1.4}$ Student's $t$-Distribution

$\text A 7.2$ Review of Linear Least Squares Theory

$\textit {A 7.2.1}$ Normal Equations

$\textit {A 7.2.2}$ Estimation of Residual Variance

$\textit {A 7.2.3}$ Covariance Matrix of Estimates

$\textit {A 7.2.4}$ Confidence Regions

$\textit {A 7.2.5}$ Correlated Errors

$\text A 7.3$ Exact Likelihood Function for Moving Average and Mixed Processes

$\text A 7.4$ Exact Likelihood Function for an Autoregressive Process

$\text A 7.5$ Examples of the Effect of Parameter Estimation

$\text A 7.6$ Special Note on Estimation of Moving Average Parameters

$\textbf 8$ MODEL DIAGNOSTIC CHECKING

$8.1$ Checking the Stochastic Model

$\textit {8.1.1}$ General Philosophy

$\textit {8.1.2}$ Overfitting

$8.2$ Diagnostic Checks Applied to Residuals

$\textit {8.2.1}$ Autocorrelation Check

$\textit {8.2.2}$ Portmanteau Lack-of-Fit Test

$\textit {8.2.3}$ Model Inadequacy Arising from Changes in Parameter Values

$\textit {8.2.4}$ Score Tests for Model Checking

$\textit {8.2.5}$ Cumulative Periodogram Check

$8.3$ Use of Residuals to Modify the Model

$\textit {8.3.1}$ Nature of the Correlations in the Residuals When an Incorrect Model Is Used

$\textit {8.3.2}$ Use of Residuals to Modify the Model

$\textbf 9$ SEASONAL MODELS

$9.1$ Parsimonious Models for Seasonal Time Series

$\textit {9.1.1}$ Fitting versus Forecasting

$\textit {9.1.2}$ Seasonal Models Involving Adaptive Sines and Cosines

$\textit {9.1.3}$ General Multiplicative Seasonal Model

$9.2$ Representation of the Airline Data by a Multiplicative $\tuple {0, 1, 1} \times \tuple {0, 1, 1}_{12}$ Seasonal Model

$\textit {9.2.1}$ Multiplicative $\tuple {0, 1, 1} \times \tuple {0, 1, 1}_{12}$ Model

$\textit {9.2.2}$ Forecasting

$\textit {9.2.3}$ Identification

$\textit {9.2.4}$ Estimation

$\textit {9.2.5}$ Diagnostic Checking

$9.3$ Some Aspects of More General Seasonal Models

$\textit {9.3.1}$ Multiplicative and Nonmultiplicative Models

$\textit {9.3.2}$ Identification

$\textit {9.3.3}$ Estimation

$\textit {9.3.4}$ Eventual Forecast Functions for Various Seasonal Models

$\textit {9.3.5}$ Choice of Transformation

$9.4$ Structural Component Models and Deterministic Seasonal Components

$\textit {9.4.1}$ Deterministic Seasonal and Trend Components and Common Factors

$\textit {9.4.2}$ Models with Regression Terms and Time Series Error Terms

$\text A 9.1$ Autocovariances for Some Seasonal Models

Part $\textbf {III}$ Transfer Function Model Building

$\textbf 10$ TRANSFER FUNCTION MODELS

$10.1$ Linear Transfer Function Models

$\textit {10.1.1}$ Discrete Transfer Function

$\textit {10.1.2}$ Continuous Dynamic Models Represented by Differential Equations

$10.2$ Discrete Dynamic Models Represented by Difference Equations

$\textit {10.2.1}$ General Form of the Difference Equation

$\textit {10.2.2}$ Nature of the Transfer Function

$\textit {10.2.3}$ First- and Second-Order Discrete Transfer Function Models

$\textit {10.2.4}$ Recursive Computation of Output for Any Input

$\textit {10.2.5}$ Transfer Function Models with Added Noise

$10.3$ Relation Between Discrete and Continuous Models

$\textit {10.3.1}$ Response to a Pulsed Input

$\textit {10.3.2}$ Relationships for First- and Second-Order Coincident Systems

$\textit {10.3.3}$ Approximating General Continuous Models by Discrete Models

$\text A 10.1$ Continuous Models With Pulsed Inputs

$\text A 10.2$ Nonlinear Transfer Functions and Linearization

$\textbf 11$ IDENTIFICATION, FITTING, AND CHECKING OF TRANSFER FUNCTION MODELS

$11.1$ Cross Correlation Function

$\textit {11.1.1}$ Properties of the Cross Covariance and Cross Correlation Functions

$\textit {11.1.2}$ Estimation of the Cross Covariance and Cross Correlation Functions

$\textit {11.1.3}$ Approximate Standard Errors of Cross Correlation Estimates

$11.2$ Identification of Transfer Function Models

$\textit {11.2.1}$ Identification of Transfer Function Models by Prewhitening the Input

$\textit {11.2.2}$ Example of the Identification of a Transfer Function Model

$\textit {11.2.3}$ Identification of the Noise Model

$\textit {11.2.4}$ Some General Considerations in Identifying Transfer Function Models

$11.3$ Fitting and Checking Transfer Function Models

$\textit {11.3.1}$ Conditional Sum of Squares Function

$\textit {11.3.2}$ Nonlinear Estimation

$\textit {11.3.3}$ Use of Residuals for Diagnostic Checking

$\textit {11.3.4}$ Specific Checks Applied to the Residuals

$11.4$ Some Examples of Fitting and Checking Transfer Function Models

$\textit {11.4.1}$ Fitting and Checking of the Gas Furnace Model

$\textit {11.4.2}$ Simulated Example with Two Inputs

$11.5$ Forecasting Using Leading Indicators

$\textit {11.5.1}$ Minimum Mean Square Error Forecast

$\textit {11.5.2}$ Forecast of $CO_2$ Output from Gas Furnace

$\textit {11.5.3}$ Forecast of Nonstationary Sales Data Using a Leading Indicator

$11.6$ Some Aspects of the Design of Experiments to Estimate Transfer Functions

$\text A 11.1$ Use of Cross Spectral Analysis of Experiment to Estimate Transfer Functions

$\textit {A 11.1.1}$ Identification of Single Input Transfer Function Models

$\textit {A 11.1.2}$ Identification of Multiple Input Transfer Models

$\text A 11.2$ Choice of Input to Provide Optimal Parameter Estimates

$\textit {A 11.2.1}$ Design of Optimal Inputs for a Simple System

$\textit {A 11.2.2}$ Numerical Example

$\textbf 12$ INTERVENTION ANALYSIS MODELS AND OUTLIER DETECTION

$12.1$ Intervention Analysis Methods

$\textit {12.1.1}$ Models for Intervention Analysis

$\textit {12.1.2}$ Example of Intervention Analysis

$\textit {12.1.3}$ Nature of the MLE for a Simple Level Change Parameter Model

$12.2$ Outlier Analysis for Time Series

$\textit {12.2.1}$ Models for Additive and Innovational Outliers

$\textit {12.2.2}$ Estimation of Outlier Effect for Known Timing of the Outlier

$\textit {12.2.3}$ Iterative Procedure for Outlier Detection

$\textit {12.2.4}$ Examples of Analysis of Outliers

$12.3$ Estimation for ARMA Models with Missing Values

Part $\textbf {IV}$ Design of Discrete Control Schemes

$\textbf 13$ ASPECTS OF PROCESS CONTROL

$13.1$ Process Monitoring and Process Adjustment

$\textit {13.1.1}$ Process Monitoring

$\textit {13.1.2}$ Process Adjustment

$13.2$ Process Adjustment Using Feedback Control

$\textit {13.2.1}$ Feedback Adjustment Chart

$\textit {13.2.2}$ Modelling the Feedback Loop

$\textit {13.2.3}$ Simple Models for Disturbances and Dynamics

$\textit {13.2.4}$ General Minimum Mean Square Error Feedback Control Schemes

$\textit {13.2.5}$ Manual Adjustment for Discrete Proportional--Integral Schemes

$\textit {13.2.6}$ Complementary Roles of Monitoring and Adjustment

$13.3$ Excessive Adjustment Sometimes Required by MMSE Control

$\textit {13.3.1}$ Constrained Control

$13.4$ Minimum Cost Control With Fixed Costs of Adjustment and Monitoring

$\textit {13.4.1}$ Bounded Adjustment Scheme for Fixed Adjustment Cost

$\textit {13.4.2}$ Indirect Approach for Obtaining a Bounded Adjustment Scheme

$\textit {13.4.3}$ Inclusion of the Cost of Monitoring

$13.5$ Monitoring Values of Parameters of Forecasting and Feedback Adjustment Schemes

$\text A 13.1$ Feedback Control Schemes Where the Adjustment Variance Is Restricted

$\textit {A 13.1.1}$ Derivation of Optimal Adjustment

$\text A 13.2$ Choice of the Sampling Interval

$\textit {A 13.2.1}$ Illustration of the Effect of Reducing Sampling Frequency

$\textit {A 13.2.2}$ Sampling an IMA $\tuple {0, 1, 1}$ Process

Part $\textbf {V}$ Charts and Tables

COLLECTION OF TABLES AND CHARTS

COLLECTION OF TIME SERIES USED FOR EXAMPLES IN THE TEXT AND IN EXERCISES

REFERENCES

Part $\textbf {VI}$

EXERCISES AND PROBLEMS

INDEX

Next

Further Editions

• 1970: George Box and Gwilym Jenkins: Time Series Analysis: Forecasting and Control
• 1976: George E.P. Box and Gwilym M. Jenkins: Time Series Analysis: Forecasting and Control (2nd ed.)
• 2008: George E.P. Box, Gwilym M. Jenkins and Gregory C. Reinsel: Time Series Analysis: Forecasting and Control (4th ed.)
• 2016: George E.P. Box, Gwilym M. Jenkins and Gregory C. Reinsel: Time Series Analysis: Forecasting and Control (5th ed.)

Source Work Progress

• 1994: George E.P. Box, Gwilym M. Jenkins and Gregory C. Reinsel: Time Series Analysis: Forecasting and Control (3rd ed.) ... (previous) ... (next):

$1$: Introduction:

$1.1$ Four Important Practical Problems:

$1.1.1$ Forecasting Time Series

Starting on Section $1.2$ with Next:

• 1994: George E.P. Box, Gwilym M. Jenkins and Gregory C. Reinsel: Time Series Analysis: Forecasting and Control (3rd ed.) ... (previous) ... (next):

$1$: Introduction:

$1.2$ Stochastic and Deterministic Dynamic Mathematical Models

$1.2.1$ Stationary and Nonstationary Stochastic Models for Forecasting and Control: Nonstationary models

Starting on Section $1.3$ with Next:

• 1994: George E.P. Box, Gwilym M. Jenkins and Gregory C. Reinsel: Time Series Analysis: Forecasting and Control (3rd ed.) ... (previous) ... (next):

Part $\text {I}$: Stochastic Models and their Forecasting:

$2$: Autocorrelation Function and Spectrum of Stationary Processes:

$2.1$ Autocorrelation Properties of Stationary Models:

$2.1.4$ Autocovariance and Autocorrelation Functions