# Book:George E.P. Box/Time Series Analysis: Forecasting and Control/Third Edition

## George E.P. Box, Gwilym M. Jenkins and Gregory C. Reinsel: Time Series Analysis: Forecasting and Control (3rd Edition)

Published $\text {1994}$, Prentice-Hall International, Inc.

ISBN 0-13-060774-6.

### Contents

PREFACE
$\textbf 1$ INTRODUCTION
$1.1$ Four Important Practical Problems
$\textit {1.1.1}$ Forecasting Time Series
$\textit {1.1.2}$ Estimation of Transfer Functions
$\textit {1.1.3}$ Analysis of Effects of Unusual Intervention Events
$\textit {1.1.4}$ Discrete Control Systems
$1.2$ Stochastic and Deterministic Dynamic Mathematical Models
$\textit {1.2.1}$ Stationary and Nonstationary Stochastic Models for Forecasting and Control
$\textit {1.2.2}$ Transfer Function Models
$\textit {1.2.3}$ Models for Discrete Control Systems
$1.3$ Basic Ideas in Model Building
$\textit {1.3.1}$ Parsimony
$\textit {1.3.2}$ Iterative Stages in the Selection of a Model

Part $\textbf {I}$ Stochastic Models and Their Forecasting
$\textbf 2$ AUTOCORRELATION FUNCTION AND SPECTRUM OF STATIONARY PROCESSES
$2.1$ Autocorrelation Properties of Stationary Models
$\textit {2.1.1}$ Time Series and Stochastic Processes
$\textit {2.1.2}$ Stationary Stochastic Processes
$\textit {2.1.3}$ Positive Definiteness and the Autocovariance Matrix
$\textit {2.1.4}$ Autocovariance and Autocorrelation Functions
$\textit {2.1.5}$ Estimation of Autocovariance and Autocorrelation Functions
$\textit {2.1.6}$ Standard Error of Autocorrelation Estimates
$2.2$ Spectral Properties of Stationary Models
$\textit {2.2.1}$ Periodogram of a Time Series
$\textit {2.2.2}$ Analysis of Variance
$\textit {2.2.3}$ Spectrum and Spectral Density Function
$\textit {2.2.4}$ Simple Examples of Autocorrelation and Spectral Density Functions
$\textit {2.2.5}$ Advantages and Disadvantages of the Autocorrelation and Spectral Density Functions
$\text A 2.1$ Link Between the Sample Spectrum and Autocovariance Function Estimate

$\textbf 3$ LINEAR STATIONARY MODELS
$3.1$ General Linear Process
$\textit {3.1.1}$ Two Equivalent Forms for the Linear Process
$\textit {3.1.2}$ Autocovariance Generating Function of a Linear Process
$\textit {3.1.3}$ Stationarity and Invertibility Conditions for a Linear Process
$\textit {3.1.4}$ Autoregressive and Moving Average Processes
$3.2$ Autoregressive Processes
$\textit {3.2.1}$ Stationarity Condituons for Autoregressive Processes
$\textit {3.2.2}$ Autocorrelation Function and Spectrum of Autoregressive Processes
$\textit {3.2.3}$ First-Order Autoregressive (Markov) Process
$\textit {3.2.4}$ Second-Order Autoregressive Process
$\textit {3.2.5}$ Partial Autocorrelation Function
$\textit {3.2.6}$ Estimation of the Partial Autocorrelation Function
$\textit {3.2.7}$ Standard Errors of Partial Autocorrelation Estimates
$3.3$ Moving Average Processes
$\textit {3.3.1}$ Invertibility Conditions for Moving Average Processes
$\textit {3.3.2}$ Autocorrelation Function and Spectrum of Moving Average Process
$\textit {3.3.3}$ First-Order Moving Average Process
$\textit {3.3.4}$ Second-Order Moving Average Process
$\textit {3.3.5}$ Duality Between Autoregressive and Moving Average Processes
$3.4$ Mixed Autoregressive--Moving Average Processes
$\textit {3.4.1}$ Stationarity and Invertibility Properties
$\textit {3.4.2}$ Autocorrelation Function and Spectrum of Mixed Processes
$\textit {3.4.3}$ First-Order Autoregressive--First-Order Moving Average Process
$\textit {3.4.4}$ Summary
$\text A 3.1$ Autocovariances, Autocovariance Generating Function, and Stationary Conditions for a General Linear Process
$\text A 3.2$ Recursive Method for Calculating Estimates of Autoregressive Parameters

$\textbf 4$ LINEAR NONSTATIONARY MODELS
$4.1$ Autoregressive Integrated Moving Average Processes
$\textit {4.1.1}$ Nonstationary First-Order Autoregressive Process
$\textit {4.1.2}$ General Model for a Nonstationary Process Exhibiting Homogeneity
$\textit {4.1.3}$ General Form of the Autoregressive Integrated Moving Average Process
$4.2$ Three Explicit Forms for the Autoregressive Integrated Moving Average Model
$\textit {4.2.1}$ Difference Equation Form of the Model
$\textit {4.2.2}$ Random Shock Form of the Model
$\textit {4.2.3}$ Inverted Form of the Model
$4.3$ Integrated Moving Average Processes
$\textit {4.3.1}$ Integrated Moving Average Process of Order $\tuple {0, 1, 1}$
$\textit {4.3.2}$ Integrated Moving Average Process of Order $\tuple {0, 2, 2}$
$\textit {4.3.3}$ General Integrated Moving Average Process or Order $\tuple {0, d, q}$
$\text A 4.1$ Linear Difference Equations
$\text A 4.2$ IMA $\tuple {0, 1, 1}$ Process with Deterministic Drift
$\text A 4.3$ ARIMA Processes with Added Noise
$\textit {A 4.3.1}$ Sum of Two Independent Moving Average Processes
$\textit {A 4.3.2}$ Effect of Added Noise on the General Model
$\textit {A 4.3.3}$ Example for an IMA $\tuple {0, 1, 1}$ Process with Added White Noise
$\textit {A 4.3.4}$ Relation Between the IMA $\tuple {0, 1, 1}$ Process and a Random Walk
$\textit {A 4.3.5}$ Autocovariance Function of the General Model with Added Correlated Noise

$\textbf 5$ FORECASTING
$5.1$ Minimum Mean Square Error Forecasts and Their Properties
$\textit {5.1.1}$ Derivation of the Minimum Mean Square Error Forecasts
$\textit {5.1.2}$ Three Basic Forms for the Forecast
$5.2$ Calculating and Updating Forecasts
$\textit {5.2.1}$ Convenient Format for the Forecasts
$\textit {5.2.2}$ Calculation of the $\phi$ Weights
$\textit {5.2.3}$ Use of the $\phi$ Weights in Updating the Forecasts
$\textit {5.2.4}$ Calculation of the Probability Limits of the Forecasts at Any Lead Time
$5.3$ Forecast Function and Forecast Weights
$\textit {5.3.1}$ Eventual Forecast Function Determined by the Autoregressive Operator
$\textit {5.3.2}$ Rule of the Moving Average Operator in Fixing the Initial Values
$\textit {5.3.3}$ Lead $1$ Forecast Weights
$5.4$ Examples of Forecast Functions and Their Updating
$\textit {5.4.1}$ Forecasting an IMA $\tuple {0, 1, 1}$ Process
$\textit {5.4.2}$ Forecasting an IMA $\tuple {0, 2, 2}$ Process
$\textit {5.4.3}$ Forecasting a General IMA $\tuple {0, d, q}$ Process
$\textit {5.4.4}$ Forecasting Autoregressive Processes
$\textit {5.4.5}$ Forecasting a $\tuple {1, 0, 1}$ Process
$\textit {5.4.6}$ Forecasting a $\tuple {1, 1, 1}$ Process
$5.5$ Use of State Model Formulation for Exact Forecasting
$\textit {5.5.1}$ State Space Model Representation for the ARIMA Process
$\textit {5.5.2}$ Kalman Filtering Relations for Use in Prediction
$5.6$ Summary
$\text A 5.1$ Correlations Between Forecast Errors
$\textit {A 5.1.1}$ Autocorrelation Function of Forecast Errors at Different Origins
$\textit {A 5.1.2}$ Correlation Between Forecast Errors at the Same Origin with Different Lead Times
$\text A 5.2$ Forecast Weights for Any Lead Time
$\text A 5.3$ Forecasting in Terms of the General Integrated Form
$\textit {A 5.3.1}$ General Method of Obtaining the Integrated Form
$\textit {A 5.3.2}$ Updating the General Integrated Form
$\textit {A 5.3.3}$ Comparison with the Discounted Least Squares Method

Part $\textbf {II}$ Stochastic Model Building
$\textbf 6$ MODEL IDENTIFICATION
$6.1$ Objectives of Identification
$\textit {6.1.1}$ Stages in the Identification Procedure
$6.2$ Identification Techniques
$\textit {6.2.1}$ Use of the Autocorrelation and Partial Autocorrelation Functions in Identification
$\textit {6.2.2}$ Standard Errors for Estimated Autocorrelations and Partial Autocorrelations
$\textit {6.2.3}$ Identification of Some Actual Time Series
$\textit {6.2.4}$ Some Additional Model Identification Tools
$6.3$ Initial Estimates for the Parameters
$\textit {6.3.1}$ Uniqueness of Estimates Obtained from the Autocovariance Function
$\textit {6.3.2}$ Initial Estimates for Moving Average Processes
$\textit {6.3.3}$ Initial Estimates for Autoregressive Processes
$\textit {6.3.4}$ Initial Estimates for Mixed Autoregressive--Moving Average Processes
$\textit {6.3.5}$ Choice Between Stationary and Nonstationary Models in Doubtful Cases
$\textit {6.3.6}$ More Formal Tests for Unit Roots in ARIMA Models
$\textit {6.3.7}$ Initial Estimate of Residual Variance
$\textit {6.3.8}$ Approximate Standard Error for $\overline w$
$6.4$ Model Multiplicity
$\textit {6.4.1}$ Multiplicity of Autoregressive--Moving Average Models
$\textit {6.4.2}$ Multiple Moment Solutions for Moving Average Parameters
$\textit {6.4.3}$ Use of the Backward Process to Determine Starting Values
$\text A 6.1$ Expected Behavior of the Estimated Autocorrelation Function for a Nonstationary Process
$\text A 6.2$ General Method for Obtaining Initial Estimates of the Parameters of a Mixed Autoregressive--Moving Average Process

$\textbf 7$ MODEL ESTIMATION
$7.1$ Study of the Likelihood and Sum of Squares Functions
$\textit {7.1.1}$ Likelihood Function
$\textit {7.1.2}$ Conditional Likelihood for an ARIMA Process
$\textit {7.1.3}$ Choice of Starting Values for Conditional Calculation
$\textit {7.1.4}$ Unconditional Likelihood; Sum of Squares Function; Least Squares Estimate
$\textit {7.1.5}$ General Procedure for Calculating the Unconditional Sum of Squares
$\textit {7.1.6}$ Graphical Study of the Sum of Squares Function
$\textit {7.1.7}$ Description of "Well-Behaved" Estimation Simulations; Confidence Regions
$7.2$ Nonlinear Estimation
$\textit {7.2.1}$ General Method of Approach
$\textit {7.2.2}$ Numerical Estimates of the Derivatives
$\textit {7.2.3}$ Direct Evaluation of the Derivatives
$\textit {7.2.4}$ General Least Squares Algorithm for the Conditional Matrix
$\textit {7.2.5}$ Summary of Models Fitted to Series A to F
$\textit {7.2.6}$ Large-Sample Information Matrices and Covariance Estimates
$7.3$ Some Estimation Results for Specific Models
$\textit {7.3.1}$ Autoregressive Processes
$\textit {7.3.2}$ Moving Average Processes
$\textit {7.3.3}$ Mixed Processes
$\textit {7.3.4}$ Separation of Linear and Nonlinear Components in Estimation
$\textit {7.3.5}$ Parameter Redundancy
$7.4$ Estimation using Bayes' Theorem
$\textit {7.4.1}$ Bayes' Theorem
$\textit {7.4.2}$ Bayesian Estimation of Parameters
$\textit {7.4.3}$ Autoregressive Processes
$\textit {7.4.4}$ Moving Average Processes
$\textit {7.4.5}$ Mixed processes
$7.5$ Likelihood Function Based on The State Space Model
$\text A 7.1$ Review of Normal Distribution Theory
$\textit {A 7.1.1}$ Partitioning of a Positive-Definite Quadratic FOrm
$\textit {A 7.1.2}$ Two Useful Integrals
$\textit {A 7.1.3}$ Normal Distribution
$\textit {A 7.1.4}$ Student's $t$-Distribution
$\text A 7.2$ Review of Linear Least Squares Theory
$\textit {A 7.2.1}$ Normal Equations
$\textit {A 7.2.2}$ Estimation of Residual Variance
$\textit {A 7.2.3}$ Covariance Matrix of Estimates
$\textit {A 7.2.4}$ Confidence Regions
$\textit {A 7.2.5}$ Correlated Errors
$\text A 7.3$ Exact Likelihood Function for Moving Average and Mixed Processes
$\text A 7.4$ Exact Likelihood Function for an Autoregressive Process
$\text A 7.5$ Examples of the Effect of Parameter Estimation
$\text A 7.6$ Special Note on Estimation of Moving Average Parameters

$\textbf 8$ MODEL DIAGNOSTIC CHECKING
$8.1$ Checking the Stochastic Model
$\textit {8.1.1}$ General Philosophy
$\textit {8.1.2}$ Overfitting
$8.2$ Diagnostic Checks Applied to Residuals
$\textit {8.2.1}$ Autocorrelation Check
$\textit {8.2.2}$ Portmanteau Lack-of-Fit Test
$\textit {8.2.3}$ Model Inadequacy Arising from Changes in Parameter Values
$\textit {8.2.4}$ Score Tests for Model Checking
$\textit {8.2.5}$ Cumulative Periodogram Check
$8.3$ Use of Residuals to Modify the Model
$\textit {8.3.1}$ Nature of the Correlations in the Residuals When an Incorrect Model Is Used
$\textit {8.3.2}$ Use of Residuals to Modify the Model

$\textbf 9$ SEASONAL MODELS
$9.1$ Parsimonious Models for Seasonal Time Series
$\textit {9.1.1}$ Fitting versus Forecasting
$\textit {9.1.2}$ Seasonal Models Involving Adaptive Sines and Cosines
$\textit {9.1.3}$ General Multiplicative Seasonal Model
$9.2$ Representation of the Airline Data by a Multiplicative $\tuple {0, 1, 1} \times \tuple {0, 1, 1}_{12}$ Seasonal Model
$\textit {9.2.1}$ Multiplicative $\tuple {0, 1, 1} \times \tuple {0, 1, 1}_{12}$ Model
$\textit {9.2.2}$ Forecasting
$\textit {9.2.3}$ Identification
$\textit {9.2.4}$ Estimation
$\textit {9.2.5}$ Diagnostic Checking
$9.3$ Some Aspects of More General Seasonal Models
$\textit {9.3.1}$ Multiplicative and Nonmultiplicative Models
$\textit {9.3.2}$ Identification
$\textit {9.3.3}$ Estimation
$\textit {9.3.4}$ Eventual Forecast Functions for Various Seasonal Models
$\textit {9.3.5}$ Choice of Transformation
$9.4$ Structural Component Models and Deterministic Seasonal Components
$\textit {9.4.1}$ Deterministic Seasonal and Trend Components and Common Factors
$\textit {9.4.2}$ Models with Regression Terms and Time Series Error Terms
$\text A 9.1$ Autocovariances for Some Seasonal Models

Part $\textbf {III}$ Transfer Function Model Building
$\textbf 10$ TRANSFER FUNCTION MODELS
$10.1$ Linear Transfer Function Models
$\textit {10.1.1}$ Discrete Transfer Function
$\textit {10.1.2}$ Continuous Dynamic Models Represented by Differential Equations
$10.2$ Discrete Dynamic Models Represented by Difference Equations
$\textit {10.2.1}$ General Form of the Difference Equation
$\textit {10.2.2}$ Nature of the Transfer Function
$\textit {10.2.3}$ First- and Second-Order Discrete Transfer Function Models
$\textit {10.2.4}$ Recursive Computation of Output for Any Input
$\textit {10.2.5}$ Transfer Function Models with Added Noise
$10.3$ Relation Between Discrete and Continuous Models
$\textit {10.3.1}$ Response to a Pulsed Input
$\textit {10.3.2}$ Relationships for First- and Second-Order Coincident Systems
$\textit {10.3.3}$ Approximating General Continuous Models by Discrete Models
$\text A 10.1$ Continuous Models With Pulsed Inputs
$\text A 10.2$ Nonlinear Transfer Functions and Linearization

$\textbf 11$ IDENTIFICATION, FITTING, AND CHECKING OF TRANSFER FUNCTION MODELS
$11.1$ Cross Correlation Function
$\textit {11.1.1}$ Properties of the Cross Covariance and Cross Correlation Functions
$\textit {11.1.2}$ Estimation of the Cross Covariance and Cross Correlation Functions
$\textit {11.1.3}$ Approximate Standard Errors of Cross Correlation Estimates
$11.2$ Identification of Transfer Function Models
$\textit {11.2.1}$ Identification of Transfer Function Models by Prewhitening the Input
$\textit {11.2.2}$ Example of the Identification of a Transfer Function Model
$\textit {11.2.3}$ Identification of the Noise Model
$\textit {11.2.4}$ Some General Considerations in Identifying Transfer Function Models
$11.3$ Fitting and Checking Transfer Function Models
$\textit {11.3.1}$ Conditional Sum of Squares Function
$\textit {11.3.2}$ Nonlinear Estimation
$\textit {11.3.3}$ Use of Residuals for Diagnostic Checking
$\textit {11.3.4}$ Specific Checks Applied to the Residuals
$11.4$ Some Examples of Fitting and Checking Transfer Function Models
$\textit {11.4.1}$ Fitting and Checking of the Gas Furnace Model
$\textit {11.4.2}$ Simulated Example with Two Inputs
$11.5$ Forecasting Using Leading Indicators
$\textit {11.5.1}$ Minimum Mean Square Error Forecast
$\textit {11.5.2}$ Forecast of $CO_2$ Output from Gas Furnace
$\textit {11.5.3}$ Forecast of Nonstationary Sales Data Using a Leading Indicator
$11.6$ Some Aspects of the Design of Experiments to Estimate Transfer Functions
$\text A 11.1$ Use of Cross Spectral Analysis of Experiment to Estimate Transfer Functions
$\textit {A 11.1.1}$ Identification of Single Input Transfer Function Models
$\textit {A 11.1.2}$ Identification of Multiple Input Transfer Models
$\text A 11.2$ Choice of Input to Provide Optimal Parameter Estimates
$\textit {A 11.2.1}$ Design of Optimal Inputs for a Simple System
$\textit {A 11.2.2}$ Numerical Example

$\textbf 12$ INTERVENTION ANALYSIS MODELS AND OUTLIER DETECTION
$12.1$ Intervention Analysis Methods
$\textit {12.1.1}$ Models for Intervention Analysis
$\textit {12.1.2}$ Example of Intervention Analysis
$\textit {12.1.3}$ Nature of the MLE for a Simple Level Change Parameter Model
$12.2$ Outlier Analysis for Time Series
$\textit {12.2.1}$ Models for Additive and Innovational Outliers
$\textit {12.2.2}$ Estimation of Outlier Effect for Known Timing of the Outlier
$\textit {12.2.3}$ Iterative Procedure for Outlier Detection
$\textit {12.2.4}$ Examples of Analysis of Outliers
$12.3$ Estimation for ARMA Models with Missing Values

Part $\textbf {IV}$ Design of Discrete Control Schemes
$\textbf 13$ ASPECTS OF PROCESS CONTROL
$13.1$ Process Monitoring and Process Adjustment
$\textit {13.1.1}$ Process Monitoring
$\textit {13.1.2}$ Process Adjustment
$13.2$ Process Adjustment Using Feedback Control
$\textit {13.2.1}$ Feedback Adjustment Chart
$\textit {13.2.2}$ Modelling the Feedback Loop
$\textit {13.2.3}$ Simple Models for Disturbances and Dynamics
$\textit {13.2.4}$ General Minimum Mean Square Error Feedback Control Schemes
$\textit {13.2.5}$ Manual Adjustment for Discrete Proportional--Integral Schemes
$\textit {13.2.6}$ Complementary Roles of Monitoring and Adjustment
$13.3$ Excessive Adjustment Sometimes Required by MMSE Control
$\textit {13.3.1}$ Constrained Control
$13.4$ Minimum Cost Control With Fixed Costs of Adjustment and Monitoring
$\textit {13.4.1}$ Bounded Adjustment Scheme for Fixed Adjustment Cost
$\textit {13.4.2}$ Indirect Approach for Obtaining a Bounded Adjustment Scheme
$\textit {13.4.3}$ Inclusion of the Cost of Monitoring
$13.5$ Monitoring Values of Parameters of Forecasting and Feedback Adjustment Schemes
$\text A 13.1$ Feedback Control Schemes Where the Adjustment Variance Is Restricted
$\textit {A 13.1.1}$ Derivation of Optimal Adjustment
$\text A 13.2$ Choice of the Sampling Interval
$\textit {A 13.2.1}$ Illustration of the Effect of Reducing Sampling Frequency
$\textit {A 13.2.2}$ Sampling an IMA $\tuple {0, 1, 1}$ Process

Part $\textbf {V}$ Charts and Tables
COLLECTION OF TABLES AND CHARTS
COLLECTION OF TIME SERIES USED FOR EXAMPLES IN THE TEXT AND IN EXERCISES
REFERENCES

Part $\textbf {VI}$
EXERCISES AND PROBLEMS

INDEX

Next

## Source Work Progress

$1$: Introduction:
$1.1$ Four Important Practical Problems:
$1.1.1$ Forecasting Time Series

Starting on Section $1.2$ with Next:

$1$: Introduction:
$1.2$ Stochastic and Deterministic Dynamic Mathematical Models
$1.2.1$ Stationary and Nonstationary Stochastic Models for Forecasting and Control: Nonstationary models

Starting on Section $1.3$ with Next:

Part $\text {I}$: Stochastic Models and their Forecasting:
$2$: Autocorrelation Function and Spectrum of Stationary Processes:
$2.1$ Autocorrelation Properties of Stationary Models:
$2.1.4$ Autocovariance and Autocorrelation Functions