Book:George E.P. Box/Time Series Analysis: Forecasting and Control/Third Edition

Subject Matter

 * Probability Theory
 * Statistics

Contents

 * PREFACE


 * $\textbf 1$ INTRODUCTION


 * $1.1$ Four Important Practical Problems
 * $\textit {1.1.1}$ Forecasting Time Series
 * $\textit {1.1.2}$ Estimation of Transfer Functions
 * $\textit {1.1.3}$ Analysis of Effects of Unusual Intervention Events
 * $\textit {1.1.4}$ Discrete Control Systems


 * $1.2$ Stochastic and Deterministic Dynamic Mathematical Models
 * $\textit {1.2.1}$ Stationary and Nonstationary Stochastic Models for Forecasting and Control
 * $\textit {1.2.2}$ Transfer Function Models
 * $\textit {1.2.3}$ Models for Discrete Control Systems


 * $1.3$ Basic Ideas in Model Building
 * $\textit {1.3.1}$ Parsimony
 * $\textit {1.3.2}$ Iterative Stages in the Selection of a Model


 * Part $\textbf {I}$ Stochastic Models and Their Forecasting


 * $\textbf 2$ AUTOCORRELATION FUNCTION AND SPECTRUM OF STATIONARY PROCESSES


 * $2.1$ Autocorrelation Properties of Stationary Models
 * $\textit {2.1.1}$ Time Series and Stochastic Processes
 * $\textit {2.1.2}$ Stationary Stochastic Processes
 * $\textit {2.1.3}$ Positive Definiteness and the Autocovariance Matrix
 * $\textit {2.1.4}$ Autocovariance and Autocorrelation Functions
 * $\textit {2.1.5}$ Estimation of Autocovariance and Autocorrelation Functions
 * $\textit {2.1.6}$ Standard Error of Autocorrelation Estimates


 * $2.2$ Spectral Properties of Stationary Models
 * $\textit {2.2.1}$ Periodogram of a Time Series
 * $\textit {2.2.2}$ Analysis of Variance
 * $\textit {2.2.3}$ Spectrum and Spectral Density Function
 * $\textit {2.2.4}$ Simple Examples of Autocorrelation and Spectral Density Functions
 * $\textit {2.2.5}$ Advantages and Disadvantages of the Autocorrelation and Spectral Density Functions


 * $\text A 2.1$ Link Between the Sample Spectrum and Autocovariance Function Estimate


 * $\textbf 3$ LINEAR STATIONARY MODELS


 * $3.1$ General Linear Process
 * $\textit {3.1.1}$ Two Equivalent Forms for the Linear Process
 * $\textit {3.1.2}$ Autocovariance Generating Function of a Linear Process
 * $\textit {3.1.3}$ Stationarity and Invertibility Conditions for a Linear Process
 * $\textit {3.1.4}$ Autoregressive and Moving Average Processes


 * $3.2$ Autoregressive Processes
 * $\textit {3.2.1}$ Stationarity Condituons for Autoregressive Processes
 * $\textit {3.2.2}$ Autocorrelation Function and Spectrum of Autoregressive Processes
 * $\textit {3.2.3}$ First-Order Autoregressive (Markov) Process
 * $\textit {3.2.4}$ Second-Order Autoregressive Process
 * $\textit {3.2.5}$ Partial Autocorrelation Function
 * $\textit {3.2.6}$ Estimation of the Partial Autocorrelation Function
 * $\textit {3.2.7}$ Standard Errors of Partial Autocorrelation Estimates


 * $3.3$ Moving Average Processes
 * $\textit {3.3.1}$ Invertibility Conditions for Moving Average Processes
 * $\textit {3.3.2}$ ''Autocorrelation Function and Spectrum of Moving Average Process
 * $\textit {3.3.3}$ First-Order Moving Average Process
 * $\textit {3.3.4}$ Second-Order Moving Average Process
 * $\textit {3.3.5}$ Duality Between Autoregressive and Moving Average Processes


 * $3.4$ Mixed Autoregressive--Moving Average Processes
 * $\textit {3.4.1}$ Stationarity and Invertibility Properties
 * $\textit {3.4.2}$ Autocorrelation Function and Spectrum of Mixed Processes
 * $\textit {3.4.3}$ First-Order Autoregressive--First-Order Moving Average Process
 * $\textit {3.4.4}$ Summary


 * $\text A 3.1$ Autocovariances, Autocovariance Generating Function, and Stationary Conditions for a General Linear Process


 * $\text A 3.2$ Recursive Method for Calculating Estimates of Autoregressive Parameters


 * $\textbf 4$ LINEAR NONSTATIONARY MODELS


 * $4.1$ Autoregressive Integrated Moving Average Processes
 * $\textit {4.1.1}$ Nonstationary First-Order Autoregressive Process
 * $\textit {4.1.2}$ General Model for a Nonstationary Process Exhibiting Homogeneity
 * $\textit {4.1.3}$ General Form of the Autoregressive Integrated Moving Average Process


 * $4.2$ Three Explicit Forms for the Autoregressive Integrated Moving Average Model
 * $\textit {4.2.1}$ Difference Equation Form of the Model
 * $\textit {4.2.2}$ Random Shock Form of the Model
 * $\textit {4.2.3}$ Inverted Form of the Model


 * $4.3$ Integrated Moving Average Processes
 * $\textit {4.3.1}$ Integrated Moving Average Process of Order $\tuple {0, 1, 1}$
 * $\textit {4.3.2}$ Integrated Moving Average Process of Order $\tuple {0, 2, 2}$
 * $\textit {4.3.3}$ General Integrated Moving Average Process or Order $\tuple {0, d, q}$


 * $\text A 4.1$ Linear Difference Equations


 * $\text A 4.2$ IMA $\tuple {0, 1, 1}$ Process with Deterministic Drift


 * $\text A 4.3$ ARIMA Processes with Added Noise
 * $\textit {A 4.3.1}$ Sum of Two Independent Moving Average Processes
 * $\textit {A 4.3.2}$ Effect of Added Noise on the General Model
 * $\textit {A 4.3.3}$ Example for an IMA $\tuple {0, 1, 1}$ Process with Added White Noise
 * $\textit {A 4.3.4}$ Relation Between the IMA $\tuple {0, 1, 1}$ Process and a Random Walk
 * $\textit {A 4.3.5}$ Autocovariance Function of the General Model with Added Correlated Noise


 * $\textbf 5$ FORECASTING


 * $5.1$ Minimum Mean Square Error Forecasts and Their Properties
 * $\textit {5.1.1}$ Derivation of the Minimum Mean Square Error Forecasts
 * $\textit {5.1.2}$ Three Basic Forms for the Forecast


 * $5.2$ Calculating and Updating Forecasts
 * $\textit {5.2.1}$ Convenient Format for the Forecasts
 * $\textit {5.2.2}$ Calculation of the $\phi$ Weights
 * $\textit {5.2.3}$ Use of the $\phi$ Weights in Updating the Forecasts
 * $\textit {5.2.4}$ Calculation of the Probability Limits of the Forecasts at Any Lead Time


 * $5.3$ Forecast Function and Forecast Weights
 * $\textit {5.3.1}$ Eventual Forecast Function Determined by the Autoregressive Operator
 * $\textit {5.3.2}$ Rule of the Moving Average Operator in Fixing the Initial Values
 * $\textit {5.3.3}$ Lead $1$ Forecast Weights


 * $5.4$ Examples of Forecast Functions and Their Updating
 * $\textit {5.4.1}$ Forecasting an IMA $\tuple {0, 1, 1}$ Process
 * $\textit {5.4.2}$ Forecasting an IMA $\tuple {0, 2, 2}$ Process
 * $\textit {5.4.3}$ Forecasting a General IMA $\tuple {0, d, q}$ Process
 * $\textit {5.4.4}$ Forecasting Autoregressive Processes
 * $\textit {5.4.5}$ Forecasting a $\tuple {1, 0, 1}$ Process
 * $\textit {5.4.6}$ Forecasting a $\tuple {1, 1, 1}$ Process


 * $5.5$ Use of State Model Formulation for Exact Forecasting
 * $\textit {5.5.1}$ State Space Model Representation for the ARIMA Process
 * $\textit {5.5.2}$ ''Kalman Filtering Relations for Use in Prediction


 * $5.6$ Summary


 * $\text A 5.1$ Correlations Between Forecast Errors
 * $\textit {A 5.1.1}$ Autocorrelation Function of Forecast Errors at Different Origins
 * $\textit {A 5.1.2}$ Correlation Between Forecast Errors at the Same Origin with Different Lead Times


 * $\text A 5.2$ Forecast Weights for Any Lead Time


 * $\text A 5.3$ Forecasting in Terms of the General Integrated Form
 * $\textit {A 5.3.1}$ General Method of Obtaining the Integrated Form
 * $\textit {A 5.3.2}$ Updating the General Integrated Form
 * $\textit {A 5.3.3}$ Comparison with the Discounted Least Squares Method


 * Part $\textbf {II}$ Stochastic Model Building


 * $\textbf 6$ MODEL IDENTIFICATION


 * $6.1$ Objectives of Identification
 * $\textit {6.1.1}$ Stages in the Identification Procedure


 * $6.2$ Identification Techniques
 * $\textit {6.2.1}$ Use of the Autocorrelation and Partial Autocorrelation Functions in Identification
 * $\textit {6.2.2}$ Standard Errors for Estimated Autocorrelations and Partial Autocorrelations
 * $\textit {6.2.3}$ Identification of Some Actual Time Series
 * $\textit {6.2.4}$ Some Additional Model Identification Tools


 * $6.3$ Initial Estimates for the Parameters
 * $\textit {6.3.1}$ Uniqueness of Estimates Obtained from the Autocovariance Function
 * $\textit {6.3.2}$ Initial Estimates for Moving Average Processes
 * $\textit {6.3.3}$ Initial Estimates for Autoregressive Processes
 * $\textit {6.3.4}$ Initial Estimates for Mixed Autoregressive--Moving Average Processes
 * $\textit {6.3.5}$ Choice Between Stationary and Nonstationary Models in Doubtful Cases
 * $\textit {6.3.6}$ More Formal Tests for Unit Roots in ARIMA Models
 * $\textit {6.3.7}$ Initial Estimate of Residual Variance
 * $\textit {6.3.8}$ Approximate Standard Error for $\overline w$


 * $6.4$ Model Multiplicity
 * $\textit {6.4.1}$ Multiplicity of Autoregressive--Moving Average Models
 * $\textit {6.4.2}$ Multiple Moment Solutions for Moving Average Parameters
 * $\textit {6.4.3}$ Use of the Backward Process to Determine Starting Values


 * $\text A 6.1$ Expected Behavior of the Estimated Autocorrelation Function for a Nonstationary Process


 * $\text A 6.2$ General Method for Obtaining Initial Estimates of the Parameters of a Mixed Autoregressive--Moving Average Process


 * $\textbf 7$ MODEL ESTIMATION


 * $7.1$ Study of the Likelihood and Sum of Squares Functions
 * $\textit {7.1.1}$ Likelihood Function
 * $\textit {7.1.2}$ Conditional Likelihood for an ARIMA Process
 * $\textit {7.1.3}$ Choice of Starting Values for Conditional Calculation
 * $\textit {7.1.4}$ Unconditional Likelihood; Sum of Squares Function; Least Squares Estimate
 * $\textit {7.1.5}$ General Procedure for Calculating the Unconditional Sum of Squares
 * $\textit {7.1.6}$ Graphical Study of the Sum of Squares Function
 * $\textit {7.1.7}$ Description of "Well-Behaved" Estimation Simulations; Confidence Regions


 * $7.2$ Nonlinear Estimation
 * $\textit {7.2.1}$ General Method of Approach
 * $\textit {7.2.2}$ Numerical Estimates of the Derivatives
 * $\textit {7.2.3}$ Direct Evaluation of the Derivatives
 * $\textit {7.2.4}$ General Least Squares Algorithm for the Conditional Matrix
 * $\textit {7.2.5}$ Summary of Models Fitted to Series A to F
 * $\textit {7.2.6}$ Large-Sample Information Matrices and Covariance Estimates


 * $7.3$ Some Estimation Results for Specific Models
 * $\textit {7.3.1}$ Autoregressive Processes
 * $\textit {7.3.2}$ Moving Average Processes
 * $\textit {7.3.3}$ Mixed Processes
 * $\textit {7.3.4}$ Separation of Linear and Nonlinear Components in Estimation
 * $\textit {7.3.5}$ Parameter Redundancy


 * $7.4$ Estimation using Bayes' Theorem
 * $\textit {7.4.1}$ Bayes' Theorem
 * $\textit {7.4.2}$ Bayesian Estimation of Parameters
 * $\textit {7.4.3}$ Autoregressive Processes
 * $\textit {7.4.4}$ Moving Average Processes
 * $\textit {7.4.5}$ Mixed processes


 * $7.5$ Likelihood Function Based on The State Space Model


 * $\text A 7.1$ Review of Normal Distribution Theory
 * $\textit {A 7.1.1}$ Partitioning of a Positive-Definite Quadratic FOrm
 * $\textit {A 7.1.2}$ Two Useful Integrals
 * $\textit {A 7.1.3}$ Normal Distribution
 * $\textit {A 7.1.4}$ Student's $t$-Distribution


 * $\text A 7.2$ Review of Linear Least Squares Theory
 * $\textit {A 7.2.1}$ Normal Equations
 * $\textit {A 7.2.2}$ Estimation of Residual Variance
 * $\textit {A 7.2.3}$ Covariance Matrix of Estimates
 * $\textit {A 7.2.4}$ Confidence Regions
 * $\textit {A 7.2.5}$ Correlated Errors


 * $\text A 7.3$ Exact Likelihood Function for Moving Average and Mixed Processes


 * $\text A 7.4$ Exact Likelihood Function for an Autoregressive Process


 * $\text A 7.5$ Examples of the Effect of Parameter Estimation


 * $\text A 7.6$ Special Note on Estimation of Moving Average Parameters


 * $\textbf 8$ MODEL DIAGNOSTIC CHECKING


 * $8.1$ Checking the Stochastic Model
 * $\textit {8.1.1}$ General Philosophy
 * $\textit {8.1.2}$ Overfitting


 * $8.2$ Diagnostic Checks Applied to Residuals
 * $\textit {8.2.1}$ Autocorrelation Check
 * $\textit {8.2.2}$ Portmanteau Lack-of-Fit Test
 * $\textit {8.2.3}$ Model Inadequacy Arising from Changes in Parameter Values
 * $\textit {8.2.4}$ Score Tests for Model Checking
 * $\textit {8.2.5}$ Cumulative Periodogram Check


 * $8.3$ Use of Residuals to Modify the Model
 * $\textit {8.3.1}$ Nature of the Correlations in the Residuals When an Incorrect Model Is Used
 * $\textit {8.3.2}$ Use of Residuals to Modify the Model


 * $\textbf 9$ SEASONAL MODELS


 * $9.1$ Parsimonious Models for Seasonal Time Series
 * $\textit {9.1.1}$  Fitting versus Forecasting
 * $\textit {9.1.2}$ Seasonal Models Involving Adaptive Sines and Cosines
 * $\textit {9.1.3}$ General Multiplicative Seasonal Model


 * $9.2$ Representation of the Airline Data by a Multiplicative $\tuple {0, 1, 1} \times \tuple {0, 1, 1}_{12}$ Seasonal Model
 * $\textit {9.2.1}$ Multiplicative $\tuple {0, 1, 1} \times \tuple {0, 1, 1}_{12}$ Model
 * $\textit {9.2.2}$ Forecasting
 * $\textit {9.2.3}$ Identification
 * $\textit {9.2.4}$ Estimation
 * $\textit {9.2.5}$ Diagnostic Checking


 * $9.3$ Some Aspects of More General Seasonal Models
 * $\textit {9.3.1}$ Multiplicative and Nonmultiplicative Models
 * $\textit {9.3.2}$ Identification
 * $\textit {9.3.3}$ Estimation
 * $\textit {9.3.4}$ Eventual Forecast Functions for Various Seasonal Models
 * $\textit {9.3.5}$ Choice of Transformation


 * $9.4$ Structural Component Models and Deterministic Seasonal Components
 * $\textit {9.4.1}$ Deterministic Seasonal and Trend Components and Common Factors
 * $\textit {9.4.2}$ Models with Regression Terms and Time Series Error Terms


 * $\text A 9.1$ Autocovariances for Some Seasonal Models


 * Part $\textbf {III}$ Transfer Function Model Building


 * $\textbf 10$ TRANSFER FUNCTION MODELS


 * $10.1$ Linear Transfer Function Models
 * $\textit {10.1.1}$ Discrete Transfer Function
 * $\textit {10.1.2}$ Continuous Dynamic Models Represented by Differential Equations


 * $10.2$ Discrete Dynamic Models Represented by Difference Equations
 * $\textit {10.2.1}$ General Form of the Difference Equation
 * $\textit {10.2.2}$ Nature of the Transfer Function
 * $\textit {10.2.3}$ First- and Second-Order Discrete Transfer Function Models
 * $\textit {10.2.4}$ Recursive Computation of Output for Any Input
 * $\textit {10.2.5}$ Transfer Function Models with Added Noise


 * $10.3$ Relation Between Discrete and Continuous Models
 * $\textit {10.3.1}$ Response to a Pulsed Input
 * $\textit {10.3.2}$ Relationships for First- and Second-Order Coincident Systems
 * $\textit {10.3.3}$ Approximating General Continuous Models by Discrete Models


 * $\text A 10.1$ Continuous Models With Pulsed Inputs


 * $\text A 10.2$ Nonlinear Transfer Functions and Linearization


 * $\textbf 11$ IDENTIFICATION, FITTING, AND CHECKING OF TRANSFER FUNCTION MODELS


 * $11.1$ Cross Correlation Function
 * $\textit {11.1.1}$ Properties of the Cross Covariance and Cross Correlation Functions
 * $\textit {11.1.2}$ Estimation of the Cross Covariance and Cross Correlation Functions
 * $\textit {11.1.3}$ Approximate Standard Errors of Cross Correlation Estimates


 * $11.2$ Identification of Transfer Function Models
 * $\textit {11.2.1}$ Identification of Transfer Function Models by Prewhitening the Input
 * $\textit {11.2.2}$ Example of the Identification of a Transfer Function Model
 * $\textit {11.2.3}$ Identification of the Noise Model
 * $\textit {11.2.4}$ Some General Considerations in Identifying Transfer Function Models


 * $11.3$ Fitting and Checking Transfer Function Models
 * $\textit {11.3.1}$ Conditional Sum of Squares Function
 * $\textit {11.3.2}$ Nonlinear Estimation
 * $\textit {11.3.3}$ Use of Residuals for Diagnostic Checking
 * $\textit {11.3.4}$ Specific Checks Applied to the Residuals


 * $11.4$ Some Examples of Fitting and Checking Transfer Function Models
 * $\textit {11.4.1}$ Fitting and Checking of the Gas Furnace Model
 * $\textit {11.4.2}$ Simulated Example with Two Inputs


 * $11.5$ Forecasting Using Leading Indicators
 * $\textit {11.5.1}$ Minimum Mean Square Error Forecast
 * $\textit {11.5.2}$ Forecast of $CO_2$ Output from Gas Furnace
 * $\textit {11.5.3}$ Forecast of Nonstationary Sales Data Using a Leading Indicator


 * $11.6$ Some Aspects of the Design of Experiments to Estimate Transfer Functions


 * $\text A 11.1$ Use of Cross Spectral Analysis of Experiment to Estimate Transfer Functions
 * $\textit {A 11.1.1}$ Identification of Single Input Transfer Function Models
 * $\textit {A 11.1.2}$ Identification of Multiple Input Transfer Models


 * $\text A 11.2$ Choice of Input to Provide Optimal Parameter Estimates
 * $\textit {A 11.2.1}$ Design of Optimal Inputs for a Simple System
 * $\textit {A 11.2.2}$ Numerical Example


 * $\textbf 12$ INTERVENTION ANALYSIS MODELS AND OUTLIER DETECTION


 * $12.1$ Intervention Analysis Methods
 * $\textit {12.1.1}$ Models for Intervention Analysis
 * $\textit {12.1.2}$ Example of Intervention Analysis
 * $\textit {12.1.3}$ Nature of the MLE for a Simple Level Change Parameter Model


 * $12.2$ Outlier Analysis for Time Series
 * $\textit {12.2.1}$ Models for Additive and Innovational Outliers
 * $\textit {12.2.2}$ Estimation of Outlier Effect for Known Timing of the Outlier
 * $\textit {12.2.3}$ Iterative Procedure for Outlier Detection
 * $\textit {12.2.4}$ Examples of Analysis of Outliers


 * $12.3$ Estimation for ARMA Models with Missing Values


 * Part $\textbf {IV}$ Design of Discrete Control Schemes


 * $\textbf 13$ ASPECTS OF PROCESS CONTROL


 * $13.1$ Process Monitoring and Process Adjustment
 * $\textit {13.1.1}$ Process Monitoring
 * $\textit {13.1.2}$ Process Adjustment


 * $13.2$ Process Adjustment Using Feedback Control
 * $\textit {13.2.1}$ Feedback Adjustment Chart
 * $\textit {13.2.2}$ Modelling the Feedback Loop
 * $\textit {13.2.3}$ Simple Models for Disturbances and Dynamics
 * $\textit {13.2.4}$ General Minimum Mean Square Error Feedback Control Schemes
 * $\textit {13.2.5}$ Manual Adjustment for Discrete Proportional--Integral Schemes
 * $\textit {13.2.6}$ Complementary Roles of Monitoring and Adjustment


 * $13.3$ Excessive Adjustment Sometimes Required by MMSE Control
 * $\textit {13.3.1}$ Constrained Control


 * $13.4$ Minimum Cost Control With Fixed Costs of Adjustment and Monitoring
 * $\textit {13.4.1}$ Bounded Adjustment Scheme for Fixed Adjustment Cost
 * $\textit {13.4.2}$ Indirect Approach for Obtaining a Bounded Adjustment Scheme
 * $\textit {13.4.3}$ Inclusion of the Cost of Monitoring


 * $13.5$ Monitoring Values of Parameters of Forecasting and Feedback Adjustment Schemes


 * $\text A 13.1$ Feedback Control Schemes Where the Adjustment Variance Is Restricted
 * $\textit {A 13.1.1}$ Derivation of Optimal Adjustment


 * $\text A 13.2$ Choice of the Sampling Interval
 * $\textit {A 13.2.1}$ Illustration of the Effect of Reducing Sampling Frequency
 * $\textit {A 13.2.2}$ Sampling an IMA $\tuple {0, 1, 1}$ Process


 * Part $\textbf {V}$ Charts and Tables


 * '''COLLECTION OF TABLES AND CHARTS


 * COLLECTION OF TIME SERIES USED FOR EXAMPLES IN THE TEXT AND IN EXERCISES


 * REFERENCES


 * Part $\textbf {VI}$


 * EXERCISES AND PROBLEMS


 * INDEX



Source Work Progress
* :
 * $1$: Introduction:
 * $1.1$ Four Important Practical Problems:
 * $1.1.1$ Forecasting Time Series

Starting on Section $1.2$ with :


 * $1$: Introduction:
 * $1.2$ Stochastic and Deterministic Dynamic Mathematical Models
 * $1.2.1$ Stationary and Nonstationary Stochastic Models for Forecasting and Control: Nonstationary models
 * $1.2.1$ Stationary and Nonstationary Stochastic Models for Forecasting and Control: Nonstationary models

Starting on Section $1.3$ with :


 * Part $\text {I}$: Stochastic Models and their Forecasting:
 * $2$: Autocorrelation Function and Spectrum of Stationary Processes:
 * $2.1$ Autocorrelation Properties of Stationary Models:
 * $2.1.4$ Autocovariance and Autocorrelation Functions
 * $2.1.4$ Autocovariance and Autocorrelation Functions