Partial Autocorrelation Function (PACF)#

Introduction#

PACF stands for Partial Autocorrelation Function. Like the Autocorrelation Function (ACF), the PACF is a tool used in time series analysis to understand the relationship between observations at different lags within a time series. However, the PACF addresses a more specific question compared to the ACF.

While the ACF measures the correlation between an observation and its lagged values, the PACF measures the correlation between an observation and its lagged values, while removing the effects of the intermediate lags. In other words, the PACF measures the direct relationship between observations at two time points, accounting for any other time points in between.

The PACF is particularly useful in identifying the appropriate order of autoregressive (AR) terms in ARIMA (AutoRegressive Integrated Moving Average) models. It helps determine the direct influence of past observations on the current observation, without the influence of intervening observations.

In summary, the PACF provides a more refined understanding of the temporal dependencies within a time series by capturing the direct relationships between observations at different lags, without the influence of other lags in between.

Here’s an example of how you can generate a Partial Autocorrelation Function (PACF) plot using Python with the help of the statsmodels library:

[1]:

import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm

# Generate some example time series data
np.random.seed(123)
data = np.random.normal(loc=0, scale=1, size=100)

# Plot the time series data
plt.figure(figsize=(10, 4))
plt.plot(data)
plt.title('Example Time Series Data')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True)
plt.show()

# Calculate PACF
pacf = sm.tsa.pacf(data, nlags=20)

# Plot PACF
plt.figure(figsize=(10, 4))
plt.stem(pacf)
plt.title('Partial Autocorrelation Function (PACF)')
plt.xlabel('Lag')
plt.ylabel('PACF Value')
plt.grid(True)
plt.show()

In this example, we first generate some example time series data using numpy. Then, we plot the time series data to visualize it. After that, we calculate the Partial Autocorrelation Function (PACF) using sm.tsa.pacf from the statsmodels library, specifying the number of lags as 20. Finally, we plot the PACF values using plt.stem to visualize the partial autocorrelation at different lags.

The Partial Autocorrelation Function (PACF) plot produced by the provided code illustrates the partial autocorrelation of the time series data at different lags. Here’s how to interpret it:

Lag: Similar to the ACF plot, the x-axis of the plot represents the lag, which is the number of time units by which the series is shifted to calculate the partial autocorrelation.
PACF Value: The y-axis of the plot represents the partial autocorrelation function (PACF) value. This value indicates the partial correlation between the series and its lagged versions, removing the effects of the intermediate lags. As with the ACF, a value close to 1 indicates a strong positive partial correlation, a value close to -1 indicates a strong negative partial correlation, and a value close to 0 indicates no partial correlation.
Stem Plot: Similar to the ACF plot, the stem plot is used to represent the PACF values. Each stem corresponds to a lag, and its height represents the magnitude of the partial autocorrelation at that lag. Positive partial autocorrelation is plotted above the x-axis, while negative partial autocorrelation is plotted below the x-axis.

Interpreting the PACF plot helps in understanding the direct influence of past observations on the current observation, without the influence of intervening observations. Here are some key points:

Significant spikes at certain lags indicate that there is a strong partial correlation between the series and its lagged versions at those particular lags. These significant spikes are often used to identify the order of autoregressive (AR) terms in ARIMA models.
A gradual decay in the PACF values as the lag increases suggests that there is a diminishing direct influence of past observations on the current observation, indicating potential stationarity in the time series data.

By analyzing the PACF plot, you can gain insights into the direct temporal dependencies within the time series data and use this information to build appropriate forecasting models or make data-driven decisions.

Key differences between ACF and PACF#

Here are the key differences between the Partial Autocorrelation Function (PACF) plot and the Autocorrelation Function (ACF) plot in Python, along with examples:

Definition:
- ACF: Measures the correlation between an observation and its lagged values.
- PACF: Measures the correlation between an observation and its lagged values, while removing the effects of the intermediate lags.
Interpretation:
- ACF: Reflects both direct and indirect correlations between observations at different lags.
- PACF: Reflects only direct correlations between observations at different lags, excluding the influence of intervening lags.
Plot Representation:
- ACF: Shows the correlation values between the time series and its lagged versions across different lags.
- PACF: Shows the partial correlation values between the time series and its lagged versions, removing the effects of intermediate lags.

Here’s an example showcasing the difference using Python code:

[2]:

import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm

# Generate example time series data with ARMA process
np.random.seed(123)
# Create ARMA(2, 1) process: AR coefficients = 0.6, 0.3, MA coefficient = 0.8
data_arma = sm.tsa.arma_generate_sample(ar=[1, -0.6, -0.3], ma=[1, 0.8], nsample=100)

# Plot the time series data
plt.figure(figsize=(10, 4))
plt.plot(data_arma)
plt.title('Example Time Series Data (ARMA(2, 1) Process)')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True)
plt.show()

# Calculate ACF
acf_arma = sm.tsa.acf(data_arma, nlags=20)

# Calculate PACF
pacf_arma = sm.tsa.pacf(data_arma, nlags=20)

# Plot ACF for ARMA process
plt.figure(figsize=(10, 4))
plt.stem(acf_arma)
plt.title('Autocorrelation Function (ACF) - ARMA(2, 1) Process')
plt.xlabel('Lag')
plt.ylabel('ACF Value')
plt.grid(True)
plt.show()

# Plot PACF for ARMA process
plt.figure(figsize=(10, 4))
plt.stem(pacf_arma)
plt.title('Partial Autocorrelation Function (PACF) - ARMA(2, 1) Process')
plt.xlabel('Lag')
plt.ylabel('PACF Value')
plt.grid(True)
plt.show()

In this code, we first generate an example time series dataset using an ARMA(2, 1) process. Then, we plot the time series data itself. After that, we calculate and plot both the ACF and PACF for the generated time series data. This way, you can observe the autocorrelation patterns in the time series data alongside the ACF and PACF plots, aiding in the understanding of how the autocorrelation functions relate to the underlying time series.

Explanation#

Let’s interpret the plots generated for the ARMA(2, 1) process:

Time Series Plot:
- The time series plot shows the values of the generated data over time. Since the data is generated using an ARMA(2, 1) process, you may observe fluctuations and trends in the data. There might also be patterns that correspond to the autoregressive (AR) and moving average (MA) components of the process.
Autocorrelation Function (ACF) Plot:
- The ACF plot shows the autocorrelation of the time series data at different lags. In the ACF plot for the ARMA(2, 1) process, you may observe a gradual decay in the autocorrelation values as the lag increases. This decay is typical in autoregressive processes. Additionally, you may notice significant autocorrelation values at the lags corresponding to the AR terms of the process (in this case, lags 1 and 2). This indicates a direct correlation between the observations at those lags.
Partial Autocorrelation Function (PACF) Plot:
- The PACF plot shows the partial autocorrelation of the time series data at different lags, removing the effects of intermediate lags. In the PACF plot for the ARMA(2, 1) process, you may observe significant spikes at the lags corresponding to the AR terms (lags 1 and 2), indicating direct correlations between the observations at those lags. The PACF values beyond the AR order typically drop off sharply, suggesting that the direct influence of previous observations diminishes beyond those lags.

Interpreting these plots helps in understanding the temporal dependencies within the time series data. The ACF and PACF plots provide insights into the underlying structure of the data, which can be valuable for model selection, forecasting, and identifying potential patterns or trends in the time series.

Interpretation of “removing the effects of intermediate lags”#

When we talk about removing the effects of intermediate lags in the context of partial autocorrelation function (PACF), it means that the partial correlation at a certain lag is calculated while controlling for the influence of the lags in between.

Let’s illustrate this with an example:

Suppose we have a time series dataset where the observation at time t is influenced by the observations at time t-1, t-2, t-3, and so on. When calculating the PACF at lag k (e.g., PACF(3)), we want to measure the direct influence of the observation at time t-k on the observation at time t, while removing the indirect influences from the observations at times t-(k-1), t-(k-2), and so on.

By removing the effects of intermediate lags, we isolate the direct relationship between observations at specific time points, which provides a clearer understanding of the temporal dependencies in the data.

In summary, removing the effects of intermediate lags in PACF means focusing solely on the direct correlation between observations at different lags, without considering the influence of other lags in between. This allows us to identify the specific lagged relationships that contribute directly to the current observation.

Mathematical Background#

The concept of “removing the effects of intermediate lags” in the Partial Autocorrelation Function (PACF) calculation can be understood through the mathematical formulation of the PACF.

The PACF at lag $ k $, denoted as $ :nbsphinx-math:`text{PACF}`(k) $, is computed as the correlation between the original time series data and the lagged series at lag $ k $, but with the linear dependence on the observations at intermediate lags (1 to $ k-1 $) removed.

Mathematically, if we have a time series $ {X_t} $, the PACF at lag $ k $ is computed as follows:

\[\text{PACF}(k) = \frac{\text{cov}(X_t, X_{t-k} - \hat{X}_{t-k|t-1})}{\sqrt{\text{var}(X_t)\text{var}(X_{t-k} - \hat{X}_{t-k|t-1})}}\]

where: - $ \text{cov}`(X_t, X_{t-k} - :nbsphinx-math:hat{X}`{t-k|t-1}) $ is the covariance between the original series $ X_t $ and the lagged series $ X{t-k} $ with the linear dependence on intermediate lags (1 to $ k-1 $) removed. - $ \text{var}`(X_t):nbsphinx-math:text{var}`(X_{t-k} - \hat{X}{t-k|t-1}) $ is the product of the variances of $ X_t $ and $ X{t-k} $ with the linear dependence on intermediate lags (1 to $ k-1 $) removed. - $ \hat{X}{t-k|t-1} $ is the predicted value of $ X{t-k} $ based on the observations at intermediate lags (1 to $ k-1 $).

In practical terms, removing the effects of intermediate lags involves fitting a regression model to the time series data up to lag $ k-1 $ and using this model to predict the value of the series at lag $ k $ ( $ \hat{X}{t-k|t-1} $ ). Then, the correlation between the original series $ X_t $ and the difference between the actual lagged value $ X{t-k} $ and the predicted value $ \hat{X}_{t-k|t-1} $ is computed, effectively removing the linear dependence on intermediate lags.

By removing the effects of intermediate lags, the PACF isolates the direct correlation between observations at lag $ k $, providing insights into the specific lagged relationships that contribute directly to the current observation.