9 Worksheet 09: Digging into the underlying time series concepts, Part 2

Before proceeding, make sure that you have finished the preceding worksheets.

So far, you have seen the extreme convenience IID normality provides, especially when generating simulated data. You have also seen complications with generating simulated data when you want to account for dependence. The most important aspect to remember is the increasing dimension of the problem. Conditional distributions, in the context of time series, depend on a conditioning set that gets extremely large as you have more trading periods or time periods. In an intuitive sense, you have to maintain a record of the past history, especially if you think that the future depends on what happened in the past.

9.1 Building the relevant concepts, continued

So far, you grappled with stationarity. It is a concept which allows one to say that “the future looks like the past”. Unfortunately, that intuition is not specific enough. To create a definition, observe that $h$ was crucial in expressing forward shifts in time. In addition, you can allow for some dependence because either the joint distribution of $\left(x_{t_1},\ldots, x_{t_m}\right)$ is left unspecified or the covariances $\mathsf{Cov}\left(x_t,x_{t+h}\right)$ only depend on $h$ and the values are unrestricted. You will explore this further in an exercise later in this worksheet.

We now introduce and work through a new concept called weak dependence. You will be working on Section 11-1b.

9.2 Exercise A

You will be exploring the concept of asymptotically uncorrelatedness as the primary weak dependence concept which is accessible at this level. Assume that you have a covariance stationary stochastic process $\{x_t:\ t=1,2,\ldots\}$.

Start with the third paragraph of Section 11-1b. How is asymptotic uncorrelatedness defined? Can you articulate the role that $h$ plays in the definition? In what sense is it asymptotic (what is getting large and larger without bound?)? In what sense is uncorrelatedness showing up?
Continue with the fifth paragraph.
1. Focus on the first sentence. Argue why an IID sequence is automatically asymptotically uncorrelated. The word “trivial” here means that there is not a lot of work to be done as the definition of asymptotically uncorrelatedness will be satisfied by default.
2. Next is a slightly complicated stochastic process called a moving average process of order 1 or MA(1). Pay attention to how a moving average process is specified. Compare $x_t$ and $x_{t+1}$. Can you understand why it is called an average and why it is moving? Observe what elements of the IID sequence you are combining.
Continue with the next paragraph. You are to do two things: check first if an MA(1) is covariance stationary and second if an MA(1) satisfies asymptotic uncorrelatedness. First, check covariance stationarity:
1. Compute $\mathrm{E}\left(x_1\right)$, $\mathrm{E}\left(x_2\right)$, and $\mathrm{E}\left(x_3\right)$. Do you see a pattern? Generalize this and justify why $\mathrm{E}\left(x_t\right)$ for any $t=1,2,\ldots$. Refer to the properties of the expected value in Appendix B. What can you conclude about
2. Compute $\mathsf{Var}\left(x_1\right)$, $\mathsf{Var}\left(x_2\right)$, and $\mathsf{Var}\left(x_3\right)$. Do you see a pattern? Generalize this and justify why $\mathsf{Var}\left(x_t\right)$ for any $t=1,2,\ldots$. Refer to the properties of the variance in Appendix B.
Now, you will compute $\mathsf{Cov}\left(x_1, x_2\right)$.
1. Unfortunately, the properties of the covariance in Appendix B do not cover this case. Explain why.
2. To compute $\mathsf{Cov}\left(x_1, x_2\right)$, you have two approaches. One is to go back to the definition of the covariance. You might need Equation (B.27) to make progress. Apply this approach to show that $\mathsf{Cov}\left(x_1,x_2\right)=\alpha_1\sigma^2_e$.
3. The other is to use an additional property: For any constants $a$, $b$, $c$, $d$ and random variables $X$, $Y$, $Z$, $W$, we have \[\begin{eqnarray*}\mathsf{Cov}\left(aX+bY, cZ+dW\right) &=&ac\mathsf{Cov}\left(X,Z\right) +ad\mathsf{Cov}\left(X,W\right)\\ && +bc\mathsf{Cov}\left(Y,Z\right)+ bd\mathsf{Cov}\left(Y,W\right)\end{eqnarray*}\] Apply this approach to show that $\mathsf{Cov}\left(x_1,x_2\right)=\alpha_1\sigma^2_e$.
Next compute $\mathsf{Cov}\left(x_2, x_3\right)$ using any of the approaches you have been exposed to. Generalize to $\mathsf{Cov}\left(x_t, x_{t+1}\right)$.
Now, compute $\mathsf{Cov}\left(x_t, x_{t+2}\right)$ using any of the approaches you have been exposed to. Generalize to $\mathsf{Cov}\left(x_t, x_{t+h}\right)$ for $h\geq 2$.
Given all the findings so far, why can you conclude that MA(1) is covariance stationary?
Use the definition of a population correlation coefficient in Equation (B.29) to compute $\mathsf{Corr}\left(x_t,x_{t+h}\right)$ for all $h\geq 1$. Do you expect the correlation coefficients to depend on $t$? Why or why not? After that, show that an MA(1) process satisfies asymptotic uncorrelatedness. What is happening to the correlation coefficients as $h\to\infty$?

9.3 Exercise B

In this exercise, you will work on Problems 2 and 3 of Wooldridge (2020) Chapter 11.

Problem 2 asks you to explore an MA(2) process.
1. Can you explain to someone who knows about MA(1) processes why this could be an MA(2) process?
2. Unlike the MA(1) proess you explored in the previous exercise, you are looking at an MA(2) where the coefficients of the $e$’s are known. Solve all the subitems of the problem using what you have learned when working on an MA(1) process.
3. How different is an MA(2) process relative to an MA(1) process? Discuss fully.
4. A general form of an MA process is an MA(q) process. Can you specify what the process is in terms of an equation and its assumptions about the $e$’s? What do you think will be the patterns for $\mathrm{E}\left(x_t\right)$? for (x_t)$? $for \mathsf{Cov}\left(x_t, x_{t+h}\right)$?
Problem 3 asks you to explore a different stochastic process. Notice that at every time period there is a common random variable. This is a simplified version of what they call a fixed effects panel data model but with only one cross-sectional unit.
1. Solve all the subitems of the problem.
2. What do you think the problem is trying to show you? Is it giving you an example? an example of what? How does this example clarify the distinction between covariance stationarity and asymptotic uncorrelatedness?
3. Return to Going Further 11.1. Is the linear time trend model feature there a covariance stationary stochastic process? Does the linear time trend model satisfy asymptotic uncorrelatedness?

9.4 A very useful stochastic process

In Section 11-1b of Wooldridge (2020), you will encounter an autoregressive process of order 1 or AR(1). The approach in the book does not actually show that an AR(1) process is covariance stationary. In fact, you will not know for sure whether an AR(1) is covariance stationary.

In the textbook, an AR(1) process was assumed to be covariance stationary first. After that, the resulting implications are drawn. You are to work this out for yourself in the next exercise.

9.5 Exercise C

How is an AR(1) process specified here? Why is there a $y_0$?
Write an expression for $y_1$ given Equation (11.2).
Why can we conclude that $\mathrm{E}\left(y_1\right)=\mathrm{E}\left(y_0\right)$? And what does this imply about $\mathrm{E}\left(y_1\right)$?
Now write an expression for $y_2$, given Equation (11.2). Why can we conclude that $\mathrm{E}\left(y_2\right)=\mathrm{E}\left(y_1\right)$? And what does this imply about $\mathrm{E}\left(y_2\right)$?
Continue this line of argumentation found in the book. Why is there a claim that $\mathrm{E}\left(y_t\right)=\mathrm{E}\left(y_{t-1}\right)$? Why will $\mathrm{E}\left(y_t\right)=0$? Why is it required for $\rho_1\neq 1$? What happens when $\rho=1$?
Why can we conclude that $\mathsf{Var}\left(y_1\right)=\mathsf{Var}\left(y_0\right)$? And what does this imply about $\mathsf{Var}\left(y_1\right)$? Can you explain why we would need a restriction $-1<\rho_1<1$?
Why can we conclude that $\mathsf{Var}\left(y_2\right)=\mathsf{Var}\left(y_1\right)$? And what does this imply about $\mathsf{Var}\left(y_2\right)$? Can you explain why we would need a restriction $-1<\rho_1<1$?
Continue this line of argumentation found in the book. Why is there a claim that $\mathsf{Var}\left(y_t\right)=\mathsf{Var}\left(y_{t-1}\right)$? Why will \[\mathsf{Var}\left(y_t\right)=\frac{\sigma^2_e}{1-\rho_1^2}?\] Why is it required for $-1< \rho_1 < 1$?
Now, you will calculate $\mathsf{Cov}\left(y_t,y_{t+h}\right)$.
1. To get a feel for the calculation, try $\mathsf{Cov}\left(y_1,y_0\right)$ first. Make sure you justify how you got your answer.
2. Next, can you figure out what $\mathsf{Cov}\left(y_2,y_1\right)$ is without doing any additional calculation? Explain how you arrived at your answer.
3. Can you generalize to $\mathsf{Cov}\left(y_t,y_{t+1}\right)$ without doing any additional calculation? Explain how you arrived at your answer.
4. Now, try $\mathsf{Cov}\left(y_2,y_0\right)$ first. Make sure you justify how you got your answer. Next, can you figure out what $\mathsf{Cov}\left(y_3,y_1\right)$ is without doing any additional calculation? Explain how you arrived at your answer. Can you generalize to $\mathsf{Cov}\left(y_t,y_{t+2}\right)$ without doing any additional calculation?
5. Given what you are seeing so far, can you guess what $\mathsf{Cov}\left(y_t,y_{t+h}\right)$ could be for $h\geq 1$? Are there additional restrictions on $\rho_1$?
6. Given your guess, can you come to a conclusion about what kind of AR(1) processes will satisfy asymptotic uncorrelatedness?
7. Now try the approach found in the book for finding $\mathsf{Cov}\left(y_t,y_{t+h}\right)$. You might find it useful to apply the covariance property introduced in this worksheet in a previous exercise.

9.6 Exercise D

Explore the case where $\rho_1=1$. Consult the first four paragraphs of Section 11-3a.

Is the resulting AR(1) process covariance stationary? What do you notice about $\mathsf{Corr}\left(y_t,y_{t+h}\right)$ as $h\to\infty$? Observe that this is Problem 4 of Wooldridge (2020) Chapter 11.
When you look at the resulting equation after repeated substitution, do you notice something familiar or something you have encountered before? Discuss with a classmate.

9.7 Visualizing the processes you have encountered

We can visualize these processes via the usual time series plot, which you have encountered before when studying stock prices and stock returns. But there is another way to visualize these processes through a correlogram or a plot of the autocorrelation function.

The quantities you have been calculating to check for asymptotic uncorrelatedness involved correlation coefficients. These correlation coefficients are sometimes called autocorrelation coefficients. The prefix “auto” means with itself. So, $\mathsf{Corr}\left(y_t,y_{t+h}\right)$ is the $h$th-order autocorrelation coefficient.

If we think of every $h$ as being connected to an $h$th-order autocorrelation coefficient, we really have an autocorrelation function or ACF. We can create a plot where $h$ is on the vertical axis and the corresponding $h$th-order autocorrelation coefficient on the horizontal axis. The resulting plot is called a correlogram.

The ACF and the correlogram have two versions – a theoretical or population version and a sample version. All the exercises in this worksheet give you the theoretical ACF and the theoretical correlogram. When you generate simulated data, you will see differences from the theoretical counterparts, but the sample counterparts should be very similar.

9.8 Exercise E

Draw the theoretical correlogram for:
1. an IID process
2. an MA(1) process where you can choose any nonzero value for $\alpha_1$ and $\sigma^2_e$
3. the MA(2) process encountered in Problem 2 of Wooldridge (2020) Chapter 11
4. the process encountered in Problem 3 of Wooldridge (2020) Chapter 11 where you can choose values for $\sigma^2_z$ and $\sigma^2_e$
Now, you will generate simulated paths for the processes you have seen so far.
1. Simulate one path from an IID process. You definitely need a known distribution for random number generation (do you know why?). Feel free to use rnorm() as a start.
2. Install the forecast package in R and use the Acf() command in R on the simulated path from item a. Repeatedly generate paths and the corresponding ACF.
3. Make sure you understand the information presented for the sample ACF. Ignore the dashed blue lines for now. How different is it from the theoretical ACF? d, Repeatedly generate paths and the sample ACFs. Compare the sample ACF against the theoretical ACF.
Now you will be learning how to generate simulated paths from an MA(1) process. We will be using arima.sim(). We have $x_t=e_t+0.5e_{t-1}$ where $e_t$ is IID normal with mean 0 and variance 1.
1. How do the lines of R code below reflect the specification provided? You may have to refer to the help file for arima.sim(). Repeatedly run the lines to get a feel for both the time series plot and the correlogram.
2. Modify the lines of R code and change the settings (for example, can you produce an IID process via arima.sim()). Explore what happens and discuss with a classmate.

x <- arima.sim(n = 100,list(ma = c(0.5)), sd = 1)
par(mfrow = c(1, 2))
plot(x, type="l")
Acf(x)

Next explore how to generate an AR(1) process with $-1<\rho_1<1$. Try values that are getting closer and closer but not equal to 1.

9.9 Looking forward

Why are we calculating these ACFs? What information do they contain? Can we compute a sample ACF for stock returns that we observe? What do we gain from this information?

To get you to start thinking about this angle, read Example 11.4 of Wooldridge (2020). What is the connection of regression with correlation? You definitely will be drawing from your past knowledge of econometrics.

We can also ask whether or not we can use linear regression when working with data from weakly dependent covariance stationary processes. How about nonstationary processes?

References

Wooldridge, Jeffrey M. 2020. Introductory Econometrics: A Modern Approach. 7th edition. Cengage Learning.