5 Worksheet 05: Prying open our imperfect model, Part 2

Before proceeding, make sure that you have finished the preceding worksheets.

In the previous worksheet, we made sense of what it means to sample (or draw) from a normal distribution. Specifically, how were the draws made is our main discussion point. In that worksheet, we focused our attention on what it means to have independent draws. Although we focused on what you might think as unrealistic, the discussion also provides a starting point for what to look for in case independence is not plausible.

In this worksheet, we will dig into the second major component underlying rnorm(). Specifically, we will look into the normal distribution itself. What information does the normal distribution contain? What convenience does it provide? What do we pay in exchange for such convenience?

Reiterating what was mentioned in the previous worksheet: We are prying open what rnorm() does because it is an important step toward building in reflection as part of the model-building process.

5.1 Why bother with the normal distribution?

In the previous worksheet, we have encountered discrete distributions for random variables which take on a countable number of values. In some of the exercises, you were asked to think of these discrete distributions as being probability distributions for returns of an asset.

In real life, returns can potentially take on a large number of possible values that you can possibly count. Therefore, a step toward reflecting real life is to consider random variables having continuous distributions. The normal distribution is probably one of the most famous and useful continuous distributions. It is not suitable for all situations, but it is so ubiquitous that a starting model for returns could start from using normality as a baseline.

A fair question to ask is why should we study a model which will eventually be replaced or is unrealistic. The first reason is to use something unrealistic as a take-off point for determining and point out more clearly what “tweaks” are needed to make things more realistic. The second reason is to provide a relatively easy baseline to “onboard” anyone who is starting to learn this topic. The third reason is that many models of returns in finance start from some form of normality assumption.

This is in contrast to what you may have been exposed to in an econometrics course. In those courses, there is a big attempt to try to not impose too many strong assumptions, especially distributional assumptions. For example, the use of OLS can be justified with a minimal assumption about the conditional expectation of a regressand given a regressor. We did not need to say that the conditional distribution of the regressand given a regressor has to be normal. But the goals in that course and what you are doing now are slightly different. There is more emphasis now on predicting the future by making probability assessments of what the future looks like, while respecting the features of financial data.

5.2 Exercise A

Review your econometrics notes. What do the OLS assumptions look like? Can you point out what supports the remarks in the preceding paragraph?
A big part of econometrics is about comparing differences between two groups (whether the groupings are hypothetical or real). When you were exposed to this topic, did you impose assumptions on the distribution of data belonging to each group? What were you usually focused on?

5.3 Setting up the normal distribution

I will use the definition of the normal distribution found in Wooldridge (2020). Specifically, let $X$ be a normally distributed random variable with parameters $\mu$ and $\sigma^2$ (or a random variable which follows a normal distribution with parameters $\mu$ and $\sigma^2$). The usual shorthand for this is $X\sim N\left(\mu, \sigma^2\right)$. What this means is

$X$ could take on any possible value on the real number line
There is a density curve or probability density function (pdf) which describes the shape of the distribution, i.e. \[f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right), \ \ \ -\infty<x<\infty\]

In contrast to discrete distributions, the random variable $X$ could take on a lot more values. This poses problems for probability assignments, which you will explore in the next exercise. But this is the reason why the shape of the distribution is described by a density curve rather than a probability curve.

5.4 Exercise B

Intuitively, the total probability you distribute over all possible values of a discrete random variable is equal to 1. The same requirement applies to the continuous case. But how can you “distribute” the total probability of 1 across a lot of possible values of a continuous random variable?

Undergo a thought experiment starting from the following simple discrete random variable. Let $X_1$ take on two values $0$ and $1$ with $\mathbb{P}\left(X_1=0\right)=0.5$ and $\mathbb{P}\left(X_1=1\right)=0.5$. Create a new random variable $X_2$ based on $X_1$ which takes on three values $-1$, $0$, and $1$.

Draw a graph of the probability distribution of $X_1$. Here the horizontal axis represent the possible values of $X_1$ and the vertical axis is the corresponding probability. What is its shape?
If you want $\mathbb{P}\left(X_2=-1\right)$ to be greater than zero, can you still retain the initial probability assignment $\mathbb{P}\left(X_2=0\right)=0.5$ and $\mathbb{P}\left(X_2=1\right)=0.5$? If yes, why? If no, what could you do so that $X_2$ will have a valid probability distribution and $\mathbb{P}\left(X_2=0\right)>0$ and $\mathbb{P}\left(X_2=1\right)>0$?
Now try pushing the previous exercise to have even more possible values beyond $-1$, $0$, and $1$, say, $-1$, $-0.5$, $0$, $0.5$, and $1$. Eventually, what will happen to the probabilities you assign to each of these possible value? What will then be the graph of the resulting probability distribution?
What if you have every real number as possible values that a random variable can take? What do you think will be the graph of the resulting probability distribution? Do you think this graph will be useful?
Can you now come to a conclusion as to why we have to change the vertical axis to something else other than probability?

5.5 Densities and probabilities

Based on the previous exercise, you will hopefully be convinced that if we want to talk about the shape of continuous distributions, we have to think of a different vertical axis. This motivates a probability density function.

The horizontal axis is the real number line and the vertical axis is a non-negative density. Because the probability at a point for a continuous random variable will be equal to zero (which you should be convinced of after doing the previous exercise), nonzero probabilities are now assigned to intervals of possible values instead. Therefore, you can think of probability as approximately being an area of a thin rectangle, where the height is the density and the width is the length of the interval. The integral would then be the related concept for probabilities.

As a result, for a generic continuous random variable with pdf $f$, we have $f(x)\geq 0$ and \[\mathbb{P}\left(-\infty < X < \infty\right)=\int_{-\infty}^{\infty} f(t)\,dt=1.\] Note that $t$ here is a dummy argument, meaning we could have used a different letter for it and the meaning of the integral stays the same. This integral is the area under the density curve $f$ above the horizontal axis and the total area should be equal to 1.

Probabilities are encoded more naturally using the cumulative distribution function (cdf). Let $a$ be any real number. The cdf is given by \[\mathbb{P}\left(X\leq a\right)=\int_{-\infty}^a f(t)\, dt.\] This integral is the area under the density curve $f$ bounded below by the horizontal axis, and bounded on the left by a vertical line $x=a$. Take note that in the continuous case, $\mathbb{P}\left(X\leq a\right)=\mathbb{P}\left(X< a\right)$, since the probability of $X=a$ is zero.

From the cdf, we can calculate probabilities of a variety of events. For example, we have, for given real numbers $a$ and $b$, \[\mathbb{P}\left(a\leq X\leq b\right)=\int_a^b f(t)\,dt=\mathbb{P}\left(X\leq b\right)-\mathbb{P}\left(X\leq a\right).\] This integral is the area under the density curve $f$ bounded below by the horizontal axis, bounded on the left by a vertical line $x=a$, and bounded on the right by a vertical line $x=b$.

For the case of a normally distributed random variable with parameters $\mu$ and $\sigma^2$, \[\mathbb{P}\left(a\leq X\leq b\right)=\int_a^b \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(t-\mu)^2}{2\sigma^2}\right)\,dt.\] Note that to obtain numerical values for probabilities from a normal random variable, we actually need the values of $\mu$ and $\sigma^2$. Fortunately, you will never be asked to compute probabilities starting from this integral anymore, especially for a special distribution like the normal.

The case where $\mu=0$ and $\sigma^2=1$ leads to a standard normal distribution. Wooldridge (2020) calls a standard normal random variable $Z$, the standard normal pdf $\phi(z)$, and the standard normal cdf $\Phi(z)$.

Table G.1 of Wooldridge (2020) is a table of cumulative probabilities of a standard normal random variable. For example, $\Phi(0)=\mathbb{P}\left(Z\leq 0\right)=0.5000$ can be read from the table directly.

5.6 Exercise C

Study the lines of code below:

dnorm(0, mean = 0, sd = 1)
1/(1*sqrt(2*pi))*exp(-(0-0)^2/(2*1^2))
curve(dnorm(x, mean = 0, sd = 1), from = -4, to = 4)

How are mean and sd related to $\mu$ and $\sigma^2$?
Articulate how these lines of code are related to the standard normal distribution and how these lines are related to each other.
Adjust the values of $\mu$, holding $\sigma^2$ constant, and trace the resulting consequences. You may have to adjust from = -4 and to = 4 to see the graph more completely.
Do the same as the previous item, but this time adjust $\sigma^2$, holding $\mu$ constant. Can you find values of $\sigma^2$ which can make the pdf sometimes exceed 1?
For the original settings and your choices in items 3 and 4, what range of values are most likely to be taken by a normal random variable for different settings of mean and sd?

Now study these lines of code:

x <- rnorm(10^4, mean = 0, sd = 1)
hist(x, freq = FALSE)
curve(dnorm(x, mean = 0, sd = 1), from = -4, to = 4, add= TRUE, lty = 2, col =  "red")

Run the lines of code multiple times. When we draw random numbers from a standard normal distribution, what numbers are being drawn? What changes and what stays the same across multiple runs? Which intervals on the real line appear more often than others?
Although there is no explicit time period which allows you to order the observations over time, it is possible to plot the observations as if it were a time series. Refer to the use of plot() in previous worksheets. For the original settings and your choices in items c and d, plot the resulting x over “time”. How is this plot different from a histogram?

5.7 Exercise D

Study the lines of code below. The histogram is drawn on a density scale. Therefore, the relative frequency of observations belonging to a particular interval is equal to the area of the rectangle. You will discover in this exercise that this will mirror our discussion of probability as area.

When we draw random numbers from a standard normal distribution, where are the numbers coming from and how are they related to $\phi(z)$ and $\Phi(z)$?

Let us return to explore the histogram of x, but this time store the values needed to construct a histogram:

x <- rnorm(10^4, mean = 0, sd = 1)
temp <- hist(x, freq = FALSE)
temp$breaks
temp$density
temp$counts

Let us explore the histogram of x more closely:
1. Match breaks to the horizontal widths of the rectangles. Match density to the heights of the rectangles. Divide the counts by 10^4 and match these with the areas of each rectangle forming the histogram.
2. By adding up the areas of specific rectangles, compute the relative frequency of finding draws from the standard normal distribution below $0$.
3. How does the result in item b compare with $\mathbb{P}\left(Z\leq 0\right)$?
4. Now, compute the relative frequency of finding draws from the standard normal distribution between $-0.5$ and $0$. How does the result compare with $\mathbb{P}\left(-0.5 \leq Z\leq 0\right)$?
5. Repeatedly run the commands for the histogram. Revisit items a to d. Document what changes and what stays the same.
Suppose you want to compute $\mathbb{P}\left(Z \leq 0.44 \right)$. There are three ways to proceed.
1. Use Table G.1 of Wooldridge (2020).
2. Use pnorm(), specifically run pnorm(0.44, mean = 0, sd = 1, lower.tail = TRUE). Does it match what you obtained in item a? Try to make sense of how the command is related to every component of the desired probability.
3. Use a simulation. Run x <- rnorm(10^4, mean = 0, sd = 1) and then mean(x <= 0.44). How is this approach connected to the insights of item 1?
Implement the three approaches in item 2 to find $\mathbb{P}\left(Z \geq 0.44\right)$ and $\mathbb{P}\left(-1 \leq Z \leq 0.5\right)$.

5.8 The meaning of the parameters of a normal distribution

You may have already gathered that the mean and sd settings refer to the $\mu$ and $\sigma$ acting as parameters in the pdf of the normal distribution. Take care and ask if the normal distribution uses parameters $\mu$ and $\sigma^2$ or normal distribution uses parameters $\mu$ and $\sigma$.

Strictly speaking, $\mu$ and $\sigma^2$ are really just some unknown constants. But it can be proven that the population mean or expected value of a normal random variable $X$ happens to be equal to $\mu$. In addition, the population variance of a normal random variable $X$ happens to be equal to $\sigma^2$. To get a sense why this could be true, you can explore in the next exercise.

5.9 Exercise E

It is not important to be able to prove mathematically what was claimed in the preceding paragraph. But it is important to gain a sense of why the result could be true.

x <- rnorm(10^4, mean = 0, sd = 1)
mean(x)
var(x)
sd(x)

Study the preceding lines of code and relate the outcomes to the parameter settings of the normal distribution. Run the lines multiple times to know which values change and which values don’t change. Adjust the values of mean and sd to explore further.

5.10 Exercise F

How do you calculate probabilities of a random variable following a normal distribution with mean $\mu$ and variance $\sigma^2$? You simply have to revisit the three approaches discussed in a previous exercise. Suppose you want to compute $\mathbb{P}\left(X\leq 1\right)$ if $X\sim N\left(\mu=3,\sigma^2=4\right)$.

To use Table G.1, you need to standardize first. The basis of this idea is Property Normal.1 of Wooldridge (2020), specifically, whenever $X\sim N\left(\mu, \sigma^2\right)$ we must have \[\frac{X-\mu}{\sigma} \sim N\left(0,1\right).\] For example, if $X\sim N\left(\mu=3,\sigma^2=4\right)$, then $Z=\dfrac{X-3}{2}\sim N\left(0,1\right)$. Specifically, whenever $X\leq 1$, we must have $\displaystyle \underbrace{\dfrac{X-3}{2}}_{=Z} \leq -1$. Therefore, to find $\mathbb{P}\left(X\leq 1\right)$ here means finding $\mathbb{P}\left(Z\leq -1\right)$. Look it up.
You can use pnorm() directly by modifying the command, which should be straightforward.
How about directly drawing randomly from the $N\left(3, 4\right)$ and estimating $\mathbb{P}\left(X\leq 1\right)$ by simulation?

5.11 Exercise G

Use the three approaches for computing normal probabilities:

(Based on Wooldridge (2020) Example B.6) $\mathbb{P}\left(2<X\leq 6\right)$ if $X\sim N\left(4, \sigma^2=9\right)$
(Based on Wooldridge (2020) Appendix B Problem 2) Let $X\sim N\left(5, \sigma^2=4\right)$
1. $\mathbb{P}\left(X\leq 6\right)$
2. $\mathbb{P}\left(X>4\right)$
3. $\mathbb{P}\left(|X-5|>1\right)$

5.12 Connection to stock returns

You might wonder why we spend so much time on fleshing out the normal distribution. One reason is that later we will be looking at departures from normality or violations of normality. For now, it is simple enough to make calculations which are relevant in real-life circumstances.

At this stage, you can already apply what you have learned to a financial context. Consider Exercise 1(a) in Section 2.5 of Ruppert and Matteson (2015). We just need a couple of definitions. Let $P_t$ be the price of an asset at time period $t$.

The simple or net return $R_t$ of an asset held from time $t-1$ to time $t$ is given by \[R_t=\frac{P_t-P_{t-1}}{P_{t-1}}=\frac{P_t}{P_{t-1}}-1.\] The simple return is usally converted into percentage terms, but it is important to recognize that a time period is attached to this return. If we talk about a daily return of $10\%$, it means $10\%$ per day.
The log return $r_t$ of an asset held from time $t-1$ to time $t$ is given by \[r_t=\log\left(1+R_t\right).\] The logarithm is calculated with respect to base $e$. Take note that log returns does not mean the logarithm of returns. It is not obvious now but this can be converted into percentage terms with a time period attached to this return.

You might ask why we even bother with another notion of return when the simple return is already available. This will be elaborated in the next worksheet. For now, it suffices to appreciate that there is another notion of a return and that we are imposing a distributional assumption on log returns, as seen in the next exercise. In effect, we are treating log returns as a random variable here.

5.13 Exercise H

Suppose that the daily log returns on a stock are independent and normally distributed with mean $0.001$ and standard deviation $0.015$. Suppose you buy $\$1,000$ worth of this stock. Your task is to find the probability that after one trading day your investment is worth less than $\$990$ by going through the following questions.

Take $\$1,000$ to be $P_0$. What is $P_1$ in terms of $R_1$? Why could we not say $P_1=990$? Why is it useful to think of $P_1$ as being a random variable?
$P_1$ in item 1 would then be how much your investment is worth after one trading day. Express the desired probability in symbols as the probability that $P_1$ will take values less than 990.
Express the probability in terms of $r_1$ and calculate the desired probability.
Reflect on why the distributional assumption on $r_1$ is useful in finding the desired probability. Reflect as well on whether independence is needed here. Reflect on why is it that the distributional assumption is imposed on log returns rather than simple returns.

5.14 Exercise I

Create your own realistic practice exercise with real-life data without the use of AI. The word “realistic” here means that you will now put in more real details, like the name of the company, the dates, etc.

From your choice of stock using previous worksheets, pattern your practice exercise after Exercise G. You are free to explore variations of the exercise. Provide your own solution to your own exercise. Choose any approach you would like to use for your solution.

References

Ruppert, David, and David S. Matteson. 2015. Statistics and Data Analysis for Financial Engineering: With r Examples. 2nd ed. Springer Texts in Statistics. Springer. https://doi.org/10.1007/978-1-4939-2614-5.

Wooldridge, Jeffrey M. 2020. Introductory Econometrics: A Modern Approach. 7th edition. Cengage Learning.