Assumptions for Regression

CEF and the assumptions for regression

Regression Assumptions

Recently was discussing with a friend (Mashrur bhai) about the assumptions for regression, and I thought it would be useful to write down some of the key points. There are essentially two approaches we can introduce the assumptions,

• Writing the linear regression model first and then imposing the assumptions on the error term (this is the most common approach in undergraduate textbooks), or
• Starting with the CEF (Conditional Expectation Function) and then deriving the assumptions from there (This approach is more common in graduate textbooks. You can look at the the beautiful book by Hansen , and also the book by Angrist and Pischke and the excellent notes by Chaisemartin).

The idea is we start from the idea of CEF, since this is the best prediction function. Here is the setup

Setup: Suppose we have a random variable $Y$ and we would like to predict it using a set of random variables $X = (X_1, X_2, \ldots, X_k)$, where $X$ is a vector of random variables, i.e.,

\[X = \begin{pmatrix} X_1 \\ X_2 \\ \vdots \\ X_k \end{pmatrix}\]

Then the CEF is defined as