Laird Stewart
7/7/26

Intuition on sampling from a multi-variate normal

Imagine drawing a sample from the standard normal distribution: $X\sim \mathcal{N}(0,1)$. Then calculate the probability density at each of those samples $f(x)=\frac{1}{\sqrt{2\pi }}{e}^{-x^2/2}$ and plot a histogram of these densities. What shape will the histogram take? Will there be more samples with density near 0 (the tails) or with density near $\frac{1}{\sqrt{2\pi }}$ (the mode $x=0$)?

Once you have a guess, launch the plot below to see for yourself

Is that what you expected? Now, what if we do the same for a 2D normal distribution? What shape will the histogram take?

Hmm ... recognize that distribution? What do you think will happen for a 3D Gaussian? Do you see a pattern?

Spoiler, in higher dimensions the pattern continues and becomes even more pronounced. I came across a histogram of densities for a 4D Gaussian at work recently and the result surprised me. I had expected it to continue to look like the 1D case. Perhaps you could attribute this to the curse of dimensionality, but I think it's really just an illustrative example of how our intuitions about smaller dimensions can betray us in higher dimensions.

I'm aware of an analytical solution for the densities $f(x)$ for one and two dimensions. Not sure about three dimensions. I'll leave that for another post.