Archive for October, 2018


I have never written anything on Statistics (coz I hate statistics) on my blog so far and I believe I am going to start writing something on it from now on. I picked Normal Distribution because it is important. By Central limit theorem, in simple terms if you have many independent random variables that are generated by all kinds of crazy distribution, the sum of those random variables will tends toward the Normal distribution.

Normal Distribution

f(x) = \displaystyle\frac{1}{\sqrt{2\pi}\sigma} e^{-\frac12(\frac{x-\mu}{\sigma})^2}

where x, \mu \in \mathbb{R}, \sigma\in\mathbb{R}^+.

Then the MGF (Moment Generating Function) would be the following:

\displaystyle M_X(t) = E(e^{tX}) = \int_{-\infty}^{\infty} e^{tx}\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac12(\frac{x-\mu}{\sigma})^2} \text{ d}x

\displaystyle = \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}\sigma} e^{tx-\frac12(\frac{x-\mu}{\sigma})^2} \text{ d}x

\displaystyle = \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}\sigma} \exp\left(\frac{2\sigma^2tx-(x^2-2x\mu+\mu^2)}{2\sigma^2}\right)\text{ d}x

\displaystyle = \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}\sigma}\exp\left(\frac{-x^2+(2\sigma^2t+2\mu)x-\mu^2}{2\sigma^2}\right) \text{ d}x

Now using completing the square,


= -(x^2-2(\sigma^2t+\mu)x+(\sigma^2t+\mu)^2-(\sigma^2t+u)^2)-\mu^2

= -(x-(\sigma^2t+\mu))^2+(\sigma^2t+\mu)^2-\mu^2

= -(x-(\sigma^2t+\mu))^2 + \sigma^4 t^2 + 2\sigma^2 t\mu


\displaystyle M_X(t) = \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}\sigma} \exp\left(\frac{-(x-(\sigma^2t+\mu))^2+\sigma^4t^2+2\sigma^2t\mu}{2\sigma^2}\right) \text{ d}x

\displaystyle = \exp\left(\frac{\sigma^2t^2+2t\mu}{2}\right) \int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}\sigma} \exp\left(-\frac12 \left(\frac{x-\sigma^2t-\mu}{\sigma}\right)^2\right) \text{ d}x

Since the integral is the cumulative distribution function of the normal distribution which shift to the right by \sigma^2t units, the integral does sum to one. Therefore we left with

\displaystyle M_X(t) = \exp\left(\frac{\sigma^2t^2}{2}+t\mu\right)

Expected Value

\displaystyle E(X) = M_X'(t)\bigg|_{t=0} = \exp\left(\frac{\sigma^2t^2}{2}+t\mu\right) \cdot (\sigma^2t+\mu)\bigg|_{t=0} = \mu


\displaystyle E(X^2) = M_X''(t)\bigg|_{t=0} =\exp\left(\frac{\sigma^2t^2}{2}+t\mu\right)(\sigma^2t+\mu)^2 + \exp\left(\frac{\sigma^2t^2}{2}+t\mu\right)\sigma^2\bigg|_{t=0}

= \mu^2 + \sigma^2


Var(X) = E(X^2)-E^2(X) = \mu^2 + \sigma^2 - \mu^2 =\sigma^2