7  Characteristic functions

Definition 7.1 ($\mathbf{\text{Characteristic function}}$)
Consider an $n$-dimensional vector $\mathbf{X}$, then the characteristic function is defined $\mathrm{\forall }t\in {\mathbb{R}}^n$ as: $${\mathrm{\Phi }}_{\mathbf{X}}(\mathbf{t})=\mathbb{E}\{e^{i{\mathbf{t}}^T\mathbf{X}}\}\text{where}{\mathbf{t}}^T\mathbf{X}=\left(\begin{array}{c}{t}_1,t_2,\dots ,t_n\end{array}\right)\left(\begin{array}{c}{X}_1\\ X_2\\ ⋮\\ X_n\end{array}\right)$$

The characteristic function always exists when treated as a function of a real-valued argument, unlike the moment-generating function. The characteristic function uniquely determines the probability distribution of the correspondent random vector $\mathbf{X}$. More precisely, saying that two random variables has the same distribution is equivalent to say that their characteristic functions are equal. It follows that we can always work under characteristic functions to prove that two distribution of some random vectors are equal or that a distribution converges to another distribution. Formally $\mathbf{X}$ and $\mathbf{Y}$ ave same distribution, i.e.  $$\mathbf{X}\sim \mathbf{Y}⟺{\mathrm{\Phi }}_{\mathbf{X}}(\mathbf{t})={\mathrm{\Phi }}_{\mathbf{Y}}(\mathbf{t})\mathrm{\forall }\mathbf{t}\in {\mathbb{R}}^n$$

Here, we list some properties considering the random variable case, i.e. $n=1$, with $t\in \mathbb{R}$.

1. Independence: $X$ and $Y$ are independent iff: $$X\perp Y⟺{\mathrm{\Phi }}_{X+Y}(t)={\mathrm{\Phi }}_X(t){\mathrm{\Phi }}_Y(t)\mathrm{\forall }t\in \mathbb{R}$$

2. Existence of the $j$-th moment: If the $j$-th moment of the random variable is finite then the characteristic function is $j$-times differentiable and continuous in 0, i.e. ${\mathrm{\Phi }}_X\in {\mathbb{C}}^{(j)}$. Formally, $$\mathbb{E}\{|X{|}^r\}<\mathrm{\infty }⟹{\mathrm{\Phi }}_X\in {\mathbb{C}}^{(r)}\text{and}{\mathrm{\Phi }}_X^{(r)}(t)=\mathbb{E}\{(i\mathbf{X}{)}^r{e}^{itX}\}r=1,2,\dots ,j$$ Note that, if $j$ is even, it became an if: $$\mathbb{E}\{|X{|}^r\}<\mathrm{\infty }⟹{\mathrm{\Phi }}_X\in {\mathbb{C}}^{(r)},{\mathrm{\Phi }}_X^{(r)}(t)=\mathbb{E}\{(iX{)}^r{e}^{itX}\}r=2,4,\dots ,j$$

3. Inversion theorem: The characteristic function uniquely determines the probability distribution of the correspondent random vector $\mathbf{X}$: $$\mathbb{F}(b)-\mathbb{F}(a)=\frac{1}{2\pi i}\underset{c\to \mathrm{\infty }}{lim}{\int }_{-c}^c\frac{e^{-ita}-e^{itb}}{t}\mathrm{\Phi }(t)dt\mathrm{\forall }a<b$$ Then, the density function is obtained as: $$f(x)=\frac{1}{2\pi }{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-it\mathbf{X}}{\mathrm{\Phi }}_{\mathbf{X}}(t)dt$$

4. Convergence in distribution: $$X_n\stackrel{d}{\underset{n\to \mathrm{\infty }}{⟶}}X⟺{\mathrm{\Phi }}_X(t)=\underset{n\to \mathrm{\infty }}{lim}{\mathrm{\Phi }}_{X_n}(t)\mathrm{\forall }t\in \mathbb{R}$$

5. Scaling and centering: Given $Y=a+bX$, the effect of scaling and centering on the characteristic function is such that: $${\mathrm{\Phi }}_Y(t)=\mathbb{E}\{e^{itY}\}=\mathbb{E}\{e^{it(a+bX)}\}=e^{itb}\mathbb{E}\{e^{i(ta)X}\}=e^{itb}{\mathrm{\Phi }}_X(at)\mathrm{\forall }a,b,t\in \mathbb{R}$$

6. Weak Law of Large Numbers: consider a sequence of IID random variables $\{X_n{\}}_{n\ge 1}$, such that exists the first derivative of the characteristic function in zero, namely $\mathrm{\exists }{\varphi }_X^{\prime }(0)$ and the first moment of $X_1$ is finite, i.e. $\mathbb{E}\{|X_1|\}=\mu <\mathrm{\infty }$, then the sample mean converges in probability to a degenerate random variable $\alpha \in \mathbb{R}$. $${\overline{X}}_n=\frac{1}{n}\sum _{i=1}^n{X}_i\underset{n\to \mathrm{\infty }}{\stackrel{\text{p}}{⟶}}\alpha$$ for some $\alpha \in \mathbb{R}$.

WLLN and characteristic function

Proof. To prove the above statement, let’s compute the characteristic function of ${\overline{\mathbf{X}}}_n$: $${\varphi }_{{\overline{\mathbf{X}}}_n}(t)=\mathbb{E}\{\exp \left(\frac{it}{n}\sum _{i=1}^n{\mathbf{X}}_n\right)\}\stackrel{\text{IID}}{=}[{\varphi }_{{\mathbf{X}}_1}(\frac{t}{n}){]}^n$$ Let’s now apply the Taylor series of a function $f(x)$ around the point $a$ (Equation 30.1) to expand till the first order term the function ${\varphi }_{{\mathbf{X}}_1}(\frac{t}{n})$ around zero ($a=0$ and $x=\frac{t}{n}$), i.e. $$\begin{array}{rl}{\varphi }_{{\overline{\mathbf{X}}}_n}(\frac{t}{n})& ={({\varphi }_{{\mathbf{X}}_1}(0)+\frac{t}{n}{\varphi }_{{\mathbf{X}}_1}^{\prime }(0)+o(\frac{t}{n}))}^n=\\ & ={(1+\frac{t{\varphi }_{{\mathbf{X}}_1}^{\prime }(0)+no(\frac{t}{n})}{n})}^n\underset{n\to \mathrm{\infty }}{⟶}\exp \{t{\varphi }_{{\mathbf{X}}_1}^{\prime }(0)\}\end{array}$$ The convergence going to the limit as $n\to \mathrm{\infty }$ follows from the fact that in general if $a_n\underset{n\to \mathrm{\infty }}{⟶}a$, then the following limit holds: $$(1+\frac{a_n}{n}{)}^n\underset{n\to \mathrm{\infty }}{⟶}{e}^a$$ Therefore, since ${\varphi }_{{\mathbf{X}}_1}^{\prime }(0)=i\alpha$ for some $\alpha \in \mathbb{R}$, it follows that: $$\underset{n\to \mathrm{\infty }}{lim}{\varphi }_{{\overline{\mathbf{X}}}_n}(t)=e^{t{\varphi }_{{\mathbf{X}}_1}^{\prime }(0)}=e^{it\alpha }\mathrm{\forall }t\in \mathbb{R}$$ Hence, since $e^{it\alpha }$ is the characteristic function of a degenerate random variable, namely a random variable that is constant almost surely, it is possible to conclude that the sample mean converges in distribution to a degenerate random variable $\alpha$. Moreover, in this specific case in which the limit is a degenerate random variable it can be proved that having convergence in distribution implies also convergence in probability, something that in general is not true.

7.1 Moment generating function

Definition 7.2 ($\mathbf{\text{Moment Generating Function}}$)
Consider an uni dimensional random variable $\mathbf{X}$, then the moment generating function is defined as: $${\psi }_{\mathbf{X}}(t)=\mathbb{E}\{e^{t\mathbf{X}}\}\mathrm{\forall }t\in \mathbb{R}-\{0\}$$

Proposition 7.1 ($\mathbf{\text{Moment generating function and sequence of moments}}$)
Consider a random variable $\mathbf{X}$, such that it’s moment generating function exists and it’s finite around zero, i.e. $${\psi }_{\mathbf{X}}(t)=\mathbb{E}\{e^{t\mathbf{X}}\}<\mathrm{\infty }ϵ>0,\mathrm{\forall }t\in (-ϵ,ϵ)$$ Then this implies that the sequence of moments are finite $\mathbb{E}\{|\mathbf{X}{|}^n\}<\mathrm{\infty }$ for all $n$ and the sequence of moments uniquely determine the distribution of $\mathbf{X}$. According to this result, if we consider another random variable $\mathbf{Y}$ such that $\mathbb{E}\{|\mathbf{X}{|}^n\}=\mathbb{E}\{|\mathbf{Y}{|}^n\}$ for all $n$, then the distribution of $\mathbf{X}$ and $\mathbf{Y}$ is the same, i.e. $\mathbf{X}\sim \mathbf{Y}$.