1  Set and events

In probability, an event is interpreted as a collection of possible outcomes of a random experiment.

Definition 1.1 ($\mathbf{\text{Random experiment}}$)
A random experiment is any repeatable procedure that results in one out of a well-defined set of possible outcomes.

• The set of possible outcomes is called sample space and denoted as $\mathrm{\Omega }$.
• A set of zero or more outcomes is an event.
• A map that goes from events to probabilities is called a probability function and it is denoted as $\mathbb{P}$. Together, sample space, event space and probability function characterize a random experiment.

1.1 Set operations

There are several definitions related to sets and their operation.

Definition 1.2 ($\mathbf{\text{Complementation}}$)
The complement of a set $A$ is denoted by $A^c$ and represents the set of elements that do not belong to $A$, i.e. $$\begin{array}{}\text{(1.1)}& A^c=\{\omega \in \mathrm{\Omega }:\omega \notin A\}\text{.}\end{array}$$

Definition 1.3 ($\mathbf{\text{Containment}}$)
A set $A$ is said to be contained in a set $B$ if every element of $A$ is also an element of $B$. Formally, $$\begin{array}{}\text{(1.2)}& A\subset B⟺\omega \in A⟹\omega \in B\mathrm{\forall }\omega \in \mathrm{\Omega }\text{.}\end{array}$$

Definition 1.4 ($\mathbf{\text{Equality}}$)
Given two sets, $A$ is equal to $B$, written $A=B$, if and only if every element of $A$ is an element of $B$ and every element of $B$ is an element of $A$. Formally, $$\begin{array}{}\text{(1.3)}& A\subset B\text{and}B\subset A\text{.}\end{array}$$

Let’s now state some elementary operations between sets.

Figure 1.1: Elementary set operations.

Definition 1.5 ($\mathbf{\text{Union}}$)
The union of two sets, written $A\cup B$, is the set of $\omega$ that belongs either to $A$ or $B$, i.e. $$\begin{array}{}\text{(1.4)}& A\cup B=\{\omega \in \mathrm{\Omega }:\omega \in A\text{ or }\omega \in B\}\text{.}\end{array}$$

As a consequence of the definition of the union the following relations holds true, i.e.  $$\begin{array}{rlrl}& A\cup A=A& & A\cup \mathrm{\Omega }=\mathrm{\Omega }\\ & A\cup \mathrm{\varnothing }=A& & A\bigsqcup A^c=\mathrm{\Omega }\end{array}$$

Definition 1.6 ($\mathbf{\text{Intersection}}$)
The intersection of $A$ and $B$ is written $A\cap B$ and is the set of elements that belongs at the same time to $A$ and $B$. $$A\cap B=\{\omega \in \mathrm{\Omega }:\omega \in A\text{ and }\omega \in B\}\text{.}$$

As a consequence of the definition of the intersection the following relations holds, i.e.  $$\begin{array}{rlrl}& A\cap A=A& & A\cap \mathrm{\Omega }=A\\ & A\cap \mathrm{\varnothing }=\mathrm{\varnothing }& & A\sqcap A^c=\mathrm{\varnothing }\end{array}$$ Moreover, let’s state the distributive laws of the union and the intersection, i.e.  $$\begin{array}{}\text{(1.5)}& \begin{array}{rlrl}& \mathbf{Intersection}.& & (A\cup B)\cap C=(A\cap C)\cup (B\cap C)\\ & \mathbf{Union}.& & (A\cap B)\cup C=(A\cup C)\cap (B\cup C)\end{array}\end{array}$$ And the De Morgan’s laws: $$\begin{array}{}\text{(1.6)}& \begin{array}{rlrl}& \mathbf{Intersection}.& & (A\cap B{)}^c=(A^c\cup B^c)\\ & \mathbf{Union}.& & (A\cup B{)}^c=(A^c\cap B^c)\end{array}\end{array}$$

Definition 1.7 ($\mathbf{\text{Difference}}$)
The difference between two sets $A$ and $B$, written $A-B$ (or also $A/B$), is the set of elements of $A$ that do not belong to $B$. Formally $$\begin{array}{}\text{(1.7)}& A-B=A\cap B^c=\{\omega \in \mathrm{\Omega }:\omega \in A\text{ and }\omega \notin B\}\text{.}\end{array}$$

Disjoint reppresentation of a set

Given two set $A$ and $B$, then each one can be written as the union of disjoint sets. In fact, their union can be decomposed into the union of three disjoint sets, i.e.  $$\begin{array}{}\text{(1.8)}& A\bigsqcup B=(A\cap B)\bigsqcup (A\cap B^c)\bigsqcup (A^c\cap B)\text{,}\end{array}$$ and therefore for example the set $A$ can be written as $$\begin{array}{}\text{(1.9)}& A=(A\cap B)\bigsqcup (A-B)=(A\cap B)\bigsqcup (A\cap B^c)\text{.}\end{array}$$

Definition 1.8 ($\mathbf{\text{Symmetric difference}}$)
The symmetric difference between two sets $A$ and $B$ is written $A\mathrm{\Delta }B$ and is the union of elements of $A$ that do not belong to $B$ and of elements of $B$ that do not belong to $A$, i.e. $$\begin{array}{rl}A\mathrm{\Delta }B& =(A-B)\bigsqcup (B-A)=\\ & =(A\cap B^c)\bigsqcup (A^c\cap B)=\\ & =\{\omega :\omega \in A,\omega \notin B\}\bigsqcup \{\omega :\omega \in B,\omega \notin A\}\end{array}$$

Proposition 1.1 Given two set $A,B$, the symmetric difference can be written as $$A\mathrm{\Delta }B=(A\bigsqcup B)\cap (A^c\bigsqcup B^c)\text{.}$$

Proof: Proposition 1.1

Proof. Let’s denote with $C=A^c\cap B$, then apply the distributive law of the union twice (Equation 1.5) and develop the computations, i.e. $$\begin{array}{rl}A\mathrm{\Delta }B& =(A\cap B^c)\bigsqcup (A^c\cap B)=\\ & =(A\cap B^c)\bigsqcup C=\\ & =(A\cup C)\cap (B^c\cup C)=\\ & =[A\cup (A^c\cap B)]\cap [B^c\cup (A^c\cap B)]=\\ & =[((A\bigsqcup A^c)\cap (A\cup B)]\cap (B^c\cup A^c)\cap (B^c\bigsqcup B)=\\ & =(A\cup B)\cap (A^c\cup B^c)\end{array}$$

1.2 Indicator function

Definition 1.9 ($\mathbf{\text{Indicator function}}$)
An indicator function is a function that associate an $\omega \in A\subset \mathrm{\Omega }$ to a real number, i.e. either 0 or 1. It is a tool that allows to transfer a computation from the set domain into the real numbers domain, i.e. $\{0,1\}$. Formally, ${\mathbb{1}}_A(\omega ):\mathrm{\Omega }\to \{0,1\}$, i.e.  $${\mathbb{1}}_A(\omega )=\{\begin{array}{l}1\omega \in A\\ 0\omega \in A^c\end{array}$$

Proposition 1.2 The containment between two sets can be equivalently written in terms of indicator functions: $$A\subset B⟺{\mathbb{1}}_A(\omega )\le {\mathbb{1}}_B(\omega ),\mathrm{\forall }\omega \in \mathrm{\Omega }\text{.}$$

Proof: Proposition 1.2

Proof. In order to prove the results in Proposition 1.2, let’s start by assuming $A\subset B$ and let’s distinguish two main cases.

1. Assuming $\omega \in A$ implies that $\omega \in B$, and therefore one have an equality $1={\mathbb{1}}_A\le {\mathbb{1}}_B=1$.
2. Assuming $\omega \in A^c$ implies $[\omega \in B]\cup [\omega \in B^c]$. In this situation for both cases one will have that ${\mathbb{1}}_A\le {\mathbb{1}}_B$, in fact:
   • Considering $\omega \in B$ implies that $0={\mathbb{1}}_A<{\mathbb{1}}_B=1$.
   • Considering $\omega \in B^c$ implies that $0={\mathbb{1}}_A\le {\mathbb{1}}_B=0$.

Hence, assuming $A\subset B$ implies that ${\mathbb{1}}_A(\omega )\le {\mathbb{1}}_B(\omega )$ for all $\omega \in \mathrm{\Omega }$. Now let’s assume the contrary: ${\mathbb{1}}_A\le {\mathbb{1}}_B$ and let’s again distinguish in two main cases:

1. Assuming $\omega \in A$, i.e. ${\mathbb{1}}_A=1$, the inequality ${\mathbb{1}}_A\le {\mathbb{1}}_B$ holds and since the indicator function is bounded by 1 by definition it is possible to write $1={\mathbb{1}}_A\le {\mathbb{1}}_B\le 1$. Therefore, one obtain ${\mathbb{1}}_B=1$ and so $\omega \in B$.
2. Assuming $\omega \in A^c$, i.e. ${\mathbb{1}}_A=0$, the inequality ${\mathbb{1}}_A\le {\mathbb{1}}_B$ holds and it is possible to write $0={\mathbb{1}}_A\le {\mathbb{1}}_B\le 1$. Hence, when $\omega \in A^c$, there are two possible cases, i.e. 
   • ${\mathbb{1}}_B=1$, but this implies that $\omega \in B$.
   • ${\mathbb{1}}_B=0$, but this implies that $\omega \in B^c$.

When an $\omega \in A$ implies that $\omega \in B$, but the contrary do not holds true. Hence, it is possible to conclude that $A\subset B$.

1.3 Limits of sets

Let’s define the infimum ($inf$) and the $liminf$ of a sequence of sets $A_k$ as: $$\underset{k\ge n}{inf}=\bigcap _{k=n}^{\mathrm{\infty }}{A}_k,\underset{n\to \mathrm{\infty }}{lim}inf{A}_n=\underset{n\ge 1}{sup}\underset{k\ge n}{inf}{A}_k=\bigcup _{n=1}^{\mathrm{\infty }}\bigcap _{k=n}^{\mathrm{\infty }}{A}_k\text{.}$$ Informally, the infimum of a sequence of sets is the smallest set in $k=n,\dots ,\mathrm{\infty }$, it follows that the limit of the infimum ($liminf$) is the biggest (union) among all the smallest (intersection) sets. Instead the supremum ($sup$) and the $limsup$ are defined as: $$\underset{k\ge n}{sup}=\bigcup _{k=n}^{\mathrm{\infty }}{A}_k,\underset{n\to \mathrm{\infty }}{lim}sup{A}_n=\underset{n\ge 1}{inf}\underset{k\ge n}{sup}{A}_k=\bigcap _{n=1}^{\mathrm{\infty }}\bigcup _{k=n}^{\mathrm{\infty }}{A}_k\text{.}$$ On the other hand, the supremum of a sequence of sets is the biggest set in $k=n,\dots ,\mathrm{\infty }$, it follows that the limit of the supremum ($limsup$) is the smallest (intersection) among all the biggest (union) sets. Moreover, by De Morgan’s laws: $$(\underset{n\to \mathrm{\infty }}{lim}sup{A}_n{)}^c={\left(\bigcap _{n=1}^{\mathrm{\infty }}\bigcup _{k=n}^{\mathrm{\infty }}{A}_k\right)}^c=\bigcup _{n=1}^{\mathrm{\infty }}\bigcap _{k=n}^{\mathrm{\infty }}{A}_k^c=\underset{n\to \mathrm{\infty }}{lim}inf{A}_n^c\text{,}$$ and similarly $$(\underset{n\to \mathrm{\infty }}{lim}inf{A}_n{)}^c={\left(\bigcup _{n=1}^{\mathrm{\infty }}\bigcap _{k=n}^{\mathrm{\infty }}{A}_k\right)}^c=\bigcap _{n=1}^{\mathrm{\infty }}\bigcup _{k=n}^{\mathrm{\infty }}{A}_k^c=\underset{n\to \mathrm{\infty }}{lim}sup{A}_n^c\text{.}$$

1.4 Fields and $\sigma$-fields

Definition 1.10 ($\mathbf{\text{Field}}$)
Let’s consider the sample space $\mathrm{\Omega }$, then a field is a non-empty class of subsets of $\mathrm{\Omega }$ closed under finite union, finite intersection and complementation. Formally, $\mathcal{A}$ is a field if and only if:

1. $\mathrm{\Omega }\in \mathcal{A}$.
2. $A\in \mathcal{A}⟹A^c\in \mathcal{A}$.
3. $A,B\in \mathcal{A}⟹A\cup B\in \mathcal{A}$.

Definition 1.11 ($\sigma \mathbf{\text{-field}}$)
Let’s consider the sample space $\mathrm{\Omega }$, then a $\sigma$-field is a non-empty class of subsets of $\mathrm{\Omega }$ closed under countable union, countable intersection and complementation. Formally, $\mathcal{B}$ is a $\sigma$-field if and only if:

1. $\mathrm{\Omega }\in \mathcal{B}$.
2. $B\in \mathcal{B}⟹B^c\in \mathcal{B}$.
3. $B_i\in \mathcal{B},i\ge 1⟹\bigcup _{n\ge 1}{B}_n\in \mathcal{B}$.

Field vs $\sigma$-Field

The main difference between a field and a $\sigma$-field is in the third property of the definitions. A field is closed under finite union, namely the union of a finite sequence of events $A_n$ indexed by $n\in \{0,1,2,\dots ,n\}$ (property 3 of Definition 1.10). On the other hand, a $\sigma$-field is closed under countable union, namely the union of an infinite sequence of events $A_n$ indexed by $n\in \{0,1,2,\dots ,n,n+1,\dots \}$ (property 3. of the Definition 1.11).