A parameter space, denoted by Θ, is the set of all possible parameter values.
Each element θ of Θ is called a parameter, which is a numerical value representing population characteristics.
A sample space, denoted by Y, is a set of all possible datasets. Each element y of Y is a possible dataset which describes the population.
In the Bayesian view, the parameter θ is nothing but a modeled random variable that reflects our uncertainty, contrary to the frequentist approach, which considers the parameter as a fixed but unknown value.
Definition.
For each θ∈Θ, the prior distributionp(θ) describes our belief that θ represents the true population characteristics.
For a given θ∈Θ, the sampling modelp(y∣θ) describes the variability of each y∈Y.
For a given y∈Y, the posterior distributionp(θ∣y) describes our belief about each θ∈Θ.
Preliminaries
Given θ∈Θ, the sampling model p(y∣θ) evaluates the suitability of each y∈Y. By Bayes’ Rule, we observe that the posterior distribution p(θ∣y) can be expressed by the prior distribution and the sampling model.
If we have a conjugacy so that we could obtain a closed form of the posterior, then we could try the followings.
Direct Calculation of the Posterior
Using Monte Carlo Method
If it is hard to derive the posterior then we are not able to sample directly. So the followings are available approaches.
Gibbs Sampler
Markov Chain Monte Carlo
Posterior Inference
Monte Carlo Approximation
Assume we have a closed form of posterior distribution, then the posterior inference using the Monte Carlo approximation is performed by the following steps.
A Markov chain Xn is homogeneous if Pr(Xn+1=j∣Xn=i) is independent of n.
Definition.
Let Xn be a homogeneous Markov chain with state space Θ.
We denote νna probability distribution on Θ at the time n and K:Θ×B(Θ)→Rthe transition kernel of Xn.
If Θ is at most countable, the transition kernel K is a matrix K={pij}i,j∈Θ such that
pij=Pr(Xn+1=j∣Xn=i).
Otherwise, for all A⊂Θ
K(x,A)=Pr(Xn+1∈A∣Xn=x)=∫Ap(Xn+1=y∣Xn=x)dy.
We have
Xn∼νnandνn=ν0Kn
for all n≥0 where νK(A)=∫ΘK(x,A)ν(x)dx.
Definition.
Let Xn be a homogeneous Markov chain with state space Θ and the transition kernel K.
A stationary distribution of Xn, denoted by π is a probability distribution satisfying
π=πK.
Xn is irreducible if, for any initial state, it has positive probability of entering any set to which assigns positive probability, that is, for all A⊂Θ with π(A)>0 and for all y∈Θ, there exists a positive integer n such that
Kn(y,A)>0.
Xn is positively recurrent if E(τ(y))<∞ for all y∈Θ where τ denotes the number of steps required to return from y to y.
Ergodic Markov Chain
Theorem.
Let Xn be a homogeneous Markov chain with state space Θ and the transition kernel K. Assume Xn satisfies the following.
irreducibility
positive recurrence
Then there exists a unique stationary distribution π and
S1k=1∑Sg(Xk)a.s.Eπ[g].
Note that we do not require aperidiocity of Xn since our claim is about almost sure convergence.
Xa.s.Xn if P(limn→∞Xn=X)=1.
Xa.s.Xn⟹XpXn
XpXn⟹XdXn
Theorem (Birkhoff).
Let (Ω,F,P) be a probability space and assume φ:Ω→Ω be a measure-preserving transformation on (Ω,F,P), that is,
P(φ−1A)=P(A)
for all A∈F. For any random variable X:Ω→R such that X∈L1,
n1k=0∑n−1X(φmω)a.s.E(X∣I)
in L1 where I={A∈F∣φ−1A=A}.
Markov Chain Monte Carlo
Markov Chain Monte Carlo
Posterior inference using the Markov Chain Monte Carlo method is performed by the following steps.
Obtaining a finite sample of parameter values using a Markov chain.
Note that Step 2 is identical to the Monte Carlo method.
MCMC Methods
Gibbs Sampling
Motivation
Now assume that our prior is p(θ) with θ={θ1,⋯,θm} and we only know the full conditional distributions. Our question is that could we sample from the joint distribution of {θ1,⋯,θm} exploiting the full conditional distributions
The Gibbs sampler generates a dependent sequence of parameters
{θ(1),θ(2),⋯}.
Metropolis Algorithm
Motivation
Our question is how could we approximate posterior when neither the posterior nor the full conditionals have closed forms.
Assume that our prior is p(θ) with θ=(θ1,⋯,θm).
Metropolis Algorithm
The Metropolis algorithm is as follows.
Definition.
Given a current state θ(s),
Sample θ∗∼J(θ∣θ(s)) as a candidate of θ(s+1).
Let r=p(θ(s)∣y)p(θ∗∣y)
Set
θ(s+1)={θ∗with probability min(r,1)θ(s)otherwise
Here the proposal distribution J must be symmetric, that is, J(α∣β)=J(β∣α) for all α,β∈Θ.
Metropolis Algorithm
J(θ∣θ(s)) suggests θ∗ in the neighborhood of the current state θ(s). Our typical choices of J are the followings.
J(θ∗∣θ(s))=Uniform(θ(s)−δ,θ(s)+δ)
J(θ∗∣θ(s))=Normal(θ(s),δ2)
Metropolis–Hastings
Metropolis–Hastings Algorithm
The Metropolis–Hastings algorithm is as follows.
Definition.
Given a current state θ(s),
Sample θ∗∼J(θ∣θ(s)) as a candidate of θ(s+1).
Let r=p(θ(s)∣y)p(θ∗∣y)×J(θ∗∣θ(s))J(θ(s)∣θ∗)
Set
θ(s+1)={θ∗with probability min(r,1)θ(s)otherwise
Note that the algorithm above is identical to the Metropolis algorithm whenever J is symmetric.
How It Works
Since the Gibbs sampler and Metropolis algorithm are special cases of the Metropolis–Hastings algorithm, we need only consider the case of Metropolis–Hastings.
Theorem.
Let {θ(1),θ(2),⋯,θ(S)} be a sequence of parameters generated by \cref{MH} and denote by π the stationary distribution of the Markov chain θ(n) with its state space Θ. Assume θ(n) satisfies the conditions of \cref{Ergodic}, then
π(θ)=p(θ∣y)
so we have
S1k=1∑Sg(θ(k))a.s.E[g(θ)].
Proof. Since we have the uniqueness of the stationary distribution, it suffices to show that the posterior p0=p(θ∣y) is stationary itself, that is, p0K=p0.
To obtain the transition kernel K(x,A), recall that we have a proposal J(θ∣x), r=p0(x)p0(θ)×J(θ∣x)J(x∣θ), and acceptance rate α(x,θ)=min{1,r}. So we have
Define yˉ=∑i=1nnyi and s2=∑i=1nn−1(yi−yˉ)2. Now we are going to show that the pair {yˉ,s2} is a two-dimensional sufficient statistic for {θ,σ2} by exploiting the following theorem.
Theorem (factorization theorem).
For a statistic T(x) to be sufficient for the parameter θ, the likelihood p(x∣θ) should be factorized as the form of
p(x∣θ)=g(T(x),θ)×h(x).
Note that it suffices to consider only the following part of our joint density.
The remaining part of the proof is so obvious that we left it as an exercise.
Group Comparisions and Hierarchial Modeling
Comparing Two Groups
Motivation
In the sense of frequentist, we compared two groups using p-value, which is the method called hypothesis testing. The following is the procedure of performing such model selection.
Setting null hypothesis H0 and alternative hypothesis H1.
Selecting the significance level α.
If p<α, then reject H0. \qquad Otherwise, accept H0.
Indeed it is a neat method to compare two groups. But this frequentist method has the limitation that it treats the two populations as completely distinct or exactly identical.
Comparing Two Groups by a Bayesian Analysis
Consider the following sampling model for data from the two groups.
Let us consider a model describing our information about a hierachical data
{Y1,⋯,Yn}whereYj={Y1,j,⋯,Ynj,j}.
Assume Y1,j,⋯,Ynj,j are random samples from group j, then it is reasonable to treat Y1,j,⋯,Ynj,j as exchangeable. Further assume that group j is large compared to the sample size nj, then de Finetti’s theorem implies that we can model the data within group j as
Y1,j,⋯,Ynj,j∣θj∼i.i.d.p(y∣θj).
Moreover, suppose the groups themselves are samples from larger population of groups. Then the exchangeability of the group-specific parameters θj can be justified.
Thus applying de Finetti’s theorem once again we obtain
Here we notice that p(y∣θ) represents variability within a group and p(θ∣ψ) represents variability across groups contrary to p(ψ) which represents unknown quantity.
Homoscedastic Case
The Hierarchical Normal Model: Homoscedastic Case
Here we introduce the hierarchical normal model which is a popular model for describing the heterogeneity of means across several populations. First assume that the within-group sampling variability σ2 is constant across groups.
Definition (the hierarchical normal model with the homoscedastic assumption).