The following thought experiment is well-known in physics. Begin with a box that is divided into two equal halves. Each half contains many air molecules. Perform the following experiment repeatedly: in each step, a molecule is chosen at random from one half and moved to the other half. If we start with unequal number of molecules in each half, what happens in the long run?
In probability texts, this experiment is conducted with "Ehrenfest Urns". There are two urns that together contain $2k$ balls. At each step, one of the balls is chosen at random and moved from its urn to the other urn.
Task 1: Model the process as a Markov chain. Choose as states the number of balls in the first urn. Write a function to make the transition matrix $P$ for general $k$. Print out your transition matrix for $k=2$ (which should be 5 x 5). Draw the directed graph of the chain for $k=2$ case.
Task 2: Is $P$ irreducible?
Task 3: Does $P^n$ converge as $n\to \infty$?
Task 4: Plot the stationary distribution of this Markov chain for $k=100$.