Type a command or search the notes. Tab cycles completions. Up and down recall history when blank. Escape dismisses, then leaves the prompt.
MATH
[1251]

Definitions from Markov systems

All definitions from Markov systems.

Back to Markov systems

Transition matrix

An n×nn\times n transition matrix for a Markov system has non-negative entries and each column sums to 11:

aij≥0,∑i=1naij=1(j=1,…,n).a_{ij}\geq0, \qquad \sum_{i=1}^n a_{ij}=1 \quad (j=1,\ldots,n).

Column jj records the probabilities of moving from state jj to each possible next state. Some texts use the transposed convention, with these probabilities in rows.

  1. [1]:Matrix representation of a linear map
  2. [2]:Markov system
  3. [3]:Probability vectors remain probability vectors
  4. [4]:The eigenvalue one
  5. [5]:A three-state stock market
  6. 2 more

Markov system

A Markov system is the discrete dynamical system

x(k+1)=Ax(k),k=0,1,2,…,\mathbf{x}(k+1)=A\mathbf{x}(k), \qquad k=0,1,2,\ldots,

where AA is a transition matrix and x(k)\mathbf{x}(k) is the state vector at time kk.

If x(0)\mathbf{x}(0) is a probability vector, then x(k)\mathbf{x}(k) records the state probabilities after kk transitions.

  1. [1]:Transition matrix
  2. [2]:Probability vectors remain probability vectors
  3. [3]:Stationary distribution
  4. [4]:Evolve a Markov system by diagonalisation
  5. [5]:A sufficient condition for long-term Markov behaviour
  6. 3 more

Stationary distribution

A stationary distribution is a probability vector p\mathbf{p} satisfying

Ap=p.A\mathbf{p}=\mathbf{p}.

Thus a stationary distribution is an eigenvector for the eigenvalue 11, scaled so that its entries sum to 11. Equivalently, it is a non-negative eigenvector for eigenvalue 11 normalized to have component sum 11.

  1. [1]:Markov system
  2. [2]:Eigenvalue
  3. [3]:A sufficient condition for long-term Markov behaviour
  4. [4]:The stock market in the long run

On this page

Transition matrixMarkov systemStationary distribution