Estimating Markov chain probabilities in Julia using proportions

Lectures
1

Greetings:

This is related to this thread.

Estimating transition probabilities in a Markov chain with maximum likelihood reduces to counting the transitions from a given state to another divided by the total number of transitions. Since the QuantEcon package in Julia seems to lack a built-in routine to do this, would someone be able to recommend an efficient way to write this from scratch?

2

Hi:

Playing around with it, it seems this Julia code estimates the Markov chain using proportions:

function estimate_markov(X::Vector, nstates::Int64)
    P = zeros(nstates, nstates)
    n = length(X) - 1
    for t = 1:n
        # add one each count is observed
        P[X[t], X[t+1]] = P[X[t], X[t+1]] + 1
    end
    for i = 1:nstates
        # divide by total number of counts
        P[i, :] .= P[i, :]/sum(P[i, :])
    end
    return P
end

Appreciate any feedback if there are any errors or significant ways to improve form/efficiency.

Best,
Mario

3

Many thanks @msilva913 !

I’ve opened an issue to track this here: Add MLE estimation of Markov chains · Issue #283 · QuantEcon/QuantEcon.jl · GitHub

We just added it to the Python side so it would be good to do the same for Julia.

4

Is this for an enormous markov chain? If not, and it is sufficiently fast for your needs, then the code is pretty clear, so no need to make it faster.