Hello everyone,
I was wondering whether the DiscreteDP interface provided by QuantEcon can also handle problems with multiple actions? For instance, the amount saved plus a portfolio problem.
I would be super grateful for any examples!
Best regards,
Ilja
My understanding on how some of those discrete problems are done in general (not specific to quantecon) is that you write each discrete combination of actions as a single action. Eg, you can take the Cartesian product of them to form a single action and rewrite your problem accordingly. Continuous states are different.
If that works, then in your implementation of the transition matrices/ that encode feasibility you can write code to unpack that single state. As an implementation detail, you may need to take some ratios and the remainder to turn the one dimensional linear index into the set of two indices for the actions consistent with hie you organized your Cartesian product.
Hi, IljaK91:
DiscreteDP assumes no structure on the action space, except that it must be a finite set. Enumerate your (multi-dimensional) actions from 0 through, say, m-1, as well as the states, from 0 to n-1. Then build the n x m reward array R, where R[s, a] is the reward to the action with index a at the state with index s, and the n x m x n transition probability array Q, where Q[s, a, s'] is the probability that the next state is the state with index s' when the action with index a is taken at the state with index s.
Thank you very much! I tried as you suggested and it works well 
Great experience using QuantEcon for value function iteration!