State-Space Search Using Numba with Parallelization

#1

Hello all,

I am solving a life-cycle model of human capital accumulation with elastic labor supply. Solving the model requires me to search over a state-space of (ability, human capital, assets). It takes quite some time for optimizing each period. I tried to use Numba for faster loops, but it still takes about 12 minutes to solve for the optimal decision rules. I want to know whether I can parallelize the for loops to make it faster (my machine has 8 cores). I tried @njit(parallel=True), but it did not make a difference. Any suggestions?

#2

Hi @Mehrdad_Esfahani. It’s hard to give general advice but the best strategy is to break the operation down in to a bunch of small functions and @njit them individually, testing for speed gains on each occasion.

Regarding parallelization, I’m not sure if the tasks you’re implementing are actually parallel. But if they are, parallelization should make a difference, after you swap range to prange in your loop. Look at your system monitor (e.g., htop for linux/Mac) and see if your cores are lighting up. If not perhaps you need to think again about whether you are actually farming out independent operations.

Hope that helps. John.