How can we formulate a continuous time optimization problem as a boundary value pde?

I would like to contribute material on continuous time optimization, but there is one thing that is not clear to me.

Consider a routine continuous time optimization problem:
\max \int_{t=0}^{T} e^{-\rho t}u(c_{t})dt \text{ s.t. }
\dot{a}_{t} = y + ra_{t} - c_{t},
a_{0} \text{ given, } a_{T}=0.
Assume y & r are constants and u(c)=\frac{c^{1-\gamma}}{1-\gamma}.
Write the value function V(t,a_{t})

Optimal control approach:
H\equiv u(c_{t}) + \lambda (y + ra_{t} - c_{t})
Boils down to a BVP (two ODEs w/ unkown c_{t}, a_{t}):
\left[\begin{array}{l} \dot{c}=\left(\frac{r-\rho}{\gamma} \right) c \\ \dot{a}_{t} = y + ra_{t} - c_{t} \\ a(0)=a_0 \\ a_{T}=0 \end{array} \right]

HJB approach:
\rho V(t,a_{t}) = \max_{c} \left\{u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) \right\}
FOC: c(t,a_{t})=u'^{-1}(V_{a}(t,a_{t})) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}}
This boils down to a PDE w/ unknown function V(t,a_{t}):
\left[\begin{array}{l} \rho V(t,a_{t}) = u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) \\ c(t,a_{t}) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}} \\ a(0)=a_0 \\ a_{T}=0 \end{array} \right]

Issue: we have boundary conditions for a_{t} (a(0)=a_0, a_{T}=0) but not for V(t,a_{t}).
To solve this PDE we need two boundary conditions for V(t,a_{t}).
Question: How do we get the two boundary conditions?
I think one of them is: V(T,a_{T}) = \psi(a_{T}) = 0 (no bequest here)

Or maybe something like:
\left[\begin{array}{l} \rho V(t,a_{t}) = u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) \\ c(t,a_{t}) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}} \\ V(T,a_{T}) = 0 \\ V_{a}(T,a_{T}) = 0 \end{array} \right]

Thanks @azev77

I think that we have some plans to eventually document some of those methods, but they tend to be pretty tricky to solve due to inherent backwards and forwards of HJBEs with a control. Sadly, the naive discretizations don’t work or convergence. At this point, the DifferentialEquations.jl package doesn’t have the algorithms that we would need to make it clean. It is something that has been discussed, but I wouldn’t hold your breath. Maybe later next year.

1 Like