A Dynamic Programming Approach to Control Heat Equation with Random Walk Process Using HJB Equation

Abstract

The heat equation is considered with Random Walk and Brownian motion under the assumption of Bernoulli's, Binomial, Geometric and Poisson distributions for Markov chain.

Some numerical methods are also used to find a numerical solution of heat equation under certain conditions as finite difference method (explicit and implicit), Crank Nicolson method and method of lines.

Separation of variables method also used to determine an analytic solution of heat equation.

In addition, we have used the Hamilton Jacobi Bellman equation (HJB) and algebraic Riccati equation that arises in the linear quadratic regulator (LQR) to obtain the optimal control function for heat equation.

Finally, a comparison between exact and approximate solution for state space equation using Euler's method.