A turnpike improvement algorithm for piecewise deterministic control

K. Boukas, A. Haurie, C. VAN DELFT

Optimal Control Applications and Methods

January-March 1991, vol. 12, n°1, pp.1-18

Departments: Information Systems and Operations Management, GREGHEC (CNRS)

Keywords: Stochastic control, Infinite horizon optimal control, Turnpike properties, Piecewise deterministic control problems, Policy improvement algorithm

This paper proposes a numerical technique, called turnpike improvement, for the approximation of the solution of a class of piecewise deterministic control problems typically associated with manufacturing flow control models. This algorithm exploits the structure of Markov decision processes with continuous state and action spaces that can be associated with piecewise deterministic control systems. The numerical method is applicable whenever a turnpike property holds for some associated infinite horizon deterministic control problem. To illustrate the approach, we use a simple model fully studied from an analytic point of view in the literature. We compare the turnpike improvement technique with a direct approximation of the solution of the continuous-time Hamilton-Jacobi dynamic programming equations inspired by Kushner's work. The two approaches agree remarkably on this simple problem. We conclude with a discussion of the relative advantages of the two approaches.