Adaptive control of discrete time Markov processes by the large deviations method

Volume 27 / 2000

T. Duncan, B. Pasik-Duncan, Łukasz Stettner Applicationes Mathematicae 27 (2000), 265-285 DOI: 10.4064/am-27-3-265-285


Some discrete time controlled Markov processes in a locally compact metric space whose transition operators depend on an unknown parameter are described. The adaptive controls are constructed using the large deviations of empirical distributions which are uniform in the parameter that takes values in a compact set. The adaptive procedure uses a finite family of continuous, almost optimal controls. Using the large deviations property it is shown that an adaptive control which is a fixed almost optimal control after a finite time is almost optimal with probability nearly 1.


  • T. Duncan
  • B. Pasik-Duncan
  • Łukasz Stettner

Search for IMPAN publications

Query phrase too short. Type at least 4 characters.

Rewrite code from the image

Reload image

Reload image