Active learning for stochastic economic model predictive control

The aim of Project P5 is to develop computationally tractable formulations and numerical methods for the online solution of stochastic optimal control formulations with nonlinear constrained models, whose state cannot be measured directly but needs to be inferred from noisy measurements. These problems are modeled in terms of a partially observable Markov decision processes (POMDP) in continuous state and action spaces where a belief of the current system state and system parameters – a state estimate – is propagated. Crucially, the prediction models under consideration might be black-box or grey-box models and shall directly encode a measure of prediction uncertainty (an example is the covariance matrix prediction in an extended Kalman filter). Given that the prediction uncertainty might have a negative impact on the economic objective, but can potentially be reduced by a smart choice of the chosen control actions, the exact solution of the stochastic optimal control problem will automatically encode a form of active learning that might be classified as implicit dual control. While it is intractable to compute the exact solution, we aim at an online optimization based model predictive control formulation that qualitatively preserves the dual control effect.

Thus, the three major aims of the proposed project are:

1. Development of tractable nonlinear programming (NLP) formulations for economic optimal control of stochastic constrained nonlinear systems with output feedback.
2. Development of tailored numerical algorithms for the formulations of Goal 1.
3. Analysis and evaluation of approximate problem formulations with respect to data informativity measures and computational cost.