Neural ODE training via stochastic control and uncertainty quantification

Data-driven and learning-based approaches for modelling of dynamic systems and for design of control laws have gained prominence in recent years. Due to their universal approximation properties, neural networks in different variants and architectures are among the most frequently considered learning methods in systems and control. In contrast to this trend, this project does not ask what machine learning can do for control. Rather we explore the question of how systems and control methods can be beneficial in the design and analysis of training formulations for neural networks.

Specifically, Project P2 considers Neural Ordinary Differential Equations (NODEs) and their explicit discretizations which take the form of Residual Networks (ResNets). We explore how generalization properties of neural networks can be directly considered in the training problems and how system-theoretic dissipativity notions of optimal control problems allow for performance-preserving pruning of trained networks. To this end, we investigate novel data informativity notions tailored to neural networks. Finally, we explore how stochastic control concepts, i.e. feedback policies, can be leveraged to design neural networks with quantifiable generalization properties. The investigated methods are evaluated on benchmark problems stemming from the machine learning literature and on systems and control specific benchmarks developed in the research unit ALeSCo.