Sensitivity analysis using Itô-malliavin calculus and martingales, and application to stochastic optimal control

• Published in: SIAM journal on control and optimization Vol. 43; no. 5; pp. 1676 - 1713
• Main Authors: GOBET, Emmanuel, MUNOS, Rémi
• Format: Journal Article
• Language: English
• Published: Philadelphia, PA Society for Industrial and Applied Mathematics 2005
• Subjects: Algorithms; Applied sciences; Approximation; Calculus; Computer science; control theory; systems; Control system synthesis; Control theory. Systems; Exact sciences and technology; Mathematical programming; Methods; Operational research and scientific management; Operational research. Management science; Optimization; Portfolio theory; Random variables; Securities prices; Sensitivity analysis; Values; Itô equation; parameterized control; Parametric programming; Numerical method; Modeling; Optimization; Stochastic integral; Time varying system; 93E20; Discretization; Smooth function; Diffusion process; Cost function; Diffusion coefficient; Random variable; Monte Carlo method; Sensitivity analysis; Probabilistic approach; 60H30; Finance; Martingale; Adjoint method; Computational complexity; weak approximation 90C31; Weak solution; Optimal control; Stochastic control; Malliavin calculus
• ISSN: 0363-0129; 1095-7138
• DOI: 10.1137/s0363012902419059

We consider a multidimensional diffusion process $(X^\alpha_t)_{0\leq t\leq T}$ whose dynamics depends on a parameter $\alpha$. Our first purpose is to write as an expectation the sensitivity $\nabla_\alpha J(\alpha)$ for the expected cost $J(\alpha)=\mathbb{E}(f(X^\alpha_T))$, in order to evaluate it using Monte Carlo simulations. This issue arises, for example, from stochastic control problems (where the controller is parameterized, which reduces the control problem to a parametric optimization one) or from model misspecifications in finance. Previous evaluations of $\nabla_\alpha J(\alpha)$ using simulations were limited to smooth cost functions $f$ or to diffusion coefficients not depending on $\alpha$ (see Yang and Kushner, SIAM J. Control Optim., 29 (1991), pp. 1216--1249). In this paper, we cover the general case, deriving three new approaches to evaluate $\nabla_\alpha J(\alpha)$, which we call the Malliavin calculus approach, the adjoint approach, and the martingale approach. To accomplish this, we leverage Ito calculus, Malliavin calculus, and martingale arguments. In the second part of this work, we provide discretization procedures to simulate the relevant random variables; then we analyze their respective errors. This analysis proves that the discretization error is essentially linear with respect to the time step. This result, which was already known in some specific situations, appears to be true in this much wider context. Finally, we provide numerical experiments in random mechanics and finance and compare the different methods in terms of variance, complexity, computational time, and time discretization error.