P. Formulations and A. .. Solutions, 193 5.2.2 Chaos Extension Setup and Approach "Coefficient by Coefficient". 193 5.2.3 The USA Algorithm

. The and . .. Converges,

. , A Truncation error for trigonometric basis

. .. Compact-;-2, 216 5.1 Introduction Since the seminal work of Robbins and Monro [RM51], the method of stochastic approximation (SA for short) has become mainstream for various applications, such as optimization, parameter estimation, signal processing, adaptive control, Monte Carlo optimization of stochastic systems (see [KY97a, BMP90]), stochastic gradient descent methods in machine 6.1 Introduction

. , 2 Example: case of polynomial sequences

, Uncertainty for SA) algorithm, to compute the chaos expansion coefficients of the SA limit as a function of the uncertain parameter and proved its a.s. and L p convergence. Our goal is to analyze the L 2-convergence rate of this algorithm. Let us briefly recall the setting of Chapter 5. We consider SA that is typically used to find zeros of an intractable function h : R q ? R q that is only available in the form of an expectation as h(z) := E, Chapter 5 we designed a new method, called the USA

, In Chapter 5 the problem (6.1.1) is considered under the presence of uncertainty. Assume for simplicity that (6.1.1) has a unique solution z, where V is some random variable

G. W. Anderson, A. Guionnet, and O. Zeitouni, An introduction to random matrices, 2009.
DOI : 10.1017/cbo9780511801334

URL : http://www.wisdom.weizmann.ac.il/%7Ezeitouni/cupbook.pdf

J. An and A. Owen, Quasi-regression, Journal of Complexity, vol.17, issue.4, pp.588-607, 2001.
DOI : 10.1006/jcom.2001.0588

URL : https://doi.org/10.1006/jcom.2001.0588

Y. Aït-sahalia, Transition densities for interest rate and other nonlinear diffusions, J. Finance, vol.45, pp.1361-1395, 1999.

Y. Aït-sahalia, Maximum likelihood estimation of discretely sampled diffusions: A closed-form approximation approach, Econometrica, vol.70, pp.223-262, 2002.

Y. Aït-sahalia, Closed-form likelihood expansions for multivariate diffusions, Ann. Statist, vol.36, pp.906-937, 2008.

Y. Ait-sahalia and J. Jacod, High-frequency financial econometrics, 2014.

Y. Ait-sahalia and P. A. Mykland, The effects of random and discrete sampling when estimating continuous-time diffusions, Econometrica, vol.71, issue.2, pp.483-549, 2003.

Y. Ait-sahalia and P. A. Mykland, Estimators of diffusions with randomly spaced discrete observations: A general theory, Annals of Statistics, vol.32, pp.2186-2222, 2004.

C. Andrieu and J. Thoms, A tutorial on adaptive mcmc, Statistics and Computing, vol.18, pp.343-373, 2008.

L. Bottou and Y. Le-cun, On-line learning for very large data sets, Applied Stochastic Models in Business and Industry, vol.21, issue.2, pp.137-151, 2005.

L. Bottou, F. E. Curtis, and J. Nocedal, Optimization methods for largescale machine learning

O. Bardou, N. Frikha, and G. Pages, Computing VaR and CVaR using stochastic approximation and adaptive unconstrained importance sampling, Monte Carlo Methods and Applications, vol.15, issue.3, pp.173-210, 2009.
URL : https://hal.archives-ouvertes.fr/hal-00497588

P. Billingsley, Probability and measure, Wiley Series in Probability and Statistics, 1995.

A. Benveniste, M. Metivier, and P. Priouret, Adaptive algorithms and stochastic approximations, Applications of Mathematics, vol.22, 1990.

. E. Bngj-+-06]-o, S. E. Barndorff-nielsen, J. Graversen, M. Jacod, N. Podolskij et al., A central limit theorem for realised power and bipower variation of continuous semimartingales. In From stochastic calculus to mathematical finance. The Shiryaev Festschrift. Allmost all papers based on the presentation at the second Bachelier colloquium on stochastic calculus and probability, pp.33-68, 2005.

O. E. Barndorff-nielsen and N. Shephard, Power variation and time change, Theory of Probability and Its Applications, vol.50, pp.1-15, 2005.

J. Bouchaud and M. Potters, Financial Applications of Random Matrix Theory: a short review. The Oxford handbook of random matrix theory, pp.824-850, 2011.

N. Berman and A. Shwartz, Abstract stochastic approximations and applications, Stochastic Processes and their Applications, vol.31, pp.133-149, 1989.

A. N. Borodin and P. Salminen, Handbook of Brownian motion: facts and formulae, 2002.

I. Babuska, R. Tempone, and G. E. Zouraris, Galerkin finite element approximations of stochastic elliptic partial differential equations, SIAM J. Numer. Anal, vol.42, pp.800-825, 2004.

M. T. Barlow and M. Yor, Semi-martingale inequalities via the GarsiaRodemich-Rumsey lemma, and applications to local times, Journal of Functional Analysis, vol.49, pp.198-229, 1982.

M. T. Barlow and M. Yor, Semi-martingale inequalities via the GarsiaRodemich-Rumsey lemma, and applications to local times, Journal of Functional Analysis, vol.49, pp.198-229, 1982.

J. Chang and S. X. Chen, On the approximate maximum likelihood estimation for diffusion processes, Ann. Statist, vol.39, pp.2820-2851, 2011.

C. Canuto, M. Y. Hussaini, A. Quarteroni, and T. A. Zang, Spectral Methods: Fundamentals in Single Domains, 2006.

X. Chen and H. White, Asymptotic properties of some projection-based Robbins-Monro procedures in a Hilbert space, Studies in Nonlinear Dynamics and Econometrics, vol.6, issue.1, 2002.

L. , L. Cam, and G. L. Yang, Asymptotics in statistics, 1990.

A. Dieuleveut and F. Bach, Nonparametric stochastic approximation with large step-sizes, The Annals of Statistics, vol.44, issue.4, pp.1363-1399, 2016.
URL : https://hal.archives-ouvertes.fr/hal-01053831

D. Duffie and P. Glynn, Estimation of Continuous-Time Markov Processes Sampled at Random Time Intervals, Econometrica, vol.72, issue.6, pp.1773-1808, 2004.

]. M. Dgm-+-01, R. Dacoronga, U. Gençay, R. Müller, O. Olsen et al., Introduction to high-frequency finance, 2001.

M. Deaconu and S. Herrmann, Hitting time for Bessel processes-walk on moving spheres algorithm, Annals of Applied Probability, vol.23, issue.6, pp.2259-2289, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00636056

G. , On estimating the diffusion coefficient, Journal of Applied Probability, vol.24, pp.105-114, 1987.

M. Duflo, Random Iterative Models, Applications of Mathematics, vol.34, 1997.

D. Dufresne and F. J. Vázquez-abad, Accelerated simulation for pricing Asian options, Proceedings of the 1998 Winter Simulation Conference, pp.1493-1500, 1998.

G. Fort, B. Jourdain, T. Lelievre, and G. Stoltz, Convergence and efficiency of adaptive importance sampling techniques with partial biasing, 2016.
URL : https://hal.archives-ouvertes.fr/hal-01389996

G. Fort, E. Moulines, and P. Priouret, Convergence of adaptive and interacting markov chain monte carlo algorithms, The Annals of Statistics, vol.39, pp.3262-3289, 2011.
URL : https://hal.archives-ouvertes.fr/hal-00695649

M. Fukasawa and J. Obloj, Efficient discretisation of stochastic differential equations, 2015.

G. Fort, Central limit theorems for stochastic approximations with controlled markov chain dynamics. ESAIM: Probability and Statistics, 2014.

M. Fukasawa and M. Rosenbaum, Central limit theorems for realized volatility under hitting times of an irregular grid, Stochastic Processes and their Applications, vol.122, pp.3901-3920, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00779747

M. C. Fu and Y. Su, Optimal importance sampling in securities pricing, Journal of Computational Finance, vol.5, issue.4, pp.27-50, 2000.

C. Fuchs, Inference for diffusion processes, 2013.

M. Fukasawa, Central limit theorem for the realized volatility based on tick time sampling, Finance Stoch, vol.14, pp.209-233, 2010.

M. Fukasawa, Stochastic analysis with Financial Applications, Progress in Probability, vol.65, pp.331-346, 2011.

M. Fukasawa, Discretization error of stochastic integrals, Annals of Applied Probability, vol.21, issue.4, pp.1436-1465, 2011.

D. Funaro, Polynomial Approximation of Differential Equations, Lecture Notes in Physics Monographs, vol.8, 1992.

D. Florens-zmirou, On estimating the diffusion coefficient from discrete observations, Journal of applied probability, vol.30, issue.4, pp.790-804, 1993.

V. Genon-catalot and J. Jacod, On the estimation of the diffusion coefficient for multidimensional diffusion processes, Ann. Inst. Henri Poincaré, vol.29, pp.119-151, 1993.

V. Genon-catalot and J. Jacod, Estimation of the diffusion coefficient for Diffusion processes: Random Sampling, Scandinavian Journal of Statistics, vol.21, pp.193-221, 1994.

C. Geiss and S. Geiss, On approximation of a class of stochastic integrals and interpolation, Stochastics and Stochastics Reports, vol.76, issue.4, pp.339-362, 2004.

A. Göing-jaeschke and M. Yor, A survey and some generalizations of Bessel processes, Bernoulli, vol.9, issue.2, pp.313-349, 2003.

E. Gobet and N. Landon, Almost sure optimal hedging strategy, Ann. Appl. Probab, vol.24, issue.4, pp.1652-1690, 2014.
DOI : 10.1214/13-aap959

URL : https://hal.archives-ouvertes.fr/hal-00657153

E. Gobet and N. Landon, Optimization of joint p-variations of Brownian semimartingales, Electronic Journal of Probability, vol.19, issue.36, 2014.
URL : https://hal.archives-ouvertes.fr/hal-00853590

E. Gobet and S. Menozzi, Stopped diffusion processes: boundary corrections and overshoot, Stochastic Processes and Their Applications, vol.120, pp.130-162, 2010.
DOI : 10.1016/j.spa.2009.09.014

URL : https://hal.archives-ouvertes.fr/hal-00157975

P. Gupta, M. K. Mehlawat, M. Inuiguchi, and S. Chandra, Portfolio Optimization: An Overview, vol.316, 2014.
DOI : 10.1007/978-3-642-54652-5_1

E. Gobet, Local asymptotic mixed normality property for elliptic diffusion: a malliavin calculus approach, Bernoulli, vol.7, pp.899-912, 2001.

L. Goldstein, Minimizing noisy functionals in hilbert space: An extension of the kiefer-wolfowitz procedure, Journal of Theoretical Probability, vol.1, issue.2, pp.189-204, 1988.

R. Ghanem and P. Spanos, Stochastic Finite Elements, 1991.

R. G. Ghanem and P. D. Spanos, Stochastic finite elements: a spectral approach, Courier Corporation, 2003.
DOI : 10.1007/978-1-4612-3094-6

E. Gobet and K. Surana, A new sequential algorithm for l2-approximation and application to Monte-Carlo integration, 2014.
URL : https://hal.archives-ouvertes.fr/hal-00972016

D. Gilbarg and N. S. Trudinger, Elliptic partial differential equations of second order, 1983.

E. Gobet and E. Teman, Discrete time hedging errors for options with irregular payoffs, Finance and Stochastics, vol.5, issue.3, pp.357-367, 2001.

S. Geiss and A. Toivola, Weak convergence of error processes in discretizations of stochastic integrals and Besov spaces, Bernoulli, vol.15, issue.4, pp.925-954, 2009.

J. Grammig and M. Wellner, Modeling the interdependence of volatility and inter-transaction duration processes, Journal of Econometrics, vol.106, pp.369-400, 2002.

P. Hall and C. C. Heyde, Martingale limit theory and its application, 1980.

M. Hairer, M. Hutzenthaler, and A. Jentzen, Loss of regularity for Kolmogorov equations, Annals of Probability, vol.43, issue.2, pp.468-527, 2015.

R. A. Horn and C. R. Johnson, Matrix analysis, 1990.

T. Hayashi, J. Jacod, and N. Yoshida, Irregular sampling and central limit theorems for power variations: The continuous case. Annales de l'Institut Henri Poincaré, Probability and Statistics, vol.47, issue.4, pp.1197-1218, 2011.

T. Hayashi and P. A. Mykland, Evaluating hedging errors: an asymptotic approach, Math. Finance, vol.15, issue.2, pp.309-343, 2005.
DOI : 10.1111/j.0960-1627.2005.00221.x

URL : http://galton.uchicago.edu/~mykland/paperlinks/hedgeerrors.pdf

N. Hoffman, T. Müller-gronbach, and K. Ritter, The Optimal Discretization of Stochastic Differential Equations, Journal of Complexity, vol.17, pp.117-153, 2001.

M. Hoffmann, Minimax estimation of the diffusion coefficient through irregular samplings, Statistics and Probability Letters, vol.32, pp.11-24, 1997.

H. Haario, E. Saksman, and J. Tamminen, An adaptive metropolis algorithm, Statistics and Computing, vol.7, pp.223-242, 2001.

T. Hayashi and N. Yoshida, Asymptotic normality of a covariance estimator for nonsynchronously observed diffusion processes, Annals of the Institute of Statistical Mathematics, vol.60, pp.357-396, 2008.

J. Jacod, Statistics and high frequency data, 2007.

A. Jentzen, T. Müller-gronbach, and L. Yaroslavtseva, On stochastic differential equations with arbitrary slow convergence rates for strong approximation, Communications in Mathematical Sciences, vol.14, issue.6, pp.1477-1500, 2016.

J. Jacod, . Ph, and . Protter, Asymptotic error distributions for the Euler method for stochastic differential equations, Annals of Probability, vol.26, issue.1, pp.267-307, 1998.

J. Jacod, . Ph, and . Protter, Discretization of processes. Stochastic Modelling and Applied Probability, vol.67, 2012.

J. Jacod and A. N. Shiryaev, Limit theorems for stochastic processes, 2002.

A. A. Kulkarni and V. S. Borkar, Finite dimensional approximation and Newtonbased algorithm for stochastic approximation in Hilbert space, Automatica, vol.45, issue.12, pp.2815-2822, 2009.

M. Kleiber and T. D. Hien, The Stochastic Finite Element Method: basic perturbation technique and computer implementation, 1992.

T. G. Kurtz and P. Protter, Weak limit theorems for stochastic integrals and stochastic differential equations, Annals of Probability, vol.19, issue.3, pp.1035-1070, 1991.

S. Kinnebrock and M. Podolskij, A note on the central limit theorem for bipower variation of general functions. Stochastic Processes and their Applications, vol.118, pp.1056-1070, 2008.

P. Krée and C. Soize, Mathematics of random phenomena, 1986.

P. Kree and C. Soize, Mathematics of random phenomena: random vibrations of mechanical structures, 2012.

J. Kiefer and J. Wolfowitz, Stochastic estimation of the maximum of a regression function, Annals of Mathematical Statistics, vol.23, issue.3, pp.462-466, 1952.

H. Kushner and G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, Application of Mathematics, vol.35, 1997.

H. Kushner and G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, Application of Mathematics, vol.35, 1997.

W. K. Liu, T. Belytschko, and A. Mani, Random field finite elements. International journal for numerical methods in engineering, vol.23, pp.1831-1845, 1986.

O. , L. Maitre, and O. Knio, Spectral Methods for Uncertainty Quantification. With Applications to Computational Fluid Dynamics, Scientific Computation, 2010.

]. Y. Lmr-+-14, P. A. Li, E. Mykland, L. Renaul, X. Zhang et al., Realized Volatility When Sampling Times are Possibly Endogenous, Econometric Theory, vol.30, issue.3, pp.580-605, 2014.

J. Li, Y. Peng, and J. Chen, Nonlinear stochastic optimal control strategy of hysteretic structures, Structural Engineering and Mechanics, vol.38, issue.1, pp.39-63, 2011.

C. Lindberg and H. Rootzén, Error distributions for random grid approximations of multidimensional stochastic integrals, Ann. Appl. Probab, vol.23, pp.834-857, 2013.

Y. Li, Z. Zhang, and X. Zheng, Volatility inference in the presence of both endogenous time and microstructure noise, Stochastic Processes and their Applications, vol.123, pp.2696-2727, 2013.

T. Müller-gronbach, Strong Approximation of Systems of Stochastic Differential Equations. Habilitation thesis. Technical University of Darmstadt, 2002.

T. Müller-gronbach, Optimal pointwise approximation of SDEs based on Brownian motion at discrete points, Annals of Applied Probability, vol.14, issue.4, pp.1605-1642, 2004.

J. Muscat, Functional Analysis. An Introduction to Metric Spaces, Hilbert Spaces, and Banach Algebras, 2014.

P. A. Mykland and L. Zhang, ANOVA for diffusions and Itô processes, Annals of Statistics, vol.34, pp.1931-1963, 2006.

P. Nevai, Géza Freud, orthogonal polynomials and Christoffel functions. a case study, Journal of Approximation Theory, vol.48, issue.1, pp.3-167, 1986.

R. Nixdorf, An invariance principle for a finite dimensional stochastic approximation method in a Hilbert space, Journal of Multivariate Analysis, vol.15, issue.2, pp.252-260, 1984.

J. R. Norris, Markov chains, 1998.

B. T. Polyak and A. B. Juditsky, Acceleration of stochastic approximation by averaging, SIAM J. Control and Optimization, vol.30, pp.838-855, 1992.

P. Protter, Stochastic integration and differential equations, 2004.

S. R. Pliska and K. Suzuki, Optimal tracking for asset allocation with fixed and proportional transaction costs, Quantitative Finance, vol.4, pp.233-243, 2004.

H. Robbins and S. Monro, A stochastic approximation method, Annals of Mathematical Statistics, vol.22, issue.3, pp.400-407, 1951.

H. Rootzen, Limit distributions for the error in approximations of stochastic integrals. The Annals of Probability, 1980.

C. Y. Robert and M. Rosenbaum, On the Microstructural Hedging Error, SIAM J. Financial Math, vol.1, pp.427-453, 2010.

C. Y. Robert and M. Rosenbaum, Volatility and covariation estimation when microstructure noise and trading times are endogenous, Mathematical Finance, vol.22, issue.1, pp.133-164, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00661645

H. Robbins and D. Siegmund, A convergence theorem for nonnegative almost supermartingales and some applications, Optimizing Methods in Statistics, pp.233-257, 1971.

D. Revuz and M. Yor, Continuous martingales and Brownian motion, 1999.

R. C. Smith, Uncertainty quantification. Theory, implementation, and applications, Computational Science & Engineering. Springer, Philadelphia, vol.12, 2014.

H. Sørensen, Parametric inference for diffusion processes observed at discrete points in time: a survey, Internat. Statist. Rev, vol.72, issue.3, pp.337-354, 2004.

S. Shalev-shwartz and N. Srebro, SVM optimization: inverse dependence on training set size, Proceedings of the 25th international conference on Machine learning, pp.928-935, 2008.

R. L. Streit, Poisson Point Processes. Imaging, Tracking, and Sensing, 2010.

D. Talay, Stochastic Hamiltonian systems: exponential convergence to the invariant measure, and discretization by the implicit Euler scheme. Markov Processes and Related Fields, vol.8, pp.163-198, 2002.

H. Walk, An invariance principle for the Robbins-Monro process in a Hilbert space, vol.39, pp.135-150, 1977.

N. Wiener, The homogeneous chaos, American Journal of Mathematics, vol.60, issue.4, pp.897-936, 1938.

T. P. Wihler, On the Hölder continuity of matrix functions for normal matrices, Journal of Inequalities in Pure and Applied Mathematics, vol.10, pp.1-5, 2009.

G. Yin, On H-valued stochastic approximation: Finite dimensional projections, Stochastic Analysis and Applications, vol.10, issue.3, pp.363-377, 1992.

G. Yin and Y. M. Zhu, On H-valued Robbins-Monro processes, Journal of Multivariate Analysis, vol.34, issue.1, pp.116-140, 1990.

L. Zhou and Z. Su, Discretization error of irregular sampling approximations of stochastic integrals, Applied Mathematics-A Journal of Chinese Universities, vol.31, pp.296-306, 2016.