Adaptive Sampling Strategies for Stochastic Composite Optimization

报告题目：Adaptive Sampling Strategies for Stochastic Composite Optimization

报告人：陈彩华   教授（南京大学 工程管理学院）

主持人：王祥丰   教授

报告时间：2024年7月13日（星期六）10:00-11:00

报告地点：华东师范大学普陀校区理科楼B504

报告摘要：

In this talk, we focus on the stochastic composite problems where the objective function comprises both smooth and nonsmooth components. When only an estimate of the gradient of the smooth component is available, we introduce adaptively sampling strategies for proximal gradient methods and its acceleration counterpart. The sample size used to estimate the gradients in each iteration is selected according to the observed trajectory of the algorithm. We develop convergence rate guarantees for unaccelerated and accelerated schemes for convex, strongly convex, and nonconvex problems, which show that the stochastic accelerated proximal gradient algorithms with adaptive sampling strategies can achieve the optimal convergence rate of first order methods. Further, under some mild conditions, we also show the asymptotic behavior of the iteration sequences. In particular, for strongly convex objectives, the iteration sequences generated by the proposed algorithms enjoy linear convergence in distribution. Our numerical experiments demonstrate the effectiveness of the proposed algorithms in both machine learning and operation management problems.

报告人简介：

Caihua Chen is a professor and associate dean in the School of Management and Engineering at Nanjing University. He received his Ph.D. in Mathematics from Nanjing University in 2012. He then joined the faculty of Nanjing University as an assistant professor and was promoted to associate professor in 2016 and professor in 2021. His research interests lie in optimization theory and algorithms, data-driven decision-making, trustworthy machine learning, human-centered AI, and their applications in revenue management, machine learning, and finance.