Stochastic optimization for learning over networks

讲座简介


Stochastic optimization methods, e.g., stochastic gradient descent (SGD), have recently found wide applications in large-scale data analysis, especially in machine learning. These methods are very attractive to process online streaming data as they scan through the dataset only once but still generate solutions with acceptable accuracy. However, it is known that classical SGDs are ineffective in processing streaming data distributed over multi-agent network systems (e.g., sensor and social networks), mainly due to the high communication costs incurred by these methods.

In this talk, we present a few new classes of SGDs which can significantly reduce the aforementioned communication costs for distributed or decentralized machine learning. We show that these methods can significantly save inter-node communications when performing SGD iterations. Meanwhile, the total number of stochastic (sub)gradient computations required by these methods are comparable to those optimal ones achieved by classical centralized SGD type methods.

This talk is based on the following two papers.

1. G. Lan and Y. Zhou, Random gradient extrapolation for distributed and stochastic optimization, SIAM Journal on Optimization, 28(4), 2753-2782, 2018.

2. G. Lan, S. Lee and Y. Zhou, Communication-efficient Algorithms for Decentralized and Stochastic Optimization, Mathematical Programming, to appear, 2018.

时间


2018-12-17

下午 14:00 ~ 15:00

主讲人


Guanghui (George) Lan, H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology

地点


信息管理与工程学院102室
上海财经大学(第三教学楼西侧)
上海市杨浦区武东路100号