Universal Features – Information Extraction in Learning Algorithms

Speaker: Prof. Lizhong Zheng
Affiliation: Massachusetts Institute of Technology

Abstract: In this talk, we study the problem of representing high dimensional data with lower dimensional features that retain the “useful” information. We formulate the problem as a special case of lossy information processing which we called “universal feature selection”, where we try to extract information that is potentially useful for an ensemble of possible inference problems. We use a geometric approach to find the solution to the optimal universal features, and show that the result has connections to a number of interesting problems in information theory and statistics, such as the HGR correlation, Wyner common information. We also show that there are a number of learning algorithms, Alternating Conditional Expectation (ACE), matrix completion and neural networks, that can be viewed as numerical solution to this problem. This insight gives rise to an information theoretic interpretation of these algorithms, as well as some novel ways to design new learning algorithms.

Biography: Lizhong Zheng received the B.S and M.S. degrees, in 1994 and 1997 respectively, from the Department of Electronic Engineering, Tsinghua University, China, and the Ph.D. degree, in 2002, from the Department of Electrical Engineering and Computer Sciences, University of California, Berkeley. Since 2002, he has been working at MIT, where he is currently a professor of Electrical Engineering. His research interests include information theory, statistical inference, communications, and networks theory. He received Eli Jury award from UC Berkeley in 2002, IEEE Information Theory Society Paper Award in 2003, and NSF CAREER award in 2004, and the AFOSR Young Investigator Award in 2007. He served as an associate editor for IEEE Transactions on Information Theory, and the general co-chair for the IEEE International Symposium on Information Theory in 2012. He is an IEEE fellow.

For more information, contact Prof. Suhas Diggavi ()

Date(s) - Nov 18, 2019
12:30 pm - 1:30 pm

EE-IV Shannon Room #54-134
420 Westwood Plaza - 5th Flr., Los Angeles CA 90095