Optimization for Machine Learning

In recent years, huge advances have been made in machine learning, which has transformed many fields such as computer vision, speech processing, and games. A key “secret sauce” in the success of these models is the ability of certain architectures to learn good representations of complex data; that is, preprocessing the data to facilitate optimization and generalization.

We investigate two instances where encoding structure in the feature variable facilitates optimization. In the first case, we look at word vectors as language-modeling mathematical constructs, and their use in data mining applications. In the second case, we investigate the curious ability of proximal methods to quickly identify sparsity patterns in an optimization variable, which greatly facilitates feature selection. These two projects give a glimpse into the key problems in the growing field of machine learning and data science in the wild.