This book is a text intended for advanced undergraduates or graduate students which provides theoretical tools for analyzing and designing a large class of supervised, unsupervised, and reinforcement statistical machine learning algorithms using classical theorems from the fields of nonlinear optimization theory and mathematical statistics.