Foundations of Machine Learning

by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar

ISBN: 9780262306577 | Copyright 2012

Click here to preview


This graduate-level textbook introduces fundamental concepts and methods in machine learning. It describes several important modern algorithms, provides the theoretical underpinnings of these algorithms, and illustrates key aspects for their application. The authors aim to present novel theoretical tools and concepts while giving concise proofs even for relatively advanced topics.

Foundations of Machine Learning fills the need for a general textbook that also offers theoretical details and an emphasis on proofs. Certain topics that are often treated with insufficient attention are discussed in more detail here; for example, entire chapters are devoted to regression, multi-class classification, and ranking. The first three chapters lay the theoretical foundation for what follows, but each remaining chapter is mostly self-contained. The appendix offers a concise probability review, a short introduction to convex optimization, tools for concentration bounds, and several basic properties of matrices and norms used in the book.

The book is intended for graduate students and researchers in machine learning, statistics, and related areas; it can be used either as a textbook or as a reference text for a research seminar.

A solid, comprehensive, and self-contained book providing a uniform treatment of a very broad collection of machine learning algorithms and problems. Foundations of Machine Learning is an essential reference book for corporate and academic researchers, engineers, and students.

Corinna Cortes Head of Google Research, NY

Finally, a book that is both broad enough to cover many algorithmic topics of machine learning and mathematically deep enough to introduce the required theory for a graduate level course. Foundations of Machine Learning is a great achievement and a significant contribution to the machine learning community.

Yishay Mansour School of Computer Science, Tel Aviv University
Expand/Collapse All
Contents (pg. v)
Preface (pg. xi)
1 Introduction (pg. 1)
2 The PAC Learning Framework (pg. 11)
3 Rademacher Complexity and VC-Dimension (pg. 33)
4 Support Vector Machines (pg. 63)
5 Kernel Methods (pg. 89)
6 Boosting (pg. 121)
7 On-Line Learning (pg. 147)
8 Multi-Class Classification (pg. 183)
9 Ranking (pg. 209)
10 Regression (pg. 237)
11 Algorithmic Stability (pg. 267)
12 Dimensionality Reduction (pg. 281)
13 Learning Automata and Languages (pg. 293)
14 Reinforcement Learning (pg. 313)
Conclusion (pg. 339)
Appendix A: Linear Algebra Review (pg. 341)
Appendix B: Convex Optimization (pg. 349)
Appendix C: Probability Review (pg. 359)
Appendix D: Concentration inequalities (pg. 369)
Appendix E: Notation (pg. 379)
References (pg. 381)
Index (pg. 397)

Mehryar Mohri

Mehryar Mohri is Professor of Computer Science at New York University’s Courant Institute of Mathematical Sciences and a Research Consultant at Google Research.

Afshin Rostamizadeh

Afshin Rostamizadeh is a Research Scientist at Google Research.

Ameet Talwalkar

Ameet Talwalkar is a National Science Foundation Postdoctoral Fellow in the Department of Electrical Engineering and Computer Science at the University of California, Berkeley.

You must have an instructor account and submit a request to access instructor materials for this book.
Go paperless today! Available online anytime, nothing to download or install.
Device Compatibility


  • Bookmarking
  • Note taking
  • Highlighting