Day/Time: | TuTh 2:00pm -- 3:15pm EST |
Location: | EGR - Room 1104 |
Armand M. Makowski | |
Office: | AVW - 2357 |
Voice: | (301) 405 - 6844 |
Fax: | (301) 314 - 9281 |
Email: | armand@umd.edu |
Day/Time: | TuTh 11:00am - 12:15pm EST |
Also by appointment (in office or by zoom) |
ENEE 634 has not been offered in a number of years with earlier editions focusing mainly on the application of adaptive learning and statistical signal processing to problems in communications engineering, e.g., blind equalization and identification (unsupervised learning), antenna array and MIMO signal processing,mspace-time and space-time-frequency coding, and neural networks (nonlinear adaptive learning).
In Fall 2023 ENEE 634 will be devoted to exploring a number of advanced topics in Machine Learning (ML); see list below. The emphasis will be on understanding the theoretical foundations of several important ideas in ML, both supervised and unsupervised, with some attention given to the needed mathematical tools. This reorientation of the course contents reflects the growing popularity of ML techniques in many application areas. The course ENEE 633/CMSC 828C (Statistical Pattern Recognition), regularly offered during Fall semesters, is not a prerequisite for ENEE 634: While some knowledge of ML may be helpful, lack of it will not be an impediment as the course will be taught in a self-contained manner.
1. | Shai Ben-David and Shai Shalev-Shwartz, Understanding Machine Learning: From Theory to Algorithms. A copy is available through this link | |
Cambridge University Press (2014), Cambridge (United Kingdom) | ||
ISBN 978-1-107-05713-5 |
2. | Martin J. Wainwright, High-Dimensional Statistics -- A Non-Asymptotic Viewpoint | |
Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press (2019), Cambridge (United Kingdom) | ||
ISBN 978-1-108-49082-9 |
3. | Roman Vershynin, High-Dimensional Probability -- An Introduction with Applications to Data Science. | |
An early draft and additional course material are available through this link | ||
Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press (2019), Cambridge (United Kingdom) | ||
ISBN 978-1-108-41519-4 |
On occasion lectures notes on certain topics and issues will be made available.
1. | Armand M. Makowski and Prakash L.K. Narayan |
Lecture Notes on the detection problem/hypothesis testing problem | |
Soft version available here | |
2. | Bruce Hajek and Maxim Raginsky |
Lecture Notes on Statistical Learning Theory | |
Soft version available here | |
3. | Gyora M. Benedek and Alon Itai |
Nonuniform Learnability, Journal of Computer and Systems Science 48 (1994), pp. 311-323 | |
4. | Anselm Blume, Andrzej Ehrenfeucht, David Haussler and Manfred K. Warmuth |
Learnability and the Vapnik-Chervonenkis dimension, JACM 36 (1994) , pp. 929-965 | |
5. | Noga Alon, Shai Ben-David, Nicolo Cesa-Bianchi and David Haussler |
Scale-sensitive dimensions, uniform convergence and learnability, JACM 44 (1997), pp. 615-631 | |
6. | Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro and Khartik Sridharan, |
Learnability, stability and uniform convergence, Journal of Machine Learning Research JMLR 11 (2010), pp. 2635-2670. |
A Formal Learning Model [Chap. 3]
Learning via Uniform Convergence [Chap. 4] -- Additional information concerning concentration inequalities can be found in Chapter 2 of the monograph by Wainwright. See also Chapter 2 of the text by Vershynin.
The VC-Dimension [Chap. 6] -- See also Chapters 4 and 5 of the monograph by Wainwright, and Sections 8.3 and 8.4 of the text by Vershynin.
Nonuniform Learnability [Chap. 7]
The Runtime of Learning [Chap. 8]
Stochastic Gradient Descent [Chap. 14]
Kernel Methods [Chap. 16] -- See also Chapter 12 of the monograph by Wainwright.
Online Learning [Chap. 21]
Complexities [Chap. 26]
Covering Numbers [Chap. 27]
Proof of the Fundamental Theorem of Learning Theory [Chap. 28]
PAC-Bayes [Chap. 31]
In the text by Ben-David and Shalev-Shwartz unless noted otherwise.
Week # 1 starting 08/28: | Chapter 2 |
Week # 2 starting 09/04: | Chapters 2 and 3 |
Week # 3 starting 09/11: | Chapter 3 |
Week # 4 starting 09/18: | Chapter 4 and Chapter 5. |
A good discussion of concentration bounds can be found in M. Wainwright's text: Chapter 2 and R. Vershynin's text: Chapter 2 | |
Week # 5 starting 09/25: | Chapter 6 |
Week # 6 starting 10/02: | Chapter 6 |
Week # 7 starting 10/09: | Chapter 6 and Chapter 7 |
R. Vershynin's text: Section 8.3 (up to 8.3.3) and Section 8.4 | |
Week # 8 starting 10/16: | Chapter 7 |
Week # 9 starting 10/23: | Chapters 13 to 15 (with particular attention to Chapter 13) |
Week # 10 starting 10/30: | Chapters 13 to 15 (with particular attention to Chapter 14) |
Week # 11 starting 11/06: | Chapters 13 to 15 (with particular attention to Chapter 14) |
Week # 12 starting 11/13: | |
Week # 13 starting 11/20: | |
Week # 14 starting 11/27: | |
Week # 15 starting 12/04: |
Grading methods include both Audit and Regular Grading. In the latter case the final grade for the course will be based on performance on either a research paper, a research project or a class presentation. Details to be discussed with instructor during the semester.
08/28/2023 | First day of classes for Fall 2023 | |
08/29/2023 | First class for ENEE 634 | Welcome to ENEE 634 |
09/04/2023 | Labor Day | Last swim of Summer 2023! |
11/21/2023 | ||
11/22/2023 | Thanksgiving break | Campus is closed and no classes. |
11/23/2023 | Thanksgiving Day | Safe travels and happy Turkey Day with your loved ones! |
11/24/2023 | Thanksgiving break | Campus is closed and no classes. |
12/07/2023 | Last class for ENEE 634 | |
12/11/2023 | Last day of classes for Fall 2023 |