CLASS SCHEDULE MTH 4/551 Fall 2023

The symbol [Lecture X] refers to the textbook, [Lecture X]. 

------------- Week 0

1. 9/27/23: Introduction. Why study Numerical Linear Algebra (in the era of Data Science). What we will cover in class. 

2. 9/29/23: Complexity of numerical computations: flops. Coding for linear algebra: operations saxpy, gaxpy,.... Start Linear Algebra notation/concept review/introduction. Use F23_MTH451551-LAreview.pdf.  

DUE:  Quiz 0: only first day questions. NO MATH!!!

-------------- Week 1

3. 10/2/23: Continue concepts from worksheet, with examples. 

4. 10/4/23: Finish worksheet. [Lectures 1-2].

DUE: QUIZ 1: available from noon till midnight. 

5. 10/6/23: Gram-Schmidt process to orthogonalize Col(A).  Vector and matrix norms. [Lecture 3].

-------------- Week 2

6. 10/9/23: More on matrix norms: induced, and not induced.  Property of being sub-multiplicative. Elementary operations on matrices represented by left or right multiplication by a diagonal or triangular matrix. Spectral decomposition of a non-defective matrix. The spectral radius and the 2-norm of a matrix. [Lecture 3 cd]

7. 10/11/23: Spectral decomposition and SVD: first examples. [Lectures 4-5]. 

DUE: QUIZ 2 available from noon till midnight. 

8. 10/13/23: Examples of SVD for square 2x2 matrices.

CLASS ONLINE: please watch the Lecture video October 13.  Here are class notes from this lecture.  

-------------- Week 3

9. 10/16/23: Examples of SVD for singular and non-square matrices. 

10. 10/18/23: Wrap-up the SVD: connection to norms; decomposition/approximation of a matrix as sum of low rank approximations; pseudo-inverse. Geometry of SVD. 

DUE:  Quiz 3 available from noon till midnight. 

11. 10/20/23: SVD geometry example. Least squares and applications. Solving LSQ via normal equations and via SVD.

-------------- Week 4

12. 10/23/23: Orthogonal projections.  [Lectures 6-11]. 

13. 10/25/23: Least squares via QR.  

DUE:  HW2. 

14. 10/27/23: Complexity of QR via GS, and complexity of bacwkward substitution. REVIEW for exam. 

EXTRA OFFICE HOURS 10/27, 1:00-4:00pm.

-------------- Week 5

OFFICE HOURS cancelled in week 5

15. 10/30/23: MIDTERM in class. 

16. 11/1/23: Watch Lecture on orthogonalization (Gram-Schmidt, and Householder)  notes are here  MTH451551_orthpgonalization.pdf

17. 11/3/23: Watch Lecture on floating point operations and sensitivity of matrix operations  notes are here MTH451551_floatingpoint.pdf

-------------- Week 6

18. 11/6/23: Symmetric positive definite matrices: definition and characterization. 

19. 11/8/23: For spd matrices, find the solution to Ax=b as the minimzer of a quadratic form. Connection to least squares and minimizaiton of norm(Ax-b)^2. 

HW3 due

(*). 11/10/23: VETERAN's Day

-------------- Week 7

20. 11/13/23: [LECTURES 12-19 lightly] Towards obtaining estimates on a solution to a problem via an algorithm: conditioning of a problem studied via condition number and relative condition number. 

21. 11/15/23: Conditioning of solving linear systems: estimates of relative error. Examples when these are "right" and when they estimate the error.  How to get LU factorization via Gauss transformations.  

22. 11/17/23: (In)stability of LU and (better) stabilty of PA=LU. LL^T (Cholesky) factorization for spd A. [Lectures 20-23].

HW4 due. 

-------------- Week 8

23. 11/20/23: Summary of stability (and b-stability) properties of direct methods for solving Ax=b. Why direct methods are now always possible. 

Introduction to iterative methods: stationary and nonstationary. For stationary methods, necessary and sufficient conditions for convergence depending on rho(G).  

Additional handout MTH451551_iterative.pdf

QUIZ 11/21/23 (on spd matrices, Cholesky, LU)

24. 11/22/23:  Gershgorin theorem which provides estimates of eigenvalues. Examples of stationary iterative methods: jacobi, Gauss-Seidel, SOR, Richardson). Sufficient conditions for convergence.  [LECTURES 32-33]. 

(*). 11/24/23: THANKSGIVING HOLIDAY

-------------- Week 9

25. 11/27/23: Nonstationary iterative methods.  

26. 11/29/23: Gradient descent algorithms: theory. Compare steepest descent and conjugate gradient. For an spd A, A-inner product, A-conjugate vectors and energy norm.  Krylov subspaces. [LECTURES 32=33; 38]. 

27. 12/1/23:  Examples with MATLAB of gradient descent algorithms including stochastic gradient. Convergence of CG. Preconditioning. Krylov subpsace methods for nonsymmetric systems.

[LECTURES 38; 40].

-------------- Week 10

28. 12/4/23: Finding eigenvalues and eigenvectors numerically. Arnoldi/Lanczos process takes us to Hessenberg form of a matrix.  Power iteration. [LECTURES 24-27]. 

29. 12/6/23: Rayleigh quotient. Linear order convergence of power iteration. Inverse power iteration. Inverse iteration with shifts plus Rayleigh quotient: cubic covnergence. Examples to work with. 

30. 12/8/23: Review.

DUE: HW5. (Extra credit due on paper in class). 

-------------- Week 11 (FINALS)

Office hours: Tuesday 12/12/23 2:00-4:00pm.  

FINAL EXAM: Thursday 12/14/23 at 18:00.