DSP Lecture 22: Least squares and recursive least squares

ECSE-4530 Digital Signal Processing
Rich Radke, Rensselaer Polytechnic Institute
Lecture 22: Least squares and recursive least squares (11/20/14)
0:00:16 Least-squares problems
0:00:52 Review of the Wiener filter
0:02:45 Setting up the problem as a linear system Ax=b
0:07:38 The least-squares (minimum norm) solution
0:10:08 Note: taking vector derivatives
0:13:18 The pseudoinverse
0:13:57 Geometric intuition and the column space
0:17:23 The structure of the least-squares solution for the Wiener filter
0:23:20 The result: like a deterministic version of Wiener-Hopf
0:25:34 Recursive least squares
0:28:31 The Matrix Inversion Lemma
0:30:30 More general least-squares problem with a forgetting factor
0:34:07 The linear system at time n-1
0:36:45 The linear system at time n
0:38:00 How are the two problems related?
0:39:23 Applying the matrix inversion lemma
0:41:52 The gain vector
0:44:19 The right-hand side
0:45:15 Putting it all together
0:49:58 The final recursive least-squares equations
0:52:17 Extensions and discussion of RLS
Follows Section 13.3 of the textbook (Proakis and Manolakis, 4th ed.).