The best thing about your lectures is, u do coding implementation along with huge maths.. That makes u different from rest of the traditional instructors. Kudos to you!!!
SVD was at the very end of my college LinAlg class so I never got a very good understanding of it before the final - this is truly amazing; you say "thank you" at the end of every video but it should be us saying it to you- keep doing your thing! I'm loving it.
you explain math in such a way as to not make someone feel stupid, but feel like their taking steps into understanding a larger concept, and the tools they need are the ones we already have, big ups
Gosh, what a class! As mr. Ayush said, this was indeed by far the best SVD explanation I've seen. You've made a such complicated subject way more affordable! I wish you all the best, Steve! Greetings from Brazil!
This was, by far, the most compensable explanation of what the SVD is mathematically and visually. The SVD is an incredible algorithm! Amazing how so little you could keep in order to understand the original system.
Please keep making these high quality lectures. They are some of the best I have seen on TH-cam and that goes a long way because I watch a lot of lectures online.
Not an engineer/student, but I'm watching this to get a better understanding of PCA in statistics. I'm going to check the book and research this, but my only complaint (nit-picky) is trying to tell the difference when Steve speaks between "M" and "N" which I know refers to the number of rows or columns of the matrix. But really, this was great and I am thankful that this is something I can study on my own. Much appreciated.
Thank you for making the linear algebra less boring and really connected to data science and machine learning, this series is so much more interpretable than what my professor explains
I find everything with these courses, even the way board arranged is just great. Many many thanks for this wonderful explanation and all your effort to make it understandable and yet complete.
Extremely clear. Just want to point out a fact which at first did not seem obvious to me: the outer product has rank 1 because all the column are proportional to the first vector of the outer product, hence they are linearly dependent.
Thank you so much for posting the course material. I was running through asking ChatGPT some questions about eigenvector/eigenvalues and revisiting some linear algebra when I stumbled upon transitional probability matrices or Markov Matrices, PCA and SVD as was getting back to my Data Science studies. This is very exciting stuff and your presentation is very clear and understandable.
This all sounded like gibberish until I started to think of the first term of the expansion (Sigma1*U1*V1T) as the (strongest) "signal" and the rest of the terms as ever decreasing amounts of "signal" and ever increasing amounts of "noise". So the last term (Sigmam*Um*VmT) is essentially all background "noise" in the data. Thinking of it that way, it all makes perfect sense.
Gentleman, you have done a very impressive job. I have just started exploring data science and have recently completed my college course in Linear Algebra. This was quite interesting.
In the example, m is the number of images you have, n is their pixel values. So m doesn't have to be enough to represent your data. It is like saying, if you have 2 photo, 2 dimensions are enough to represent image features.
Thank you for this great explanation. I just lost you on one point, why is this matrix multiplication equals sig1U1V1T + sig2U2V2T + ... + sigmUmVmT Can someone explain how does it complete the entire matrix multiplication? I somehow lost in this columns of U and row of V
The assumption n >> m is contrary to what we have quite often in data sciences. In many problems, the number of samples (here m) is bigger than number of features (here n). In such a case, we just take the transpose and keep going the same way? Or there are some additional considerations (of course except of swapping interpretations of eigen vectors etc)?
This is so excellent. I just have one question you keep saying that you will multiply a column of u with a row of v. But matrix multiplication is row by column. So how are you doing it the other way around ? What am I missing ?
It is very clear when you compare the vector x with the water flow that, 'U' is the eigen flows and 'V' is the time the eigen flows evolves in the flow. In contrast to the face, also it ise very clear in the part of 'U' as the eigen faces but I wonder what will be the 'V' vector ?
Great video in linear space point of view. One naive question: can we make use of Lanczos algorithm such that we can pick up the most significant eigenvalues for approximation in order to circumvent the full diagonalization of the whole large matrix?
Hello, I can't find a way to compress all my images into one X matrix, the only examples I've seen are for doing SVD for one image at a time. How do I make one X matrix from images please?
others : "wow great explanation thanks for the lsson" me : how is this man writing perfectly backwards onto thin air ? mr bruton : I'm glad the green screen, glass board, and $1000 dollars of adobe products really paid off
i had some time to accept 1:22 conclusion, since if i understood you right, we have n vector space, in which our data can be, so it should be okay to use the whole n vectors of U as new basis, unless we want dimensionality reduction and not just matrix decomposition, or i'm just missing something?
I'm confused with the T-transpose notation. Dose the SVD of X give U*Sigma* V, for which you manually have to transpose V afterwards, or does the SVD of X give you U*Sigma*V-transpose, such that the SVD transposes V for you automatically upon completion of its calculation.
Such a lucid explanation! A question though : If I do an SVD on this toy matrix X = [1, 2; 1,2], then the second singular value is zero as expected. But if I am unable to reconstruct X using sigma1*U1*V1^T, i.e. just the first singular value (The matrix is reconstructed as [1, -2; 1 -2]). But if I add an infinitesimal noise to X though, it works. Is this a well documented numerical issue?
This channel is so underrated, your explanations and overal video presentation is really good!
Don't know why you think it's underrated...
Everyone who is watching this videos knows how great they are.
The best thing about your lectures is, u do coding implementation along with huge maths.. That makes u different from rest of the traditional instructors. Kudos to you!!!
It's my pleasure
SVD was at the very end of my college LinAlg class so I never got a very good understanding of it before the final - this is truly amazing; you say "thank you" at the end of every video but it should be us saying it to you- keep doing your thing! I'm loving it.
The best explanation of SVD. Your videos are excellent. Thank you very much!
You have a talent for taking complicated topics and breaking them down into digestible pieces. That's the sign of a good teacher. Thank you.
you explain math in such a way as to not make someone feel stupid, but feel like their taking steps into understanding a larger concept, and the tools they need are the ones we already have, big ups
Gosh, what a class! As mr. Ayush said, this was indeed by far the best SVD explanation I've seen. You've made a such complicated subject way more affordable! I wish you all the best, Steve! Greetings from Brazil!
Thanks so much! That is great to hear!!
This was, by far, the most compensable explanation of what the SVD is mathematically and visually. The SVD is an incredible algorithm! Amazing how so little you could keep in order to understand the original system.
Please keep making these high quality lectures. They are some of the best I have seen on TH-cam and that goes a long way because I watch a lot of lectures online.
It doesn't get better than this. I am so thankful to you. I don't know how to repay this help.... And yes, this is a highly underrated channel
What a wonderful way to simplify a complicated topic such as SVD--I wish more people in academia emulated your way of teaching, Mr. Brunton.
Not an engineer/student, but I'm watching this to get a better understanding of PCA in statistics. I'm going to check the book and research this, but my only complaint (nit-picky) is trying to tell the difference when Steve speaks between "M" and "N" which I know refers to the number of rows or columns of the matrix. But really, this was great and I am thankful that this is something I can study on my own. Much appreciated.
One of the best channels I have ever followed, appreciate it so much!
Thank you for making the linear algebra less boring and really connected to data science and machine learning, this series is so much more interpretable than what my professor explains
Hey I know it's been nine months but I just came across your comment and was curious. How'd the rest of your class go?
I find everything with these courses, even the way board arranged is just great. Many many thanks for this wonderful explanation and all your effort to make it understandable and yet complete.
This series is by far the best explanation of SVD that I have seen.
every time I think it's time to pause and comment this video with "awesome", it surprises me with more informative perspective. great job
The best explanation of SVD i have ever seen !
Wonderful explanation, clear and easy to understand. Thank you very much
Can you do a series on QR decomposition as well? This is so useful!
Lightbulbs are finally going off when it comes to SVD cant thank you enough!
Steve is magician of explanation.
Thanks so much!
It was pleasure to watch. You should do more educational videos, mr. Brunton.
Thank you for presenting us an amazing experience to learn about SVD!
Extremely clear. Just want to point out a fact which at first did not seem obvious to me: the outer product has rank 1 because all the column are proportional to the first vector of the outer product, hence they are linearly dependent.
Literally the best svd explained, so meaningful
Thank you very much for those videos , they are very explanatory . Keep up the good work, we need you lessons for our academic improvement.
These are the best videos out there. Biggest fan Steve, keep it up.
Thank you so much for posting the course material. I was running through asking ChatGPT some questions about eigenvector/eigenvalues and revisiting some linear algebra when I stumbled upon transitional probability matrices or Markov Matrices, PCA and SVD as was getting back to my Data Science studies. This is very exciting stuff and your presentation is very clear and understandable.
To the point. Answers all the important questions. I mean you should come to the party knowing some lin alg but great for intermediate level
best teacher that l have ever met
Thank you so much for this easy to understand explanation. I was really struggling with the topic and this helped a lot. Thanks again 😊
Glad it was helpful!
This is also the best explanation of SVG I have seen! Thanks for sharing!
I've read about and even used SVD. But I never really understood it in this way. Thank you!
I never *really* appreciated SVD until I saw this video. Mind blown!
This all sounded like gibberish until I started to think of the first term of the expansion (Sigma1*U1*V1T) as the (strongest) "signal" and the rest of the terms as ever decreasing amounts of "signal" and ever increasing amounts of "noise". So the last term (Sigmam*Um*VmT) is essentially all background "noise" in the data. Thinking of it that way, it all makes perfect sense.
the best lecture of SVD I have ever seen!
superinformative series of SVD
for mn ? (For example, what happen if my dataset is composed by 5000 images of 32x32?)
Very nice lecture and clearly understantable...Thanks Steve...🤗
Such a cool concept of decomposition and brilliantly explained here. Big thumbs up!!
Very, very nice explanation and presentation. Thank you!
@14:52
No Steve, thank YOU!
This is answering a lot of my questions on SVD.
Gentleman, you have done a very impressive job. I have just started exploring data science and have recently completed my college course in Linear Algebra. This was quite interesting.
Thank you Prof. Brunton, excellent explanation! Just come from MIT 18.06.
This is everything that I need. Thanks for nice explanation .
You are welcome!
Great video. Should be required viewing for anybody learning the SVD!
really appreciate your efforts. wish u all the best
Top rate education, I'm happily learning a lot.
Nicely done. Thank you
In the example, m is the number of images you have, n is their pixel values. So m doesn't have to be enough to represent your data.
It is like saying, if you have 2 photo, 2 dimensions are enough to represent image features.
Wow, what a wonderful presentation. Congratulations.
you are at the tip-top i like your explanation
Love his explanation style
Good explanation! Many thanks ! how could one manage to get these stuffs explained in such an elegant way.
The explanation is really wow !! Very intuitive ... thank you so much !!
Thank you for this great explanation. I just lost you on one point, why is this matrix multiplication equals sig1U1V1T + sig2U2V2T + ... + sigmUmVmT
Can someone explain how does it complete the entire matrix multiplication? I somehow lost in this columns of U and row of V
love the video, well explained and aesthetically good.
Beautifully explained.
That's so helpful! thank you!
The assumption n >> m is contrary to what we have quite often in data sciences. In many problems, the number of samples (here m) is bigger than number of features (here n). In such a case, we just take the transpose and keep going the same way? Or there are some additional considerations (of course except of swapping interpretations of eigen vectors etc)?
This is so excellent. I just have one question you keep saying that you will multiply a column of u with a row of v. But matrix multiplication is row by column. So how are you doing it the other way around ?
What am I missing ?
How to determine the criterion to truncate please? It might depend on the specific case but is there a general guideline for it?
incredicle explanation!
For the last point, (u~)*transpose(u~) is still a identity matrix, but it is a n by n matrix instead of a r by r matrix. Am I right?
Thank you so much professor.
I don't understand why we are multiplying the columns of U by the rows of V... shouldn't it be the opposite?
Excellent, excellent content. Thank you so much!
Just amazing explanation.
Thank you for this video series.
My man writting in mirror mode. Absolute legend
I was just thinking about that! But I think he just write in normal mode and mirror the video
Mindblowing!
Thank you very much. Excellent explanations.
Is truncating at 'r' similar to filtering out the highest frequencies in the FFT ?
It is very clear when you compare the vector x with the water flow that, 'U' is the eigen flows and 'V' is the time the eigen flows evolves in the flow. In contrast to the face, also it ise very clear in the part of 'U' as the eigen faces but I wonder what will be the 'V' vector ?
Great video in linear space point of view. One naive question: can we make use of Lanczos algorithm such that we can pick up the most significant eigenvalues for approximation in order to circumvent the full diagonalization of the whole large matrix?
Prof.Steve, Thanks for your explanations. But what is the difference between the POD and SVD for aerodynamics analyeses?
Why is the X matrix n by m but your SVD written out for X as m by n matrix?
Shouldn't u vectors be row vectors? How do you multiply column vector with another column vector in other matrix?
Hello, I can't find a way to compress all my images into one X matrix, the only examples I've seen are for doing SVD for one image at a time. How do I make one X matrix from images please?
Great lectures. Thanks so much. Kind of hard to distinguish the m's and n's sometimes. Hoped you'd use some other letters :D
Wow, you are really good teacher.
others : "wow great explanation thanks for the lsson"
me : how is this man writing perfectly backwards onto thin air ?
mr bruton : I'm glad the green screen, glass board, and $1000 dollars of adobe products really paid off
Hello, do we pass the U, S or VT as input into the SINDy algorithm or we pass in the approximation of X gotten using the U,S and VT?
What if u = 2x2 matrix, and V a 3x3 matrix. Then how would you calculate that outer product uV
thank u so much sir, very helpful
i had some time to accept 1:22 conclusion, since if i understood you right, we have n vector space, in which our data can be, so it should be okay to use the whole n vectors of U as new basis, unless we want dimensionality reduction and not just matrix decomposition, or i'm just missing something?
Is the economy SVD not also an approximation of X? (Since we lose some columns of U)
what if we change our norm. is this remain the best approximation or not?
I'm confused with the T-transpose notation. Dose the SVD of X give U*Sigma* V, for which you manually have to transpose V afterwards, or does the SVD of X give you U*Sigma*V-transpose, such that the SVD transposes V for you automatically upon completion of its calculation.
I am interested in knowing what software he is using to write.
incredible!
What happens when m >> n ? Does something change in all that has been said?
found this hard to follow without teaching any ways to think of this intuitively.
I wish i was his student during my college days ,my grades had been improved ..
Such a lucid explanation! A question though : If I do an SVD on this toy matrix X = [1, 2; 1,2], then the second singular value is zero as expected. But if I am unable to reconstruct X using sigma1*U1*V1^T, i.e. just the first singular value (The matrix is reconstructed as [1, -2; 1 -2]). But if I add an infinitesimal noise to X though, it works. Is this a well documented numerical issue?
if the U columns after m dont matter .. why is U unique ?
Wow superb!
@11:08 his face when he said Frobenius norm xD
why could he write in mirror image? Amazing!
I also feel amazed by that
@@NG-lx1kx Remember that image processing is one of the applications of SVD. Now think, how does image processing relate to this video?
Post processing he flips left-right. Notice which hand has the wedding ring.