I first watched this series 3 years ago. But each year, I rewatch it as I learn more and more maths. Each time, I realize that when I previously watched it, I missed something important and beautiful. It's amazing how much information there is in such a work, that can be easely be missed by the untrained eye, who can't appreciate the hidden connections of math.
yup : there's quite a lot to think about in this particular episode. Had to stop a few times. Prob rewatch part of it tomorrow morning. Can't believe we weren't taught any of this in Math1...the formulas, yes, but zero intuition like this. No wonder most of us find Math really hard.
This highlights another thing that makes the video series good: The ideas are presented in a way where they can be accessed easily by people with different levels of experience and familiarity with the topic.
I have the exact same feeling. I keep coming back to these videos every once in a while for the past 2-3 years. And I get a new perspective, new understanding every single time. Just amazingly beautiful. Math is beautiful !! Thanks for making such a nice series.
This series is basically perfect, congratulations. It's strange that this subject is usually taught so badly. I attended a world class university and at no point did they properly discuss the intuition and motivation for the formal concepts.
The problem is, that visual understanding is seen as something, people should figure out themselves, mainly because it is faster (and easier for the lecturer) to show to rigorous proofs, if you _only_ show the proof, and don't show the intuition behind. However, I guess, one could try and change some of the exercises to help with visual understanding. One could e.g. every once in a while instead of asking for the fourty-third inverse matrix ask the students to draw the effect of a matrix-vector multiplication into a coordinate system. Or, one could leave out coordinates (to indicate their arbitrariness) and only give them vectors and their images drawn into space/the plane and ask for properties of the (or all) linear operation(s) which make the given vectors land on the indicated ones. (So, you could draw two basis vectors of the plane, let there image be on one line and without knowing any coordinates, students now know, that (e.g.) the determinant will be zero)
Well, the problem is that all of this stuff is impossible to show on a blackboard. If a program existed which was both powerful enough to illustrate these concepts and easy enough to use on the fly, mathematics understanding through using it would take off like a rocket. That's why you keep seeing people what program 3B1B is using to make them. They're hoping for something they can play and experiment with.
carrionpigeons Don't entirely agree. For one thing, I think all of the visual concepts in this video could be comfortably presented on a blackboard. Secondly, lots of it isn't even visual, it's just explanations. For example, the video where matrix*vector multiplication was explained. Many courses just give you the definition in subscript notation and never motivate the concept. This course gave the "proper" explanation in terms of the intermediate step of linear combinations of matrix vectors, which makes everything a hundred times clearer. I have seen one course (MIT Linear Algebra) spend a long while explicating that idea, but most courses do not.
see also lem.ma for a great set of videos that (also) communicate these ideas well - especially the interplay of geometric and algebraic aspects - but using a traditional blackboard approach.
Its a shame that mathematics has become such an infamous subject for the average person, otherwise this channel would be far more popular, which it really deserves.
Mathematicians are the secret society of people who know how the world works. We are the 1%. Consider this video series as an invitation to the silent ruling class.
I have a phD in physics. I was really good at geometry during my study years... and I have NEVER seen this. And not one of my (very good) professors talked about this in this way. I am REALLY shocked, in a good way. You are the best Math Theacher on line that there is. We need one of you for every language on Earth!.
I say the same thing about the determinant analogy to area shifting! Through my bachelor and masters in physics! No one ever made that connection for me!
"Sometimes you realize that it's easier to understand it (a vector) not as an arrow in space, but as the physical embodiment of a linear transformation - it's as if a vector is a conceptual shorthand for a linear transformation." Definitely one of the most beautiful ideas I have ever learned, thank you for articulating it so well.
ABo NaBiL Gamer ! It took me three watches with a couple of weeks in between to understand this fully. And even then it's not quite simple when you try to apply it on your own
Sincerely, this series of videos has taught me more than the whole Linear Algebra subject I took many years ago when I was studying Computer Science. It addresses precisely the most important problem I found back them and that is the lack of geometrical interpretation. Now I'm a Postdoc in Computer Graphics and still find myself finally understanding some details here and there. Thank you very much. I would gladly pay tons for something like this applied to more advanced concepts.
The much simpler real life application, I think, for finding a "dot product": Imagine you and your friend are pulling a big rock, tied by 2 ropes. Your friend's rope is making an angle ø, with your rope, like V shape. Now the *dot product* gives the actual amount of force your friend is adding to yours, given there is ø angle between you and your friend's direction of rope pulling. If your friend stands in same line as you, pulls exactly in the same direction where you want to move the stone towards, (or both are pulling the same rope), angle ø=0, i.e Cos(0)=1, then thats the most efficient way of pulling the stone. Bcz all his force magnitude will absolutely add up to yours. Otherwise, atleast, the lesser the angle ø between the forces, the higher the total magnitude will be. But if your friends rope (force) direction is making some angle, ø with yours, then though he's putting some X amount of absolute Force, only a fraction of X (projection of your friend's on yours) will add up to yours. If your friend makes an angle 90 degree with yours, i.e. he is pulling in perpendicular, then he's not adding up anything to your effort, cos(90)=0. But if your friend is pulling the rope in opposite direction, cos(180)=-1==>Means your friend is working against you. Then the rock moves in the direction where the force is higher, but the amount of displacement would be much lesser, yourForce-friendForce. This is also used in the Newton's second law of motion= F = m*a cos@. So basically the Dot Product tells us, how much a vector is working FOR/AGAINST the other. It's called cosine similarities! This is also used in comparing 2 things (vectors) like how similar are they, do they add up to the entirety or negating each other.. Man.. How I wish I had a board to write down, draw and explain..
This is a nice way to understand what projection does but it doesn't explain why the dot product is used, why it is used the way it is and why it works and makes sense at all.
@@group555_ the dot product is basically asking this question, you and your friend pull a thing really heavy and you ask yourself: Does this guy help me at all or just faking, the value reflect the "help", and why projection , because the use of cos(x), cos(x) basically mean x direction loosely speaking, you can check more on Unit circle . Projection is basically mean take cos(x) of the other one and why cos(x) is used because it reflect the 'true' value of the other one ASSUME they are one the same vector, in this case x.
This series of videos is going to last. I think presents an almost purely geometrical explanation of linear algebra that can't be found anywhere else at the moment. I believe this could be done with every math subject. As an undergraduate student of mathematics, I often find myself struggling to translate what the books say into a more intuitive mental image of what's going on. Of course, some subjects (e.g. linear algebra) are easier to visualize than others. But then again, mathematicians always try to build mental images of what they're working on, even if it is a very abstract subject. These mental images are crucial, and in my opinion they should be taught along with the (undeniably necessary) formal arguments. These videos show that animations are a very powerful tool to do this. All of this could be said about your Multivariable Calculus Course as well. Now, I realize it is a lot of work, but it would be great if you did the same thing with some other undergraduate subjects. Have you already considered this? If so, what subjects?
Thank Alvaro! I'm definitely planning to do more of these. Long term, it would be cool to tackle all undergrad topics one by one, but the ones closer to the top of the list right now are calculus, real analysis, and probability.
Calculus and real analysis? What? I was under the impression that what we call analysis in our courses is called calculus in English. What is real analysis, then?
The word analysis, or analyse in english is just the process of understanding something, through the application of "pure" (i.e. undeniable) logic. Real analysis, in mathematics, refers to trying to define properties of the "real" numbers (i.e. numbers with potentially unending digits) and functions of those numbers. Calculus is the numerical/analytic description of change. Although often this results in functions of the real numbers, there are other uses for calculus outside of the real numbers. I imagine it must be quite difficult to learn about mathematics in another language, as the english part of mathematics, is often ill formed to say the least...
+Alex D. Bryan you seem to misunderstand my situation, or is it me misunderstanding your comment? I study maths in German (mostly) which is my mother tongue and the occasional course is in English, but that's not a problem for me. It seems, though, that our university lumps analysis and calculus (and topography) into one 2-year course named Analysis 1 through 4. So I didnt know how to distinguish between the two. The language is not the problem.
It took me a while to understand the point he was making(with regards to how projection occurs), but it makes sense in the context of previous chapters: _(3-D Example below, but it generalizes pretty intuitively imo)_ 1) *Linear Transformations and Matrix Multiplication→* Multiplying two matrices can be conceptualized as applying one linear transformation after another. I will make sense of the dot product in this context. 2) *Column Space→* The non-square vector on the right side of the dot product has column space one(rank = 1); thus, applying it as a linear transformation will shrink your original vector down into 1-dimensional space. → This is what Grant was getting at using û as the basis vector for 1-dimensional space. 3) *Basis→* The non-square matrix on the left can be conceptualized as the linearly-transformed î-intermediate, ĵ-intermediate, and k̂-intermediate coordinates(where the basis vectors land during the second linear transformation). The non-square matrix on the right can be conceptualized as just 1 basis vector(û) with 3 coordinates of landing. 4) *Dot Product→* Conceptualized as the projection of a 3-d vector onto 1-d space(the right matrix) where the product of transformed basis vectors(left matrix * right matrix) determines the extent to which each dimension contributes. *What this means→ * 1) The dot product represents the extent to which 2-vectors combine in each dimension, represented by a singular value. 2) Column vectors can be conceptualized as a linear transformation from n-dimensional space onto 1-dimensional space.
For anyone who's lost like I was hours ago, I think I'm beginning to grasp it. See the number line as a 1D vector space with basis vector u-hat, where u-hat has coordinates ux and uy. The basis vectors i-hat and j-hat define the 2D coordinate system. The coordinates ux and uy are projections of u-hat onto the x-axis and y-axis respectively. Say you want to transform this 2D space into the 1D number line, such that L(i-hat) and L(j-hat) are projections of i-hat and j-hat onto the number line. You'll need a 1x2 matrix that will look like [L(i-hat) L(j-hat)]. As I wrote earlier, ux and uy are projections of u-hat onto the x-axis (i-hat) and y-axis (j-hat). Because of symmetry, the projections of i-hat and j-hat onto u-hat (the number line) will also be ux and uy. So, L(i-hat) = ux, L(j-hat) = uy! The matrix will be [ux uy]. Note that ANY vector, v, in the original 2D space is a linear combination of the basis vectors i-hat and j-hat. After the transformation, this still holds true: v = c1 i-hat + c2 j-hat and L(v) = c1 L(i-hat) + c2 L(j-hat). So, because L(i-hat) and L(j-hat) are projections of i-hat and j-hat, L(v) is a projection of v onto the number line! Now, the vector equivalent of the transformation matrix [ux uy] is just u-hat. If you take the dot product between a vector v and u-hat, you do exactly the same as you do with the transformation; you project v onto u-hat! So, to generalize: Taking the dot product of vectors v and w is equivalent to transforming the vector v by the matrix [wx wy]. But also equivalent to transforming the vector w by [vx vy].
I think I am starting to understand it now. It really helps me to see it this way. I finally got more than halfway through your series and my brain is finally starting to put pieces together. I will keep watching and re-watching until it sticks. I'm gonna try to solve some problems now to see if I can solve them and also understand them better than I have before. Thank you for these!
I went through it for more than 1 hour pausing at each point made. I have never attended college and learned through textbooks recommended by first class colleges and the Khan Academy teachings (in particular) to get my engineering credentials for a living. I jammed into my head all these math proofs like an Ape and try to relate them to the real world of tangible things. Thank God that I found You to light up the bulbs and cast away the shadows (vague conjectures of realities). Example on dot product. You sum it beautifully as a linear transformation from 2D space not defined as numerical vectors but projecting space onto a diagonal copy of a number line. I take this and others as wings to fly now. Yes. I signed onto Patreon as a small token of appreciation.
In all honesty grant, you have made me realize that a solid foundation in math can allow you to explore many concepts by yourself, the fact you frame things so beautifully that it makes everything click in place, it is admirable, you are a great teacher, fueling the next generation of math lovers, and inspiring many, specifically your way of teaching, it doesn't give you the answer, it gives you a way to understand it and then using that understanding figure out something, your style of teaching makes math very fun, and i will always appreciate it.
To those who don't quite get the bit about the duality and how dot product could occur from that, after hours of thinking, I think I might have figured out a more direct explanation that could help you. 7:48 || 3b1b takes a copy of the number line and pastes it on the two-dimensional matrix grid, such that it is slanted and the "0" is at the origin. It is important to note that U-hat (the unit vector of the new number line) has the same length as I-hat and J-hat. We will see why in a moment. 7:50 || 3b1b explains why the new number line is a legal move, and that a function that converts 2-D vectors into a number on the new number line exists. 8:53 || 3b1b tells us that it is important to know where I-hat and J-hat land in the new number line. He explained why earlier, from 5:21 to 6:18. To add on to what 3b1b said, knowing where I-hat and J-hat lands in the new number line allows us to define ALL vectors that exist in the 2-D space just as they are in the form a * I-hat + b * J-hat. Suppose L(V) is the function to transform a given vector V onto the new number line, where V is a 2-D vector of the form a * I-hat + J-hat. Let L(I-hat) and L(J-hat) be where the I-hat and J-hat lands on the new number line respectively. Thus, L(V) = a * L(I-hat) + b * L(J-hat). a and b are given from vector V. If we can find what L(I-hat) and L(J-hat) are, then we know L(V) is possible. 8:59 || 3b1b explains to us that an arbitrary U-hat (with length equal to the lengths of I-hat and J-hat) with an x-component of Ux and y-component of Uy, has L(I-hat) and L(J-hat) as its x-component and y-component respectively. This proof can only work if U-hat has the same length as the lengths of I-hat and J-hat. Suppose otherwise, then the line of symmetry would no longer exist and the proof no longer exists. 10:07 || 3b1b shows us where the dot product comes in. At this point, 3b1b has shown us the dot product between U-hat and another vector, say, V, is really just V projected onto the number line that U-hat sits on. We are meant to prove for two vectors, say V and W, and not one of them with an arbitrary unit vector. At 10:33, 3b1b tackles this by considering projections onto non-unit vectors, i.e, projecting V on W or vice versa. He said that we should scale up the U-hat by a factor. In his example, he used 3. This took me a while to understand, but I think I finally got it. I don't think he explicitly stated this, but the new number line with U-hat, should lie on one of the vector you are trying to project on. Let's say that we want to project vector W on vector V, then the new number line should be on vector V. We need to use a unit vector, in this case U-hat, for this to work. This was explained in 8:59, I have written about it above as well. Now, we have a projection of vector W on unit vector U-hat. However, we want the projection of vector W on vector V, and not its unit vector. This is where the scaling is necessary. Since U-hat is a unit vector of the number line that vector V sits on, that would mean that vector V is a multiple of U-hat. We know this is true because they are perfectly coincident, i.e, they share the same gradient and points. Now that we know that vector V is a multiple of U-hat, how much do we need to scale U-hat by to get vector V? The answer is just the length of vector V divided by the length of U-hat. It is like when a * b = c, and you know a and c, and b is the scaling factor, then you can divide a from both sides to get b = c / a. Recall that the length of U-hat is the same as the length of I-hat and J-hat. I-hat and J-hat both have a length of one, therefore, U-hat also has a length of one. Thus, we need to scale U-hat by the length of vector V divided by one, i.e length of vector V. For the dot product, I am going to be using the asterisk (*). So A * B is the dot product between A and B. Currently, we have (U-hat) * (vector W), or [Ux Uy] [A B] (A and B are meant to be on top of each other, A and B are arbitrary letters that I have picked to represent the x and y components of vector W respectively). Scaling up U-hat by length of vector V (denoted by |V|), |V| x (U-hat) * vector W. This becomes |V|(Ux x A + Uy x B) = |V| x Ux x A + |V| x Uy x B. Let C and D be the x and y components of vector V respectively. C = |V| x Ux and D = |V| x Uy. Finally, we have AC + BD, which is the outcome of vector V * vector W. The new number line can be placed on vector V instead, and by the same arguments, we should have the same results. To summarise, (1) get U-hat and place it on one of the vectors, say V. (2) Project other vector onto U-hat, so that it is projected onto the new number line. (3) Scale U-hat up to get the original vector V. If it helps anyone, I could have this down on paper and link it here.
@@themangobui1474 because in the normal orthogonal coordinate system it's a 2 dimensional vector. We are relating the two dimensional space to the one dimensional number line, quich in the two dimensional space is pointing in direction u. To me it seems that the vector u in 2d space has the coordinates (0.5,sqrt(0.75)). Try to draw it and youll see!
Man, even with your fancy well-animated and explained examples, I was having a very hard time to understand this concept and took some days when I finally got it. No, it's not your fault. I think it's simply because english is not my main language so my learning curve rely more on visual examples than anything (my english is not poor, but it's just not fluent enough to understand everything at the same pace of natural english speakers). However, I took this difficulty as good thing, as I REALLY learned all the subject of your linear algebra series by wathing every video multiple times. I just want to thank you for your effort to bring us this different method of teaching math which is actually much more stimulant than the old fashion way most of us are used to (copy and paste formulas). And your passion about how cool is this subject gives me even more inspiration to learn. Thanks from a brazilian guy.
TheAwakeningMission same condition here bro ...I was just thinking that I'm too dum to under stand this concept but as u said I also have this problem...His linear transformation video I have to watched 4 times to fully understand. sometimes I feel to give up bcz I don't understand after so many times ...But truly I want to learn this concept .....
And that's okay, keep going :) Sometimes the brain needs a day or 2 to wrap itself around some idea. If you don't get it in the beginning, that doesn't make you stupid. Stopping to try to understand is what makes you stupid.
have the same problem man. can you direct me on what to do when i have this problem. i mean what have you done to make it easier for yourself? have you just been watcing theese videos multiple times, or did you write something on paper?
You've done a truly excellent job with these videos. The geometric insight into linear algebra is wonderful, and I say that as someone who used linear algebra almost daily during my career at NASA.
It's also his channel symbol, a circle that's 3/4 blue and 1/4 brown (3blue1brown) which is all in turn referencing the fact that his eyes are 3/4ths blue and 1/4th brown. kinda neat lol
I've been a graphics programmer for over 8 years, I've also written countless collision systems, animation engines, physics engines, and your videos are still blowing my mind. How is this not the standard way of teaching linear algebra in school/uni?
If anyone's having trouble understanding, this is how I've come to think of it after watching the video a few times: 1. To turn 1 unit (u) of the number line into a vector, you'll have to project it from the number line onto the xy-plane. That projection will be marked by u's coordinates (ux, uy), as shown at 9:30 2. To go back to the number line from the xy-plane, move ux units along the x-axis, and then move uy units parallel to the y-axis, and you'll get back to u on the number line. 3. To turn any vector on the xy-plane into a number on the number line, you'll also have to mark its location in the xy-plane by taking note of its coordinates, (x,y). Then if you scale those coordinates by u's coordinates, you'll be able to get to the number line by walking x*ux units along the x-axis, then y*uy units parallel to the y-axis. And x*ux + y*uy is basically just the dot product of [x, y] and u.
This is just . . . Beautiful! I just feel so lucky to have discovered your channel, because, no teacher, no professor, no matter how we advance in our fields that are related to maths, explains like this and shows the deep essence of the science (except if your main specialization is mathematics).
I've had rough 4 years of undergraduate studies and got two more semesters extended. In my last semester before graduation, watching your videos on vectors gives me strength, 'cause I'm taking Vector Calculus. I would get nothing out of my course, if not for your beautiful explanation and eye-opening visuals. Now I am sure that if I can understand these videos, then practically anyone can enjoy, let alone learn, mathematics. Thank you so much! P.S. I keep rewatching your course playlists to root the concepts and entertain, as well.
I once had a friend who took his own life. He was very religious, myself scientific. The last background he had on his Facebook was a Calvin and Hobbes comic. I think about that man a lot and miss our talks. Seeing this quote really made me consider some of those conversations and their impact on me, and I can't help but think his life helps drive me, even in his death, to see through the lens of maths what my existence, and his, might mean. Thank you for your videos, Grant.
If this feels a bit overwhelming, ask drunken Grant to explain this to you. You reach him by setting the video speed at 0.5x. He's just as smart as regular Grant but has a more laid back style.
I've watched it 5 times and I'm still not able to understand what is the use of taking a dot product. Please treat me like a 5-year child and give me some real world example. Thanks, brother
I opened 12 tabs with different websites explaining dot product. 1 minute into this video, closed the other 11 tabs. A big fat thank you for helping us realize the 'why' part of it!
This feels like achieving enlightenment. After all this time you mean to tell me that this is not just: "Funny numbers go brrr and it's useful for some reason" but that it actually makes sense?! This is genuinely reviving my love for mathematics and science again after being buried under test scores and grades.
Thank you. Thank you so much! Dot products finally make sense on what they physically represent. However, I will say that this video is presented in a confusing order: it mentions that dot products relate to projections before proving why much later. I had to watch the second half of the video first in order to understand the first part's discussion of why order does not matter.
This one took some time to wrap my mind around but it is truly amazing! Thank you for this series, it's a great help to develop an intuition about linear algebra and all its applications.
In computer graphics, I think there is a _far_ more intuitive way of understanding the dot product - it is a measure of similarity between two unit length vectors, 1 when identical, 0 when orthogonal, and -1 when perfectly opposite. (This can be intuited by the fact that a . b = |a||b|cos(theta)). This comes up _constantly_ when writing lighting shaders, since the amount of diffuse light that bounces off a surface is generally L . N, where L is the direction from the surface to the light source and N is the normal of the surface. (This is fantastic because it has a direct visual interpretation.) Also, interestingly enough, I believe Google employs a concept of "cosine similarity" on vectors of words when ranking websites for search.
Yep. I think of dot products as a measure of how much two vecors are going in the same direction, with the cosine of the angle between them acting sort of like a percentage that can also be negative.
The dot product can also rank the similarity of vectors in any dimension. For example: in natural language processing, you can create a vector full of numerical context values for each of the words in a sentence, or even a paragraph. Then you're taking the dot product of two vectors in dimensions upwards of 100, and using interpreting the result in the same way. The intuition comes very nicely with geometric interpretation, but spans much further.
Woow just wow I have seen this video so many times already and it just keeps blowing my mind This is so beautiful I hope that second series is coming soon for all the Jordan, IP and spectral stuff Thank you very much!
Bravo sir. This was absolutely brilliant! Having a degree in math from one of the most trending universities, but seeing the first time in years this actually not-so-simple concept the way it was intended to be... Here is my deepest bow.
Man, this is the best explanation I have seen for Linear Algebra, excellent series, I can't wait for the next video. It felt great to find the series just at the moment I'm starting to learn Linear Algebra at the University, it has helped me to understand the true meaning of what I have been learning. Keep it up!
The definition you gave of dot product is only valid under the right hypotheses, which are not always validated in linear algebra since you could take any base of vectors for the set you're representing. Instead, the more general geometric definition gives a clearer view of the geometric properties of the dot product, and makes the formula you give trivial to find under the right circumstances. Great series, thanks a lot for your work.
Hey this series has helped me understand linear algebra to a degree that I didn’t think I ever could. That being said, this video is the point where I got confused. I see a lot of people in this comment section with the same visualization problem I had and I thought this might help you because it helped me. The dot product alone isn’t the projection of one vector onto another in itself. The formula for projection of u onto v is ( (u•v)/|v^2| )*v where |v^2| is the length of v squared. So this means that after you take the dot product you still need to divide the result by the magnitude of the initial vector and then set it in the direction of the initial vector (hence the *v at the end of the formula and the second division of the magnitude of v resulting in v^2 being on the bottom) Hope this helps
For those who don't know there is a different notation for the dot product. Imagine that you have a vector x and y (of same size), the dot product of those vectors can be noted as x • y but also as x^Ty. The T means transpose which means that you turn the vector *x* on it's side to get the associated matrix as shown on the video.
These videos are amazing! My only criticism is that you started using "projection" and "unit vector" without fulling explaining what they were. As someone with no background in linear algebra, I had to look these up.
Similar here: I thought I was following this series quite well, but then he threw in projection without really explaining it, and I was lost for a bit. I did Maths A-Level but don't recall learning anything about projection. Better go look it up!
Yeah same here, although I think unit vector was explained earlier referring to i-hat and j-hat, but with u-hat I'm not sure what is means. I'm thinking it's a vector with a magnitude of 1.
Your channel should trend monthly. I want to believe there is a reason why most people find math repulsive. A reason somehow more meaningful than the way it is taught in school. I think that knowing these types of things gives you a satisfying and powerful understanding of almost everything around you. Maybe through some divine phenomenon, that power is kept in the hands of few people, because if everyone found this stuff easy, well, they might not use it for good. But then again what am i saying think of how advanced we would be as a civilization there would be no room for evil. I dont kno
While watching this I suddenly understood why transforming a vector into a lower dimension and then back into a higher dimension always causes some loss of information. Genuinely some of the most intuitive explanations of linear algebra I’ve ever heard
Suddenly, the way the dot product was first introduced in my maths textbook - the length of the first vector times the length of the second vector times the cosine of the angle between them - makes perfect sense!
Mariusz Wiesiolek as we know, cosine is the ratio of adjacent side to hypotenuse. You can think of w as the hypotenuse and of its projection as of this adjacent side. Since we’re projecting w onto v, then the angle between v and w is the same as between w and its projection, therefore we can use the cosine. Hope you understand my attempt at explaining this since English isn’t my native language.
Good explanation Mark, I too was confused about that point. My understanding is that if the vectors are forces then the dot product is the resultant force (a*b*cos(theta)). The resultant force ends up somewhere between the two vectors (e.g. 2 horses pulling a barge), it doesn't end up going in the same direction as the other force - as it seems from the video.
@@helenkirby2539 I'm not sure if your comment refers to your prior understanding or your new understanding. If it's the latter, it is not correct: If the vectors represent forces, the resultant force is the sum of the vectors.
Currently learning about this in much less detail in my trig class, but I really appreciate the number line translation you're explaining here. I typically need to hear/see a concept a few different ways to really get it hard wired into the brain and this number line translation helped a lot, thank you!
If you're struggling, do not be ashamed, I've scrolled the comments, many many people struggled with this video and so did I. It'll click just keep watching and asking questions.
2:37 My intuition for "Why the order doesn't matter" was to consider the area of the triangle bounded by the 2 vectors. What all of this means is that you get the area of that triangle by 2 ways.
The area of the triangle bounded by the 2 vectors is half the absolute value of the determinant of the matrix of the two vectors (in 2-d) or magnitude of cross product (in 3-d).
I wish I watched your videos back when I was taking my linear algebra course. Life would have been really simple! You are an Ubermensch by all means, Cheers!
The duality of convolution has always amazed me. The method of fractional resampling for signal timing correction is the best example. In the end rather than adjusting the time index of the sample you adjust the filter to its location at the time that index of signal would have occured because of the duality... i used to explain it as like the farnsworth engine in futurama... it doesnt move the ship forward, it pulls space backward.
I'm no mathematician and I'm just a dilettante when it comes to the subject matter. But boy is it interesting. I always had respect for math and my math professors. Got me all the way to calc III. Love this series of videos. Also completely approve of the Calvin and Hobbes dialogue at the beginning!
But maths is completely unlike religion. When you have two apples and get two more, you have four apples. When you have two dollars and get two more, you have four dollars. 2+2=4 is just an abstraction over this observation. You don't need to believe anything. You don't even need to believe an axiom in order to work with it.
because all religions are burdened with superstitions, religion has almost become a synonymous of "believing blindly in something". Actually religion means "binding together" "bridging yourself with the universe"/"realizing the unity of everything" (not believing in it, but Realizing it, experiencing it) Pseudo-religions like Christianity have been preaching belief and have been exploiting people, and that is so stupid and ugly. Although blind belief and other such things have nothing to do with religion, they have been happening IN The NAME of religion, that's why the association (in our minds) is so strong. You can observe that 2 apples + 2 apples = 4 apples , but essentially this is very mysterious. Everything in this world is quite mysterious and will remain mysterious no matter how much we sugar-coat everything something with explanations. Explanations are utilitarian. They are very useful since they can provide frameworks on which we can create technology, but explanations should not have the pretension that they represent ___the supreme absolute "truth" that must never be doubted___ or that they demystified existence. These pretensions are dangerous, because then people will behave exactly like the religious fanatics do, but now IN THE NAME of science. PS: Mathematical proof that you can't demystify existence: If you want to explain what A is, you'll have to use another term B. But then you're caught in an infinite loop, then you'll have to demystify B as well. You'll have to describe what B is: "B is C and D in such and such way", and then you'll have to explain C and D and so on. All explanations of an object/phenomena X are expressed in terms of other object/phenomena Y1,Y2,...YN,... . If you managed to explain away an infinite number of phenomena, you'll have created an infinite more phenomena which are waiting to be explained. PPS: I'd like to hear some criticisms of the above proof.
i found 9:30 super hard to understand visually with what is being said. What i see happening is that to project i-hat onto u you make a 90 degree line from u onto the head of i-hat. To project u onto i-hat you make a 90 degree line from i-hat that crosses the head of u. Super great videos and representations. Its a very nice way of thinking about the cases
Ok, I love the series so far. Chapter 7, however, took me by surprise. I spent three days and scribbled out 17 pages of notes trying to assimilate Chapter 7. I still catch myself daydreaming, trying to visualize the contortions those poor unit vectors must go through to exit 2-D space and find their new home on the number line. I admit I'm a little fearful of tackling cross products in Chapter 8.
You Sir/Ma'am seems to be a wonderful writer. And your comment is absolutely relatable. Until now I thought I was the only one spending entire couple of days on a single video, taking notes, uprooting hairs and slowly slowly visualizing every aspect of the explanations. Seems I am not alone. I hope you have completed the series by now and have absorbed all its beauty. Will take me couple more weeks to do that.
you nailed this one! i also learned this concept almost at the end of my lineair algebra course, it just makes way more sense that way, and you can really dive into the concepts of inner product spaces
This video is a gem, this is what is needed in education, I understood each and every part and really say this is never taught it is also hard to teach these topics without actual visualization, there were parts where the graph was not meant to be scaled, I also like the simplicity and focus not giving irrelevant examples, when I will grow I will surely donate to this channel so that it can keep up with these videos for the future generation, I am still 16 and fully support this guy, Hats off huge respect from India sir🎉🎉🎉
Something eluded me the first time a saw this. However, after learning about "change of basis" in a rigorous manner, I think the explanations that used "symmetry" and "projections" finally clicked. The diagonal blue line that is spanned by u-hat is a vector (sub)space U; the projections are linear transformations. Since the length of u-hat, i-hat, and j-hat are all the same, then the length of the projection of u-hat to i-hat is equal to i-hat's projection to u-hat, i.e. u_x. Note that the projection of i-hat to u-hat is a projection of i-hat to U. Thus, in a "change of basis", i-hat has a "coordinate" of u_x in U with respect to U's basis which contains one vector, that is {u-hat}. Similarly, the coordinate of j-hat in U with respect to {u-hat} is u_y.
I'm totally in love with this series, but this is the first video I felt a little lost on - I hadn't encountered the notion of 'projection' before, and while I can Google it and get a formula, that hasn't helped me with the whole 'get a visual intuition' thing, that this series is all about. If you ever revisit this, I'd love if you could add a footnote video about that.
this might be a little late, but let me try to help you out. Think of projection as the way you see your shadow under sunlight. Your shadow is just a projection of your body onto the ground. In this case, the sunlight that helps projecting your body onto the ground is always perpendicular to ground, so if you stand up straight (meaning your body is also perpendicular to the ground), you won't see your shadow on the ground. If you lean over a little, you will start seeing your projection (or shadow). Hope this helps
This is extremely beautiful. I am taking the course Mathematics for Machine Learning: Linear Algebra on Coursera and it is really cool. At some point, it explains the symmetry or duality of matrix-vectors and dot product and the professor describes as something beautiful, a duality of geometry and calculations. The thing is that the explanation was short and visually hard to understand. So I find this beautiful video explains it perfectly with the spot-on visual for understanding it. I think this is the real value of this well explain and visually perfect videos. Clarify the abstract concepts through visual and intuitive examples Books, lectures, online courses sometimes fails on that. So this is were these types of videos shine for actual meaningful academic formation. Books and courses are getting aware of it, I have a book on Deep Learning for Computer Vision that literally references 3Blue1Brown videos for concept clarification. Thanks for the amazing work!
For anyone who is still struggling to grasp, what finally made this concept click for me was to think of the "dual" of the vector as the UNIQUE LINE in space that now represents your 1D number line. A number line can live anywhere in a 2D space (a 1,0 number line would be the straight line along the X-axis, a 0,1 number line would be the straight line along the Y-axis, or the U-hat number line that was diagonal through the origin looked like coordinates of ~(0.5,0.5)) So, 2D space in a sense remains absolute. When we project onto a 1D number line, we're essentially saying "this is your PARTICULAR 1D "playing field" (number line) right now and it lives in a unique place within 2D space" but there are an truly infinite number of 1D number lines capable of being generated from the 2D space. The three examples I gave are just a subset of all the possibilities. Thus, because each number line is unique (the 1,0 number line doesn't LOOK like the 0.5,0.5 number line in 2D space), each projection onto 1D space is unique, as represented by the unique 2D vector that codifies where the number line "originally lived" in 2D space.
i feel we are saying the same thing it but I've worded it different. Each vector can be seen as a line/arrow in 2D space, going on its merry way. A vector is also a transformation that can squish 2D space to 1D(by taking any other vector and dot product with it). Interesting to note that once we see the vector as a transformation, it no longer makes sense to think of it as a vector, since 2D space is now a line.
Quick caviat: not all 2d vectors transform into unique number lines. In fact, there are an infinite number of 2d vectors that could be transformed into EACH number line. For example, [1, 1] would be the same as [2,2]. For any two 2d vectors W-> and V-> where W-> = [Wx,Wy] and V->=[Vx,Vy], if Wy/Wx=Vy/Vx, they will transform into the same number line. This becomes intuitive if you consider that if you have a point (x,y) on any line that crosses the origin, you can calculate the slope of that line as y/x. So basically, if the "slope" of the two vectors is the same, they will transform into the same number line.
I LOVE LOVE LOVE your "alternative pacings" to things because it helps set such a CEMENT understanding of things to know "mathematically yeah sure, but *why*?" You do such a good job helping conceptualize such nebulous topics.
I think I have watched the part between 6:30 and 10:00 at least 8 times, but it finally clicked and made geometric sense. Thank you for the cool new way to look at linear algebra. I love how it gives us a more intuitive way to think about vectors.
same .I still dont know why i should project a vector onto anoter in the first place.To know if they point in the same direction.Y is it so useful or why do i have to project them
To Ismir Eghal, I hope this helps: Imagine you were imposed upon with the task of pushing a large boulder up a hill. You and I know the steeper the hill, the tougher the task. That's because gravity is exerting a force on the boulder. It turns out it makes more sense to think of gravity, not just as exerting any ol' force... but a "downward force." We can think of gravity here as a vector pointing downward with the size of the vector reflecting gravity's (CONSTANT) strength. Notice something subtly odd here, changing the incline of the hill changes the difficulty of the task even though *gravity itself remains the same* .... What's going on here? Well, to push the boulder, you must apply a force to it and that force, like gravity, has a direction. Notice how, if you want to push the boulder directly up, you must fight 100% against gravity. If the hill lacks steepness at all, gravity plays no role in the task (it's essentially only friction making your job tough here). You may begin to wonder if there's a precise way to describe this phenomenon. The more your force is against gravity, the more gravity fights it. If the ground is flat (so you're pushing *perpendicular* to gravity), gravity doesn't make your job tough at all.... Can you figure out how the dot product might be of some use here...?
So if we consider that the gravity vector and the vector along which we are applying the force to be perpendicular (meaning flat surface) we have to put in zero effort, so the dot product is zero. Correctme if i m wrong.
Fantastic. Another simple geometric explanation for the fact, that "order does not matter", is that the projections create "two similar right triangles." Thanks-a-million for the excellent series.
Absolutely bloody amazing, it took me almost an hour to pause and rewind multiple times throughout the video, but now that I GET IT, i feel enlightened!
One easy way to understand dot product is to relate it with the formula of _cos(a-b)_ , i.e. _Cos(a-b)= Cos(a)Cos(b)+Sin(a)Sin(b)_ if you replace *Sin(a)Sin(b)* with *Cos(90-a)Cos(90-b)* which are essentially the angles the vectors make with y axes if a,b are angles with X-axis. This is exactly the same as dot product of the vector matrices of two unit vectors, as in both cases we are multiplying X and Y component of unit vectors and adding them. You can look up the geometric proof _Cos(a-b)= Cos(a)Cos(b)+Sin(a)Sin(b)_ and think of the line segments as unit vectors and you will understand that we are dong exactly the same thing in both the processes but under different names, that is line segments and vectors. You can do the same for higher dimensions.
This is absolutely great! I think I see for the first time "intuitive" explanation for what det and rank is. I took linear algebra course at the university but we were just using them to solve systems of equations. They just were. I feel almost betrayed that nobody told me this explanation then. (and nobody mentioned anything about eigenvalues !!!) This series is just wonderful! Keep doing the great stuff! And I really like it is strongly focused on essence not on computing: computing I can find anywhere.
Your teaching is unparalleled. The best explanations, with the animations to build the best intuition. When I think mathematically, I think in terms of 3B1Bs animations.
This chapter was bit harder than other lectures, and I had to watch 3 times but still having difficulty in understanding completely. I had to pause the video multiple times to understand... Why !!
It feels to me like an intuition about projection, itself, is necessary to understand the implications being described in this video. This is the first video in this series that didn't click for me.
This video helped me to understand directional derivatives and gradient. There is a just incredibly deep connection between them, because in a way any directional derivative is a projection of unit vector of derivative on the number line formed by the gradient . Thank you so much for this video!
Wow! This one was quite tough but really awesome once I stopped to think about it. The toughest bit was understanding why exactly duality holds for non-unit vectors, but I think I've finally got it. Thanks for this series, it's great and really approachable, like most of the stuff on your channel.
@@robinswamidasanUse common sense, dude. If the guy says he spent his entire life around linear algebra n is into data science(probably data scientist) & given that linear algebra course comes only after 12th, then *it's obvious that data science has something to do with linear algebra* 🙄
i'll attempt to phrase the duality. Each vector can be seen as a line/arrow in 2D space, going on its merry way. A vector is also a transformation that can squish 2D space to 1D(by taking any other vector and projecting/dot producting with it). Interesting to note that once we see the vector as a transformation, it no longer makes sense to think of it as a vector, since there is no more 2D space, just a line. Is my understanding right?
Learned about matrix multiplication first so i asked the professor why we don´t have to transpose the vector before we multiply or if its simple because the output would be the same regardless. He just looked at me like i was an idiot and said well they are vectors and not matricies. Makes so much more sense to me after watching this video, love this series
Only looked this up in preparation for my linear algebra intro course, now I am fully subsumed by the beauty in mathematical manipulations themselves..because of you Grant!!
![[MV] A leap of faith...ความเชื่อใจครั้งสุดท้าย (From "Petrichor the series")](http://i.ytimg.com/vi/U1piZH2CNXA/mqdefault.jpg)
I first watched this series 3 years ago. But each year, I rewatch it as I learn more and more maths. Each time, I realize that when I previously watched it, I missed something important and beautiful. It's amazing how much information there is in such a work, that can be easely be missed by the untrained eye, who can't appreciate the hidden connections of math.
I feel the same way
yup : there's quite a lot to think about in this particular episode. Had to stop a few times. Prob rewatch part of it tomorrow morning. Can't believe we weren't taught any of this in Math1...the formulas, yes, but zero intuition like this. No wonder most of us find Math really hard.
This highlights another thing that makes the video series good: The ideas are presented in a way where they can be accessed easily by people with different levels of experience and familiarity with the topic.
I have the exact same feeling. I keep coming back to these videos every once in a while for the past 2-3 years. And I get a new perspective, new understanding every single time. Just amazingly beautiful. Math is beautiful !! Thanks for making such a nice series.
This series is basically perfect, congratulations.
It's strange that this subject is usually taught so badly. I attended a world class university and at no point did they properly discuss the intuition and motivation for the formal concepts.
Yep. We badly need reform in Mathematics education from "K through post-grad." I'm thrilled that 3B1B is calling attention to this issue.
The problem is, that visual understanding is seen as something, people should figure out themselves, mainly because it is faster (and easier for the lecturer) to show to rigorous proofs, if you _only_ show the proof, and don't show the intuition behind.
However, I guess, one could try and change some of the exercises to help with visual understanding. One could e.g. every once in a while instead of asking for the fourty-third inverse matrix ask the students to draw the effect of a matrix-vector multiplication into a coordinate system.
Or, one could leave out coordinates (to indicate their arbitrariness) and only give them vectors and their images drawn into space/the plane and ask for properties of the (or all) linear operation(s) which make the given vectors land on the indicated ones. (So, you could draw two basis vectors of the plane, let there image be on one line and without knowing any coordinates, students now know, that (e.g.) the determinant will be zero)
Well, the problem is that all of this stuff is impossible to show on a blackboard. If a program existed which was both powerful enough to illustrate these concepts and easy enough to use on the fly, mathematics understanding through using it would take off like a rocket.
That's why you keep seeing people what program 3B1B is using to make them. They're hoping for something they can play and experiment with.
carrionpigeons Don't entirely agree. For one thing, I think all of the visual concepts in this video could be comfortably presented on a blackboard. Secondly, lots of it isn't even visual, it's just explanations. For example, the video where matrix*vector multiplication was explained. Many courses just give you the definition in subscript notation and never motivate the concept. This course gave the "proper" explanation in terms of the intermediate step of linear combinations of matrix vectors, which makes everything a hundred times clearer. I have seen one course (MIT Linear Algebra) spend a long while explicating that idea, but most courses do not.
see also lem.ma for a great set of videos that (also) communicate these ideas well - especially the interplay of geometric and algebraic aspects - but using a traditional blackboard approach.
Its a shame that mathematics has become such an infamous subject for the average person, otherwise this channel would be far more popular, which it really deserves.
Mathematicians are the secret society of people who know how the world works. We are the 1%. Consider this video series as an invitation to the silent ruling class.
Math has just become synonymous with computations
The problem is that most teachers suck.
Oobaga !! (Random drum sounds)
It's not completely bad that mathematicians are rare, it means we are much more valuable, which is nice.
I have a phD in physics. I was really good at geometry during my study years... and I have NEVER seen this. And not one of my (very good) professors talked about this in this way. I am REALLY shocked, in a good way. You are the best Math Theacher on line that there is. We need one of you for every language on Earth!.
You learn this in undergrad quantum mechanics though (Riesz representation theorem).
Well it seems that my professors didn't make the nail go through my skull then. Or i am reaaally good at forgeting QM. Which is possible.
I say the same thing about the determinant analogy to area shifting! Through my bachelor and masters in physics! No one ever made that connection for me!
"Sometimes you realize that it's easier to understand it (a vector) not as an arrow in space, but as the physical embodiment of a linear transformation - it's as if a vector is a conceptual shorthand for a linear transformation." Definitely one of the most beautiful ideas I have ever learned, thank you for articulating it so well.
here I was thinking "understand vectors as coordinates and vice versa instead of arrows" was revolutionary... And now were back to arrows lol
I wish there was a 3blue1brown of physics also
There are many good books to develop an intuition for physics though. Something lacking in hardcore math, I feel.
nothing beats quality animations for explaining concepts tho
@Bronn - you can say that again!
@@FullHouseFanatic I agree
@@Brono25 minutephysics had a series on special reletivity that was really good like this
"Unlearn what you've just learned" - finally something i can do
no you can't unlearn it if you haven't learned it in the first place.
@@ishan_murjhani lol. I mean these videos simplified all the concepts but still it is difficult to visualize it. Espcially duality ughhhg
ABo NaBiL Gamer ! It took me three watches with a couple of weeks in between to understand this fully. And even then it's not quite simple when you try to apply it on your own
@@ishan_murjhani vacuous truth
The first teacher to ever tell that intently to their students. Man that was even hard to write, what a bothersome alliteration :P
Sincerely, this series of videos has taught me more than the whole Linear Algebra subject I took many years ago when I was studying Computer Science. It addresses precisely the most important problem I found back them and that is the lack of geometrical interpretation. Now I'm a Postdoc in Computer Graphics and still find myself finally understanding some details here and there. Thank you very much. I would gladly pay tons for something like this applied to more advanced concepts.
I'm saving this playlist for my children.
Hope you ain't to hard on them ;)
The much simpler real life application, I think, for finding a "dot product":
Imagine you and your friend are pulling a big rock, tied by 2 ropes. Your friend's rope is making an angle ø, with your rope, like V shape.
Now the *dot product* gives the actual amount of force your friend is adding to yours, given there is ø angle between you and your friend's direction of rope pulling.
If your friend stands in same line as you, pulls exactly in the same direction where you want to move the stone towards, (or both are pulling the same rope), angle ø=0, i.e Cos(0)=1, then thats the most efficient way of pulling the stone. Bcz all his force magnitude will absolutely add up to yours.
Otherwise, atleast, the lesser the angle ø between the forces, the higher the total magnitude will be.
But if your friends rope (force) direction is making some angle, ø with yours, then though he's putting some X amount of absolute Force, only a fraction of X (projection of your friend's on yours) will add up to yours.
If your friend makes an angle 90 degree with yours, i.e. he is pulling in perpendicular, then he's not adding up anything to your effort, cos(90)=0.
But if your friend is pulling the rope in opposite direction, cos(180)=-1==>Means your friend is working against you. Then the rock moves in the direction where the force is higher, but the amount of displacement would be much lesser, yourForce-friendForce.
This is also used in the Newton's second law of motion= F = m*a cos@.
So basically the Dot Product tells us, how much a vector is working FOR/AGAINST the other. It's called cosine similarities! This is also used in comparing 2 things (vectors) like how similar are they, do they add up to the entirety or negating each other..
Man.. How I wish I had a board to write down, draw and explain..
Excellent addition.
it s a cool explanation to give when explaining to someone else.
with example in real life it s always easier to understand thanks
This is a nice way to understand what projection does but it doesn't explain why the dot product is used, why it is used the way it is and why it works and makes sense at all.
really good
@@group555_ the dot product is basically asking this question, you and your friend pull a thing really heavy and you ask yourself: Does this guy help me at all or just faking, the value reflect the "help", and why projection , because the use of cos(x), cos(x) basically mean x direction loosely speaking, you can check more on Unit circle . Projection is basically mean take cos(x) of the other one and why cos(x) is used because it reflect the 'true' value of the other one ASSUME they are one the same vector, in this case x.
This series of videos is going to last. I think presents an almost purely geometrical explanation of linear algebra that can't be found anywhere else at the moment.
I believe this could be done with every math subject. As an undergraduate student of mathematics, I often find myself struggling to translate what the books say into a more intuitive mental image of what's going on. Of course, some subjects (e.g. linear algebra) are easier to visualize than others. But then again, mathematicians always try to build mental images of what they're working on, even if it is a very abstract subject. These mental images are crucial, and in my opinion they should be taught along with the (undeniably necessary) formal arguments. These videos show that animations are a very powerful tool to do this.
All of this could be said about your Multivariable Calculus Course as well.
Now, I realize it is a lot of work, but it would be great if you did the same thing with some other undergraduate subjects. Have you already considered this? If so, what subjects?
Thank Alvaro! I'm definitely planning to do more of these. Long term, it would be cool to tackle all undergrad topics one by one, but the ones closer to the top of the list right now are calculus, real analysis, and probability.
Calculus and real analysis? What? I was under the impression that what we call analysis in our courses is called calculus in English. What is real analysis, then?
The word analysis, or analyse in english is just the process of understanding something, through the application of "pure" (i.e. undeniable) logic. Real analysis, in mathematics, refers to trying to define properties of the "real" numbers (i.e. numbers with potentially unending digits) and functions of those numbers. Calculus is the numerical/analytic description of change. Although often this results in functions of the real numbers, there are other uses for calculus outside of the real numbers. I imagine it must be quite difficult to learn about mathematics in another language, as the english part of mathematics, is often ill formed to say the least...
I'd love for you to go into hyperdeterminants and multidimensional matrices!
+Alex D. Bryan you seem to misunderstand my situation, or is it me misunderstanding your comment? I study maths in German (mostly) which is my mother tongue and the occasional course is in English, but that's not a problem for me. It seems, though, that our university lumps analysis and calculus (and topography) into one 2-year course named Analysis 1 through 4. So I didnt know how to distinguish between the two. The language is not the problem.
Holy moly, this was deep. I feel as if I don't have to memorize anything for my upcoming linear algebra test -- it's all intuition! Thanks so much!
@Robin Rastle 👍
must have had an easy professor. it’s still all greek to me - and not only because Pi is teaching multiple Pi’s.
It took me a while to understand the point he was making(with regards to how projection occurs), but it makes sense in the context of previous chapters:
_(3-D Example below, but it generalizes pretty intuitively imo)_
1) *Linear Transformations and Matrix Multiplication→* Multiplying two matrices can be conceptualized as applying one linear transformation after another. I will make sense of the dot product in this context.
2) *Column Space→* The non-square vector on the right side of the dot product has column space one(rank = 1); thus, applying it as a linear transformation will shrink your original vector down into 1-dimensional space. → This is what Grant was getting at using û as the basis vector for 1-dimensional space.
3) *Basis→* The non-square matrix on the left can be conceptualized as the linearly-transformed î-intermediate, ĵ-intermediate, and k̂-intermediate coordinates(where the basis vectors land during the second linear transformation). The non-square matrix on the right can be conceptualized as just 1 basis vector(û) with 3 coordinates of landing.
4) *Dot Product→* Conceptualized as the projection of a 3-d vector onto 1-d space(the right matrix) where the product of transformed basis vectors(left matrix * right matrix) determines the extent to which each dimension contributes.
*What this means→ *
1) The dot product represents the extent to which 2-vectors combine in each dimension, represented by a singular value.
2) Column vectors can be conceptualized as a linear transformation from n-dimensional space onto 1-dimensional space.
After viewed almost 10 times, finally I interpret the essence of this episode .
You got me beat. I watched it probaby 15 times to understand what he was saying lol
Yep, I'm going to try the fourth time.
I thought i was alone but now i am happy
哈哈哈
I did the same with the determinant video, I could never understand how the 3D determinant formula actually worked
For anyone who's lost like I was hours ago, I think I'm beginning to grasp it.
See the number line as a 1D vector space with basis vector u-hat, where u-hat has coordinates ux and uy. The basis vectors i-hat and j-hat define the 2D coordinate system. The coordinates ux and uy are projections of u-hat onto the x-axis and y-axis respectively.
Say you want to transform this 2D space into the 1D number line, such that L(i-hat) and L(j-hat) are projections of i-hat and j-hat onto the number line. You'll need a 1x2 matrix that will look like [L(i-hat) L(j-hat)]. As I wrote earlier, ux and uy are projections of u-hat onto the x-axis (i-hat) and y-axis (j-hat). Because of symmetry, the projections of i-hat and j-hat onto u-hat (the number line) will also be ux and uy. So, L(i-hat) = ux, L(j-hat) = uy! The matrix will be [ux uy].
Note that ANY vector, v, in the original 2D space is a linear combination of the basis vectors i-hat and j-hat. After the transformation, this still holds true: v = c1 i-hat + c2 j-hat and L(v) = c1 L(i-hat) + c2 L(j-hat). So, because L(i-hat) and L(j-hat) are projections of i-hat and j-hat, L(v) is a projection of v onto the number line!
Now, the vector equivalent of the transformation matrix [ux uy] is just u-hat. If you take the dot product between a vector v and u-hat, you do exactly the same as you do with the transformation; you project v onto u-hat!
So, to generalize: Taking the dot product of vectors v and w is equivalent to transforming the vector v by the matrix [wx wy]. But also equivalent to transforming the vector w by [vx vy].
Clear, simple explanation. Thank you!
That's it, it took me three replays to understand it lmao
Fantastic summary. Thanks! 💜
Your explanation makes it easier to understand. Thank you.
Excellent and clear explanation
at this point, i've been pavlov conditioned to associate that piano sound with "you gon learn some shit"
I love the small pauses that give you a chance to reflect on maths' significance and beauty.
Yessss!❤
Hey pal, you did understand the topic. Would you mind to explain me since I'm having trouble?
I just stare into space thinking "maybe I should rewind a bit" 😂
I think I am starting to understand it now. It really helps me to see it this way. I finally got more than halfway through your series and my brain is finally starting to put pieces together. I will keep watching and re-watching until it sticks. I'm gonna try to solve some problems now to see if I can solve them and also understand them better than I have before. Thank you for these!
I d say do khan academy first then watch these...
I went through it for more than 1 hour pausing at each point made. I have never attended college and learned through textbooks recommended by first class colleges and the Khan Academy teachings (in particular) to get my engineering credentials for a living. I jammed into my head all these math proofs like an Ape and try to relate them to the real world of tangible things. Thank God that I found You to light up the bulbs and cast away the shadows (vague conjectures of realities).
Example on dot product. You sum it beautifully as a linear transformation from 2D space not defined as numerical vectors but projecting space onto a diagonal copy of a number line. I take this and others as wings to fly now.
Yes. I signed onto Patreon as a small token of appreciation.
In all honesty grant, you have made me realize that a solid foundation in math can allow you to explore many concepts by yourself, the fact you frame things so beautifully that it makes everything click in place, it is admirable, you are a great teacher, fueling the next generation of math lovers, and inspiring many, specifically your way of teaching, it doesn't give you the answer, it gives you a way to understand it and then using that understanding figure out something, your style of teaching makes math very fun, and i will always appreciate it.
To those who don't quite get the bit about the duality and how dot product could occur from that, after hours of thinking, I think I might have figured out a more direct explanation that could help you.
7:48 || 3b1b takes a copy of the number line and pastes it on the two-dimensional matrix grid, such that it is slanted and the "0" is at the origin. It is important to note that U-hat (the unit vector of the new number line) has the same length as I-hat and J-hat. We will see why in a moment.
7:50 || 3b1b explains why the new number line is a legal move, and that a function that converts 2-D vectors into a number on the new number line exists.
8:53 || 3b1b tells us that it is important to know where I-hat and J-hat land in the new number line. He explained why earlier, from 5:21 to 6:18. To add on to what 3b1b said, knowing where I-hat and J-hat lands in the new number line allows us to define ALL vectors that exist in the 2-D space just as they are in the form a * I-hat + b * J-hat. Suppose L(V) is the function to transform a given vector V onto the new number line, where V is a 2-D vector of the form a * I-hat + J-hat. Let L(I-hat) and L(J-hat) be where the I-hat and J-hat lands on the new number line respectively. Thus, L(V) = a * L(I-hat) + b * L(J-hat). a and b are given from vector V. If we can find what L(I-hat) and L(J-hat) are, then we know L(V) is possible.
8:59 || 3b1b explains to us that an arbitrary U-hat (with length equal to the lengths of I-hat and J-hat) with an x-component of Ux and y-component of Uy, has L(I-hat) and L(J-hat) as its x-component and y-component respectively. This proof can only work if U-hat has the same length as the lengths of I-hat and J-hat. Suppose otherwise, then the line of symmetry would no longer exist and the proof no longer exists.
10:07 || 3b1b shows us where the dot product comes in.
At this point, 3b1b has shown us the dot product between U-hat and another vector, say, V, is really just V projected onto the number line that U-hat sits on. We are meant to prove for two vectors, say V and W, and not one of them with an arbitrary unit vector. At 10:33, 3b1b tackles this by considering projections onto non-unit vectors, i.e, projecting V on W or vice versa.
He said that we should scale up the U-hat by a factor. In his example, he used 3. This took me a while to understand, but I think I finally got it. I don't think he explicitly stated this, but the new number line with U-hat, should lie on one of the vector you are trying to project on. Let's say that we want to project vector W on vector V, then the new number line should be on vector V. We need to use a unit vector, in this case U-hat, for this to work. This was explained in 8:59, I have written about it above as well.
Now, we have a projection of vector W on unit vector U-hat. However, we want the projection of vector W on vector V, and not its unit vector. This is where the scaling is necessary. Since U-hat is a unit vector of the number line that vector V sits on, that would mean that vector V is a multiple of U-hat. We know this is true because they are perfectly coincident, i.e, they share the same gradient and points. Now that we know that vector V is a multiple of U-hat, how much do we need to scale U-hat by to get vector V? The answer is just the length of vector V divided by the length of U-hat. It is like when a * b = c, and you know a and c, and b is the scaling factor, then you can divide a from both sides to get b = c / a. Recall that the length of U-hat is the same as the length of I-hat and J-hat. I-hat and J-hat both have a length of one, therefore, U-hat also has a length of one. Thus, we need to scale U-hat by the length of vector V divided by one, i.e length of vector V.
For the dot product, I am going to be using the asterisk (*). So A * B is the dot product between A and B.
Currently, we have (U-hat) * (vector W), or [Ux Uy] [A B] (A and B are meant to be on top of each other, A and B are arbitrary letters that I have picked to represent the x and y components of vector W respectively). Scaling up U-hat by length of vector V (denoted by |V|), |V| x (U-hat) * vector W. This becomes |V|(Ux x A + Uy x B) = |V| x Ux x A + |V| x Uy x B. Let C and D be the x and y components of vector V respectively. C = |V| x Ux and D = |V| x Uy. Finally, we have AC + BD, which is the outcome of vector V * vector W.
The new number line can be placed on vector V instead, and by the same arguments, we should have the same results.
To summarise, (1) get U-hat and place it on one of the vectors, say V. (2) Project other vector onto U-hat, so that it is projected onto the new number line. (3) Scale U-hat up to get the original vector V. If it helps anyone, I could have this down on paper and link it here.
Thanks bruh
Why did he call "u hat" a two dimensional vector when it is the unit vector of the numberline?
@@themangobui1474 because in the normal orthogonal coordinate system it's a 2 dimensional vector. We are relating the two dimensional space to the one dimensional number line, quich in the two dimensional space is pointing in direction u. To me it seems that the vector u in 2d space has the coordinates (0.5,sqrt(0.75)). Try to draw it and youll see!
Finally! Thank you very much for the explanation
Thanks bro
Man, even with your fancy well-animated and explained examples, I was having a very hard time to understand this concept and took some days when I finally got it. No, it's not your fault. I think it's simply because english is not my main language so my learning curve rely more on visual examples than anything (my english is not poor, but it's just not fluent enough to understand everything at the same pace of natural english speakers).
However, I took this difficulty as good thing, as I REALLY learned all the subject of your linear algebra series by wathing every video multiple times.
I just want to thank you for your effort to bring us this different method of teaching math which is actually much more stimulant than the old fashion way most of us are used to (copy and paste formulas). And your passion about how cool is this subject gives me even more inspiration to learn. Thanks from a brazilian guy.
TheAwakeningMission É BR no bagulho! Tamo junto! xD
TheAwakeningMission same condition here bro ...I was just thinking that I'm too dum to under stand this concept but as u said I also have this problem...His linear transformation video I have to watched 4 times to fully understand. sometimes I feel to give up bcz I don't understand after so many times ...But truly I want to learn this concept .....
And that's okay, keep going :) Sometimes the brain needs a day or 2 to wrap itself around some idea.
If you don't get it in the beginning, that doesn't make you stupid. Stopping to try to understand is what makes you stupid.
LunnarisLP -- As someone who only speaks English, I can tell you that your last sentence is (unintentionally?) very funny.
have the same problem man. can you direct me on what to do when i have this problem. i mean what have you done to make it easier for yourself? have you just been watcing theese videos multiple times, or did you write something on paper?
You've done a truly excellent job with these videos. The geometric insight into linear algebra is wonderful, and I say that as someone who used linear algebra almost daily during my career at NASA.
i know this has been five years but out of curiosity what do you, sir, do at NASA?
I appreciate that you use a black background. Its easy on my eyes because i get headaches very easily
7:02 i feel stupid for just realizing that theres always "three blue and one brown" π in those talking animations
Oh my god I just realized that too
I feel more stupid now I understood this
It's also his channel symbol, a circle that's 3/4 blue and 1/4 brown (3blue1brown) which is all in turn referencing the fact that his eyes are 3/4ths blue and 1/4th brown. kinda neat lol
Actually it is 3 brown and 1 grey
Lol , I get it now 😅
I've been a graphics programmer for over 8 years, I've also written countless collision systems, animation engines, physics engines, and your videos are still blowing my mind.
How is this not the standard way of teaching linear algebra in school/uni?
Simply because is a huge amount of work to make videos like these and teacher don't have time to make them at a large scale.
If anyone's having trouble understanding, this is how I've come to think of it after watching the video a few times:
1. To turn 1 unit (u) of the number line into a vector, you'll have to project it from the number line onto the xy-plane. That projection will be marked by u's coordinates (ux, uy), as shown at 9:30
2. To go back to the number line from the xy-plane, move ux units along the x-axis, and then move uy units parallel to the y-axis, and you'll get back to u on the number line.
3. To turn any vector on the xy-plane into a number on the number line, you'll also have to mark its location in the xy-plane by taking note of its coordinates, (x,y). Then if you scale those coordinates by u's coordinates, you'll be able to get to the number line by walking x*ux units along the x-axis, then y*uy units parallel to the y-axis. And x*ux + y*uy is basically just the dot product of [x, y] and u.
Very good explanation. That does make everything a bit clearer.
The last part (you will be able to walk to the number line) is wrong, the dot product just gives the value on the number line directly.
This is just . . . Beautiful! I just feel so lucky to have discovered your channel, because, no teacher, no professor, no matter how we advance in our fields that are related to maths, explains like this and shows the deep essence of the science (except if your main specialization is mathematics).
I've had rough 4 years of undergraduate studies and got two more semesters extended. In my last semester before graduation, watching your videos on vectors gives me strength, 'cause I'm taking Vector Calculus. I would get nothing out of my course, if not for your beautiful explanation and eye-opening visuals. Now I am sure that if I can understand these videos, then practically anyone can enjoy, let alone learn, mathematics. Thank you so much!
P.S. I keep rewatching your course playlists to root the concepts and entertain, as well.
I once had a friend who took his own life. He was very religious, myself scientific. The last background he had on his Facebook was a Calvin and Hobbes comic. I think about that man a lot and miss our talks. Seeing this quote really made me consider some of those conversations and their impact on me, and I can't help but think his life helps drive me, even in his death, to see through the lens of maths what my existence, and his, might mean. Thank you for your videos, Grant.
If this feels a bit overwhelming, ask drunken Grant to explain this to you. You reach him by setting the video speed at 0.5x. He's just as smart as regular Grant but has a more laid back style.
I love this comment, idk why it is on the top when it only has 4 likes, but i love this comment
You get my like
This is the most enlightening video I’ve ever watched on YT and probably in my entire life... I still can’t stop smiling :’) thanks a ton, Grant! 🤲🏼
Me too!!! 🤯✨🕊️
Had to watch this multiple times to understand.
Congrats! You blew my mind.
I've watched it 5 times and I'm still not able to understand what is the use of taking a dot product. Please treat me like a 5-year child and give me some real world example. Thanks, brother
@@imrsk Sorry late comment but read @Bhargavi Suhas' comment down below.
I opened 12 tabs with different websites explaining dot product. 1 minute into this video, closed the other 11 tabs. A big fat thank you for helping us realize the 'why' part of it!
This feels like achieving enlightenment. After all this time you mean to tell me that this is not just: "Funny numbers go brrr and it's useful for some reason" but that it actually makes sense?! This is genuinely reviving my love for mathematics and science again after being buried under test scores and grades.
Could you give me easiest definition of duality
Funny, how the things we have been taught for years finally makes sense. Beautiful work.
Thank you. Thank you so much! Dot products finally make sense on what they physically represent. However, I will say that this video is presented in a confusing order: it mentions that dot products relate to projections before proving why much later. I had to watch the second half of the video first in order to understand the first part's discussion of why order does not matter.
This one took some time to wrap my mind around but it is truly amazing! Thank you for this series, it's a great help to develop an intuition about linear algebra and all its applications.
I really don't know how to thank you. I am not even getting words. you are really a gift to mankind.
The teacher deserves a nobel prize for teaching these concepts 🙏
Sal introduced me to math, 3Blue1Brown made me appreciate and fall in love with it
In computer graphics, I think there is a _far_ more intuitive way of understanding the dot product - it is a measure of similarity between two unit length vectors, 1 when identical, 0 when orthogonal, and -1 when perfectly opposite. (This can be intuited by the fact that a . b = |a||b|cos(theta)). This comes up _constantly_ when writing lighting shaders, since the amount of diffuse light that bounces off a surface is generally L . N, where L is the direction from the surface to the light source and N is the normal of the surface. (This is fantastic because it has a direct visual interpretation.) Also, interestingly enough, I believe Google employs a concept of "cosine similarity" on vectors of words when ranking websites for search.
Or just think of it as the length of a force vector.
Yep. I think of dot products as a measure of how much two vecors are going in the same direction, with the cosine of the angle between them acting sort of like a percentage that can also be negative.
Cosine Similarity is what used in many Recommendation Systems too!
The dot product can also rank the similarity of vectors in any dimension. For example: in natural language processing, you can create a vector full of numerical context values for each of the words in a sentence, or even a paragraph. Then you're taking the dot product of two vectors in dimensions upwards of 100, and using interpreting the result in the same way. The intuition comes very nicely with geometric interpretation, but spans much further.
Thanks. This made a lot more sense to me
Woow just wow
I have seen this video so many times already and it just keeps blowing my mind
This is so beautiful
I hope that second series is coming soon for all the Jordan, IP and spectral stuff
Thank you very much!
Bravo sir. This was absolutely brilliant! Having a degree in math from one of the most trending universities, but seeing the first time in years this actually not-so-simple concept the way it was intended to be... Here is my deepest bow.
Every time I re-watch this video, I feel happier and more realized, I feel hope again, and I am not exaggerating. Thanks for your work, sir.
I am surprised how these 15-20 minutes of videos are teaching me more than my high school courses that lasted for more than 9 months.
Man, this is the best explanation I have seen for Linear Algebra, excellent series, I can't wait for the next video. It felt great to find the series just at the moment I'm starting to learn Linear Algebra at the University, it has helped me to understand the true meaning of what I have been learning. Keep it up!
The definition you gave of dot product is only valid under the right hypotheses, which are not always validated in linear algebra since you could take any base of vectors for the set you're representing. Instead, the more general geometric definition gives a clearer view of the geometric properties of the dot product, and makes the formula you give trivial to find under the right circumstances. Great series, thanks a lot for your work.
This is the BEST explanation I’ve ever seen. This is exactly what’s happening behind primal-dual optimization. Thank you!
Hey this series has helped me understand linear algebra to a degree that I didn’t think I ever could. That being said, this video is the point where I got confused. I see a lot of people in this comment section with the same visualization problem I had and I thought this might help you because it helped me.
The dot product alone isn’t the projection of one vector onto another in itself. The formula for projection of u onto v is ( (u•v)/|v^2| )*v where |v^2| is the length of v squared.
So this means that after you take the dot product you still need to divide the result by the magnitude of the initial vector and then set it in the direction of the initial vector (hence the *v at the end of the formula and the second division of the magnitude of v resulting in v^2 being on the bottom)
Hope this helps
For those who don't know there is a different notation for the dot product. Imagine that you have a vector x and y (of same size), the dot product of those vectors can be noted as x • y but also as x^Ty. The T means transpose which means that you turn the vector *x* on it's side to get the associated matrix as shown on the video.
Oh nice
These videos are amazing!
My only criticism is that you started using "projection" and "unit vector" without fulling explaining what they were.
As someone with no background in linear algebra, I had to look these up.
Similar here: I thought I was following this series quite well, but then he threw in projection without really explaining it, and I was lost for a bit. I did Maths A-Level but don't recall learning anything about projection. Better go look it up!
Yeah same here, although I think unit vector was explained earlier referring to i-hat and j-hat, but with u-hat I'm not sure what is means. I'm thinking it's a vector with a magnitude of 1.
@@inxiveneoy that’s exactly what it means yes
Your channel should trend monthly. I want to believe there is a reason why most people find math repulsive. A reason somehow more meaningful than the way it is taught in school. I think that knowing these types of things gives you a satisfying and powerful understanding of almost everything around you. Maybe through some divine phenomenon, that power is kept in the hands of few people, because if everyone found this stuff easy, well, they might not use it for good. But then again what am i saying think of how advanced we would be as a civilization there would be no room for evil. I dont kno
I've come back to this video countless times, and each time it helps just as much
While watching this I suddenly understood why transforming a vector into a lower dimension and then back into a higher dimension always causes some loss of information. Genuinely some of the most intuitive explanations of linear algebra I’ve ever heard
It took me this much to realize why your function always makes
2
7
To [1.8]. It's e 2.71828...
so you chose the red one
Welcome to matrix
(7+2)/(7-2)
@@svsrkpraveen could you please explain the meaning of this quotient? Thanks!
@@f1cti 9/5 =1.8
7:22 "unlearn what you have learned" makes me want to smash the like button more times than TH-cam will allow, Master Yoda.
Suddenly, the way the dot product was first introduced in my maths textbook - the length of the first vector times the length of the second vector times the cosine of the angle between them - makes perfect sense!
Nillie this part still didn’t click to me
Mariusz Wiesiolek as we know, cosine is the ratio of adjacent side to hypotenuse. You can think of w as the hypotenuse and of its projection as of this adjacent side. Since we’re projecting w onto v, then the angle between v and w is the same as between w and its projection, therefore we can use the cosine. Hope you understand my attempt at explaining this since English isn’t my native language.
Good explanation Mark, I too was confused about that point. My understanding is that if the vectors are forces then the dot product is the resultant force (a*b*cos(theta)). The resultant force ends up somewhere between the two vectors (e.g. 2 horses pulling a barge), it doesn't end up going in the same direction as the other force - as it seems from the video.
@@helenkirby2539 I'm not sure if your comment refers to your prior understanding or your new understanding. If it's the latter, it is not correct: If the vectors represent forces, the resultant force is the sum of the vectors.
Currently learning about this in much less detail in my trig class, but I really appreciate the number line translation you're explaining here. I typically need to hear/see a concept a few different ways to really get it hard wired into the brain and this number line translation helped a lot, thank you!
If you're struggling, do not be ashamed, I've scrolled the comments, many many people struggled with this video and so did I. It'll click just keep watching and asking questions.
Could you give me easiest definition of duality
wow, just wow.
this was amazing. I never understood why scalar product actually works...
2:37
My intuition for "Why the order doesn't matter" was to consider the area of the triangle bounded by the 2 vectors. What all of this means is that you get the area of that triangle by 2 ways.
The area of the triangle bounded by the 2 vectors is half the absolute value of the determinant of the matrix of the two vectors (in 2-d) or magnitude of cross product (in 3-d).
I wish I watched your videos back when I was taking my linear algebra course. Life would have been really simple! You are an Ubermensch by all means, Cheers!
4+ years later...still my favorite video on TH-cam.
The duality of convolution has always amazed me. The method of fractional resampling for signal timing correction is the best example. In the end rather than adjusting the time index of the sample you adjust the filter to its location at the time that index of signal would have occured because of the duality... i used to explain it as like the farnsworth engine in futurama... it doesnt move the ship forward, it pulls space backward.
I'm no mathematician and I'm just a dilettante when it comes to the subject matter. But boy is it interesting. I always had respect for math and my math professors. Got me all the way to calc III. Love this series of videos.
Also completely approve of the Calvin and Hobbes dialogue at the beginning!
But maths is completely unlike religion.
When you have two apples and get two more, you have four apples. When you have two dollars and get two more, you have four dollars.
2+2=4 is just an abstraction over this observation. You don't need to believe anything. You don't even need to believe an axiom in order to work with it.
because all religions are burdened with superstitions, religion has almost become a synonymous of "believing blindly in something".
Actually religion means "binding together" "bridging yourself with the universe"/"realizing the unity of everything" (not believing in it, but Realizing it, experiencing it)
Pseudo-religions like Christianity have been preaching belief and have been exploiting people, and that is so stupid and ugly. Although blind belief and other such things have nothing to do with religion, they have been happening IN The NAME of religion, that's why the association (in our minds) is so strong.
You can observe that 2 apples + 2 apples = 4 apples , but essentially this is very mysterious. Everything in this world is quite mysterious and will remain mysterious no matter how much we sugar-coat everything something with explanations. Explanations are utilitarian. They are very useful since they can provide frameworks on which we can create technology, but explanations should not have the pretension that they represent ___the supreme absolute "truth" that must never be doubted___ or that they demystified existence.
These pretensions are dangerous, because then people will behave exactly like the religious fanatics do, but now IN THE NAME of science.
PS: Mathematical proof that you can't demystify existence:
If you want to explain what A is, you'll have to use another term B. But then you're caught in an infinite loop, then you'll have to demystify B as well. You'll have to describe what B is: "B is C and D in such and such way", and then you'll have to explain C and D and so on. All explanations of an object/phenomena X are expressed in terms of other object/phenomena Y1,Y2,...YN,... . If you managed to explain away an infinite number of phenomena, you'll have created an infinite more phenomena which are waiting to be explained.
PPS: I'd like to hear some criticisms of the above proof.
nobody needds your approval, you dumbass furry
completely disapprove mate!
i found 9:30 super hard to understand visually with what is being said. What i see happening is that to project i-hat onto u you make a 90 degree line from u onto the head of i-hat. To project u onto i-hat you make a 90 degree line from i-hat that crosses the head of u.
Super great videos and representations. Its a very nice way of thinking about the cases
Me too
Ok, I love the series so far. Chapter 7, however, took me by surprise. I spent three days and scribbled out 17 pages of notes trying to assimilate Chapter 7. I still catch myself daydreaming, trying to visualize the contortions those poor unit vectors must go through to exit 2-D space and find their new home on the number line. I admit I'm a little fearful of tackling cross products in Chapter 8.
SierraSlim1 Hi, bro! Can you send me a copy of your notes? I feel the same as you about this subject. Greetings from Brazil! Thanks!
It's really just about using an ordered group of numbers in two different ways.
You Sir/Ma'am seems to be a wonderful writer. And your comment is absolutely relatable. Until now I thought I was the only one spending entire couple of days on a single video, taking notes, uprooting hairs and slowly slowly visualizing every aspect of the explanations. Seems I am not alone. I hope you have completed the series by now and have absorbed all its beauty. Will take me couple more weeks to do that.
thanks for the commenting this. untill now i thought i was the only one who spent a lot of time in a single video but your comment gave me hope.
Yep chapter 7 was a little more difficult than the others and needed some time to comprehend.
you nailed this one! i also learned this concept almost at the end of my lineair algebra course, it just makes way more sense that way, and you can really dive into the concepts of inner product spaces
This video is a gem, this is what is needed in education, I understood each and every part and really say this is never taught it is also hard to teach these topics without actual visualization, there were parts where the graph was not meant to be scaled, I also like the simplicity and focus not giving irrelevant examples, when I will grow I will surely donate to this channel so that it can keep up with these videos for the future generation, I am still 16 and fully support this guy, Hats off huge respect from India sir🎉🎉🎉
Something eluded me the first time a saw this. However, after learning about "change of basis" in a rigorous manner, I think the explanations that used "symmetry" and "projections" finally clicked. The diagonal blue line that is spanned by u-hat is a vector (sub)space U; the projections are linear transformations. Since the length of u-hat, i-hat, and j-hat are all the same, then the length of the projection of u-hat to i-hat is equal to i-hat's projection to u-hat, i.e. u_x. Note that the projection of i-hat to u-hat is a projection of i-hat to U. Thus, in a "change of basis", i-hat has a "coordinate" of u_x in U with respect to U's basis which contains one vector, that is {u-hat}. Similarly, the coordinate of j-hat in U with respect to {u-hat} is u_y.
I'm totally in love with this series, but this is the first video I felt a little lost on - I hadn't encountered the notion of 'projection' before, and while I can Google it and get a formula, that hasn't helped me with the whole 'get a visual intuition' thing, that this series is all about. If you ever revisit this, I'd love if you could add a footnote video about that.
this might be a little late, but let me try to help you out. Think of projection as the way you see your shadow under sunlight. Your shadow is just a projection of your body onto the ground. In this case, the sunlight that helps projecting your body onto the ground is always perpendicular to ground, so if you stand up straight (meaning your body is also perpendicular to the ground), you won't see your shadow on the ground. If you lean over a little, you will start seeing your projection (or shadow). Hope this helps
@@Spaaardaaa Thanks, that's a really nice visualisation :)
Thanks, now I understand.
Also, a vector is a n x 1 matrix projecting 1D space into n-D space with Rank one.
This is extremely beautiful. I am taking the course Mathematics for Machine Learning: Linear Algebra on Coursera and it is really cool. At some point, it explains the symmetry or duality of matrix-vectors and dot product and the professor describes as something beautiful, a duality of geometry and calculations. The thing is that the explanation was short and visually hard to understand. So I find this beautiful video explains it perfectly with the spot-on visual for understanding it.
I think this is the real value of this well explain and visually perfect videos. Clarify the abstract concepts through visual and intuitive examples Books, lectures, online courses sometimes fails on that. So this is were these types of videos shine for actual meaningful academic formation. Books and courses are getting aware of it, I have a book on Deep Learning for Computer Vision that literally references 3Blue1Brown videos for concept clarification. Thanks for the amazing work!
Please please trust me this video is much deeper than you think it is!!
Love you 3b1b!!!!
Now I know why dot product of two perpendicular vectors is 0.
Such an amazing series, loving it.
I really wish they'd emphasised these more in school in the 90's, given how important they are for game programming today.
For anyone who is still struggling to grasp, what finally made this concept click for me was to think of the "dual" of the vector as the UNIQUE LINE in space that now represents your 1D number line. A number line can live anywhere in a 2D space (a 1,0 number line would be the straight line along the X-axis, a 0,1 number line would be the straight line along the Y-axis, or the U-hat number line that was diagonal through the origin looked like coordinates of ~(0.5,0.5)) So, 2D space in a sense remains absolute. When we project onto a 1D number line, we're essentially saying "this is your PARTICULAR 1D "playing field" (number line) right now and it lives in a unique place within 2D space" but there are an truly infinite number of 1D number lines capable of being generated from the 2D space. The three examples I gave are just a subset of all the possibilities.
Thus, because each number line is unique (the 1,0 number line doesn't LOOK like the 0.5,0.5 number line in 2D space), each projection onto 1D space is unique, as represented by the unique 2D vector that codifies where the number line "originally lived" in 2D space.
i feel we are saying the same thing it but I've worded it different. Each vector can be seen as a line/arrow in 2D space, going on its merry way. A vector is also a transformation that can squish 2D space to 1D(by taking any other vector and dot product with it). Interesting to note that once we see the vector as a transformation, it no longer makes sense to think of it as a vector, since 2D space is now a line.
i like the "originally lived" - once that vector is seen as a transformation, there is only a number line... no space.
Quick caviat: not all 2d vectors transform into unique number lines. In fact, there are an infinite number of 2d vectors that could be transformed into EACH number line. For example, [1, 1] would be the same as [2,2]. For any two 2d vectors W-> and V-> where W-> = [Wx,Wy] and V->=[Vx,Vy], if Wy/Wx=Vy/Vx, they will transform into the same number line. This becomes intuitive if you consider that if you have a point (x,y) on any line that crosses the origin, you can calculate the slope of that line as y/x. So basically, if the "slope" of the two vectors is the same, they will transform into the same number line.
Thanks for this comment. Somehow this just helped me 'click'. Kudos!
How was the i and j projected onto a number line and got new basis (2, 1). Can anyone pls help me with this. I don't understand it.
I LOVE LOVE LOVE your "alternative pacings" to things because it helps set such a CEMENT understanding of things to know "mathematically yeah sure, but *why*?" You do such a good job helping conceptualize such nebulous topics.
I think I have watched the part between 6:30 and 10:00 at least 8 times, but it finally clicked and made geometric sense.
Thank you for the cool new way to look at linear algebra. I love how it gives us a more intuitive way to think about vectors.
Gonna have to rewatch this one a few times before i understand what is going on.
aaaaand this is where you lost me. lol. back to review i go.
aaaand a week later and i'm still lost.
lol yep it was all going so well until here..
same .I still dont know why i should project a vector onto anoter in the first place.To know if they point in the same direction.Y is it so useful or why do i have to project them
To Ismir Eghal, I hope this helps: Imagine you were imposed upon with the task of pushing a large boulder up a hill. You and I know the steeper the hill, the tougher the task. That's because gravity is exerting a force on the boulder. It turns out it makes more sense to think of gravity, not just as exerting any ol' force... but a "downward force." We can think of gravity here as a vector pointing downward with the size of the vector reflecting gravity's (CONSTANT) strength.
Notice something subtly odd here, changing the incline of the hill changes the difficulty of the task even though *gravity itself remains the same* .... What's going on here? Well, to push the boulder, you must apply a force to it and that force, like gravity, has a direction. Notice how, if you want to push the boulder directly up, you must fight 100% against gravity. If the hill lacks steepness at all, gravity plays no role in the task (it's essentially only friction making your job tough here).
You may begin to wonder if there's a precise way to describe this phenomenon. The more your force is against gravity, the more gravity fights it. If the ground is flat (so you're pushing *perpendicular* to gravity), gravity doesn't make your job tough at all....
Can you figure out how the dot product might be of some use here...?
So if we consider that the gravity vector and the vector along which we are applying the force to be perpendicular (meaning flat surface) we have to put in zero effort, so the dot product is zero. Correctme if i m wrong.
Hey Grant I know you're not gonna read this but you're my favorite person after Einstein
Fantastic. Another simple geometric explanation for the fact, that "order does not matter", is that the projections create "two similar right triangles." Thanks-a-million for the excellent series.
Absolutely bloody amazing, it took me almost an hour to pause and rewind multiple times throughout the video, but now that I GET IT, i feel enlightened!
This is real beauty , my eyes tears sometimes from such elegance !
I find it amazing how the cauchy-schwarz inequality is a simple consequence of the dot product
"simple" may be an overstatement here xd
Actually, it is enough to say that Chauchy-Schwarz is a consequence of having a positive semidefinite symmetric bilinear form.
One easy way to understand dot product is to relate it with the formula of _cos(a-b)_ , i.e. _Cos(a-b)= Cos(a)Cos(b)+Sin(a)Sin(b)_ if you replace *Sin(a)Sin(b)* with *Cos(90-a)Cos(90-b)* which are essentially the angles the vectors make with y axes if a,b are angles with X-axis. This is exactly the same as dot product of the vector matrices of two unit vectors, as in both cases we are multiplying X and Y component of unit vectors and adding them. You can look up the geometric proof _Cos(a-b)= Cos(a)Cos(b)+Sin(a)Sin(b)_ and think of the line segments as unit vectors and you will understand that we are dong exactly the same thing in both the processes but under different names, that is line segments and vectors. You can do the same for higher dimensions.
That's a great way to remember the trig identity too!
Can you please tell me how to get the "length" of a vector, projected onto another vector in a plane???
@@-Momin literally just Google it. I would help you but I would have to Google it anyways lol. Just Google vector projection
This is absolutely great! I think I see for the first time "intuitive" explanation for what det and rank is. I took linear algebra course at the university but we were just using them to solve systems of equations. They just were. I feel almost betrayed that nobody told me this explanation then. (and nobody mentioned anything about eigenvalues !!!) This series is just wonderful! Keep doing the great stuff! And I really like it is strongly focused on essence not on computing: computing I can find anywhere.
Your teaching is unparalleled. The best explanations, with the animations to build the best intuition. When I think mathematically, I think in terms of 3B1Bs animations.
This chapter was bit harder than other lectures, and I had to watch 3 times but still having difficulty in understanding completely. I had to pause the video multiple times to understand... Why !!
You weren't alone. All the other ones I breezed through. Has taken me multiple hours of contemplation for this one.
I'm not going to lie. This one fried my brain a little bit. I need to take a nap then come back to fully process this haha. It seems amazing though.
It feels to me like an intuition about projection, itself, is necessary to understand the implications being described in this video. This is the first video in this series that didn't click for me.
This video helped me to understand directional derivatives and gradient. There is a just incredibly deep connection between them, because in a way any directional derivative is a projection of unit vector of derivative on the number line formed by the gradient . Thank you so much for this video!
Lol, I just added similar comment as you : )
@@zma4543 In that case I can be happy that there is at least one person who also understood beaty of it;)
Wow! This one was quite tough but really awesome once I stopped to think about it. The toughest bit was understanding why exactly duality holds for non-unit vectors, but I think I've finally got it. Thanks for this series, it's great and really approachable, like most of the stuff on your channel.
This is so unfair. All my life I suffered with linear algebra (I am into data science) and only now the does the author post these videos. :P
Saptarshi Mitra just curious .... r u watching this after completing University ?
Saptarshi Mitra: also curious. Did you start Linear Algebra in kindergarten? Cheers!
@@robinswamidasanUse common sense, dude. If the guy says he spent his entire life around linear algebra n is into data science(probably data scientist) & given that linear algebra course comes only after 12th, then *it's obvious that data science has something to do with linear algebra* 🙄
you're a TH-cam God, thank you
i'll attempt to phrase the duality. Each vector can be seen as a line/arrow in 2D space, going on its merry way. A vector is also a transformation that can squish 2D space to 1D(by taking any other vector and projecting/dot producting with it). Interesting to note that once we see the vector as a transformation, it no longer makes sense to think of it as a vector, since there is no more 2D space, just a line. Is my understanding right?
Learned about matrix multiplication first so i asked the professor why we don´t have to transpose the vector before we multiply or if its simple because the output would be the same regardless. He just looked at me like i was an idiot and said well they are vectors and not matricies. Makes so much more sense to me after watching this video, love this series
Only looked this up in preparation for my linear algebra intro course, now I am fully subsumed by the beauty in mathematical manipulations themselves..because of you Grant!!