Its kind of cool and odd that someone who has taught this subject for so long can keep it so fresh...like he's stumbling across the Null Space Matrix for the first time. Thank you Dr. Strang and thank you MITOCW.
Conclusion: 1. To calculate Ax=0 in other words calculate the null space of A, we can use 'reduce row echelon form' (rref) method. 2. The rank of A equal to the number of pivots or rows after reducing row echelon, notation as r. The column of A equal to the number of variables, notation as n. So n-r is equal to the number of free variables. 3. Consider the solution of Ax=0, if we calculate the reduced row echelon form of A consisting of [T F], the solution matrix will be the transform of [-F T], where T stands for identity matrix and F stands for free matrix. The solution matrix would be n*(n-r) shape.
This is more than a lecture on linear algebra, it's a demo on perfect teaching presentation. His way of pinpointing each question along the way that our brains need to ask and then solve is truly beautiful.
Every student should have at least one professor like Prof Strang. Motivating, illuminating and such great energy. I truly appreciate these classes. Thank you!
This guy is giving me such a good intuitive understanding of linear algebra, rather than just presenting seemingly semi-random algorithms without explanation.
For anyone confused by the block matrix explanation --- the I and F and blocks of zeros --- hang in there until lecture 8 where it all becomes clearer. And yes, F may be interspersed with the I, and, contrary to the top rated answer on Stack Exchange, this cannot be remedied with permutation matrices. Basically it's just a visual cue that allows you to pluck out the relevant numbers.
This is exactly what got me confused at first. I really appreciate professor Strang and MIT for making this gem of a lecture available online, but I felt he presented a few tricks like the block matrix one for finding the spanning set of the null space of a linear map (a linearly independent one*, too, because of that I block) in a rather hand-wavy manner. Perhaps a better way to visualize it is as follows: 1. Draw the RREF matrix as staircases with pivots, preferably with interlaced free columns for generality. 2. If there are any all-zero rows at the bottom of the RREF matrix, trim off that part. 3. Pluck out a free column from the staircase, then turn it sideways (90 degrees counterclockwise.) 4. Multiply each component in the free column by -1, to reverse their signs. (This is for building the -F block) 5. Insert the "selector" (coefficient of 1) component at the same index as the index of the extracted free column in the RREF matrix. 6. Insert "N/A" (coefficient of 0) components at the same indices as the indices of the rest of the free columns in the RREF matrix. 7. Now turn the free column back to its original position (90 degrees clockwise) 8. Put the finished column in the "special solutions" matrix. 9. Do the same with the rest of the free columns in the RREF. 10. In the special cases where F is NOT interspersed with the I in the original RREF matrix, what you get is a matrix with the -F block stacked on top of the I block. P.S. The point of plucking out a free column and then laying it on its side is to make the step 5 and 6 easier to visualize. *If only one column vector has a non-zero entry at a specific row index in a set of columns, then there is no linear combination of the rest of columns in the set that is equal to that column. That is why the special solutions matrix built this way always contains a linearly independent set of columns.
yes, I'm a bit confused what to do with matrices which rreff is something like [ 1 * 0 * 0 [ 0 0 1 * 0 [ 0 0 0 0 1 they are clearly not [I F I will check this answer
@@eulerappearethwhat i've noticed is how switching columns 2 and 3 in this scenario's rref of A (compared to the usual [I F] form), caused the ROWS 2,3 to be switched in the special solutions. so since the solutions would be [-2 0 1 0] and [2 -2 0 1] if the rref was just [I F], then we switch rows 2,3 and get [-2 1 0 0] and [2 0 -1 2] instead. pretty late but i hope this helps somebody
For those like me, who did not get about the free columns and pivot columns fiasco at first. Firstly, note that the free columns are linear combinations of the pivot columns (you can do some scribbling to confirm this). This gives some intuition as to why we can allow the free columns to be scaled freely by any number, then solving for the scalars of the pivot columns, such that you get zero column/vector if you add all these scaled columns. Pivot variables and free variables are the names for those respected scalars. I hope this cleared some doubts...wish you the best of luck.
The second part of the lecture going from Ux = 0 to Rx = 0 and further on to RN = 0, and proposing what is N, seemed to be full of leaps that I could not follow completely. I have done a ML course, and a Neural network course without a deeper knowledge of Linear Algebra. I thought of filling that gap. The rabbit hole seems to go deep, and again I seem to be taking a few magical things that happen to be as axiomatic. I will persist. If I can not get it from Prof Strang, I may not get it at all. Hope the pennies will drop as I move forward, and I will get rich!
Dr. Strang wanted us to realize that the reduced row echelon form of the original matrix consisted of the identity matrix (when only looking at the pivot columns) and some other matrix, which he called F, when only looking at the free columns. He generalized this notion by defining the matrix R using placeholders I and F for the identity matrix (I) and the matrix formed by the free columns (F), with possible rows of 0s beneath. Since he was generalizing, he wrote R as a block matrix (where I and F represent matrices). We know I has dimensions r x r (since I is the identity matrix formed by the pivot columns, and the number of pivot columns = number of pivot variables = rank = r) We know F has dimensions n - r x n - r (since F is the matrix formed by the free columns, and we know there are n - r free columns). So our original Ax = 0 can be rewritten -- throughout the whole process of his lecture -- as Rx = 0. He then wonders what would the solution of this matrix equation would be. Well, since defined R generally using I and F, he unintentionally (I am assuming given how pleasantly surprised he sounded) was defining R as a block matrix, he decided to find all the special solutions at once in which he called a null space matrix N. This N would solve the Rx=0 equation, i.e., would make RN = 0 true. Well, knowing how matrix multiplication works, N needs to be a matrix that, when multiplied with the row(s) of R, would produce 0's. Since the first row of R is I F, what linear combination of I F would = 0? We would need to multiply I by -F, and F by I (because then we'd have -F + F = 0). This is how to look at it pure algorithmically. Dr. Strang actually uses wonderful logic. If the first row of R = [I F], then of course we want I in the free variable row (the second row of N) in order to preserve it, and to cancel it out, of course we would need -F in the identity row (the first row of N) in order to cancel out the F in the free variable block of R. This is how he knows the null space matrix N is always going to be [-F I] (obviously written as a column, but I can't type that out in this comment). He then goes further to show us how this actually is not surprising. Going back to Rx = 0, remember that R (as a block matrix) = [I F]. x = [x_pivot, x_free] (as a column matrix). If we actually did the matrix multiplication we would have: I * x_pivot + F * x_free = 0. Solving for x_pivot we get: x_pivot = -F * x_free So, if in our solution we make our free variables the identity (remember when Dr. Strang said "hey, these are free variables. Let's make them whatever we want. Let's make x_2 = 1 and x_4 = 0" and later he said "hey, let's make x_2 = 0 and x_4 = 1"), then by the above equation, of course x_pivot HAS to be -F.
I believe that Mr. Strang is really amazing and incredible, but I got stuck when he said of "the pivot and free variables" , I mean he told us about the algorithm very well, but I am not able to connect these with that what does it mean to?? Why does non pivot columns can be considered anything?? And what are its effect on my graph if I wish to plot it on?? And what's the concept behind these pivot and free variables, how did it occur from anywhere?? So , if you guys could help me out with this , it would really be appreciated!!
The pivot is the first non-zero entry on a row. If the system has a unique (single) solution, each row will have exactly one non-zero entry when the matrix is in reduced row echelon form, and there won't be any free variables. In the null space example, if each row has exactly one non-zero entry, the only vector that solves the system when the RHS is zero is the zero vector.
You are right about pivots because he did not really explain that. The idea of only two pivots is that columns of this matrix are not linearly independent, bu I agree it should be explained much better and more detailed then in this lecture.
this is the best way possible to describe the rank of a matrix! for so long I have struggled with this concept! And now it feels so rudimentary, so basic! Thank you professor strange for such a fantastic way of explaining things
I've my linear algebra class early in the morning, and I never make it to the class was frustrated to catch up all this stuff but watching these videos are helping me so much Sincerely thank professor Strang and this channel!
Seriously the best thing that I could have found on the Internet. Too bad my final is in 4 days. Naturally I will be staying on youtube for quite a few hours this week
learning linear algebra with you is like watching movies. it's fascinating, exciting, convincing and fun. thank you so much Professor Strang! Im so lucky to be learning this subject with you!
Indeed! It's like a story - with characters, and plot, and plot twists...Mr Strang is a shining example of what education should be - accessible, engaging and with the sense of disccovery!
I was studying for my engineering degree when this was filmed. I just wish I was having professors like Dr. Strang and Dr. Lewin. Clear cut and practical explanations of the most abstruct branch of Mathematics!
9:11 "I can assign anything that I like for X2 and X4..." So, what's stopping us from choosing the free variables as X2 and X3? Because, it seems clear from the equations that they can be assigned any value arbitrarily. Somebody!
at 26:13 I dont get how he can switch column 2 and column 3 to get the identity matrix in the first block of [ I F] ? You cant chage the order of pivot columns just like that ? PLEASE answer this for me someone! 1love
I've never understood null space, rref, and how null basis is immediate from rref better. I'd recommend Dr. Strang to anyone that tries to learn linear algebra.
I just read the latest edition of the book for this course and it is brilliant , the best Linear Algebra textbook! Thank you Dr. Gilbert Strang! But I like the cover of older edition with the houses being transformed. Is there any software for online hw for the book? Please let me know.
In the original example we treated used all of the identity elements (1,0;0,1) in the solution (x), but in the transpose we just reduce it to "1". Why?
The identity Matrix of a 1x1 Matrix is just 1. Since the result was a 1x3 and not a 4x2, the identity part only had a 1x1 spot in the result, thus it just gives 1.
Why does my university not allow for students to record the lectures? It is so good to have those at home in video format. You can re-watch them and rewind time anytime you missed something because you weren't paying attention. Mighty helpful.
i think Iam having some trouble with the terminology here..so suppose we have 5 eqs and 10 unknowns, we know that null space will be a subspace of R^10..do we say its a subspace of 10 dimensional?..also let say all the 5 vectors are independent, so they span like 5 dimensions out of 10?? i mean idk
You can also use this algorithm to get the cross product of two vectors in R3. Solve Ax=0 for 2x3 matrix. The dotproduct of each rows and x will be 0. Can you derive the cross product matrix from this algorithm?
I study at a german uni and everything is so god damn formal I couldnt fathom up unti today why the core plus the rank is equal the number of culmns thank you mit opencourseware abd thank you dr. Strang
didn't get the last bit before seeing it 5 times. Now I see that he re-orders the columns of A (now called R), in a way that the x vector is now x = , so if you reorder cols, you reorder x, and then is just remembering things. When you do this, R is [I F] and x = [pivs, frees] so then pivs = -F frees N is just a way to condense all solutions. RN = 0
They are free because they can be chosen to be anything you like and you can still find a solution. I think it's simplest to see why this is the case in reduced echelon form, as opposed to echelon form, as I'll explain: Remember that we're just solving a bunch of coupled linear equations represented by Ax = 0. Around 9:57 he shows you how the matrix equation Ax = 0 is just encoding these linear equations. So we start off with a matrix A that has numbers everywhere in it, which means x_1, x_2, x_3, and x_4 appear in all three equations. We can add these equations together and multiply them by numbers without changing the solution set, which is what row operations do. However, by doing this we can simplify the equations so they're easier to solve. For instance, we started with three equations but after doing row operations found we really only had two. So far so good. The most you can simplify the equations with row operations is called "reduced echelon form", which he starts talking about at 19:02. Notice that, unlike at the start, there are two variables, x_1 and x_3, that now appear in ONLY ONE equation each. We call such variables "pivot variables". Appearing in only one equation each makes them special because it allows us to obtain solutions in the following way: imagine starting off with the values of the all the variables at 0, so the equations are balanced. Now make the rest of the variables (x_2 and x_4) to be whatever you want, which will make each equation no longer equal to 0 as it should be. But then, since the pivot variables only show up in one equation each, you can then tune them individually to re-balance each equation again to 0. To find out what to tune them to, you just solve for them in terms of the non-pivot variables. Essentially, appearing in only one equation each allows you to "undo" the mess you made by choosing the other variables to be whatever you want. This imaginary process will clearly always work, so we can always choose the non-pivot variables to be whatever we want, and hence they are "free". Hope this helps!
@@physicsguy877 well, after deducing the equations from reduced row echelon form I can also set the coefficient of the pivot columns to a random number and still find a solution. For example 5《1 0 0》 + x_2《2 0 0》 + 7《0 2 0》 + x_4《-2 4 0》=《0 0 0》 X_4 = -14/4 , X_2 = 1
@@makecocgreatagain-g2c that's not surprising. If I have an equation x_1 + x_2 = 0, I can choose x_2 to be anything, and then solve for x_1, or I can choose x_1 to be anything, and solve for x_2. The important thing is that I have only one degree of freedom to play with, so the solution is a line. With multiple equations, it can be harder to tell how many degrees of freedom you actually have because any given variable can appear in more than one equation, so by choosing some set of variables to be free may be fine in one equation but break another. Deciding that the leading non-zero entries in a row denote the pivot variables and everything else is free will always work though, by the reasoning I gave you above. Of course, you could have chosen different variables to be "free", but you'd need to be careful. If you like, you could imagine first interchanging the columns of the matrix, which doesn't change the solution set (x_1 + x_2 = x_2 + x_1). Then you could follow the same algorithm Strang gives you, and as long as you know that the n'th column no longer denotes x_n, you'd find some of what you used to call pivot variables are now free and what you used to call free are now pivot. Note however that the number of free variables and pivot variables will never change, and it isn't always possible to change one to another (for example x_5 = 0 is never free because it's always 0). The important point is that the number of independent solutions is invariant no matter how you choose free and pivot. The number of free variables both represent the number of independent solutions and the number of numbers necessary to specify any particular solution. When you get further you'll learn that this is equivalent to saying the dimension of the null space is equal to the number of free variables.
I think free variables & pivot variables are used as a standardization technique.Like if you and me got a set of dependent matrices how we will compare our solution?I think free variables would naturally be the least number of random numbers that we have to input.
Why are free variables chosen for assuming 0 & 1 to solve the equation, why not pivot. If I do, I am getting a different null space & its valid null space.Can someone help to understand why only free variables are chosen why cant we continue with same process with pivot variables to be 0&1 and hence obtain more solutions
The part I found confusing is when we write [I F]*[-F,I] = 0 F and -F are the same dimension but the identity on the left hand side is rxr and on the right hand side (n-r)x(n-r), Thinking it through, it makes sense. The RHS has the same number of columns as the number of free variables. It's just a little unusual to see the same letter on both sides meaning slightly different things.
Just wanted to say that blocking the rref matrix into [[I F], [0...]] form and solving for the nullspace matrix like that is one of the greatest things I've ever seen. It seems like it shouldn't work because F could have different shape than I, but it does. And it generalizes to when F doesn't exist, which helps you remember the ideas in the next lecture.
F will have the same number of rows as I, but maybe not the same number of columns. So to make N, you just put -F on top and fill in the bottom with the identity matrix of the correct size (the number of columns of F). So say I is m by m and F is m by n, then N will have (m + n) rows and R will have (m + n) columns, so it works out. And each block multiplication (I * -F and F * I) also work out.
Yes. So the dimension of the identity matrix in R is not the same as the dimension of the identity matrix in N. And the sum of the dimensions of these identity matrix should be equal to the number of columns in A.
Can't I sub 0 and 1 into the pivot variables just as easily as into the free variables? It just gives new vectors and all these vectors are just linear combinations.
One thing I didn't understand. When he solved for A transpose ..2nd row becomes 0 because it's dependent on 1st row ..4th row also became 0 but it doesn't depend on any other row.. he said that row of 0's means that original row is a combination of other rows ..but 4th row is not a combination so why it became 0??? Please help if anyone knows the answer!
So the independent columns are associated with dependent variables whereas dependent columns give us free variables that can be arbitrarily assigned in the equation.
In the last prob we had one free variable and 2 pivot variables, but while concluding XN=0, indicating N = [ -F , I] where in two numbs were filled by free variables each as "-1" and the last unknown X3 = 1 !! Can anyone explain me ab this??
+MIT OpenCourseWare good day, how is that book called which they use for solving problems ? How could we get that books with theory and practics ? Is it possible? Could you share with us? Thank you so much for these videos
Why should we choose only between zero and one for the free variables? Can we not choose any set of variables for these? Is it impossible for these free variables to be any thing else? If I choose 0, and 0 as the free variables, I get [1, 1, -2. 1 ]T as the solution in the case under discussion, and that fits too. However, as suggested, that becomes a combination of these special solutions. Ok. Would it always happen to be this way, irrespective of which free variable I choose? For example, if free variables were to be 1 and 1, it does not fit a solution.
If you think of each row as a polynomial equation, then the zero pivots mean the coefficients of those variables are zero. So it doesn't matter what value you assign the variable, it doesn't change the result. I think of it this way. Imagine a straight line going through the X-Y plane of your computer screen. ax + by = d. Then imagine that line in 3-D (X-Y-Z) space. What is the z-coefficient? The z-coefficient is zero. a*x + b*y + 0*z = d. So it doesn't matter what value I use for the variable z because the z-coefficient is zero, and I get the same straight line that never leaves the X-Y plane. He has an equation with zero coefficients for two of the variables, so he can put any value for those variables without changing the result. So he just picks the values that make his calculations easier. You could use other values for the special solution, but it won't change the result because the coefficients are zero. So he picks 1 for one of the zero pivot variables to make his calculations easier by yielding the identity matrix, and he picks zero for the rest of the zero-pivot variables just to get rid of them. I think if he had three zero pivots, then he'd pick [1,0,0] or a variation of that. For four zero pivots, he'd pick [1,0,0,0], and so forth. One value for the special solution would be 1 to get the identity matrix, and all the rest would be zeros. I've gotten that far, but I still don't get what happens after he has the identity matrix.
Why he is talking about it is he wants us to understand on a deeper level so we can apply the concepts creatively. As for what he is talking about, I have to listen to each lecture several times.
Can someone please explain to me, Why do we get all the Null space of A just by taking all the combinations of only two special solutions. What is the proof of this?. Shouldn't we take all possibilities of special solutions.
@Janmejay Singh I think because there is 2 free variables. One special solution To change first free variable and second special solution to chge second free variable. As in next example free variable is 1 so only 1 special solution(vector) needed.
I believe all special solutions are linear combinations of the two special solutions obtained with (0,1) and (1,0) for X2 and X4. Therefore the total Null space can be formed by the linear combination of the the special solutions discussed by the professor (we don't need any other special solutions). For example, try solving the problem with a special solution X2=3 and X4=2. This will yield X1=-2 and X3=-4, or X = (-2, 3, -4, 2). This solution is a linear combination of the two special solutions derived by the professor: (-2, 3, -4, 2) = 3*(-2, 1, 0, 0) + 2*(2, 0, -2, 1). I also verified another special solution with X2=10 and X4=-11. The final solution is also a linear combination of the two special solutions (-2,1,0,0) and (2,0,-2,1). Actually, if we think about it any X2 can be formed by some linear combinations of 1 and 0, similarly any X4 can be formed by some linear combinations of 0 and 1. Therefore, we don't need any more special solutions to determine the Null space. I hope I got my arithmetic right...but please do the calculations yourself and let me know. Thanks.
That's the beauty of setting all but one of the free variables to zero in each special solution. That way, each respective special solution can represent any possible value for its respective free variable by multiplying. If you set x2 to 1 and all the other free variables to zero, then you can multiply that special solution by 7 to set x2 to 7 in another possible solution. Since you've set x2 to 0 in all the other special solutions, it doesn't matter what you multiply those special solutions by. When you combine all the special solutions, you will get x2 = 7, or any other possibility. You do the same for the respective free variable in the other special solutions to get any possible value for that free variable. So from the special solutions, all possible solutions can be derived. The key is that each special solution can return all possible values for its respective free variable set to 1. Just to clarify the terminology, I wouldn't say "Shouldn't we take all possibilities of special solutions." I'd say "Shouldn't we take all possibilities of solutions." That's exactly what you get. You can get any possible solution by multiplying the respective special solution for that respective free variable by whatever, and combining all the scaled special solutions into one linear combination.
Its kind of cool and odd that someone who has taught this subject for so long can keep it so fresh...like he's stumbling across the Null Space Matrix for the first time. Thank you Dr. Strang and thank you MITOCW.
Well said.
one is 9 yrs ago and the other one is 9 months ago lol
@@tianjoshua4079
I wonder if the couple at 15:03 in the second row is still together.
lol
i was noticing them
Why don't we ask them?
Makes you feel like you're in the classroom even more...
Exactly haha they’re in most of the lectures
infinite blackboards...
zhen de ei
domain expansion typa shit
long live professor strang!
May glory come to him.
given how old he looks now I suppose you're right
mit has high quality of blackboard
+YOUWEI QIN I thought the same thing. There's just like infinite sliding blackboards stacked on top of each other :)
quantity?
MIT has everything of a very high quality! Its such a pity that you only noticed the blackboards!
Agree, no body mentions the chalks.
Conclusion: 1. To calculate Ax=0 in other words calculate the null space of A, we can use 'reduce row echelon form' (rref) method.
2. The rank of A equal to the number of pivots or rows after reducing row echelon, notation as r.
The column of A equal to the number of variables, notation as n.
So n-r is equal to the number of free variables.
3. Consider the solution of Ax=0, if we calculate the reduced row echelon form of A consisting of [T F], the solution matrix will be the transform of [-F T], where T stands for identity matrix and F stands for free matrix.
The solution matrix would be n*(n-r) shape.
bro i think its I instead of T for Identity matrix
@@muhammadwahajkhalil6577 it just notion we can use any variable to represent identity matrix for us convince it is not mandatory to stick with I
This is more than a lecture on linear algebra, it's a demo on perfect teaching presentation. His way of pinpointing each question along the way that our brains need to ask and then solve is truly beautiful.
When you think there is no more sliding boards 33:16
Truly agree. It's quite impressing how MIT has so many sliding boards... the # of blackboards in MIT is INFINITE. LOL
Every student should have at least one professor like Prof Strang. Motivating, illuminating and such great energy. I truly appreciate these classes. Thank you!
This guy has figured out how to access the 12 dimension. Infinite chalkboards; some crazy wizardry shit.
The last thing he wrote at the end of the lecture was "FIN"... like the end of an old-fashioned French film.
I thought it was funny.
readap427
Or Spanish film.
Both from Latin
fin means the end
Does anyone else feel a nice smooth buttery feeling when the chalk glides against the board?
This guy is giving me such a good intuitive understanding of linear algebra, rather than just presenting seemingly semi-random algorithms without explanation.
I literally want to cry after watching this. Thank you so much for saving my ass.
Have u observed
First lecture got Million's of views
And views count is slowly reduced from video to video
It's fun pausing the video and trying to figure out how the process ends before he's shown it ...
I took linear algebra 30 years ago and I thought it was pretty hard at the time. Prof. Strang makes it easy!
For anyone confused by the block matrix explanation --- the I and F and blocks of zeros --- hang in there until lecture 8 where it all becomes clearer. And yes, F may be interspersed with the I, and, contrary to the top rated answer on Stack Exchange, this cannot be remedied with permutation matrices. Basically it's just a visual cue that allows you to pluck out the relevant numbers.
This is exactly what got me confused at first. I really appreciate professor Strang and MIT for making this gem of a lecture available online, but I felt he presented a few tricks like the block matrix one for finding the spanning set of the null space of a linear map (a linearly independent one*, too, because of that I block) in a rather hand-wavy manner.
Perhaps a better way to visualize it is as follows:
1. Draw the RREF matrix as staircases with pivots, preferably with interlaced free columns for generality.
2. If there are any all-zero rows at the bottom of the RREF matrix, trim off that part.
3. Pluck out a free column from the staircase, then turn it sideways (90 degrees counterclockwise.)
4. Multiply each component in the free column by -1, to reverse their signs. (This is for building the -F block)
5. Insert the "selector" (coefficient of 1) component at the same index as the index of the extracted free column in the RREF matrix.
6. Insert "N/A" (coefficient of 0) components at the same indices as the indices of the rest of the free columns in the RREF matrix.
7. Now turn the free column back to its original position (90 degrees clockwise)
8. Put the finished column in the "special solutions" matrix.
9. Do the same with the rest of the free columns in the RREF.
10. In the special cases where F is NOT interspersed with the I in the original RREF matrix, what you get is a matrix with the -F block stacked on top of the I block.
P.S. The point of plucking out a free column and then laying it on its side is to make the step 5 and 6 easier to visualize.
*If only one column vector has a non-zero entry at a specific row index in a set of columns, then there is no linear combination of the rest of columns in the set that is equal to that column. That is why the special solutions matrix built this way always contains a linearly independent set of columns.
This really bothered me. The block presentation wasn't exactly blocked. But I'll stick with it.
yes, I'm a bit confused what to do with matrices which rreff is something like
[ 1 * 0 * 0
[ 0 0 1 * 0
[ 0 0 0 0 1
they are clearly not [I F
I will check this answer
@@eulerappearethwhat i've noticed is how switching columns 2 and 3 in this scenario's rref of A (compared to the usual [I F] form), caused the ROWS 2,3 to be switched in the special solutions. so since the solutions would be [-2 0 1 0] and [2 -2 0 1] if the rref was just [I F], then we switch rows 2,3 and get [-2 1 0 0] and [2 0 -1 2] instead. pretty late but i hope this helps somebody
For those like me, who did not get about the free columns and pivot columns fiasco at first.
Firstly, note that the free columns are linear combinations of the pivot columns (you can do some scribbling to confirm this).
This gives some intuition as to why we can allow the free columns to be scaled freely by any number, then solving for the scalars of the pivot columns, such that you get zero column/vector if you add all these scaled columns.
Pivot variables and free variables are the names for those respected scalars.
I hope this cleared some doubts...wish you the best of luck.
thank you so much for clearing this doubt.
For a second, I'm always surprised that people don't clap at the end of the lecture...
I know!!😂 He's the best professor I know.
The second part of the lecture going from Ux = 0 to Rx = 0 and further on to RN = 0, and proposing what is N, seemed to be full of leaps that I could not follow completely. I have done a ML course, and a Neural network course without a deeper knowledge of Linear Algebra. I thought of filling that gap. The rabbit hole seems to go deep, and again I seem to be taking a few magical things that happen to be as axiomatic. I will persist. If I can not get it from Prof Strang, I may not get it at all. Hope the pennies will drop as I move forward, and I will get rich!
Any updates?
same issue with me
Dr. Strang wanted us to realize that the reduced row echelon form of the original matrix consisted of the identity matrix (when only looking at the pivot columns) and some other matrix, which he called F, when only looking at the free columns.
He generalized this notion by defining the matrix R using placeholders I and F for the identity matrix (I) and the matrix formed by the free columns (F), with possible rows of 0s beneath. Since he was generalizing, he wrote R as a block matrix (where I and F represent matrices).
We know I has dimensions r x r (since I is the identity matrix formed by the pivot columns, and the number of pivot columns = number of pivot variables = rank = r)
We know F has dimensions n - r x n - r (since F is the matrix formed by the free columns, and we know there are n - r free columns).
So our original Ax = 0 can be rewritten -- throughout the whole process of his lecture -- as Rx = 0. He then wonders what would the solution of this matrix equation would be.
Well, since defined R generally using I and F, he unintentionally (I am assuming given how pleasantly surprised he sounded) was defining R as a block matrix, he decided to find all the special solutions at once in which he called a null space matrix N.
This N would solve the Rx=0 equation, i.e., would make RN = 0 true.
Well, knowing how matrix multiplication works, N needs to be a matrix that, when multiplied with the row(s) of R, would produce 0's.
Since the first row of R is I F, what linear combination of I F would = 0? We would need to multiply I by -F, and F by I (because then we'd have -F + F = 0).
This is how to look at it pure algorithmically. Dr. Strang actually uses wonderful logic. If the first row of R = [I F], then of course we want I in the free variable row (the second row of N) in order to preserve it, and to cancel it out, of course we would need -F in the identity row (the first row of N) in order to cancel out the F in the free variable block of R.
This is how he knows the null space matrix N is always going to be [-F I] (obviously written as a column, but I can't type that out in this comment).
He then goes further to show us how this actually is not surprising. Going back to Rx = 0, remember that R (as a block matrix) = [I F]. x = [x_pivot, x_free] (as a column matrix).
If we actually did the matrix multiplication we would have:
I * x_pivot + F * x_free = 0. Solving for x_pivot we get:
x_pivot = -F * x_free
So, if in our solution we make our free variables the identity (remember when Dr. Strang said "hey, these are free variables. Let's make them whatever we want. Let's make x_2 = 1 and x_4 = 0" and later he said "hey, let's make x_2 = 0 and x_4 = 1"), then by the above equation, of course x_pivot HAS to be -F.
@@toanvo2829 F has dimensions r x n - r (NOT n - r x n -r), no?
Anyone understand the equation at 32:15? I think x_free should be above x_pivot?
15:06 lovebirds
lol would be annoying af if sittin behind em.
i still do not understand how it works , F and I definitely can have different shapes ? This part is not clear from the video.
I believe that Mr. Strang is really amazing and incredible, but I got stuck when he said of "the pivot and free variables" , I mean he told us about the algorithm very well, but I am not able to connect these with that what does it mean to?? Why does non pivot columns can be considered anything?? And what are its effect on my graph if I wish to plot it on?? And what's the concept behind these pivot and free variables, how did it occur from anywhere??
So , if you guys could help me out with this , it would really be appreciated!!
same here! been listening to this part again and again, but having hard time understanding the logic behind pivot and free variables
The pivot is the first non-zero entry on a row. If the system has a unique (single) solution, each row will have exactly one non-zero entry when the matrix is in reduced row echelon form, and there won't be any free variables. In the null space example, if each row has exactly one non-zero entry, the only vector that solves the system when the RHS is zero is the zero vector.
You are right about pivots because he did not really explain that. The idea of only two pivots is that columns of this matrix are not linearly independent, bu I agree it should be explained much better and more detailed then in this lecture.
He is the best teacher i've ever had. How can i get in touch with him? Please!!! Thank you so much.
+hoan huynh See his department page for contact information: www-math.mit.edu/~gs/
this is the best way possible to describe the rank of a matrix! for so long I have struggled with this concept! And now it feels so rudimentary, so basic! Thank you professor strange for such a fantastic way of explaining things
can someone please explain how he got from R = [1 2 0 -2; 0 0 1 2; 0 0 0 0] to R = [I F; 0 0] ?
I've my linear algebra class early in the morning, and I never make it to the class
was frustrated to catch up all this stuff but watching these videos are helping me so much
Sincerely thank professor Strang and this channel!
I have never seen teacher like u.....ur way of teaching and clearing the concepts of students is amazing sir..
...
36:20 "I quit without trying, I shouldn't have done that." So true
hahahahahah me with linear algebra 2 years ago. will get a 10 now easy
Seriously the best thing that I could have found on the Internet. Too bad my final is in 4 days. Naturally I will be staying on youtube for quite a few hours this week
learning linear algebra with you is like watching movies. it's fascinating, exciting, convincing and fun. thank you so much Professor Strang! Im so lucky to be learning this subject with you!
Indeed! It's like a story - with characters, and plot, and plot twists...Mr Strang is a shining example of what education should be - accessible, engaging and with the sense of disccovery!
Yes! this is what I felt as I was watching! And I felt that I was as happy as I would be watching a favourite movie.
Is this the only lecture of the series in which he wears a different colored shirt (sky blue in rest all)😉?
There's a lot of magic going on here that Dr. Strang doesn't state explicitly. It makes this lecture worth a couple listen throughs.
definitely more than a couple. I dont know why people are saying its magical
A magician telling all his secret tricks...
It is mind blowing how elegant linear algebra really is
I was studying for my engineering degree when this was filmed. I just wish I was having professors like Dr. Strang and Dr. Lewin. Clear cut and practical explanations of the most abstruct branch of Mathematics!
I LOVE THIS GUY
9:11 "I can assign anything that I like for X2 and X4..."
So, what's stopping us from choosing the free variables as X2 and X3? Because, it seems clear from the equations that they can be assigned any value arbitrarily.
Somebody!
Mit has done wonderful job to give us quality education for free veryyyyy thanks
at 26:13 I dont get how he can switch column 2 and column 3 to get the identity matrix in the first block of [ I F] ? You cant chage the order of pivot columns just like that ? PLEASE answer this for me someone! 1love
if you changed the order of pivot columsn you would also have to change the order of the variables in the solution vector
I've never understood null space, rref, and how null basis is immediate from rref better.
I'd recommend Dr. Strang to anyone that tries to learn linear algebra.
I just read the latest edition of the book for this course and it is brilliant , the best Linear Algebra textbook! Thank you Dr. Gilbert Strang! But I like the cover of older edition with the houses being transformed. Is there any software for online hw for the book? Please let me know.
When we write X as [ -F I] T, the number of cols in F is n-r and number of cols in I is r so wont there be a condition for n-r=r??? Confused, pls help
enjoying these lectures tremendously - cant say I expected to find linear algebra that interesting
W. Gilbert Strang, you are a gem of a teacher! Thank you so very much!!
I'm in love with these lecs
No way there is anyone can explain linear algebra like professor Strang!
every math loving student would love this great man
what is pivot row
34:13 how many pivot variableS?
In the original example we treated used all of the identity elements (1,0;0,1) in the solution (x), but in the transpose we just reduce it to "1". Why?
The identity Matrix of a 1x1 Matrix is just 1. Since the result was a 1x3 and not a 4x2, the identity part only had a 1x1 spot in the result, thus it just gives 1.
how can i find the number of pivot in a matrix ?
Why does my university not allow for students to record the lectures? It is so good to have those at home in video format. You can re-watch them and rewind time anytime you missed something because you weren't paying attention. Mighty helpful.
i think Iam having some trouble with the terminology here..so suppose we have 5 eqs and 10 unknowns, we know that null space will be a subspace of R^10..do we say its a subspace of 10 dimensional?..also let say all the 5 vectors are independent, so they span like 5 dimensions out of 10?? i mean idk
You can also use this algorithm to get the cross product of two vectors in R3. Solve Ax=0 for 2x3 matrix. The dotproduct of each rows and x will be 0.
Can you derive the cross product matrix from this algorithm?
The way he connects the flow of ideas..........
26:31 even the ghost gets mindblowned
I study at a german uni and everything is so god damn formal I couldnt fathom up unti today why the core plus the rank is equal the number of culmns
thank you mit opencourseware abd thank you dr. Strang
i did not get the 23:52
19:54 "Let me suppose I got as far as u" lol
Transpose([0 1 -2 1])
is also nullspace....why didn't he include that??
17:13 how many free variable
Love you Prof. Strang.....I am beginning to fall in love with Linear Algebra....You are a genius Prof. Strang....
a great prof.. abstract maths can so easily taught .... Its amazing...... Great ..... hats off to u
U should come up with simillar lect in analysis
Strang loses me toward the end with the [ F N ] matrix and the flipping of signs etc.
Using Null matrics the sound waves can be converted as no sound domain as zero power of sound as switch application in hearing aids.
all of a sudden i'm heavily interested in linear algebra and math
didn't get the last bit before seeing it 5 times.
Now I see that he re-orders the columns of A (now called R), in a way that the x vector is now x = , so if you reorder cols, you reorder x, and then is just remembering things.
When you do this, R is [I F] and x = [pivs, frees] so then pivs = -F frees
N is just a way to condense all solutions. RN = 0
just hope you remember the minus F on the test.
Wow!
How are those variables “FREE”? Can someone explain
They are free because they can be chosen to be anything you like and you can still find a solution. I think it's simplest to see why this is the case in reduced echelon form, as opposed to echelon form, as I'll explain:
Remember that we're just solving a bunch of coupled linear equations represented by Ax = 0. Around 9:57 he shows you how the matrix equation Ax = 0 is just encoding these linear equations.
So we start off with a matrix A that has numbers everywhere in it, which means x_1, x_2, x_3, and x_4 appear in all three equations. We can add these equations together and multiply them by numbers without changing the solution set, which is what row operations do. However, by doing this we can simplify the equations so they're easier to solve. For instance, we started with three equations but after doing row operations found we really only had two. So far so good.
The most you can simplify the equations with row operations is called "reduced echelon form", which he starts talking about at 19:02. Notice that, unlike at the start, there are two variables, x_1 and x_3, that now appear in ONLY ONE equation each. We call such variables "pivot variables". Appearing in only one equation each makes them special because it allows us to obtain solutions in the following way: imagine starting off with the values of the all the variables at 0, so the equations are balanced. Now make the rest of the variables (x_2 and x_4) to be whatever you want, which will make each equation no longer equal to 0 as it should be. But then, since the pivot variables only show up in one equation each, you can then tune them individually to re-balance each equation again to 0. To find out what to tune them to, you just solve for them in terms of the non-pivot variables. Essentially, appearing in only one equation each allows you to "undo" the mess you made by choosing the other variables to be whatever you want. This imaginary process will clearly always work, so we can always choose the non-pivot variables to be whatever we want, and hence they are "free".
Hope this helps!
@@physicsguy877 well, after deducing the equations from reduced row echelon form I can also set the coefficient of the pivot columns
to a random number and still find a solution.
For example 5《1 0 0》 + x_2《2 0 0》 + 7《0 2 0》 + x_4《-2 4 0》=《0 0 0》
X_4 = -14/4 , X_2 = 1
@@makecocgreatagain-g2c that's not surprising. If I have an equation x_1 + x_2 = 0, I can choose x_2 to be anything, and then solve for x_1, or I can choose x_1 to be anything, and solve for x_2. The important thing is that I have only one degree of freedom to play with, so the solution is a line.
With multiple equations, it can be harder to tell how many degrees of freedom you actually have because any given variable can appear in more than one equation, so by choosing some set of variables to be free may be fine in one equation but break another. Deciding that the leading non-zero entries in a row denote the pivot variables and everything else is free will always work though, by the reasoning I gave you above. Of course, you could have chosen different variables to be "free", but you'd need to be careful. If you like, you could imagine first interchanging the columns of the matrix, which doesn't change the solution set (x_1 + x_2 = x_2 + x_1). Then you could follow the same algorithm Strang gives you, and as long as you know that the n'th column no longer denotes x_n, you'd find some of what you used to call pivot variables are now free and what you used to call free are now pivot. Note however that the number of free variables and pivot variables will never change, and it isn't always possible to change one to another (for example x_5 = 0 is never free because it's always 0).
The important point is that the number of independent solutions is invariant no matter how you choose free and pivot. The number of free variables both represent the number of independent solutions and the number of numbers necessary to specify any particular solution. When you get further you'll learn that this is equivalent to saying the dimension of the null space is equal to the number of free variables.
I think free variables & pivot variables are used as a standardization technique.Like if you and me got a set of dependent matrices how we will compare our solution?I think free variables would naturally be the least number of random numbers that we have to input.
@@physicsguy877 understood .Thank you very much sir.
Why are free variables chosen for assuming 0 & 1 to solve the equation, why not pivot. If I do, I am getting a different null space & its valid null space.Can someone help to understand why only free variables are chosen why cant we continue with same process with pivot variables to be 0&1 and hence obtain more solutions
The part I found confusing is when we write [I F]*[-F,I] = 0 F and -F are the same dimension but the identity on the left hand side is rxr and on the right hand side (n-r)x(n-r),
Thinking it through, it makes sense. The RHS has the same number of columns as the number of free variables. It's just a little unusual to see the same letter on both sides meaning slightly different things.
Thank you! This helped enormously! And 9 years later!
34:00 If rank(A) = 3, then U*x = 0 has only trivial solution. But A*(-1, -1, 1)^T = 0. So rank(A) is not equal to 3.
Just wanted to say that blocking the rref matrix into [[I F], [0...]] form and solving for the nullspace matrix like that is one of the greatest things I've ever seen. It seems like it shouldn't work because F could have different shape than I, but it does. And it generalizes to when F doesn't exist, which helps you remember the ideas in the next lecture.
F will have the same number of rows as I, but maybe not the same number of columns. So to make N, you just put -F on top and fill in the bottom with the identity matrix of the correct size (the number of columns of F). So say I is m by m and F is m by n, then N will have (m + n) rows and R will have (m + n) columns, so it works out. And each block multiplication (I * -F and F * I) also work out.
Yes. So the dimension of the identity matrix in R is not the same as the dimension of the identity matrix in N. And the sum of the dimensions of these identity matrix should be equal to the number of columns in A.
Can't I sub 0 and 1 into the pivot variables just as easily as into the free variables? It just gives new vectors and all these vectors are just linear combinations.
Are you guys talking later about the Ker and Im of a matrix ?
One thing I didn't understand. When he solved for A transpose ..2nd row becomes 0 because it's dependent on 1st row ..4th row also became 0 but it doesn't depend on any other row.. he said that row of 0's means that original row is a combination of other rows ..but 4th row is not a combination so why it became 0??? Please help if anyone knows the answer!
@ 32:10 X subscript "pivot" and X subscript "free" are being treated as submatrices to enable block multiplication. Hope I'm right
This lecture taught the algorithm but did not explain or prove why it will work... Anybody know where I can find the proof?
You mentioned that Null space =[-F I]^T. But what about X=zero vector? Because [-F I]^T does not cover X=zero vector. Explain this please
Awesome, i´m glad I found this videos =D
So the independent columns are associated with dependent variables whereas dependent columns give us free variables that can be arbitrarily assigned in the equation.
In the last prob we had one free variable and 2 pivot variables, but while concluding XN=0, indicating N = [ -F , I] where in two numbs were filled by free variables each as "-1" and the last unknown X3 = 1 !! Can anyone explain me ab this??
a great prof.. abstract maths can so easily taught .... Its amazing...... Great ..... hats off to u
U should come up with simillar lect in analysis
non-trivial solutions doesn't exist for nxn matrix if their columns are linearly independent
correct
just saw a funny way to write "8" lol
+MIT OpenCourseWare
good day, how is that book called which they use for solving problems ? How could we get that books with theory and practics ? Is it possible? Could you share with us?
Thank you so much for these videos
Why should we choose only between zero and one for the free variables? Can we not choose any set of variables for these? Is it impossible for these free variables to be any thing else? If I choose 0, and 0 as the free variables, I get [1, 1, -2. 1 ]T as the solution in the case under discussion, and that fits too. However, as suggested, that becomes a combination of these special solutions. Ok. Would it always happen to be this way, irrespective of which free variable I choose? For example, if free variables were to be 1 and 1, it does not fit a solution.
If you think of each row as a polynomial equation, then the zero pivots mean the coefficients of those variables are zero. So it doesn't matter what value you assign the variable, it doesn't change the result.
I think of it this way. Imagine a straight line going through the X-Y plane of your computer screen. ax + by = d. Then imagine that line in 3-D (X-Y-Z) space. What is the z-coefficient? The z-coefficient is zero. a*x + b*y + 0*z = d. So it doesn't matter what value I use for the variable z because the z-coefficient is zero, and I get the same straight line that never leaves the X-Y plane. He has an equation with zero coefficients for two of the variables, so he can put any value for those variables without changing the result.
So he just picks the values that make his calculations easier. You could use other values for the special solution, but it won't change the result because the coefficients are zero. So he picks 1 for one of the zero pivot variables to make his calculations easier by yielding the identity matrix, and he picks zero for the rest of the zero-pivot variables just to get rid of them.
I think if he had three zero pivots, then he'd pick [1,0,0] or a variation of that. For four zero pivots, he'd pick [1,0,0,0], and so forth. One value for the special solution would be 1 to get the identity matrix, and all the rest would be zeros.
I've gotten that far, but I still don't get what happens after he has the identity matrix.
Is he just making this up?I have no idea what he is talking about, or why he is talking about it
Why he is talking about it is he wants us to understand on a deeper level so we can apply the concepts creatively. As for what he is talking about, I have to listen to each lecture several times.
yeah mr.white, yeah science
If Professor Strang were to break bad...imagine him launching all the nukes on earth because you look at him wrong.
At 32:10 shouldn't the x be [ xfree xpiviot]? So that xfree + F * xpivot = 0.
why x-free is identity?
Can someone please explain to me, Why do we get all the Null space of A just by taking all the combinations of only two special solutions. What is the proof of this?. Shouldn't we take all possibilities of special solutions.
@Janmejay Singh
I think because there is 2 free variables. One special solution
To change first free variable and second special solution to chge second free variable.
As in next example free variable is 1 so only 1 special solution(vector) needed.
I believe all special solutions are linear combinations of the two special solutions obtained with (0,1) and (1,0) for X2 and X4. Therefore the total Null space can be formed by the linear combination of the the special solutions discussed by the professor (we don't need any other special solutions).
For example, try solving the problem with a special solution X2=3 and X4=2. This will yield X1=-2 and X3=-4, or X = (-2, 3, -4, 2). This solution is a linear combination of the two special solutions derived by the professor: (-2, 3, -4, 2) = 3*(-2, 1, 0, 0) + 2*(2, 0, -2, 1).
I also verified another special solution with X2=10 and X4=-11. The final solution is also a linear combination of the two special solutions (-2,1,0,0) and (2,0,-2,1).
Actually, if we think about it any X2 can be formed by some linear combinations of 1 and 0, similarly any X4 can be formed by some linear combinations of 0 and 1. Therefore, we don't need any more special solutions to determine the Null space.
I hope I got my arithmetic right...but please do the calculations yourself and let me know. Thanks.
That's the beauty of setting all but one of the free variables to zero in each special solution. That way, each respective special solution can represent any possible value for its respective free variable by multiplying. If you set x2 to 1 and all the other free variables to zero, then you can multiply that special solution by 7 to set x2 to 7 in another possible solution. Since you've set x2 to 0 in all the other special solutions, it doesn't matter what you multiply those special solutions by. When you combine all the special solutions, you will get x2 = 7, or any other possibility. You do the same for the respective free variable in the other special solutions to get any possible value for that free variable. So from the special solutions, all possible solutions can be derived.
The key is that each special solution can return all possible values for its respective free variable set to 1.
Just to clarify the terminology, I wouldn't say "Shouldn't we take all possibilities of special solutions." I'd say "Shouldn't we take all possibilities of solutions." That's exactly what you get. You can get any possible solution by multiplying the respective special solution for that respective free variable by whatever, and combining all the scaled special solutions into one linear combination.
hmmm thanks for the explanation. I had to play with examples for some time to get a hang of it.
Like ainesten, what was 1?
At 7:24 he changed Ax=0 to ux=0, why?
Previously in lecture 4, he taught EA = u. Confused here, please enlighten.
EA=U means elimination and elimination does not change the solution so Ax=0 has same solution as Ux=0
Me when I saw the deduction of F and I:
"It's not math, it's magic" lol