The 100k Q and A form is here: forms.gle/dHnWwszzfHUqFKny7 Please consider sharing, liking, commenting on the video - this is probably one of my favorite videos on my channel. By the way, if you are wondering why (A^T)^T = A, this will require the double duals, or co-covectors and the "canonical isomorphism". Essentially, the co-covectors are machines that measures covectors, and they can be thought of as vectors themselves. The explanation is not that visual, and so I am probably not making videos on the double duals.
Standard linear algebra course should start from a very nice amateur 😂definition of vector as a co-covector - (1,0) tensor. Besides jokes if you could give some visual intuition for a general tensor definition - it would be cool !)
3b1b should be watching while learning each topic from your textbook or other video. Their main focus is granting you more intution and understanding. Not fully learning a concept.
This makes it a little more complicated then it actually is. Here it is simply - Every vector space V has a dual space V* - Elements of the dual space are called dual vectors or co-vectors. When you think of dual space, just think of this as the space of row vectors, while the original space is the space of column vectors. - Every linear map from V -> W induces a linear map between the dual spaces W* -> V* called the transpose. - Every finite dimensional vector space is isomorphic to its dual, so we can really view the transpose as going from W -> V One more point: Row vectors eat column vectors and return a scalar. In R^2, you can visualize row vectors as equally spaced stacks of planes with one through the origin, and the application of the row vector to the column vector is the number of planes that the column vector pierces. This is the intuition the video is trying to get at.
Thanks for clarifying! I had a big "sigh" while the chosen way to present the topic was the "physics-inspired" one, when the algebraic one would have been much easier like you wrote. Of course visualization is important, but sometimes it just clobbers things up IMHO.
I think there’s something worth to be mentioned about the second to last point. In general vector spaces, the isomorphism between a vector space and its dual is not natural. One would need to specify a basis for the vector space. Only when the vector space is endowed with an inner product is the isomorphism natural.
I would definitely call that a "simple but doesn't explain anything". It doesn't convey any meaning, you don't even say how these things exist, but more importantly why we would care about doing such things. That's probably how classic textbook math presents it, but it doesn't explain much, that's why visual explainatory math videos by the likes of him or 3B1B are amazing, they actually explain the intuition behind these ideas.
@@JthElement Eh, it really depends on the person. I actually found this comment easier to understand than the video, but different people prefer different methods, so I'm glad both of them exist. The problem to me is that I have to keep translating the geometric explanation to the algebraic one while watching the video. The problem is that this concept is just too complex for a geometric explanation to really explain it well, (at least for me, you may feel differently). And just so it's clear, I appreciated the video, but it made more sense after I read this comment. For example, the explanation of why A^T * A = I for orthogonal matrices was really insightful.
I commend you for a simple and overlooked process and revealing the hidden intricacies as to why it happens to look like ‘just switching rows and columns’. There’s alway a reason we do something in math and when we are first introduced to new concepts it are often give the how before we are given the why. Keep up the good work and please continue on this trend of explaining the hidden reasons why we some procedures that are surely taken for granted
I've always had a hard time visualizing matrices as vector transformations, but you do a very good job with that here. You also do a terrific job of explaining the transpose as an inverse vector transformation rather than as a new matrix. Great video!
In Haskell matrix transpose is called `zip`, because of how it unzips a list of M N-tuples into a N-tuple of N lists of scalars. Very useful in general programming, even if you never think about it being linear algebra.
Could you please make a video about tensors? I don't like the usual introduction to it, and I feel that you could give a more intuitive and tangible approach. Thank you for your work, it is amazing. The 13:30 got me real good 😄
The main issue with learning about tensors is that the word means different things in different fields, so when you look for information on tensors you receive conflicting preliminaries and definitions. In physical science, tensors are considered a "generalization" of vectors in the sense that they have more indices, but their mathematical properties are identical to (mathematical) vectors. For mathematicians, contravariant tensors are best considered as the dual of multilinear forms: for a vector space V and field F, - A multilinear k-form or (covariant) k-tensor is a function f: Vᵏ -> F that varies linearly with respect to each of its components - A contravariant k-tensor is a member of Vᵏ, often considered as the input to a multilinear k-form (or covariant tensor). In mathematics, a tensor is *almost always* a covariant tensor, while in physical science, it is *always* a contravariant tensor.
Check out the Tensor Calculus series by Prof Pavel Grinfeld on his YT channel MathTheBeautiful. I deal with tensors since 2011 and seeing his lectures was totally eye-opening. I've never ever seen such a clear and didactic but also deep and rigorous approach. In all I do there's a clear distinction between before and after I came across his channel. Only now I know what Einstein meant by "horse of true mathematics" in a letter he wrote to Levi-Civita (as the tale tells, at least). I cannot recommend Prof Grinfeld's channel enough.
Oh boy you have no idea how much I’ve needed a video about this exact series of topics. Combining this with differential forms + knot theory and we about to *transcend.*
There is a difference between transpose and adjoint that should not be swept under the rug. It all boils down to the inner product. If the inner product is just multiply and sum, then yes, transpose is adjoint. If it's more complicated though, the adjoint follows suit.
To be more specific about the "more complicated", for example, the dot product in the complex vector space is not just a multiply and sum, since you have to take conjugates. But in this case, the adjoint is the conjugate transpose, so not too different.
This is why I said something like you have to restrict the space of covectors the "transpose" acts on. That refers to covectors of the form , where you can directly make a 1-1 correspondence with the vectors, even though in infinite dimensional spaces, there are covectors not of this specific form. In fact, the whole last chapter is actually about that inner product! (actually, more generally, a symmetric bilinear form) I just did not call it that way, and said something about the "pairing" between vectors and covectors, which is only doable with some notion of inner product anyway (or generally, a symmetric bilinear form).
@@mathemaniac Well I was referring to covectors of this form specifically, just with a different inner product. I didn't even think about the crazier stuff in infinite dimensions! I love these deep things lurking in corners of seemingly mundane stuff.
@@btnt5209 That's not coincidental, you can derive the adjoint from the inner product and it will follow its form. For instance, if = x^T Hy, then A* = H^-1 A^T H.
Agreed. Of all you want is the transpose, stay with duals (as row vectors) and not their adjoints: w=vA Then just transpose the whole equation. But better: keep the rows and just multiply backwards.
Well, the frustration is still there: the main thing is I am, at least algorithmically, discouraged to experiment with new things. But I just put down the frustration a little bit, and return to what I'm always doing.
This was just a bit difficult to grasp at first, but then I got the feel for it. Linear Algebra never stops asking more and more questions, and no wonder why I like it so much. A video on metrics would really help, considering the fact that I have to tame Special Relativity next semester.
The point of the transpose is to move the measuring device back to the original space, so that you can do the measurements in the original space. So you ask, what is the operation so that the measurements remain the same, but the basis vectors have not changed? That would be the inverse. As I said later on in the video, it only looks like inverse, but you have to treat those gridlines as covectors.
In my opinion, the explanations are a bit ambiguous, at least to me. I am not a native English speaker, I don't know how other people feel, but I think what confuses me the most is how you chose very long, compound sentences, which made it hard for me to follow. I also feel like I was not explained the terms like "co-vector", "measurement",... and sometimes I have no idea what is being talking about. I really appreciate the hard work you put into this video though.
Okay i just watched the video a few more times. I think i should have focused more on the visualization. It is a lot easier to understand with the animation.
to me linearity and its duality between vector spaces is the most mesmerizing thing in all of math- especially because it is the motivation behind so many more complex entities like differential equations and laplace transforms which rely on those principles for their unshakeable power and accuracy for everything from interpolation to analysis
Keep up the great work Mathemaniac as an early undergrad your content is at a perfect level and it’s so interesting different interpretations of linear algebra from those I’ve already seen.
Try and look up a category theory approach if you want to learn a new perspective. There you could for example learn that what he this video is giving intuition for is the way the dual functor maps lineare maps to maps in the dual space. In general it is a very abstract viewpoint and can be incredibly beuatiful if you understand it.
18:00 the fact that when refering to 'rotational' matrixes the transpose is the inverse implies that QtQ = I (if Q is orthogonal) because orthogonal matrixes can be viewed as rotations. Really great video, greetings from Spain
The time when i am looking for all resources available on internet for covectors , i am learning Special and General relativity kind of in transitioning between them , this video helped me🥰
Very good explanation, but at 1:20 you say that the measuring device takes a vector and takes it into a number. Then in subsequent slides, the device is treated as something that takes a vector to a convector. Any chance of explaining that?
These past two semesters at uni, we covered the book "Linear algebra done right", by Sheldon Axler (it is free, btw. def recommend it!). As suggested in the preface to the teacher at the beginning of the book, we skipped over the section detailing quotients and duality due to the limited time span. This concept of covectors and transposition gives off the vibes of adjoint, which I had trouble grasping. The mathematics is really interesting, especially in the context of inner products. But it is very difficult to digest. This is a good video and will undoubtedly benefit others in the future. I love algebra and hope to see more of it! Edit: I picked up my text, and I am thinking of the matrix conjugate transpose, which is also related to the adjoint. For a linear mapping T, with its adjoint T*, there exists a matrix M which is the matrix representation of T (to and from some bases). E.g, if we perform a change of basis from B to C, then the matrix representation of T is written M(T; B, C). Then, the matrix conjugate transpose is M(T*; C, B). This can be found on page 208 of Axler's third edition book.
The reason why you think of adjoint as conjugate transpose is only because in complex spaces, the usual inner product is = u(dagger)v. In real spaces, the adjoint is exactly the transpose!
We used all of axler except quotients in our second sem first year course. I thought the dual spaces chapter was the most fascinating and strongly recommend it. I have summarised it in the my reply to you elsewhere though. It was interesting watching this video as the scales analogy screamed dual map to me but the later video screamed adjoint map. From my other comment the matrix for the dual map with respect to the canonical dual basis is always the transpose even under C whilst the adjoint map’s matrix as you said is conjugate transpose
@@jacb2997 Yeah, same with us! Some of the later parts of the book are skipped for brevity, but we go through to SVD decomposition. I believe my uni introduces quotients in our standalone algebra course, but I am not too sure. I will certainly be giving the chapter a read after my end-of-semester exams! Thanks for your replies. They have been great
My heart still lies with the view of the transpose that comes out of categorical quantum mechanics (somewhat akin to this one, given the emphasis on the notion of "measurement"), but this one is very nice (and nearly as intuitive) as well. Great video!
It feels so satisfying after understanding the idea of transpose.. Keep up the amazing work. Also, are you planning on explaining Adjoint in the similar way? It would be a great help!!
Nice geometric interpretation of the transpose. Two comments: 1) you can get the components of the one form directly from the intercepts of the c=1 hyperplane. This can be used for a straightforward demonstration that the adjoint matrix is the transpose. 2) you mentioned linearity of the forms as functions, but they are also linear objects in the sense that they add like vectors. Both properties were used.
You say in infinite dimensional vector spaces we have many more vectors than co-vectors. Is this true in general? I was under the impression that it is essentially arbitrary which space we consider the dual. And in infinite dimensional (complex) vector spaces we encounter another quirk. We find operators that have no Eigenvectors in either the space or the dual. They either have left or right eigenvectors. An example from quantum mechanics are the lowering operator, which only has right eigenvectors, and its adjoint, the raising operator, which only has left eigenvectors. This is a concept that I am having trouble understanding, as the characteristic polynomial should just give us the Eigenvalues and not care about left or right?
Many more *covectors* than vectors (the other way around). See the description for the explanation. There are more covectors than vectors precisely when the vector space is infinite-dimensional. How do you have the characteristic polynomial in infinite dimensions?
@@narfwhals7843 That's not all the duals in the infinite dimensional case. If we go back to the wavefunctions in 1 dimension (to make things a bit more concrete), then the function psi --> psi (0) is a perfectly good dual [it is linear, and it sends a wavefunction to a number], but you can't write it in terms of
The ladder operators are defective matrices in finite dimensional spaces (thus missing eigenvectors), but not in infinite spaces because it takes infinite operations to kill arbitrarily excited states, one level at a time. See the comment above on Hilbert spaces about th different sizes of spaces.
If you consider the space of square integrable functions (countably infinite, Hilbert space, for example, vectors of Fourier coefficients), the dual vectors include non-funtion distributions like the Dirac delta function. This is a dense, uncountable basis (one for each real number).
I believe this is the corollary of Hahn-Banach Theorem. Corollary is as follows; For a given m > 0 and a vector x_0 in the normed space, there exists a functional(covector) such that ||f|| = m and f(x_0) = m ||x_0|.
Could you do a video on visualising the transformations performed by (A transpose times A) and (A times A transpose), and perhaps extend that to the general projection matrix?
Who would have umagined I would be binge watching math videos on late weekend nights, they are addictive! Understanding is addiction - a good one though.
i have difficulty conceptually separating vectors and covectors when, in nearly all vector algebra i've used, there's been zero meaningful distinction between them in practice for one: with matrix notation, whether covectors are the "row vectors" or the "column vectors" is entirely arbitrary, and likewise with "bras" and "kets" in bra-ket notation - covectors with no corresponding vector didn't seem to come up in QM, for me, but maybe i missed them - but since the algebra doesn't seem to suggest an inherent distinction on its own, it feels like weird human artifice to impose it for another: when dealing with special relativity, you can bake the metric directly into your basis elements if you don't restrict yourself to standard linear algebra, removing the metric tensor from consideration entirely, and allowing every vector and its corresponding "covector" to be fundamentally equal - this was extra useful to me, because many applications of the metric tensor are very dependent on situational "covariance" and "contravariance", and these concepts were _never_ explained in a useful manner or clearly demonstrated within the algebra itself, so it just looked highly arbitrary at worst and fudged by-eye at best baking the metric directly into the basis also allowed me to discard the cross-product in favour of an operation that makes sense and works in more than 3 dimensions, but that's a different matter (also, i have a somewhat better grasp of covariance and contravariance now, because using explicit bases instead of implicit ones seems almost mandatory in genuinely demonstrating the covariant and contravariant parts of a "basis transformation", which it turns out, when done correctly, is less a transformation than it is a fun notational trick that transforms nothing except the way things are written down)
expanding slightly on the second point, about the metric tensor, now that i remember it slightly better: when taking a vector-gradient of the electromagnetic 4-potential, versus taking the divergence of the resulting electromagnetic tensor, the metric tensor would switch between "covariant" and "contravariant" forms (by inverting the tensor in some capacity, i think), and most everywhere treated this like it should be absolutely obvious when and why it is doing that - not to mention how unintuitive the missing minus-signs were for one of the forms when i construct an equivalent gradient operator using geometric algebra, with the metric baked into the basis elements (their "quadratic form"), i need only understand that the basis vectors in each term are actually _multiplicatively inverse*_ (not obvious in most explanations of the gradient): we don't have x̂ d/dx, but rather 1/x̂ d/dx, for example, and the necessary signs emerge naturally from the metric thereafter and neither do i need to do anything to that operator before also applying it to the resulting electromagnetic bivector (for its divergence): the elements handle the metric _inherently_ it's much cleaner, and its easier for me to understand the math because it feels like the math better understands itself *there're some details to this i have yet to work out, because i've been very depressed and possibly the only person on the planet who cares; if i am wrong in some capacity here, the easiest way for me to understand it might be to understand how (or whether!) the gradient operator can meaningfully exist for elementary null-vectors (that is, null-vectors that aren't just a sum of elements with opposing non-null metrics) - given a null-metric shouldn't have an invertible metric tensor either, i've been assuming it cannot
12:58 guess you made the script calling the 2nd vector "green" before deciding to use RGB for the 3-d axes (seen at 5:20 and later on) and thus changed to using cyan for that 2nd vector? :-)
The concept presentation is novel and wonderful. Deep insight. I did not get the opportunity to learn these concepts in my student and research scholar days. A great service to workers in basic sciences. We pay our sincere gratitude to 3b1b.
And actually, pretty annoyed at myself for not having subbed before, ive watched (and remember watching!) a few of your videos before. Thanks for the content
Why use a column and not a row vector inside the measuring device at 4:50? In fact, why are you using columns to denote covectors in general? They are naturally rows. Everything fits together better that way.
the covectors of a measuring machine seem to be a dot product, like so wTv. if v = Ax then we have wTAx = (ATw)Tx, and thus the AT is the transform for covectors.
For a given vector space (a collection of vectors that can be added together and multiplied by scalars to give other vectors in the vector space) the dual space is the collection of covectors corresponding to these vectors
@@merlynbarrer1018 but how does that mean that the amount of covectors is larger than the number of vectors? Are there covectors that can’t be expressed by a vector? (Currently I’m visualizing this as R2 vectors so please specify if this does actually hold up on R2 or R3 but falls apart in some higher dimension)
As I said in the video, this breaks down in infinite-dimensional vector spaces (and in fact, the number of covectors is far greater than that of vectors precisely when the vector space is infinite-dimensional, see description), and so if you just visualise in R2 or R3, that works perfectly.
It decomposes the determinant into a contraction, giving A B = |A| I. The determinant can be visualized as the matrix acting on p-forms (cross or triple products). See Flanders for a beautiful introduction.
Very good video! First one I see from you, but you earned my subscription. I’ll have to rewatch a couple of times because I’m a little rusty on the abstract side of Linear Algebra and I’m sure I’m missing some important subtleties. I loved the effort you put into the “big picture” chapter! If I may, I’d like to make a couple suggestions: - Master “the pause”, by allowing a few seconds when you deliver punchlines, to allow us to digest what was just said. Your rhythm overall is great, but I felt you just kept talking non-stop at those moments. (I’m sure that’s also a consequence of my rusty L.A.) - In your intro, please mention the “prerequisites” and the target audience. These concepts are deceptively simple at first glance, but it became clear I was missing something as the video progressed, due to my somewhat forgotten command of precise definitions. Of course, the lack of knowledge is entirely my fault, but I didn’t realize I needed it before starting watching the video (after all, I thought I knew perfectly well what transposes were, lol). Keep up the good work!
It's also interesting to consider cases, when matrices linearly map vectors to bivectors and back. Such as when dealing with transformation of a force into a torque (in 2d, or in 4d, since amount of bivector components are different there from amount of vector components). I've seen transpose of such matrix being used to figure out effective mass from the inertia tensor and position to which force is applied (1/effectiveMass = transpose(toTorque) × inertia × toTorque), where inertia is just a symmetric matrix (aka orthogonal resizer) that turns torque bivector into a resized angular velocity bivector. ToTorque represents geometric algebra wedge of a force-vector by offset-vector. And inverse effective mass just shows how much of tangential velocity unit force imparts onto the object (ignoring linear velocity). Anyway, sorry for the unclear monologue; I'm just too tired after a long day and I wanted to share a little piece of cool info I recently learned.
Hello, I am a bit confused with your introduction, it looks like you are looking for a substitute to composition, which you do not mention at all. However i really like the column vector representation for linear forms as their matrix expressions involve the transpose of these columns for a dot product: let µ be our measurement, of vector X, t is the transposition, then µ(Y) = t(X) . Y and µ(alpha(Y)) = t(X) . A . Y = t(t(A) . X) . Y therefore t(A) . X is the vector of our "same measurement" in the starting space and the rank of A is not a limitation.
I always wondered why linear algebra cannot be illustrated using real life example with numbers that represent real objects such as how a tradsperson would make measurements and then transpose these measurements for other construction. For instance, a 2×2 matrix as two restaurant selling two different meals, and when it's transpose or its inverse is required, what are those changes in respect to the restaurants and the meals. Hope I have made myself reasonably clear. When too many math jargons are used, it may become lots of " hand wavings" and novice like me will get lost. Thank you.
I loved watching this video but I need something that will lead into it for me as I never studied this topic before. I studied Physics 50 years ago and am currently taking a QM class so need to understand matrices more deeply particularly Hermitian operators and what adjoint, transpose etc reaally mean. This seemed clear as I watched it but I haven't really grasped the meaning.
Very nice video about not so easy to grasp concepts: The content of this video and more on coordinate transform, dual spaces, transpose maps, kinematics with a rigorous full-concept-notation (rather than proof complete) for mechanics and robotics can be found in appendix D in DOI: 10.3929/ethz-a-010662262
Thank you for making this video. Is there any methods understanding why detA=detA^t intuitively? I think it is possible if using the idea introduced in this video. But I have no more plan. 😢
You will have to think about what does det (A^T) means. For now, I am not sure if we can think about what that really means without a circular reasoning that leads to det A = det (A^T). I think the idea of an adjoint *might* help.
I’ve been studying Lin Alg for a little while now but I never quite saw transpose that way.. this was incredibly refreshing.. and for a moment I thought you were going to bring Eigenvalues and Eigenvectors into the fold, but you didn’t (even though we both know they were hiding there in plain sight now, didn’t they? 😉)
Thanks for a video with great visuals. To add to it, I strongly recommend the Tensor Calculus series from the YT channel MathTheBeautiful by Prof Pavel Grinfeld. I've been studying these topics for many years and, to me, there is currently a plethora of notations and names that shuffle conventions from vector calculus, linear algebra, etc. - often making the topic very confusing. He employs a clear and unifying notation that it makes it VERY EASY to understand covectors, metrics, inverses, transposes... I cannot recommend it enough.
@21:38 This looks like a 2-category, where the functor between objects in covectors and vectors have a 2-morphism that maps between them. Category theory has ruined me
People start doing matrices and totally forget that in the basics. matrices is just a bunch of linear equations But with coefficients only.... and realizing that, [A] and [A]^t is either one function out of a list of functions of some sort, or a list of all linear components that correspond with its basis vector. if you were to read only the row of [A] and [A]^t.
The 100k Q and A form is here: forms.gle/dHnWwszzfHUqFKny7
Please consider sharing, liking, commenting on the video - this is probably one of my favorite videos on my channel.
By the way, if you are wondering why (A^T)^T = A, this will require the double duals, or co-covectors and the "canonical isomorphism". Essentially, the co-covectors are machines that measures covectors, and they can be thought of as vectors themselves. The explanation is not that visual, and so I am probably not making videos on the double duals.
Standard linear algebra course should start from a very nice amateur 😂definition of vector as a co-covector - (1,0) tensor. Besides jokes if you could give some visual intuition for a general tensor definition - it would be cool !)
3b1b linear algebra series saved my life during college, but oh boy do i wish this have seen this video sooner
the 3b1b series was great but definitely left me wanting more
It was amazing but crucially left out transposition, SVD and QR decomposition
3b1b should be watching while learning each topic from your textbook or other video. Their main focus is granting you more intution and understanding. Not fully learning a concept.
@@TheFlemishNut definitely. Don’t rely only on his videos but they are great. They did help me a lot in understanding
This makes it a little more complicated then it actually is. Here it is simply
- Every vector space V has a dual space V*
- Elements of the dual space are called dual vectors or co-vectors. When you think of dual space, just think of this as the space of row vectors, while the original space is the space of column vectors.
- Every linear map from V -> W induces a linear map between the dual spaces W* -> V* called the transpose.
- Every finite dimensional vector space is isomorphic to its dual, so we can really view the transpose as going from W -> V
One more point:
Row vectors eat column vectors and return a scalar. In R^2, you can visualize row vectors as equally spaced stacks of planes with one through the origin, and the application of the row vector to the column vector is the number of planes that the column vector pierces. This is the intuition the video is trying to get at.
Thanks for clarifying! I had a big "sigh" while the chosen way to present the topic was the "physics-inspired" one, when the algebraic one would have been much easier like you wrote. Of course visualization is important, but sometimes it just clobbers things up IMHO.
I think there’s something worth to be mentioned about the second to last point. In general vector spaces, the isomorphism between a vector space and its dual is not natural. One would need to specify a basis for the vector space. Only when the vector space is endowed with an inner product is the isomorphism natural.
I would definitely call that a "simple but doesn't explain anything". It doesn't convey any meaning, you don't even say how these things exist, but more importantly why we would care about doing such things. That's probably how classic textbook math presents it, but it doesn't explain much, that's why visual explainatory math videos by the likes of him or 3B1B are amazing, they actually explain the intuition behind these ideas.
I agree. This was quite useless.
@@JthElement Eh, it really depends on the person. I actually found this comment easier to understand than the video, but different people prefer different methods, so I'm glad both of them exist. The problem to me is that I have to keep translating the geometric explanation to the algebraic one while watching the video. The problem is that this concept is just too complex for a geometric explanation to really explain it well, (at least for me, you may feel differently). And just so it's clear, I appreciated the video, but it made more sense after I read this comment. For example, the explanation of why A^T * A = I for orthogonal matrices was really insightful.
This video is EXACTLY what I've been looking for! Please don't stop making these!
I commend you for a simple and overlooked process and revealing the hidden intricacies as to why it happens to look like ‘just switching rows and columns’. There’s alway a reason we do something in math and when we are first introduced to new concepts it are often give the how before we are given the why.
Keep up the good work and please continue on this trend of explaining the hidden reasons why we some procedures that are surely taken for granted
I've always had a hard time visualizing matrices as vector transformations, but you do a very good job with that here. You also do a terrific job of explaining the transpose as an inverse vector transformation rather than as a new matrix. Great video!
In Haskell matrix transpose is called `zip`, because of how it unzips a list of M N-tuples into a N-tuple of N lists of scalars. Very useful in general programming, even if you never think about it being linear algebra.
This was actually very refreshing and a great way to visualise what a transpose really is, even if there are limitations.
What limitations?
Could you please make a video about tensors? I don't like the usual introduction to it, and I feel that you could give a more intuitive and tangible approach.
Thank you for your work, it is amazing. The 13:30 got me real good 😄
I will have to when I do general relativity some time in the future, but really tensor should not be described as "transforms like a tensor".
Eigenchris's explanations are the best I've found. I'm excited for Mathemaniac's take, though
The main issue with learning about tensors is that the word means different things in different fields, so when you look for information on tensors you receive conflicting preliminaries and definitions.
In physical science, tensors are considered a "generalization" of vectors in the sense that they have more indices, but their mathematical properties are identical to (mathematical) vectors. For mathematicians, contravariant tensors are best considered as the dual of multilinear forms: for a vector space V and field F,
- A multilinear k-form or (covariant) k-tensor is a function f: Vᵏ -> F that varies linearly with respect to each of its components
- A contravariant k-tensor is a member of Vᵏ, often considered as the input to a multilinear k-form (or covariant tensor).
In mathematics, a tensor is *almost always* a covariant tensor, while in physical science, it is *always* a contravariant tensor.
Check out the Tensor Calculus series by Prof Pavel Grinfeld on his YT channel MathTheBeautiful. I deal with tensors since 2011 and seeing his lectures was totally eye-opening. I've never ever seen such a clear and didactic but also deep and rigorous approach. In all I do there's a clear distinction between before and after I came across his channel. Only now I know what Einstein meant by "horse of true mathematics" in a letter he wrote to Levi-Civita (as the tale tells, at least). I cannot recommend Prof Grinfeld's channel enough.
Why don’t you make a video on tensors?
Oh boy you have no idea how much I’ve needed a video about this exact series of topics. Combining this with differential forms + knot theory and we about to *transcend.*
There is a difference between transpose and adjoint that should not be swept under the rug. It all boils down to the inner product. If the inner product is just multiply and sum, then yes, transpose is adjoint. If it's more complicated though, the adjoint follows suit.
To be more specific about the "more complicated", for example, the dot product in the complex vector space is not just a multiply and sum, since you have to take conjugates. But in this case, the adjoint is the conjugate transpose, so not too different.
This is why I said something like you have to restrict the space of covectors the "transpose" acts on. That refers to covectors of the form , where you can directly make a 1-1 correspondence with the vectors, even though in infinite dimensional spaces, there are covectors not of this specific form.
In fact, the whole last chapter is actually about that inner product! (actually, more generally, a symmetric bilinear form) I just did not call it that way, and said something about the "pairing" between vectors and covectors, which is only doable with some notion of inner product anyway (or generally, a symmetric bilinear form).
@@mathemaniac Well I was referring to covectors of this form specifically, just with a different inner product. I didn't even think about the crazier stuff in infinite dimensions! I love these deep things lurking in corners of seemingly mundane stuff.
@@btnt5209 That's not coincidental, you can derive the adjoint from the inner product and it will follow its form. For instance, if = x^T Hy, then A* = H^-1 A^T H.
Agreed. Of all you want is the transpose, stay with duals (as row vectors) and not their adjoints: w=vA Then just transpose the whole equation. But better: keep the rows and just multiply backwards.
I have been searching for an explanation about transposé for à decade, thank you so much 😁
I’m glad that you didn’t get frustrated by the low view of videos! Keep going man I love your math videos
Low views ? He's getting at least 100 k per video that's incredible for a math channel
@@vinvic1578 He once posted something in the "community" go check it out
Well, the frustration is still there: the main thing is I am, at least algorithmically, discouraged to experiment with new things. But I just put down the frustration a little bit, and return to what I'm always doing.
@@zhuolovesmath7483 ohh I see thanks for the information, apologies i didn't know
@@mathemaniac Always support you!
This was just a bit difficult to grasp at first, but then I got the feel for it. Linear Algebra never stops asking more and more questions, and no wonder why I like it so much. A video on metrics would really help, considering the fact that I have to tame Special Relativity next semester.
12:18 why do the basis vectors go back to their starting points? That's should only happen if you multiply by A^-1 right? Not A^T
The point of the transpose is to move the measuring device back to the original space, so that you can do the measurements in the original space. So you ask, what is the operation so that the measurements remain the same, but the basis vectors have not changed? That would be the inverse.
As I said later on in the video, it only looks like inverse, but you have to treat those gridlines as covectors.
Msterpiece!
Thank you for this huge work with the topic and animations!
In my opinion, the explanations are a bit ambiguous, at least to me. I am not a native English speaker, I don't know how other people feel, but I think what confuses me the most is how you chose very long, compound sentences, which made it hard for me to follow. I also feel like I was not explained the terms like "co-vector", "measurement",... and sometimes I have no idea what is being talking about. I really appreciate the hard work you put into this video though.
Okay i just watched the video a few more times. I think i should have focused more on the visualization. It is a lot easier to understand with the animation.
to me linearity and its duality between vector spaces is the most mesmerizing thing in all of math- especially because it is the motivation behind so many more complex entities like differential equations and laplace transforms which rely on those principles for their unshakeable power and accuracy for everything from interpolation to analysis
Keep up the great work Mathemaniac as an early undergrad your content is at a perfect level and it’s so interesting different interpretations of linear algebra from those I’ve already seen.
Try and look up a category theory approach if you want to learn a new perspective. There you could for example learn that what he this video is giving intuition for is the way the dual functor maps lineare maps to maps in the dual space. In general it is a very abstract viewpoint and can be incredibly beuatiful if you understand it.
@@roybean9983 I intend to eventually
18:00 the fact that when refering to 'rotational' matrixes the transpose is the inverse implies that QtQ = I (if Q is orthogonal) because orthogonal matrixes can be viewed as rotations. Really great video, greetings from Spain
The time when i am looking for all resources available on internet for covectors , i am learning Special and General relativity kind of in transitioning between them , this video helped me🥰
Very good explanation, but at 1:20 you say that the measuring device takes a vector and takes it into a number. Then in subsequent slides, the device is treated as something that takes a vector to a convector. Any chance of explaining that?
These past two semesters at uni, we covered the book "Linear algebra done right", by Sheldon Axler (it is free, btw. def recommend it!). As suggested in the preface to the teacher at the beginning of the book, we skipped over the section detailing quotients and duality due to the limited time span. This concept of covectors and transposition gives off the vibes of adjoint, which I had trouble grasping. The mathematics is really interesting, especially in the context of inner products. But it is very difficult to digest. This is a good video and will undoubtedly benefit others in the future. I love algebra and hope to see more of it!
Edit: I picked up my text, and I am thinking of the matrix conjugate transpose, which is also related to the adjoint. For a linear mapping T, with its adjoint T*, there exists a matrix M which is the matrix representation of T (to and from some bases). E.g, if we perform a change of basis from B to C, then the matrix representation of T is written M(T; B, C). Then, the matrix conjugate transpose is M(T*; C, B). This can be found on page 208 of Axler's third edition book.
The reason why you think of adjoint as conjugate transpose is only because in complex spaces, the usual inner product is = u(dagger)v. In real spaces, the adjoint is exactly the transpose!
@@mathemaniac Thanks for that! I will certainly give the section in textbook a read when I have the time. It is really interesting stuff!
We used all of axler except quotients in our second sem first year course. I thought the dual spaces chapter was the most fascinating and strongly recommend it. I have summarised it in the my reply to you elsewhere though. It was interesting watching this video as the scales analogy screamed dual map to me but the later video screamed adjoint map. From my other comment the matrix for the dual map with respect to the canonical dual basis is always the transpose even under C whilst the adjoint map’s matrix as you said is conjugate transpose
@@jacb2997 Yeah, same with us! Some of the later parts of the book are skipped for brevity, but we go through to SVD decomposition. I believe my uni introduces quotients in our standalone algebra course, but I am not too sure. I will certainly be giving the chapter a read after my end-of-semester exams! Thanks for your replies. They have been great
My heart still lies with the view of the transpose that comes out of categorical quantum mechanics (somewhat akin to this one, given the emphasis on the notion of "measurement"), but this one is very nice (and nearly as intuitive) as well. Great video!
It feels so satisfying after understanding the idea of transpose.. Keep up the amazing work.
Also, are you planning on explaining Adjoint in the similar way? It would be a great help!!
Brilliant video
Deep insights in Linear Algebra ( Reminds of Essence of Linear Algebra series on 3B1B)
Nice geometric interpretation of the transpose. Two comments: 1) you can get the components of the one form directly from the intercepts of the c=1 hyperplane. This can be used for a straightforward demonstration that the adjoint matrix is the transpose. 2) you mentioned linearity of the forms as functions, but they are also linear objects in the sense that they add like vectors. Both properties were used.
You say in infinite dimensional vector spaces we have many more vectors than co-vectors. Is this true in general? I was under the impression that it is essentially arbitrary which space we consider the dual.
And in infinite dimensional (complex) vector spaces we encounter another quirk. We find operators that have no Eigenvectors in either the space or the dual. They either have left or right eigenvectors. An example from quantum mechanics are the lowering operator, which only has right eigenvectors, and its adjoint, the raising operator, which only has left eigenvectors.
This is a concept that I am having trouble understanding, as the characteristic polynomial should just give us the Eigenvalues and not care about left or right?
Many more *covectors* than vectors (the other way around). See the description for the explanation. There are more covectors than vectors precisely when the vector space is infinite-dimensional.
How do you have the characteristic polynomial in infinite dimensions?
@@mathemaniac Again in QM we have the state space with elements |x> and the dual space with the
@@narfwhals7843 That's not all the duals in the infinite dimensional case. If we go back to the wavefunctions in 1 dimension (to make things a bit more concrete), then the function psi --> psi (0) is a perfectly good dual [it is linear, and it sends a wavefunction to a number], but you can't write it in terms of
The ladder operators are defective matrices in finite dimensional spaces (thus missing eigenvectors), but not in infinite spaces because it takes infinite operations to kill arbitrarily excited states, one level at a time. See the comment above on Hilbert spaces about th different sizes of spaces.
This is amazing! I didn't expect this to help in understanding rigid monoidal categories! AMAZING!!!
Your intention to teach deeper meaning of transpose is awesome, but lot of viewers are more confused now.
Do you have a brief explanation as to why there are more covectors than vectors in infinite dimensional spaces?
If you consider the space of square integrable functions (countably infinite, Hilbert space, for example, vectors of Fourier coefficients), the dual vectors include non-funtion distributions like the Dirac delta function. This is a dense, uncountable basis (one for each real number).
I believe this is the corollary of Hahn-Banach Theorem. Corollary is as follows;
For a given m > 0 and a vector x_0 in the normed space, there exists a functional(covector) such that ||f|| = m and f(x_0) = m ||x_0|.
Could you do a video on visualising the transformations performed by (A transpose times A) and (A times A transpose), and perhaps extend that to the general projection matrix?
"t" is not supposed to go BEFORE the matrix we want to transpose ?
I'd never considered that a transpose might be more than a mere "matrix reflection". Thanks for another enlightening video.
1:05 to 1:52 best explanation of duality in linear algebra.❤
Who would have umagined I would be binge watching math videos on late weekend nights, they are addictive! Understanding is addiction - a good one though.
it's interesting how I'm currently studying 4 topics and this video managed to get into all of them! Badass
i have difficulty conceptually separating vectors and covectors when, in nearly all vector algebra i've used, there's been zero meaningful distinction between them in practice
for one: with matrix notation, whether covectors are the "row vectors" or the "column vectors" is entirely arbitrary, and likewise with "bras" and "kets" in bra-ket notation - covectors with no corresponding vector didn't seem to come up in QM, for me, but maybe i missed them - but since the algebra doesn't seem to suggest an inherent distinction on its own, it feels like weird human artifice to impose it
for another: when dealing with special relativity, you can bake the metric directly into your basis elements if you don't restrict yourself to standard linear algebra, removing the metric tensor from consideration entirely, and allowing every vector and its corresponding "covector" to be fundamentally equal - this was extra useful to me, because many applications of the metric tensor are very dependent on situational "covariance" and "contravariance", and these concepts were _never_ explained in a useful manner or clearly demonstrated within the algebra itself, so it just looked highly arbitrary at worst and fudged by-eye at best
baking the metric directly into the basis also allowed me to discard the cross-product in favour of an operation that makes sense and works in more than 3 dimensions, but that's a different matter
(also, i have a somewhat better grasp of covariance and contravariance now, because using explicit bases instead of implicit ones seems almost mandatory in genuinely demonstrating the covariant and contravariant parts of a "basis transformation", which it turns out, when done correctly, is less a transformation than it is a fun notational trick that transforms nothing except the way things are written down)
expanding slightly on the second point, about the metric tensor, now that i remember it slightly better: when taking a vector-gradient of the electromagnetic 4-potential, versus taking the divergence of the resulting electromagnetic tensor, the metric tensor would switch between "covariant" and "contravariant" forms (by inverting the tensor in some capacity, i think), and most everywhere treated this like it should be absolutely obvious when and why it is doing that - not to mention how unintuitive the missing minus-signs were for one of the forms
when i construct an equivalent gradient operator using geometric algebra, with the metric baked into the basis elements (their "quadratic form"), i need only understand that the basis vectors in each term are actually _multiplicatively inverse*_ (not obvious in most explanations of the gradient): we don't have x̂ d/dx, but rather 1/x̂ d/dx, for example, and the necessary signs emerge naturally from the metric thereafter
and neither do i need to do anything to that operator before also applying it to the resulting electromagnetic bivector (for its divergence): the elements handle the metric _inherently_
it's much cleaner, and its easier for me to understand the math because it feels like the math better understands itself
*there're some details to this i have yet to work out, because i've been very depressed and possibly the only person on the planet who cares; if i am wrong in some capacity here, the easiest way for me to understand it might be to understand how (or whether!) the gradient operator can meaningfully exist for elementary null-vectors (that is, null-vectors that aren't just a sum of elements with opposing non-null metrics) - given a null-metric shouldn't have an invertible metric tensor either, i've been assuming it cannot
12:58 guess you made the script calling the 2nd vector "green" before deciding to use RGB for the 3-d axes (seen at 5:20 and later on) and thus changed to using cyan for that 2nd vector? :-)
Great! Please more about matrix operations and the idea behind it! :)
16:15 hmmm I think this is why robot kinematics uses Jacobean transpose as an approximation to inverse
18:57 is more to the point I think. In robotics you are generally dealing with rotations, in which case the transpose really _is_ the inverse.
The concept presentation is novel and wonderful. Deep insight. I did not get the opportunity to learn these concepts in my student and research scholar days. A great service to workers in basic sciences. We pay our sincere gratitude to 3b1b.
the borrowed scale metaphor is just brilliant
I’ve always learnt transpose as rotation along the diagonal. Even in high school.
I will have to rewatch this video a few times.
what has the length of a covector got to do with density?
6:08 up&down or left&right?
Im only 4 minutes in and ive done this kind of math for years and so far this has hit like 3 or 4 of those "ohhh" moments. Subbed.
And actually, pretty annoyed at myself for not having subbed before, ive watched (and remember watching!) a few of your videos before. Thanks for the content
Really clarified why transposing the matrix of a linear map has to do with the matrix dual linear map!! Nice video well done
14:44
How did you calculate the gap size to be 1/√2
Awesome vedio.....Waiting for your special and general relativity vedios....
You just made the best youtube explanation of machine learning svm without even mentioning svm in the title xd
that presentation is really good. how could you do that? I mean is there a special program you use? thanks
Why use a column and not a row vector inside the measuring device at 4:50? In fact, why are you using columns to denote covectors in general? They are naturally rows. Everything fits together better that way.
Agreed.
Can you please make a video on the proof of Laplace Expansion formula
On Earth, what is 'geographic north' (in general 3d unit vector form)?
Also, what is 'magnetic north' (in gen 3d uvf)?
Off-diagonal matrix coefficients can be also kinda thought of as bivector components btw
Can u make videos about Polya Enumeration Theorem ?
this is amazing. please make more math concepts. thankyou!!
the covectors of a measuring machine seem to be a dot product, like so wTv. if v = Ax then we have wTAx = (ATw)Tx, and thus the AT is the transform for covectors.
exactly what I needed
Are we going to get Hermitian here?
Can you talk about dual spaces? This part of linear algebra doesn’t make sense to me
That's the "covector' part he mentions here
For a given vector space (a collection of vectors that can be added together and multiplied by scalars to give other vectors in the vector space) the dual space is the collection of covectors corresponding to these vectors
@@merlynbarrer1018 but how does that mean that the amount of covectors is larger than the number of vectors? Are there covectors that can’t be expressed by a vector? (Currently I’m visualizing this as R2 vectors so please specify if this does actually hold up on R2 or R3 but falls apart in some higher dimension)
As I said in the video, this breaks down in infinite-dimensional vector spaces (and in fact, the number of covectors is far greater than that of vectors precisely when the vector space is infinite-dimensional, see description), and so if you just visualise in R2 or R3, that works perfectly.
Is there a visual way of understanding the matrix of cofactors as well?
It decomposes the determinant into a contraction, giving A B = |A| I. The determinant can be visualized as the matrix acting on p-forms (cross or triple products). See Flanders for a beautiful introduction.
Very good video! First one I see from you, but you earned my subscription. I’ll have to rewatch a couple of times because I’m a little rusty on the abstract side of Linear Algebra and I’m sure I’m missing some important subtleties. I loved the effort you put into the “big picture” chapter!
If I may, I’d like to make a couple suggestions:
- Master “the pause”, by allowing a few seconds when you deliver punchlines, to allow us to digest what was just said. Your rhythm overall is great, but I felt you just kept talking non-stop at those moments. (I’m sure that’s also a consequence of my rusty L.A.)
- In your intro, please mention the “prerequisites” and the target audience. These concepts are deceptively simple at first glance, but it became clear I was missing something as the video progressed, due to my somewhat forgotten command of precise definitions. Of course, the lack of knowledge is entirely my fault, but I didn’t realize I needed it before starting watching the video (after all, I thought I knew perfectly well what transposes were, lol).
Keep up the good work!
It's also interesting to consider cases, when matrices linearly map vectors to bivectors and back. Such as when dealing with transformation of a force into a torque (in 2d, or in 4d, since amount of bivector components are different there from amount of vector components).
I've seen transpose of such matrix being used to figure out effective mass from the inertia tensor and position to which force is applied (1/effectiveMass = transpose(toTorque) × inertia × toTorque), where inertia is just a symmetric matrix (aka orthogonal resizer) that turns torque bivector into a resized angular velocity bivector. ToTorque represents geometric algebra wedge of a force-vector by offset-vector. And inverse effective mass just shows how much of tangential velocity unit force imparts onto the object (ignoring linear velocity).
Anyway, sorry for the unclear monologue; I'm just too tired after a long day and I wanted to share a little piece of cool info I recently learned.
Takes some effort to understand but really insightful. Thanks 😊:)
Can you do more videos on Matrices and Vectors? This video was awesome!
I have enjoyed it and learn new things today. Thank you ❤️
Hello,
I am a bit confused with your introduction, it looks like you are looking for a substitute to composition, which you do not mention at all.
However i really like the column vector representation for linear forms as their matrix expressions involve the transpose of these columns for a dot product:
let µ be our measurement, of vector X, t is the transposition,
then µ(Y) = t(X) . Y
and µ(alpha(Y)) = t(X) . A . Y = t(t(A) . X) . Y
therefore t(A) . X is the vector of our "same measurement" in the starting space and the rank of A is not a limitation.
Extraordinary explanation
Really great video! One technical recommendation though: think about adding some background music - it'll make it flow more smoothly :)
I never understood tensors quite well, but in this video I smelled the tensors cooking up!, still don’t know where though!.
Just wonderful! Thanks
glad i could understand this stuff. 3b1b really helps! amazing stuff!
Yes, all of this comes directly from the transpose, so what do you mean?
thank you for the video, but I didn't get it. Can you recommend any materials/video on covectors?
I always wondered why linear algebra cannot be illustrated using real life example with numbers that represent real objects such as how a tradsperson would make measurements and then transpose these measurements for other construction. For instance, a 2×2 matrix as two restaurant selling two different meals, and when it's transpose or its inverse is required, what are those changes in respect to the restaurants and the meals. Hope I have made myself reasonably clear. When too many math jargons are used, it may become lots of " hand wavings" and novice like me will get lost. Thank you.
If a covector is a just a linear map from vectors to scalars, isn't each covector just a dot product with a particular vector?
It is basically what Riesz representation theorem says.
I loved watching this video but I need something that will lead into it for me as I never studied this topic before. I studied Physics 50 years ago and am currently taking a QM class so need to understand matrices more deeply particularly Hermitian operators and what adjoint, transpose etc reaally mean. This seemed clear as I watched it but I haven't really grasped the meaning.
This just left me with more confusion. It replaces a simpler meaning with a complex one but does not make it deeper.
Same dude😅. And it seems like his explanation is longer than it needs to be
eigenchis has a great series on this sorta stuff
Can you share video code?
Very nice video about not so easy to grasp concepts:
The content of this video and more on coordinate transform, dual spaces, transpose maps, kinematics with a rigorous full-concept-notation (rather than proof complete)
for mechanics and robotics can be found in appendix D in
DOI: 10.3929/ethz-a-010662262
so convinced in first two minutes.....and also figured out what the key feature of the video is going to be like. Thank you so much!
Thank you for making this video. Is there any methods understanding why detA=detA^t intuitively? I think it is possible if using the idea introduced in this video. But I have no more plan. 😢
You will have to think about what does det (A^T) means. For now, I am not sure if we can think about what that really means without a circular reasoning that leads to det A = det (A^T). I think the idea of an adjoint *might* help.
I’ve been studying Lin Alg for a little while now but I never quite saw transpose that way.. this was incredibly refreshing.. and for a moment I thought you were going to bring Eigenvalues and Eigenvectors into the fold, but you didn’t (even though we both know they were hiding there in plain sight now, didn’t they? 😉)
Beautiful explanation 👌 I would love that 10years ago 😆 still fun to watch
Thanks, King :)
Great video, well done
Great video. Thank you
Tristan Needham is a legend!
Thanks for a video with great visuals. To add to it, I strongly recommend the Tensor Calculus series from the YT channel MathTheBeautiful by Prof Pavel Grinfeld. I've been studying these topics for many years and, to me, there is currently a plethora of notations and names that shuffle conventions from vector calculus, linear algebra, etc. - often making the topic very confusing. He employs a clear and unifying notation that it makes it VERY EASY to understand covectors, metrics, inverses, transposes... I cannot recommend it enough.
@21:38 This looks like a 2-category, where the functor between objects in covectors and vectors have a 2-morphism that maps between them. Category theory has ruined me
Wow. I understood this topic before, but the ideas never really felt intuitive. This video is amazing.
People start doing matrices and totally forget that in the basics. matrices is just a bunch of linear equations But with coefficients only.... and realizing that, [A] and [A]^t is either one function out of a list of functions of some sort, or a list of all linear components that correspond with its basis vector. if you were to read only the row of [A] and [A]^t.
_Aren't the best transposes when they bring shivers to the diagonals of your matrix?_
THANK YOU
Great video!
perfect timing