The Math behind (most) 3D games - Perspective Projection
ฝัง
- เผยแพร่เมื่อ 9 พ.ค. 2024
- Perspective matrices have been used behind the scenes since the inception of 3D gaming, and the majority of vector libraries will have built-in helper functions to construct them for you. But what if you wanted to know the reasoning behind how these matrices are constructed.
I start off with a brief introduction of computer graphics and the key ideas behind 3D rendering. I differentiate between Image Order Rasterization typically used by Ray tracers, and object order rasterization as used in most video games. I then present an animated walkthrough showing the full derivation of both the orthographic and perspective projection matrices.
My derivation here is focused on the Vulkan API, but the reasoning applies just as well to other API's such as OpenGL or direct X, with the only notable differences being the size of the canonical viewing volumes, and the handedness and conventions of the xyz coordinate systems.
0:00 How does 3D graphics work?
2:05 Image versus object order rendering
2:51 The Orthographic Projection matrix
5:17 The perspective transformation
7:08 Homogeneous Coordinate division
8:27 Constructing the perspective matrix
10:29 Non-linear z depths and z fighting
11:30 The perspective projection transformation
** Resources **
Fundamentals of Computer Graphics by Peter Shirley & Steve Marschner
www.songho.ca/opengl/gl_projec...
matthewwellings.com/blog/the-...
www.insider.com/pixars-animat...
* Attributions*
Ray tracing graphics cards - Photo by Nana Dua from Pexels
Rubiks cube - Photo by Mathias P.R. Reding from Pexels
Ray tracing in video games - www.digitaltrends.com/gaming/...
en.wikipedia.org/wiki/Line%E2...
For those of you who have been following the vulkan tutorial series, this video is a slight departure from the normal format. The coding portion for the tutorial will be released separately as soon as I’m finished. Future tutorials will return to the more typical structure, with code and theory interleaved. Enjoy!
i loved the Homogeneous Coordinates Hero 😂
@@yaircambordamorocho4506 hahaha thank you :)
I really like the way you explain the theory behind these computer graphics topics without mudding with the programming part.
Amazing design of video, very accessible and interesting explanation, thanks a lot.
Crystal clear as usual! Now I know what z-fighting really means! Take that wikipedia!
Cleanest description of the math behind view projection I've seen! Nice.
Agreed! I've been reading and watching almost everything I can find on the topic and this is by far the best explanation I've seen to date.
WHAT THE FUCK IS HAPPENING GOD HELP THE MATHS IS HELL, KILL ME NOW.
I’m writing a rendering engine from scratch in C and this was exactly the resource I was looking for. Wonderful visualization and explanation!
I remember, when first learning programming I tried to get (any level of) 3D rendering to work, and discovering the math of perspective transformation (e.g. x1/z1 = x2/z2, thus x(screen) = x(world) * (z1 / z2) ) was a critical breakthrough. Even if the only thing I could actually code with it was an imitation of the "starfield" Windows screen saver.
question : are you using SDL ?
I’m working on the same project with SDL and quaternions right now and it has been a whole rabbit hole to dive into!
After watching 3blue1browns "Essence of Linear Algebra" Playlist, taking the linear algebra module in my university AND watching this video I was finally able to implement these concepts all while deriving the used methods by myself too. Thanks for this awesome video
Thank you!! glad you found it helpful!
Wow, these drawings are so cool, they have a 2000's vibe. Nice.
Thank you for this video!! I don’t code much, but I love art and math and was so curious on the math behind game engines / 3d graphics and wanted to see how close or far off my guesses were. this makes me want to code!
Thank you so much. Comments like this are what keeps me motivated to keep making videos.
Found this, Great explanation this topic was killing me, I was not able to understand it before where they get the values and a visual representation of it. You have everything. Great job!
Another master piece of 3d explanation. I really never understood well the black magic of the perspective matrix. It was in somehow just diving by z. Yet, your explanation is so clean and so perfect. It makes it really easy to understand. Please post more tutorials like these. You should definitely write a book about it too.
Beautiful, right to the point. Not a lot of extra theory. Easy to follow and understand. Keep them coming 🙏
Absolutely loved this explanation and would recommend it to anyone trying to understand the nuts and bolts of graphics programming.
Very well explained!! Good work!! Look forward to your game engine and computer graphics series!!
Probably the best in depth tutorial i have ever seen on TH-cam for graphics rendering. Even better than Cherno IMO
Two videos in a week! So glad you're making this series, I was struggling a lot with the vulkan documentation before I found your videos!
This is amazing!! I always wondered what was behind it, great Job!
This video on its own is incredible, educational, and just all around superb. I learned so much and tied together so many loose ends I had dangling in my brain because of this video. Thank you.
This simultaneously looks easily and neatly explained and not understandable at all without an already pretty advanced level
Even though I thought I had already understood this before I watched, I still learn something from it. Thanks for making this video, I think it's the clearest one about how to get the Perspective Projection
This is very very very good. I’ve seen too many people code in opengl etc without really understanding what is going on. Sure it works, but this intuition as to what these matrices are doing is good
Thank you!
The best mathematical explanation. In most cases, the equations come out of the blue like from the magician's hat. Thank you very much
THIS WAS SO AWESOME! Keep up the great work!
Brilliant video. I did a computer graphics course a few years ago and we never actually went into what makes the matrix. Thanks for explaining it!
Awesome video! Thanks a lot for this!
Just wow. Fantastic explanation!
Wow, thank you so much for explaining these topics so well!
I have exactly no background in computer graphics, only linear algebra. Your video is masterful in its explanation. I find the tempo of your speech helps alot!
The best explanation of perspective projection. Thank you
homogeneous coordinates swooping in lmao. Great video! I've been meaning to understand this better for about a year now and finally searched it up and I'm thrilled to have found this, perfect for my level.
This channel is just awesome! Thank you so much.
Thank you very much! I needed to rewind parts of the video to fully digest the idea, but finally i did. :) Please, keep up the quality work!
Ya Ive gotten some good feedback that maybe I went a little fast 😅 but really glad you were able to get it! This has definitely been the hardest video for me to make so far
great explanation. i love the visuals; it makes it so much easier to understand.
Thank you!! this was definitely the hardest video I've yet to make.
Time to drop everything for the new Brendan Galea video, love your stuff man!
I’m working on a career change right now, going back to school in the near future for an MS in CS with hopes to be a graphics engineer. I’m reviewing all the prerequisite math right now so having an understanding of what kind of math I’ll need to learn in the future is very helpful. Even though I didn’t really understand 95% of this it was still a great video 😁
Thanks and best of luck with your studies!!
Just a question, do you have relevant CS or math experience? Because just going "I want to program 3d space vectors" is not something the average Joe just gets up and does?
@@codejunki567 no I don’t, that’s why I’m going back and taking all the prerequisites before doing the grad program. I have to do calc 1, 2, linear algebra, physics, C++ programming, etc. I’ve done a lot of Python, R and SQL but that’s not really relevant. Believe me, I understand how difficult is and how much I have left to learn. I probably won’t be able to start the grad program for another year and half or so.
@@stef8776honestly if it's something you're interested in it's not very difficult. I think people who complete certain qualification programs try to oversell it's difficulty.
As a complete average Joe myself I haven't really had problems with the maths side of 3d graphics. It's actually helped me improve my intuitive understanding of certain concepts.
If you enjoy the work, you'll be thinking about it enthusiastically when you're out and about, going on a walk, doing whatever. When things are on your mind out of pure intrigue, you are probably in the right field and will definitely succeed.
Thank you so much for this video! Finally made me understand the perspective projection matrix
12:33 The vertical FOV is actually the angle from the center to the top (or center to the bottom, same thing). It's a half angle. Hence the theta/2.
Also the the height he marks at the illustration should be height/2.
never knew there is even so much more math beyond the math that my simulation itself needs, but it finally explains a lot
so clear and crisp. Thanks for this video
Glad you liked it!
It's very useful and eay to understand, thank you for your work.
Honestly I've probably watched this video around 4 times. Best explanation on the topic and a super good reference to hold on to. Thanks a lot!
Wow, thanks!
Man you saved my life. I was cramming my 3D book and completely lost in projection stuff then found your vid. Thank you so much aa
My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life th-cam.com/play/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B-.html
th-cam.com/video/7d16CpWp-ok/w-d-xo.html
Thank you! Amazingly helpful video!!!
This is a Beautiful Explanation
thank you so much for the explanation. thanks to you I also understood new things about z-fighting.
Love your explanations man, you make the documentation easy S/o
Nice Explanation video !!! Somehow I had to watch in 2x to not get distracted and understand better
Very nice and detailed explanation. Thank you for the video.👏
Glad it was helpful! Thanks!
Excellent video! Can't thank you enough!
this is the type of stuff you understand when you're learning it, but when you're programming it makes you wanna die
Hahaha ya exactly. The main problem is that you rarely get useful error messages as feedback. Things will just look wrong.
My first school project working on projection matrices I was passing in the matrix in row major order and didn’t realize OpenGL wanted column major… took me hours to figure that one out
@@BrendanGalea any tips for getting my brain to actually work with all these different orderings? It's 99% of what i spend my time debugging in my brain
Thanks again for continuing the series! I have something important (potentially) to add about future 3D animations:
the cube demo ran poorly for me, low framerate, creating an ugly jitter. Unusual, since my hardware is fairly fast. Fixed it by disabling the "apidump" layer I had defined in the "VK_INSTANCE_LAYERS" system environment variable. However, this made my framerate skyrocket to above 3000 FPS, making the spinning cube become an imperceivable blur. Your demo implementation is framerate dependent, I have since created a lightweight way to measure timing (delta) and adjust the speed of animations to any system. Just thought I should remind you of this just in case. I can provide my implementation if you'd like to use it.
Edit: having apidump enabled will not only slow you down, but also spam the console on every api call. Hope it helps. Sorry for the long post!
Ya this is something I should have mentioned. The easiest way to limit frame rate for now is update the chooseSwapPresentMode function in the swapchain by commenting out the mailbox present mode, and using FIFO (vsync).
Your solution is the ultimately correct one, where the game loop needs some timing mechanism. We'll cover the different methods of game loop timing pretty soon.
I really really needed this video. Thank you !!!!!
I think i finally get it! Thank you for this video
One question, the vertical field of view as you display considers the view center point to be at Y = 0 and not at h/2, thus theta is incorrect in your statement, or did i miss something? (Coming from a bloke who used to have to draw these projections on paper back in the early 80's).
I have been searching for a clear explanation of NDC system and the projective transformation. This is such a good video with illustrations that explain the concepts beautifully!
This is what I was looking for...thank you.♥♥
This is a masterpiece. Thx!
Amazing video to learn 3d projection!!
Thank you very much for making this quality tutorial series, I've been following along slowly for some time and really appreciate the detail and clarity. I will say that, for me personally, the videos which focus more on the mathematical background are less useful, but I may not be representative of the audience; just thought the feedback could be useful.
Looking forward to more lessons!
Thank you! And I appreciate the feedback. It's good to know what's working for some people and what isn't. Well, some good news then is this is the most math heavy video for the foreseeable future. Tutorial 14 will have just a bit more and then it's predominantly coding after that point.
@@BrendanGalea Might wanna do a poll, I enjoyed this video! Helps me understand why I put in a matrix, without blindly copy-pasta-ing it :) I personally prefer both!
@@dexterman6361 - ya that's a good idea. Something I'll have to think about
My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life th-cam.com/play/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B-.html
th-cam.com/video/7d16CpWp-ok/w-d-xo.html
how different would be this perspective projection matrix for OpenGL ? it is having -z...z range so i assume it will be different on z component of the matrix
Thanks! Very good explenation.
Currently learning basic math, but I watched this whole video for some reason and I really hope that someday I can understand it.
That intro omfg 🔥🔥🔥🔥
Just wow, thank you so much!!
Era exatamente esse tipo de vídeo que eu tava procurando. Muito obrigado!! 👏😁 ➕1 subscriber!
thanks for great explaination
I'm a bit confused. In the first example, the origin "orthographic view volume" is in world-space? Therefore this is valid for a camera looking down the z-axis, but if you wanted to have a camera looking towards anywhere you would need to add a rotation matrix in there right?
I don't know why, but every explanation online of how and why the z order is preserved ends up in a shady and not very explanatory way. At 10:35 you literally made me get it instantly! thank you for this great content, keep it up brother!
thank you so much for this
I think that both OpenGL and Vulkan use right-handed coordinate axis. The coordinates just differ by rotation by 180 degrees around x axis.
Amazing again, was nice to get to some theory.
My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life th-cam.com/play/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B-.html
th-cam.com/video/7d16CpWp-ok/w-d-xo.html
Brendan, you're a gem.
thank you :)
Thank you so much!
Right...
Definitely understood what being said.
New video, Awesome!
If you use the right-hand rule with X pointing to the right and Y pointing up, positive Z goes into the screen and -Z goes out. If that matches up, it is right-handed. Otherwise it is left.
wow this is amazing thank you very much
My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life.. th-cam.com/play/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B-.html
th-cam.com/video/7d16CpWp-ok/w-d-xo.html
Interesting. I've tried to derive it using transform to ortogonal frustum first (far goes to one, near goes to near/far) and then solving equations to open the wedge and map right, near point to 1,0.
One slight inaccuracy is z buffer precision. It's from times when z buffer was stored as fixed point number. Today it can be switched to floating point. Also range can be both -1 to 1 or 0 to 1 in modern OpenGL and z test can be negated.
awesome video!
Valuable videos.
Good job man
All that Math Library Programmed in the Engine is Insane. Layers and Layers of Codes writing New Codes to render the output, Just like Neural Network!
I have a 3x4 matrix obtained from a camera calibration method. The matrix includes the intrinsic and extrinsic of the camera but not really a model view projection matrix. How do I dare to convert this raw matrix into a 4x4 projection matrix?
finally a tutorial that actual explains the maths
Amazing
This is actually a general derivation that holds for a skewed view frustrum. The picture shows a regular frustrum. Skewed frustra you need when you want to make a huge picture so that you have to chop up the initial regular frustrum.
Great video, ty!!!
Thank you so much! You explained everything I needed.
Amazing!
Thanks!
Great video, making LA way more interesting to me personally
This is almost the perfect derivation of perspective projection (especially since it includes both the off-axis and FOV versions!). However, I wish you would have talked more about why applying the first perspective transform gets us into a space that allows for trivially transforming with the orthographic projection matrix. It doesn't seem intuitive to me. Firstly, I think it would have helped to emphasize that multiplying by n and dividing by z actually turns the slanted edge of the frustum into the right-angled straight edge of the desired rectangular prism because everything on that will end up landing at the exact same x value (or y, for the top, respectively). Secondly, however, we should note that what we get after multiplying with our perspective matrix is *not* a 3D rectangular prism, since its components haven't been divided by the w component (i.e. z value) yet. It's a seemingly somewhat arbitrary 4-dimensional entity, and so it's unclear from my limited linear algebra skills why doing the perspective divide and then applying the orthographic projection is equivalent to doing the orthographic projection and then the perspective divide. There seems to be a deeper "truth" to homogeneous coordinate equivalence that is more than just a notational convenience. If you had touched a little on that, this would have been the de-facto, perfect explanation.
Hmm, trying to figure it out more, it seems like the "divide-by-w" can really just be considered to be multiplying the matrix-vector multiplication by the scalar 1/w. And if you have a scalar a and a matrix-vector multiplication M times v, M[av] = a[Mv] (as well as [aM]v). I think that's the last piece of the puzzle.
This is amazing!! I always wondered what was behind it, great Job!. Love your explanations man, you make the documentation easy S/o.
omg, you are awesome!
Except 1 thing that I don't understand: from the final output of perspective projection matrix, do you need to divide x and y by w from the final output of perspective projection matrix to get the correct x and y? Thanks for potential answer 😊
No, in the shader when you set gl_position it will automatically divide by the w component.
Awesome thanks 👍
how does the non linear z buffering effects the light calculations? or are they too small to care about?
When doing light calculations you want to use the world space x,y,z values so the non linear z value (screen space) in the depth buffer does not play a role in the calculation.
The z depth buffering value is only really used for determining which fragment is closest so only the ordering matters.
However for some more advanced techniques such as screen space ambient occulsion or when using shadow maps the non linear z buffering does start to matter and is something that should be kept in mind. It’s a bit beyond the scope of a TH-cam comment to be able to go into. But any good tutorial that depends on depth dependent information should cover how it is relevant in the context of the tutorials topic.
I like your video, but I have a question.
In 3d video games with true perspective, a square is orthogonal to the distance to another square.
But in the isometric video game, the rhombus is instead orthogonal to the distance to another rhombus.
What would have happened if we enlarged one of the rhombus, so that it became true perspective, instead of isometric?
I understand now! thanks
Nice, I can use these math for making LiDAR sensors.
You are the best!!!
Great! I think I'll not watch it though; I've been for a few years just figuring this out mostly on my own, I don't really want to spoil anything for myself! I think I've figured out UV mapping or at least something akin to it, I figured that if you know which triangle a point is being projected from, then you can change the central point, in other words shift every 3d object's coordinates aside from Z position and reproject the point again, if you do this twice making sure that you're projecting to central positions such that the orthographic projection of the de-projected 3d point can't be on a line from the two 2d projected points, then you can now draw a line from the two projected positions to the point that was their central position, find the line intersection and you'll now have the X and Y coordinates of the de-projected point, all that's left is to simply figure the Z coordinate based on X and Y coordinates.
Good explanation of the frustrum matrix... but I can imagine total newbies would not take much out of it, unless you show them some code and what effects can be achieved with manipulating the frustrum parameters. One thing which tripped me up is what are n and f values and how they are chooses... but this is probably just me not paying attention.
Damn this was good.