Patrons can now vote for the next video! Thank you for your support. ❤ Support me on Patreon: www.patreon.com/simondevyt 🌍 Live Demo + Courses: simondev.io
I actually had to write my own sine function for the BigDecimal class in Java, which is used to represent floating points with a variable amount of precision. It already had addition, subtraction, multiplication, division, integer powers, and on Java 9+, square roots. But it didn't have transcendental functions, so I had to code those in manually. My method was first to calculate π to the specified precision, using some method with cubic convergence that I forgot the name of. It then stores π in a hash table which correlates numbers of digits to approximations of π, so we don't have to recalculate π every time for trig functions of the same precision. After this, I perform range reduction, like you said. Although in this case, I actually limit the range to (-π/2,π/2], since as you'll see later, my approximation is an odd function and has the same precision for x as it does -x. After that, I acknowledge the fact that angle duplication, triplication, quadruplication, etc formulas exist, and since BigDecimal stores everything in decimal, I figured it'd be trivial to simply multiply the angle by 0.2, calculate the sine, then apply an angle quintuplification formula at the end. Now our range of calculation has been reduced to (-π/10,π/10]. I then tell it to calculate the sine of our 1/5 angle using a certain number of iterations of the Taylor's series, the number depending on how many digits we have, then apply the quintuplification formula, and we're done. The cosine is calculated in a similar way. Luckily, multiplying an angle by an odd number does not require knowing the other number. That is, knowing the sine of nx does not require knowing the cosine of x, and knowing the cosine of nx does not require knowing the sine of x. Furthermore, it's easy to find how many iterations you need, since π/10 ≈ √(1/10) (slightly less, actually), and n! goes up super-exponentially, so worst case, your Taylor series adds slightly more than 1 digit each time.
Sorry I missed this Simon - I do follow your channel and would have probably gone with this … I’ve already bought ThreeJS journey by Simon Bruno so can’t really afford yours as well but let’s hope there’s another discount in the future?
As someone learning game programming with almost zero useful formal mathematical education (through no ones fault other than my own - I was just a terrible math student), this was really accessible. Thank you for taking the time to put this stuff together in such a nice format.
Game math isn't as bad as the stuff they teach in school -- since it's visualized and meaningful. Of course it's still difficult, but knowing what a dot product is is a lot better than solving a dot product on paper. Same with cross and all that. You mainly just need to know what the functions do. It does help if you understand how to read math notation and even better if you know the little tricks of everything like a mathematician so you can figure out solutions to problems that other people haven't encountered or find ways to optimize certain things. I used to be an F student for my whole life, but found after learning math through games that I am very good at it and even decided to take some classes and got great grades. With classes there is a ton of memorization and formulas, and you have to know the basics extremely well. Good luck though, game programming is very difficult. And debugging and testing are a lot harder in game dev than most other types of software dev
Having been a bad math student isn't the end, you can still improve through TH-cam vids and stuff, in fact, you can be pretty good at math and not score well in exams, this was a common issue I had. So don't say it's over
It's so strange to hear someone admit to doing bad at maths online. First thing people do online is usually is blame literally everything except themselves lol
Inverse-tangent is part of the formula for angular size. theta = 2*atan((r/2)/d), where r is the radius of the object and d is the distance to it. Something you'll need to approximate if you want to do sprites in 3D.
In defense of the triangle, everything with the unit circle is literally just an application of the triangle! And in my opinion the most intuitive explanation of the weird tan graph actually comes from looking at the triangle: since tan is opp/adj, what happens when the adjacent side gets shorter and shorter? The output is going to get bigger and bigger until you get ridiculously big numbers (ie: 10/1=10, 10/0.00001=100000, 10/0.00000000001=100000000000, etc.), then when the length adjacent side is 0... (Also, quick correction at 4:47, the visuals for the Bhaskara I's Approximation show it matching for [0,2π], though it actually breaks off pretty soon after π.)
Yeah, even just slightly different presentations can make a huge different people. Oops, my bad, I thought I drew it on 0, 2pi but said the range was only 0, pi, lemme go back and check.
Yeah, a circle was what I taught myself what sin and cos were about when programming in Scratch as a child. And when school started explaining sin and cos with triangles I was like "huh?"
@@Afreshio That seems weird, the triangles in computer programming are not necessarily right triangles, which is a prerequisite for the definition of sin/cos/tan
Digital logic designer here. I use the CORDIC algorithm to calculate sine and cosines because it is simple, fast, and small (in terms of chip area) and involves only bitshifts additions and subtraction. You can also use it to calculate absolute magnitude of a vector, or atan2() of an x/y pair.
Very good video. I appreciated the part where you show how the symmetries of the sine function can lead to very efficient and accurate Taylor expansion computations. A note I will make is while a circle can be divided into any number of parts, the choice of 360 wasn't completely arbitrary historically. 360 is a highly composite number, that is it has more factors than any number before it. You can easily divide it by 2, 3, 4, 5, 6, 8, ,12 ,30, 60, 120... (there are several more). It's easy to imagine why this was useful for say, cartographers and navigators trying to make easy shortcuts for computing angles.
It is mathematically arbitrary, as it is not more valid than any other number in any true mathematical sense. It makes things nicer for us, and the way we represent things with our symbols. Just like how we use base ten number system, as opposed to any other base for our number system.
@@Supreme_Lobster we didn’t invent math, we discovered it. We invented the symbols and system we use to represent it, but the mathematical truths were not “invented.” Math is about truth. In math, what symbols or system you use has no effect on the truth. The only thing our choice has done is make things aesthetically pleasing to us given the system we already chose.
@@andrewkarsten5268 Yeah, but you don’t realize that truth is arbitrary. If humans were to have defined math using different axioms, truth would be defined differently. Math isn’t discovered, just like language isn’t “discovered.” Just because you can use language to describe things well doesn’t mean youre “discovering” language. Math describes things better, but that doesn’t make it discovery. Humans intentionally casted off mathematical constructions and concepts that don’t end up proving to be useful, through a process of knowledge evolution. Did humans also “discover” opposable thumbs? No, they were evolved and persist because they are useful. Math looks like discovery because you only are taught about the concepts that work.
I have been wondering what Sin Cos and Tan were for ages. I could never find any halfway descent explanation of what they were only that they were. Thank you immensely.
5:35, Recently seen this concept again in the Kaze Emanuar M64 optimization videos. Using a folded polynomial solution, with 1/8th of the sine wave to reconstruct the whole sin/cos graph.
Love the video. Unlike many commenters here I learned trigonometry exactly as explained in the video with a circle of radius 1. In fact trigonometry was the first time that math "clicked" for me. I guess having a good way of visualizing what is being explained helps me a ton.
I've fiddled around with approximating sines and cosines. I like the idea of reducing things to the range from 0 to pi/2. One more step, I've found, is to care only about the range from 0 to pi/4. What happens from pi/4 to pi/2? Well, that'd be the complementary function of the complementary angle. Like, the sine of 80 degrees would be the same as the cosine of 10 degrees. Polynomial expansions close to zero tend to get very accurately very quickly. So that region around pi/4 (45 degrees) is where things would converge the slowest, but even then, I think we've got an approach for that. Remember that sin(a - b)= sina*cosb + cosa*sinb, so if you wanted to calculate the sine of 40 degrees, that's the sine of (45 - 5) degrees. Okay then, so we'd have to calculate sin(45)*cos(5) - cos(45)*sin(5), and sin(45) = cos(45) = 0.7071etc, so we've got very little to calculate. The answer will be 0.7071etc*(cos(5) - sin(5)), and those will converge quickly. At this point we're approximating just from 0 to pi/8.
Okay, I did some more mathing on this. If we take the measures as described above, the most error-prone calculation we could perform is cos(pi/8); and yet, even at three Taylor terms (1 - x^2/2 + x^4/24), the error is only 0.00000508. That's 5 parts per million. Add a fourth term (-x^6/720) and the error drops to 0.000000014, which is 14 parts per billion. And that's the worst case scenario: this converges to a reliable value pretty fast. I imagine a person could keep going with this basic approach, where one reckons against angles with known values, and thus has to Taylor Expand against only an offset. If we reckon against pi/16, even three terms will get us to an error of 80 parts per billion. (A fourth term will take us to 55 parts per trillion. Holy smokes, that's got to be accurate enough for essentially every purpose.)
@@kingbeauregard correct me if I'm wrong but, if I understand floating point numbers correctly, to be as precise as can be for a float (32 bit with 24 significant bit), you need the error to be less than 1/2^24 (~0.0000000596), and for a double (64 bit with 53 significant bit) 1/2^53 (~0.00 000 000 000 000 0111).
@@meandyours I don't really know how floating point numbers are represented internally. I guess that the degree of precision required is dependent upon the application. I do know this much though (and, arguably, not much more). Let's say you were trying to figure out the deviation between two functions and there was some trig involved, and the differences would be small, like in the "thousandth of a percent" range. The smart approach would be to do Taylor Series of the two functions and then subtract one series from the other; you'd probably find that most of the initial terms would go away altogether, and you'd be left with terms that start at the order of magnitude that matters.
@@kingbeauregard Basically, a floating point number is represented in scientific notion, except in binary. The significant bit (binary digit) is the equivalent to the significant digit of a scientific notation. However, a floating point number can only be so precise as its size is limited, that's what I mean on the precision needed, as anymore wouldn't be representable sry, i was looking at this more from the computer science angle
I hated trigonometry in high school because any time I asked what the trig functions were I would just get a vague hand-wavy "ratio of triangle sides" explanation. Even though I eventually learned all of this on my own eventually I still appreciate you giving a more intuitive explanation.
I didn’t find that hand-wavy at all. The easy way to remember which triangle sides are involved is, from 0°, sine is the one that gets larger as its corresponding side gets longer, and cosine is the one that gets smaller. Meantime, the hypotenuse (longest side) stays the same length as it rotates to describe a circle.
@@lawrencedoliveiro9104 But that's hand wavy. That's just a high level view. It doesn't explain why the Taylor Series would need to exist, or Bhaskara I's sine approximation, etc.
@@lawrencedoliveiro9104 Maybe read all the words next time. It doesn't say "the approximations are the answer" it says "what you claimed as an easy understanding is not at all the full understanding, note how there are complex approximation, and more, a full response is much more intricate,"
I have spent a year trying to find a video that explains what “radians” are as a unit of measure and you give it as a 1-sentence aside! Such a clear explanation, thank you.
This is one of the best videos I've seen on this topic! As an oldskool gamedev, the dot product, lookuptables, and quadrantifying/octantifying periodic values were very useful. This video just makes understanding them easier even for us retired users. Have you made videos on splines? I have been using both the Bezier and Catmull-Rom to do enemy patterns in SHMUPS. They're way easier to predict than using a combination of trig functions. A piece on how to derive an angle in regard to "t" as well as reparameterizing them so that points across splines are uniform in distance would be swell. Looking forward to your next vid!
I love old school dev tricks, miss being around a lot of the more senior coders who had crazy stories of gamedev back in the days. Was always impressed with their breadth of knowledge, would love to hear about some of your experiences!
This method works pretty well for sine, cosine, and tangent. There’s some other optimizations you can do, such as representing the Taylor polynomial with nested multiplication. Also, for arcsine, arccosine, arctangent, and log, some basic identities come in handy.
I never understood trigonometry before and even in coding all I needed to know that they make waves that I can exploit to make something move smoothly back and forth. But now I finally understand... until I forget again because I still won't use them to their fullest.
I heard my maths professor mention this YESTERDAY. I did zero research, didn't look anything about it yet youtube still recommended this to me. Jeez... how am I supposed to believe I don't live in simulation?
I spent a whole day trying to figure out how to convert my X,Y speed parameters into a simple vector that just described the angle in 0-360 degrees and the amplitude. I then came across and remembered the old trig formulas and bodged my way through. I laughed so hard when I saw the circle diagram here.
i always found it easier to view tan as the slope of the hypotenuse. it has asymptotes at pi/2 and 3pi/2 because the hypotenuse is vertical there. the reason the slope is equal to that length you showed is because slope is change in y over change in x, but when the radius is 1, the change in x becomes 1, so it’s just the change in y that matters. this is the length of that vertical line it sound strange but it you draw it all out and try to find the slope of a triangle in a circle manually and use 1 radius to be 1 unit, you’ll see it really easily. just remember to extend the hypotenuse
This is how sin and cos routines in various math libraries work, and it is often very possible to write an algorithm for trig operations which is lower latency than using a single CPU instruction with the trade off being less precision. Though latency can also be a little misleading at times since those operators are highly parallelized. If the FSIN instruction takes 170 cycles to give a result that doesn't mean your computer spends all that time waiting for an answer. It will try to do other operations with that core while it waits and can even have several FSIN operations being calculated simultaneously. So which is faster in practice can be more nuanced, as always seems to be the case with modern super scalar architectures. In hardware these operators are usually implemented using the CORDIC algorithm. The resources you linked discuss that a little. Essentially you implement a more general function that rotates a vector (x, y) by an arbitrary angle. For anyone with textheworld installed: [;\begin{bmatrix}x_{out} \\y_{out}\end{bmatrix}=ROT(\theta)\begin{bmatrix}x_{in}\\y_{in}\end{bmatrix}=\begin{bmatrix}x_{in} \\y_{in}\end{bmatrix}\begin{bmatrix}cos(\theta) & -sin(\theta)\\ sin(\theta) & cos(\theta)\end{bmatrix};] Sine and cosine in that situation are just the x and y coordinates that result from rotating the vector (1,0). Using only rotation in various ways it's possible to implement several other tricky functions as well including inverse trig functions, logarithms, hyperbolic trig functions, square roots, and even standard multiplication and division. The documentation for the Xilinx CORDIC IP core is pretty good for seeing how all that works in practice. The CORDIC algorithm takes advantage of the fact that rotation by an arbitrary angle can be expressed using a series of rotations by different angles which sum to that angle. For example to rotate by 67 degrees you could rotate a vector by 45 degrees, then by 30 degrees, then by -8 degrees. These smaller angles are chosen using powers of 2 so that the algorithm can use a bit shift operation rather than a more expensive multiplication. With this structure CORDIC calculates an answer to N bits of precision with N iterations of shift-addsub. The output is still an approximation, but its as precise an implementation of the trigonometric operations as floating point numbers are able to express.
I remember seeing a discussion on a game dev forum where the approach of 6:38 was used to fit a quadratic to that part of the graph, and the fit was quite good despite the simplicity. You could then use Newton's method to refine the result if you wanted more accuracy. OTOH if you're working at the hardware level, CORDIC is the usual algorithm of choice since it avoids any multiplies.
The legendary Clive Sinclair brought out a low-cost scientific calculator in 1975, complete with log and trig functions. Somehow his engineer, Nigel Searle, was able to implement all these functions in a ROM that could only hold 320 instructions. The algorithms he came up with were a bit slower than regular CORDIC, and had lower precision, but they worked. No other product of the time could offer that kind of functionality at that price point.
@@toasteduranium CORDIC means: coordinate rotation digital computer. If I recall correctly, it's an algorithm that iterates and only uses bit-shifts and add operations. More iterations means better approximations.
Well this video is really intuitive and useful, people use these built in libraries directly in their code and they don't know how the background process goes, thanks a lot for this video
lol, wow... That paper you showed at 7:05... I worked as a graduate intern on that exact team (especially, Shane and Ted), and I personally rewrote the entire test suite software for the extremely high performance code they were developing for the original Itanium (all of the "math.h functions", not just sine). Work included testing on beta versions of Win64 (and Linux), emulated IA-64 hardware, early IA-64 hardware revs, prototype compilers, and more. The team literally counted clock cycles for code paths through functions, maximally shaving them off.
@@simondev758 It's a long story, but no. They wanted me to stay (Ted in particular came back to me to [re-re-]confirm that I was sure), but I eventually went to graduate school instead, then wound up elsewhere. (Btw, "graduate intern" here means an internship immediately following graduation with an undergraduate degree. I'd already done two previous "undergraduate technical internships" there during earlier summers, performing other work, though still sort of tangentially related... the first one doing some research on dgemm (double-precision generalized matrix-matrix multiplication) for the team developing Intel's Math Kernel Library---a very, very high performance linear algebra suite that got incorporated under the hood into enterprise software such as MATLAB.)
This was so helpful. I don't know if I missed it, but I don't think I've ever seen tan explained like that, and I certainly had no idea why it made such weird shapes on the graph, but now it makes simple and intuitive sense, thank you!
Its funny how when you need to learn something you learn it alot easier than when youre forced to learn it. I willing learnt about the different trig functions a while back. Nice to see the OG making a vid on it😆
this is a really well made video, and I actually liked it....I honestly, like a lot of people, copy a lot of math functions from other code when I am less familiar with it...then I store it away in a kind of cheat sheet until I get comfortable with the function....
I was diving into this recently, particularly Robin Green's writing on the subject, as I am - out of curiosity and for fun - building a simple game engine in Rust, and have found that Rust - as of writing - seems to lack bindings for SIMD-accelerated sine, cosine, and sincos functions, which I seem to remember having seen in C++, so I decided to try and see if I could create an efficient implementation using SSE or AVX intrinsics. In the document Robin Green lists at the end of his blog I saw code that superficially resembles the code shown at 6:46 though the switch statement seems different. Green uses 'switch(k & 3)' and then three case statements (with constants 0, 1 and 2) and then a 'default' fallback case. I think this works because it is guaranteed to be exhaustive, the AND with 3 means that all but the lowermost 2 bits are zeroed, effectively leaving a 2-bit integer, which of course can only encode 0, 1, 2 and 3. There's another difference though that confuses me. In the code shown in the video, integer 'quadrant' is defined as k mod 4, presumably to wrap k to range [0, 3], but then the switch statement refers to k and not quadrant. As far as I can tell, this means that k is not wrapped to the range of [0, 3] and 'quadrant' is not used. Am I correct in this, or am I misreading something?
Yeah, I screwed up the image of the code. The shader itself was actually using the corrected version, but I never updated the imaged to reflect it, sigh.
Being an old time dev, I have always worried about how sin & cos are being computed in case they are expensive, as I'd heard how early 3D engines used the lookup table approach as a speed hack. But seeing how it's done these days has set my mind at ease. Thanks! :)
Thank you, short and concise, as I like it. Me being older, I try to pick up geometric algebra from TH-cam videos, at high school level. It took me a while to figure out why if u = ae1+be2 and v = ce1+de2, then u.v is both ac+bd and |u||v|cos(th). I did, and your video shows it clearly. From a stupid looking formula, the inner product proves to be very useful, I really start to like geometric algebra!
Nice video and well explained. I really like that you also showed some code. One thing I noticed is that in the code at 6:50 it should be “switch (quadrant)”, but I guess you added this to check whether or not we’re paying attention? 🤓
In the 70's I was slinging Fortran for Space Shuttle Simulators. When I started running the guidance software to support landing, everything I was monitoring was going along fine until just before landing when the simulated shuttle did a maneuver to transition from the steep glide slope to the shallow inner glide slope (-1.5 degrees) just before touchdown. The shuttle would suddenly climb , slow down, stall, and crash. I got with the vendor of the computer we were using and he pulled out the scientific subroutine library code-we were looking at hard copies here, no CRTs yet. It turned out that the sin function would start out by testing the value of the argument you were handing it then go to a unique Taylor Series expansion for that sized angle-this was partly necessary to get everything to execute in real-time. Sure enough, for certain small negative angles, the particular Taylor series it used was wrong and would return a positive sin instead of a negative one which is what it should be for a -1.5 degree angle. The guidance algorithm would then generate a positive (upward) altitude rate reference and issue commands to the control effectors to try to fly to it, which it did.
Thank you. This remind me that I could implement various versions of approximate sin and cos functions depending on the accuracy I need for my application, and thus get close enough results faster than when using standard sin and cos function defined by the language or the default Math library. Standard sin and cos function defined by the language, its standard library or the ALU are in most cases compromises that are at the same time way to slow for real time artistic applications and way to inaccurate for science research. It's good to master this so we know what we do, always do enough without overdoing.
One of the neatest tricks I ever learned was called CORDIC, which is something like Coordinate Rotation Digital Computation. It expresses an angle as sums and differences of smaller angles which are chosen so that rotation by those smaller operations are trivially simple, operations on bits. Since a rotation of the vector (1,0) can give you sine and cosine, those are easy to compute. But suppose you are given x and y and you want to compute the angle that rotates it onto sqrt(x^2+y^2) - e.g. find the angle. It's all additions and subtractions. You can also get any of the trig functions with similar tricks. If you have to build the hardware implementation of trig computations, it's really easy.
Super neat, I'm reading a bit about it now. Any idea if modern hardware still uses CORDIC? I found references for intel using it on older generations of hardware, 486, and the paper I cited comes out in 1999.
@@simondev758 Simon, I haven't worried about that for a long time. I first learned about it when Hewlett Packard came up with a hand-held digital calculator which could do trig functions, etc. The trig functions were all base don CORDIC techniques. In those days, a hand-held calculator hadn't gotten the years of advantage of Moore's law, so things like lookup tables were pretty burdensome.
Yes, CORDIC was used on handheld calculators. It is a much friendlier way to take a small, carefully crafted lookup table and create solutions for any precision you desire. Modern algorithms include the Chebychev series. Taylor series are too slow to converge, so they are not used for trig functions.
I've been working on low level graphics functions for some ten years now and it blows me away how bad people are at explaining trig functions. You, my dude, have nailed it. The unit circle is easily the best way to visualize how they work. Looking forward to see how you simplify matrices and quaternions, assuming you'll be doing those. The best way I've found *personally* to represent the former is using basis vectors and showing how it's like a box within a box within a box, if that makes sense. The latter's a bit trickier, but I usually imagine it as a ray emitted from a given point, with everything rotated a given number of radians around that ray. Hope this channel grows. I'll do my part. :p
I think I really learned to appreciate trigonometry after learning it in school. It's just a beautiful concept that just is so satisfying to watch. (And it has some cool ways to apply it in engineering to make fun stuff)
School does everything wrong. This is a much better explanation than anything school has ever taught me. I just luckily happened to figure out these circle rules on my own when playing with code, but the school system doesn't provide the intuitive or useful explanations of math 😔
Actually our school uses unit circles to show trig and their derived properties, but the last part (about LUT and approximation) is completely new to me 😮
In fact you can reduce the range even further, by noticing that the sin function, for x ranging from pi/4 to pi/2 is a mirror of the cos function between 0 and pi/4. And the Taylor series in the 0 to pi/4 range converges to an optimal precision after very few terms, both for sin and cos. This is what I did, without suggestions from anyone, on my Z80, in assembly code, back in the early eighties.
Never for a millisecond even questioned the ability for a computer to calculate sin(x) with perfect accuracy until I saw this thumbnail Then you explained how some games and stuff approximate it for speed, but now I'm realizing even on a *supercomputer* we've *never* been able to perfectly calculate sin(x), because it's a function based on π, an irrational number of infinite length which can't be represented in a computer. This existential crisis quickly led to another, realizing that we *will never be able to perfectly represent sin or any trig function because pi goes on forever, and that in a sense ALL of the maths we do, both on the computer and off, is like this: only theoretical on nature and every actual calculated number we use is just an approximation of some unobtainable truth of the universe and...* I think I need a drink
But the grat benefit of these algorithms and formulas ( Baskhara, CORDIC, Chevyshev, etc.) Is that you use only the precision that you need. And talking about precision, there are fórmulas with very fast convergence for computing billions and trillions of digits of π (I forgot the Authors of the paper from 2004 or 2008) that even can compute segments of digits (for example between 1 million to 10 million, or between 1 billion and 10 billion, etc,) without even computing the precedent digits. They say the techniques used to get those fórmulas come from INTERVAL ARITHMETIC.
Great video, easy to learn from. I personally, find it much easier to see tanθ as the slope of the terminal arm. That is, once you place your θ and extend the terminal arm to the circle, the arm's rise/run is tanθ.
it’s really the same idea though. he explained tangent as the height of the line when you connect it with the unit circle on the x-axis. this height is the rise. the unit circle, with radius 1, is the run
I went with this because I like this version, it makes the most sense to me in terms of how it's calculated. Maybe next time, I'll put every possible visualization.
In first year university calculus we imagined that there are two functions s(x) and c(x) such that the first derivative (slope) of s is given by c and the 1st derivative of c is -s. It is demonstrated that s is sine and c is cosine. In Intel/AMD processors the microprocessor provides all of the primary mathematical functions. This is because it is more efficient to perform the calculations using the underlying hardware directly rather than exploiting the complex instruction set implementations of multiplication and division. This is not an advantage on reduced instruction set computers (RISC) such as ARM or RISC5 so they provide only the basic floating point operations. When you look at the x86 "hardware" instruction that provides sine you will observe that it returns *both* the sine and cosine values for the argument. Although this does permit obtaining the values of tangent and cotangent that is not why this is done. It is an artifact of the use of the Newton-Rapson algorithm to refine an initial approximation together with permitting further restricting the range of values that must be supported to just 0 through pi/4 by observing that sin(x) is cos(pi/2 - x) and cos(x) is sin(pi/2 - x). With modern inexpensive memory it is cheaper to use a table of, for example 256, values of sin and cos or any other transcendental function and then use the Newton-Rapson method to refine the value, especially to get 64 or 128 bit extended floating point values, than to use a polynomial. Each loop through the Newton-Rapson method increases the precision by a number of digits. All arithmetic operations more complex than multiplication depend intrinsically on gradually refining an approximation, usually by a variant of Newton-Rapson. For example look up videos on how to extract a square root by hand. Why do you divide the remainder by 2 times the current estimate? Because that is the 1st derivative of x^2! Also recall that you all learned how to do arithmetic by a table lookup. 7 times 8 is 56.
This comment kinda brings back memory of this super obscure function in the quake source code I ran into way back, this fast inverse square root that used, aside from some floating point bit wizardry, pretty sure there was a newton iteration in there.
My issue with the unit circle explanation is that it's so often framed as "the triangle explanation doesn't make intuitive sense" or some variation thereof, but that just creates (pardon the pun) circular logic: Lets say we want to figure out the sine and cosine of some angle using nothing but pen and paper. Using the circle we know the radius, the arc length and... now what? Using the arc length and radius, we could find 1/2 the length of the chord that spans from the point we want to know to it's mirror about the x-axis (to get the y-value) or the y-axis (to get the x-value)... but that requires inverse sine, and how do we find that by hand? Or say we have the radius, the angle AND the x-value. Cosine = x, but what's sine? How do we know that? Well, see cos = adjacent / hypotenuse and... dangit those are triangle terms. Okay let's just take it as a given that cos = x. How do we find the y-value? Well if we know x and radius, we could say that r^2 = x^2 + y^2 and solve from there... but that's the Pythagorean formula and it was, surprise surprise, derived from right triangles! Even the CS solution is either "look it up i guess" or "approximate without calculating it," so, near as I can tell, we're stuck with triangles and the circle just gives a more pleasant visual representation. All that being said, this was an simple to understand and visually pleasant overview of the underlying trig and how it's enacted in computers. I'm not a game dev, and still I'm considering purchasing your course because it might contain equally enlightening topics told in a simple to understand, yet still concise way
I think with the unit circle and using more lines you can also visualize COT, SEC, and CSC. If I am not mistaken, the unit hyperbola is used to visualize the HYPERBOLIC TRIG FUNCTIONS, Sinh, Cosh, Tanh, etc
After reading many comments here by different persons, i collected a brief summary about the algorithm CORDIC (not the algorithm itself): X The CORDIC algorithm was used to compute Trig functions in hand-held calculators (HP and others), and computers with slow processors and small memory, but is specially useful for implementation in hardware as bit-shift operations and additions and substractions, which is very easy. You can also use CORDIC to compute the absolute value of a vector, and the function atan2() of any pair of x/y . The Baskhara Approximation formula can be used to compute sin(x), too (but maybe with less precision). A more modern algorithm than CORDIC ( and vastly superior, they say) uses Chevysev Series (or Approximation). Taylor series are very slow to converge, so they are not used to compute Trig functions.
Congrats on 100k subscribers :) Have been following your channel for a while now and absolutely love your content. Even the video about NFT minting, people were just bashing everything NFT related. The tech behind it is still interesting so please don't stop making videos about topics you like to learn about even if they are not graphics/game-dev related
That Jigsaw comment. 🤣 I wonder what level of precision he would want for a trig approximation. "Hello, Reginald. I want to play a game. For the past semester you have neglected your studies of series expansions by relying on Mathematica and Desmos. Inside this cabinet you will find paper and a pencil and three trig problems to solve. You will have to approximate your answers to four decimal places, within an error of..."
Not somebody with any formal Math or CS education or anything _what-so-ever,_ but I remember reading about the use of "Chebyshev polynomials" for optimization on Stackoverflow, and how the Taylor series approximation has been rejected many times for use in, say, a standard library. Many more tricks were mentioned, and it's just a bunch of tricks about crazy/trivial micro-optimizations. Thoughts?
Great video as I’m going into calculus and hadn’t quite grasped the relationships and just been working out their inverses & antiderivatives & derivatives. I love a similar videos on csc, sec, and cot if your up for it
0:32 What do you mean with come in handy close to zero times? The unit circle is basically the same thing, only scaled one side to 1. If you use the formula on it you get the same thing. :D
Yeah, they're the same, just presented in different ways. I've always worked with vectors for the most part, so working in the context of the circle and the relationships therein is what I'm used to.
Definitely! I have a really good idea for some other math ones. I'd also like to get into some more advanced math topic in game development like spherical harmonics.
Patrons can now vote for the next video! Thank you for your support.
❤ Support me on Patreon: www.patreon.com/simondevyt
🌍 Live Demo + Courses: simondev.io
I actually had to write my own sine function for the BigDecimal class in Java, which is used to represent floating points with a variable amount of precision. It already had addition, subtraction, multiplication, division, integer powers, and on Java 9+, square roots. But it didn't have transcendental functions, so I had to code those in manually.
My method was first to calculate π to the specified precision, using some method with cubic convergence that I forgot the name of. It then stores π in a hash table which correlates numbers of digits to approximations of π, so we don't have to recalculate π every time for trig functions of the same precision.
After this, I perform range reduction, like you said. Although in this case, I actually limit the range to (-π/2,π/2], since as you'll see later, my approximation is an odd function and has the same precision for x as it does -x. After that, I acknowledge the fact that angle duplication, triplication, quadruplication, etc formulas exist, and since BigDecimal stores everything in decimal, I figured it'd be trivial to simply multiply the angle by 0.2, calculate the sine, then apply an angle quintuplification formula at the end. Now our range of calculation has been reduced to (-π/10,π/10]. I then tell it to calculate the sine of our 1/5 angle using a certain number of iterations of the Taylor's series, the number depending on how many digits we have, then apply the quintuplification formula, and we're done. The cosine is calculated in a similar way. Luckily, multiplying an angle by an odd number does not require knowing the other number. That is, knowing the sine of nx does not require knowing the cosine of x, and knowing the cosine of nx does not require knowing the sine of x. Furthermore, it's easy to find how many iterations you need, since π/10 ≈ √(1/10) (slightly less, actually), and n! goes up super-exponentially, so worst case, your Taylor series adds slightly more than 1 digit each time.
Sorry I missed this Simon - I do follow your channel and would have probably gone with this … I’ve already bought ThreeJS journey by Simon Bruno so can’t really afford yours as well but let’s hope there’s another discount in the future?
But what you did IS A REAL MATH PROOF..just a visual geometric one..see what I mean?
As someone learning game programming with almost zero useful formal mathematical education (through no ones fault other than my own - I was just a terrible math student), this was really accessible. Thank you for taking the time to put this stuff together in such a nice format.
I feel you. I jumped In hard into wanting to learn machine learning. The grind is real
Game math isn't as bad as the stuff they teach in school -- since it's visualized and meaningful. Of course it's still difficult, but knowing what a dot product is is a lot better than solving a dot product on paper. Same with cross and all that. You mainly just need to know what the functions do. It does help if you understand how to read math notation and even better if you know the little tricks of everything like a mathematician so you can figure out solutions to problems that other people haven't encountered or find ways to optimize certain things. I used to be an F student for my whole life, but found after learning math through games that I am very good at it and even decided to take some classes and got great grades. With classes there is a ton of memorization and formulas, and you have to know the basics extremely well.
Good luck though, game programming is very difficult. And debugging and testing are a lot harder in game dev than most other types of software dev
Having been a bad math student isn't the end, you can still improve through TH-cam vids and stuff, in fact, you can be pretty good at math and not score well in exams, this was a common issue I had. So don't say it's over
It's so strange to hear someone admit to doing bad at maths online. First thing people do online is usually is blame literally everything except themselves lol
I've listened while school was teaching math and still find youtube to be much better at making me actually understand the subject.
Tangent is very useful for converting angles to slopes. Tangent of an angle is literally precisely the slope of any line with that angle from +x.
i use inverse tan and then cos and sin to find points on lines given the slope but not the line equation
Inverse-tangent is part of the formula for angular size. theta = 2*atan((r/2)/d), where r is the radius of the object and d is the distance to it. Something you'll need to approximate if you want to do sprites in 3D.
Oh my gosh. That is why they are both respectively called TANGENT. How have I NEVER put two and two together until now
Indeed. It’s been hiding in plain sight since algebra. “rise over run”
Yeah slope is just hight over distance. Sin over cos
In defense of the triangle, everything with the unit circle is literally just an application of the triangle! And in my opinion the most intuitive explanation of the weird tan graph actually comes from looking at the triangle: since tan is opp/adj, what happens when the adjacent side gets shorter and shorter? The output is going to get bigger and bigger until you get ridiculously big numbers (ie: 10/1=10, 10/0.00001=100000, 10/0.00000000001=100000000000, etc.), then when the length adjacent side is 0...
(Also, quick correction at 4:47, the visuals for the Bhaskara I's Approximation show it matching for [0,2π], though it actually breaks off pretty soon after π.)
Yeah, even just slightly different presentations can make a huge different people.
Oops, my bad, I thought I drew it on 0, 2pi but said the range was only 0, pi, lemme go back and check.
Yeah, a circle was what I taught myself what sin and cos were about when programming in Scratch as a child. And when school started explaining sin and cos with triangles I was like "huh?"
No idea why all the love for the triangle version, feels unintuitive
@@simondev758 i've heard trig favores triangles in Comp graphics because 3D graphics are made of tiny triangles or something like that
@@Afreshio That seems weird, the triangles in computer programming are not necessarily right triangles, which is a prerequisite for the definition of sin/cos/tan
@@simondev758 it's easier to show and follow as people are more familiar with basic shapes lile right triangles
@@Afreshio thats not true at all
Digital logic designer here. I use the CORDIC algorithm to calculate sine and cosines because it is simple, fast, and small (in terms of chip area) and involves only bitshifts additions and subtraction.
You can also use it to calculate absolute magnitude of a vector, or atan2() of an x/y pair.
Very good video. I appreciated the part where you show how the symmetries of the sine function can lead to very efficient and accurate Taylor expansion computations.
A note I will make is while a circle can be divided into any number of parts, the choice of 360 wasn't completely arbitrary historically. 360 is a highly composite number, that is it has more factors than any number before it. You can easily divide it by 2, 3, 4, 5, 6, 8, ,12 ,30, 60, 120... (there are several more). It's easy to imagine why this was useful for say, cartographers and navigators trying to make easy shortcuts for computing angles.
It is mathematically arbitrary, as it is not more valid than any other number in any true mathematical sense. It makes things nicer for us, and the way we represent things with our symbols. Just like how we use base ten number system, as opposed to any other base for our number system.
@@andrewkarsten5268 well, we invented math for us. It is supposed to work for us, not against us lol
@@Supreme_Lobster we didn’t invent math, we discovered it. We invented the symbols and system we use to represent it, but the mathematical truths were not “invented.” Math is about truth. In math, what symbols or system you use has no effect on the truth. The only thing our choice has done is make things aesthetically pleasing to us given the system we already chose.
@@andrewkarsten5268 Okay, but you know they weren't saying we invented mathematical truths. You're being petty for the sake of it
@@andrewkarsten5268 Yeah, but you don’t realize that truth is arbitrary. If humans were to have defined math using different axioms, truth would be defined differently. Math isn’t discovered, just like language isn’t “discovered.” Just because you can use language to describe things well doesn’t mean youre “discovering” language. Math describes things better, but that doesn’t make it discovery. Humans intentionally casted off mathematical constructions and concepts that don’t end up proving to be useful, through a process of knowledge evolution. Did humans also “discover” opposable thumbs? No, they were evolved and persist because they are useful. Math looks like discovery because you only are taught about the concepts that work.
I have been wondering what Sin Cos and Tan were for ages. I could never find any halfway descent explanation of what they were only that they were. Thank you immensely.
5:35, Recently seen this concept again in the Kaze Emanuar M64 optimization videos. Using a folded polynomial solution, with 1/8th of the sine wave to reconstruct the whole sin/cos graph.
Love the video. Unlike many commenters here I learned trigonometry exactly as explained in the video with a circle of radius 1. In fact trigonometry was the first time that math "clicked" for me. I guess having a good way of visualizing what is being explained helps me a ton.
I've fiddled around with approximating sines and cosines. I like the idea of reducing things to the range from 0 to pi/2. One more step, I've found, is to care only about the range from 0 to pi/4. What happens from pi/4 to pi/2? Well, that'd be the complementary function of the complementary angle. Like, the sine of 80 degrees would be the same as the cosine of 10 degrees. Polynomial expansions close to zero tend to get very accurately very quickly.
So that region around pi/4 (45 degrees) is where things would converge the slowest, but even then, I think we've got an approach for that. Remember that sin(a - b)= sina*cosb + cosa*sinb, so if you wanted to calculate the sine of 40 degrees, that's the sine of (45 - 5) degrees. Okay then, so we'd have to calculate sin(45)*cos(5) - cos(45)*sin(5), and sin(45) = cos(45) = 0.7071etc, so we've got very little to calculate. The answer will be 0.7071etc*(cos(5) - sin(5)), and those will converge quickly. At this point we're approximating just from 0 to pi/8.
Good stuff
Okay, I did some more mathing on this. If we take the measures as described above, the most error-prone calculation we could perform is cos(pi/8); and yet, even at three Taylor terms (1 - x^2/2 + x^4/24), the error is only 0.00000508. That's 5 parts per million. Add a fourth term (-x^6/720) and the error drops to 0.000000014, which is 14 parts per billion. And that's the worst case scenario: this converges to a reliable value pretty fast.
I imagine a person could keep going with this basic approach, where one reckons against angles with known values, and thus has to Taylor Expand against only an offset. If we reckon against pi/16, even three terms will get us to an error of 80 parts per billion. (A fourth term will take us to 55 parts per trillion. Holy smokes, that's got to be accurate enough for essentially every purpose.)
@@kingbeauregard correct me if I'm wrong but, if I understand floating point numbers correctly, to be as precise as can be for a float (32 bit with 24 significant bit), you need the error to be less than 1/2^24 (~0.0000000596), and for a double (64 bit with 53 significant bit) 1/2^53 (~0.00 000 000 000 000 0111).
@@meandyours I don't really know how floating point numbers are represented internally. I guess that the degree of precision required is dependent upon the application.
I do know this much though (and, arguably, not much more). Let's say you were trying to figure out the deviation between two functions and there was some trig involved, and the differences would be small, like in the "thousandth of a percent" range. The smart approach would be to do Taylor Series of the two functions and then subtract one series from the other; you'd probably find that most of the initial terms would go away altogether, and you'd be left with terms that start at the order of magnitude that matters.
@@kingbeauregard Basically, a floating point number is represented in scientific notion, except in binary. The significant bit (binary digit) is the equivalent to the significant digit of a scientific notation. However, a floating point number can only be so precise as its size is limited, that's what I mean on the precision needed, as anymore wouldn't be representable
sry, i was looking at this more from the computer science angle
I hated trigonometry in high school because any time I asked what the trig functions were I would just get a vague hand-wavy "ratio of triangle sides" explanation. Even though I eventually learned all of this on my own eventually I still appreciate you giving a more intuitive explanation.
I didn’t find that hand-wavy at all. The easy way to remember which triangle sides are involved is, from 0°, sine is the one that gets larger as its corresponding side gets longer, and cosine is the one that gets smaller. Meantime, the hypotenuse (longest side) stays the same length as it rotates to describe a circle.
@@lawrencedoliveiro9104 But that's hand wavy. That's just a high level view. It doesn't explain why the Taylor Series would need to exist, or Bhaskara I's sine approximation, etc.
@@error.418 All those are approximations, which are just as hand-wavy.
He asked what the functions were, not the ratios of side lengths; therefore, a proper answer to the actual question being asked was not given.
@@lawrencedoliveiro9104 Maybe read all the words next time. It doesn't say "the approximations are the answer" it says "what you claimed as an easy understanding is not at all the full understanding, note how there are complex approximation, and more, a full response is much more intricate,"
I have spent a year trying to find a video that explains what “radians” are as a unit of measure and you give it as a 1-sentence aside! Such a clear explanation, thank you.
This is one of the best videos I've seen on this topic!
As an oldskool gamedev, the dot product, lookuptables, and quadrantifying/octantifying periodic values were very useful.
This video just makes understanding them easier even for us retired users.
Have you made videos on splines? I have been using both the Bezier and Catmull-Rom to do enemy patterns in SHMUPS. They're way easier to predict than using a combination of trig functions. A piece on how to derive an angle in regard to "t" as well as reparameterizing them so that points across splines are uniform in distance would be swell.
Looking forward to your next vid!
I love old school dev tricks, miss being around a lot of the more senior coders who had crazy stories of gamedev back in the days. Was always impressed with their breadth of knowledge, would love to hear about some of your experiences!
This 5 minute of trig taught me more than my math teacher could in 6 months
This method works pretty well for sine, cosine, and tangent. There’s some other optimizations you can do, such as representing the Taylor polynomial with nested multiplication. Also, for arcsine, arccosine, arctangent, and log, some basic identities come in handy.
Definitely, I feel like a full on function approximation video would be interesting
@@simondev758 Please do that one for now I’ll be liking and subscribing
Thanks! Solid video as always! Can't wait for the new course
I never understood trigonometry before and even in coding all I needed to know that they make waves that I can exploit to make something move smoothly back and forth.
But now I finally understand... until I forget again because I still won't use them to their fullest.
I heard my maths professor mention this YESTERDAY. I did zero research, didn't look anything about it yet youtube still recommended this to me.
Jeez... how am I supposed to believe I don't live in simulation?
Eerie!
I spent a whole day trying to figure out how to convert my X,Y speed parameters into a simple vector that just described the angle in 0-360 degrees and the amplitude. I then came across and remembered the old trig formulas and bodged my way through. I laughed so hard when I saw the circle diagram here.
i always found it easier to view tan as the slope of the hypotenuse. it has asymptotes at pi/2 and 3pi/2 because the hypotenuse is vertical there.
the reason the slope is equal to that length you showed is because slope is change in y over change in x, but when the radius is 1, the change in x becomes 1, so it’s just the change in y that matters. this is the length of that vertical line
it sound strange but it you draw it all out and try to find the slope of a triangle in a circle manually and use 1 radius to be 1 unit, you’ll see it really easily. just remember to extend the hypotenuse
This is how sin and cos routines in various math libraries work, and it is often very possible to write an algorithm for trig operations which is lower latency than using a single CPU instruction with the trade off being less precision. Though latency can also be a little misleading at times since those operators are highly parallelized. If the FSIN instruction takes 170 cycles to give a result that doesn't mean your computer spends all that time waiting for an answer. It will try to do other operations with that core while it waits and can even have several FSIN operations being calculated simultaneously. So which is faster in practice can be more nuanced, as always seems to be the case with modern super scalar architectures.
In hardware these operators are usually implemented using the CORDIC algorithm. The resources you linked discuss that a little.
Essentially you implement a more general function that rotates a vector (x, y) by an arbitrary angle.
For anyone with textheworld installed:
[;\begin{bmatrix}x_{out} \\y_{out}\end{bmatrix}=ROT(\theta)\begin{bmatrix}x_{in}\\y_{in}\end{bmatrix}=\begin{bmatrix}x_{in} \\y_{in}\end{bmatrix}\begin{bmatrix}cos(\theta) & -sin(\theta)\\ sin(\theta) & cos(\theta)\end{bmatrix};]
Sine and cosine in that situation are just the x and y coordinates that result from rotating the vector (1,0). Using only rotation in various ways it's possible to implement several other tricky functions as well including inverse trig functions, logarithms, hyperbolic trig functions, square roots, and even standard multiplication and division. The documentation for the Xilinx CORDIC IP core is pretty good for seeing how all that works in practice.
The CORDIC algorithm takes advantage of the fact that rotation by an arbitrary angle can be expressed using a series of rotations by different angles which sum to that angle. For example to rotate by 67 degrees you could rotate a vector by 45 degrees, then by 30 degrees, then by -8 degrees. These smaller angles are chosen using powers of 2 so that the algorithm can use a bit shift operation rather than a more expensive multiplication. With this structure CORDIC calculates an answer to N bits of precision with N iterations of shift-addsub. The output is still an approximation, but its as precise an implementation of the trigonometric operations as floating point numbers are able to express.
I remember seeing a discussion on a game dev forum where the approach of 6:38 was used to fit a quadratic to that part of the graph, and the fit was quite good despite the simplicity. You could then use Newton's method to refine the result if you wanted more accuracy. OTOH if you're working at the hardware level, CORDIC is the usual algorithm of choice since it avoids any multiplies.
The legendary Clive Sinclair brought out a low-cost scientific calculator in 1975, complete with log and trig functions. Somehow his engineer, Nigel Searle, was able to implement all these functions in a ROM that could only hold 320 instructions. The algorithms he came up with were a bit slower than regular CORDIC, and had lower precision, but they worked. No other product of the time could offer that kind of functionality at that price point.
What do OTOH and CORDIC mean?
@@toasteduranium CORDIC seems to be an algorithm
@@toasteduranium CORDIC means: coordinate rotation digital computer. If I recall correctly, it's an algorithm that iterates and only uses bit-shifts and add operations. More iterations means better approximations.
all my life ive never needed this video until literally last week this is perfect
amazing video my friend
Well this video is really intuitive and useful, people use these built in libraries directly in their code and they don't know how the background process goes, thanks a lot for this video
As a math teacher, this is a spectacular video. Awesome work.
Glad you liked it!
The title of this video is exactly the question that's been annoying me for a really long time. Thanks so much for making this.
this video answers one of my questions i’ve been wondering for years! beautiful video
lol, wow... That paper you showed at 7:05... I worked as a graduate intern on that exact team (especially, Shane and Ted), and I personally rewrote the entire test suite software for the extremely high performance code they were developing for the original Itanium (all of the "math.h functions", not just sine). Work included testing on beta versions of Win64 (and Linux), emulated IA-64 hardware, early IA-64 hardware revs, prototype compilers, and more. The team literally counted clock cycles for code paths through functions, maximally shaving them off.
VERY cool! Did you end up there after graduating?
@@simondev758 It's a long story, but no. They wanted me to stay (Ted in particular came back to me to [re-re-]confirm that I was sure), but I eventually went to graduate school instead, then wound up elsewhere.
(Btw, "graduate intern" here means an internship immediately following graduation with an undergraduate degree. I'd already done two previous "undergraduate technical internships" there during earlier summers, performing other work, though still sort of tangentially related... the first one doing some research on dgemm (double-precision generalized matrix-matrix multiplication) for the team developing Intel's Math Kernel Library---a very, very high performance linear algebra suite that got incorporated under the hood into enterprise software such as MATLAB.)
That is very cool, as well. I hope your "elsewere" and your actual work had been, and are, more interesting and better.
This was so helpful. I don't know if I missed it, but I don't think I've ever seen tan explained like that, and I certainly had no idea why it made such weird shapes on the graph, but now it makes simple and intuitive sense, thank you!
Its funny how when you need to learn something you learn it alot easier than when youre forced to learn it. I willing learnt about the different trig functions a while back. Nice to see the OG making a vid on it😆
5:27 lovely reference!
anyway, i do loved the whole explanation and you showing the actual code / mathematical definition
this is a really well made video, and I actually liked it....I honestly, like a lot of people, copy a lot of math functions from other code when I am less familiar with it...then I store it away in a kind of cheat sheet until I get comfortable with the function....
Thanks alot Simon!!! Sent to my friend who's stuck on same subject what a neat synchronicity 😃👍. Happy Coding
Good luck to your friend!
@@simondev758 your most welcome!! ☺
That was the most useful explanation on the topic i've seen in a while. Congrats!!!
I'm so glad I watched this before my kids. I can't have them hear such an explicit description of how radians are made.
Can't thank you enough
I've been searching for this since forever
Just found this channel - and while I do a lot of math for a living, this was such a nice presentation that you earned a subscriber
Welcome aboard!
Me too. And I am not a game developer, or coder, or mathematician, but I love mathematics and how it may be implemented in smart and clever algorithms
I was diving into this recently, particularly Robin Green's writing on the subject, as I am - out of curiosity and for fun - building a simple game engine in Rust, and have found that Rust - as of writing - seems to lack bindings for SIMD-accelerated sine, cosine, and sincos functions, which I seem to remember having seen in C++, so I decided to try and see if I could create an efficient implementation using SSE or AVX intrinsics. In the document Robin Green lists at the end of his blog I saw code that superficially resembles the code shown at 6:46 though the switch statement seems different.
Green uses 'switch(k & 3)' and then three case statements (with constants 0, 1 and 2) and then a 'default' fallback case. I think this works because it is guaranteed to be exhaustive, the AND with 3 means that all but the lowermost 2 bits are zeroed, effectively leaving a 2-bit integer, which of course can only encode 0, 1, 2 and 3.
There's another difference though that confuses me. In the code shown in the video, integer 'quadrant' is defined as k mod 4, presumably to wrap k to range [0, 3], but then the switch statement refers to k and not quadrant. As far as I can tell, this means that k is not wrapped to the range of [0, 3] and 'quadrant' is not used. Am I correct in this, or am I misreading something?
Yeah, I screwed up the image of the code. The shader itself was actually using the corrected version, but I never updated the imaged to reflect it, sigh.
I didn’t know Bob from bobs burgers was teaching math now
Being an old time dev, I have always worried about how sin & cos are being computed in case they are expensive, as I'd heard how early 3D engines used the lookup table approach as a speed hack. But seeing how it's done these days has set my mind at ease. Thanks! :)
this, this is exactly what I needed, I've had this question for a while now so thanks so much for this honestly awesome video
Great video! Would love to see an explanation of how logs are computed!
Thank you, short and concise, as I like it. Me being older, I try to pick up geometric algebra from TH-cam videos, at high school level. It took me a while to figure out why if u = ae1+be2 and v = ce1+de2, then u.v is both ac+bd and |u||v|cos(th). I did, and your video shows it clearly. From a stupid looking formula, the inner product proves to be very useful, I really start to like geometric algebra!
Thank God it is not Algebraic Geometry, which is probably the hardest branch of mathematics.
Nice video and well explained. I really like that you also showed some code.
One thing I noticed is that in the code at 6:50 it should be “switch (quadrant)”, but I guess you added this to check whether or not we’re paying attention? 🤓
Also shouldn't it be PI - y and not PI / 2 - y?
@@johannbauer2863 I think PI/2 - y is correct, because it mirrors the y around π/2 into Q1 which sin_kinda expects.
Sigh yes, always a mistake somewhere in the video!
In the 70's I was slinging Fortran for Space Shuttle Simulators. When I started running the guidance software to support landing, everything I was monitoring was going along fine until just before landing when the simulated shuttle did a maneuver to transition from the steep glide slope to the shallow inner glide slope (-1.5 degrees) just before touchdown. The shuttle would suddenly climb , slow down, stall, and crash. I got with the vendor of the computer we were using and he pulled out the scientific subroutine library code-we were looking at hard copies here, no CRTs yet. It turned out that the sin function would start out by testing the value of the argument you were handing it then go to a unique Taylor Series expansion for that sized angle-this was partly necessary to get everything to execute in real-time. Sure enough, for certain small negative angles, the particular Taylor series it used was wrong and would return a positive sin instead of a negative one which is what it should be for a -1.5 degree angle. The guidance algorithm would then generate a positive (upward) altitude rate reference and issue commands to the control effectors to try to fly to it, which it did.
Hah nice find!
Thank you. This remind me that I could implement various versions of approximate sin and cos functions depending on the accuracy I need for my application, and thus get close enough results faster than when using standard sin and cos function defined by the language or the default Math library.
Standard sin and cos function defined by the language, its standard library or the ALU are in most cases compromises that are at the same time way to slow for real time artistic applications and way to inaccurate for science research. It's good to master this so we know what we do, always do enough without overdoing.
One of the neatest tricks I ever learned was called CORDIC, which is something like Coordinate Rotation Digital Computation. It expresses an angle as sums and differences of smaller angles which are chosen so that rotation by those smaller operations are trivially simple, operations on bits. Since a rotation of the vector (1,0) can give you sine and cosine, those are easy to compute. But suppose you are given x and y and you want to compute the angle that rotates it onto sqrt(x^2+y^2) - e.g. find the angle. It's all additions and subtractions. You can also get any of the trig functions with similar tricks. If you have to build the hardware implementation of trig computations, it's really easy.
Super neat, I'm reading a bit about it now. Any idea if modern hardware still uses CORDIC? I found references for intel using it on older generations of hardware, 486, and the paper I cited comes out in 1999.
@@simondev758 Simon, I haven't worried about that for a long time. I first learned about it when Hewlett Packard came up with a hand-held digital calculator which could do trig functions, etc. The trig functions were all base don CORDIC techniques. In those days, a hand-held calculator hadn't gotten the years of advantage of Moore's law, so things like lookup tables were pretty burdensome.
Yes, CORDIC was used on handheld calculators. It is a much friendlier way to take a small, carefully crafted lookup table and create solutions for any precision you desire.
Modern algorithms include the Chebychev series. Taylor series are too slow to converge, so they are not used for trig functions.
I've been working on low level graphics functions for some ten years now and it blows me away how bad people are at explaining trig functions. You, my dude, have nailed it. The unit circle is easily the best way to visualize how they work. Looking forward to see how you simplify matrices and quaternions, assuming you'll be doing those. The best way I've found *personally* to represent the former is using basis vectors and showing how it's like a box within a box within a box, if that makes sense. The latter's a bit trickier, but I usually imagine it as a ray emitted from a given point, with everything rotated a given number of radians around that ray.
Hope this channel grows. I'll do my part. :p
Definitely, basis vectors are THE way to go for matrices. You can even simplify multiplication that way.
@@simondev758 Heck yeah, my dude. Lookin forward to the videos when they come out!
I think I really learned to appreciate trigonometry after learning it in school. It's just a beautiful concept that just is so satisfying to watch. (And it has some cool ways to apply it in engineering to make fun stuff)
Love seeing the visualisation of this stuff!
School does everything wrong. This is a much better explanation than anything school has ever taught me. I just luckily happened to figure out these circle rules on my own when playing with code, but the school system doesn't provide the intuitive or useful explanations of math 😔
Actually our school uses unit circles to show trig and their derived properties, but the last part (about LUT and approximation) is completely new to me 😮
Where are you from? That's what they teach in middle school in my country
@@roastyou666 because it's taught in calculus 1 in university/college, not school
@@badgermcbadger1968 they didn't even teach the unit circle in my uni calc 1 because it was assumed knowledge, it definitely isn't universal
@@badgermcbadger1968 Ah I see
Very interesting. Do you expect to make a similar video about logarithms? I'm quite curious as to how a computer computes log(x) efficiently.
I could take a look. I believe log is covered in the paper referenced.
I would like to see going over modulo, since there are 2 definitions of it and different engines uses one of the two definintions.
In fact you can reduce the range even further, by noticing that the sin function, for x ranging from pi/4 to pi/2 is a mirror of the cos function between 0 and pi/4. And the Taylor series in the 0 to pi/4 range converges to an optimal precision after very few terms, both for sin and cos. This is what I did, without suggestions from anyone, on my Z80, in assembly code, back in the early eighties.
im so glad bob burger taught me how to compute sine and cosine
this was amazing and very helpful
Well. Damn. That was a very educational visualization of SinCosTan. Especially the movement to graph. Thank you.
Thank you! Looking forward to the math course from you.
Yamaha's FM implementation used a quarter of a sine wave (premultiplied to ease of certain other calculations) as a lookup table.
Brilliant video! You've answered the question in an interesting, complete and easy to follow way 😁
Never for a millisecond even questioned the ability for a computer to calculate sin(x) with perfect accuracy until I saw this thumbnail
Then you explained how some games and stuff approximate it for speed, but now I'm realizing even on a *supercomputer* we've *never* been able to perfectly calculate sin(x), because it's a function based on π, an irrational number of infinite length which can't be represented in a computer. This existential crisis quickly led to another, realizing that we *will never be able to perfectly represent sin or any trig function because pi goes on forever, and that in a sense ALL of the maths we do, both on the computer and off, is like this: only theoretical on nature and every actual calculated number we use is just an approximation of some unobtainable truth of the universe and...*
I think I need a drink
This got deep, quickly
But the grat benefit of these algorithms and formulas ( Baskhara, CORDIC, Chevyshev, etc.) Is that you use only the precision that you need.
And talking about precision, there are fórmulas with very fast convergence for computing billions and trillions of digits of π (I forgot the Authors of the paper from 2004 or 2008) that even can compute segments of digits (for example between 1 million to 10 million, or between 1 billion and 10 billion, etc,) without even computing the precedent digits. They say the techniques used to get those fórmulas come from INTERVAL ARITHMETIC.
holy shit this just explained everything that my pre calc teacher isn’t explaining
Looking forward to this course
I look forward to picking up the math course!
quick question at timestamp 01:19 you say if we move one UNIT along the circle... a "unit" is x=0 to the edge of the circle?
Starting from (1, 0)
Great video, easy to learn from.
I personally, find it much easier to see tanθ as the slope of the terminal arm. That is, once you place your θ and extend the terminal arm to the circle, the arm's rise/run is tanθ.
Yeah, I feel like I should have included multiple interpretations of tan instead of settling on one.
it’s really the same idea though. he explained tangent as the height of the line when you connect it with the unit circle on the x-axis. this height is the rise. the unit circle, with radius 1, is the run
This is the exact question I've been trying to figure out for weeks. Like what actually is in the black box of sin and cos.
1:52 it's much easier (and more useful) to say that tan is the slope of the line
I went with this because I like this version, it makes the most sense to me in terms of how it's calculated. Maybe next time, I'll put every possible visualization.
Could you please make a video about application of calculus in game development?
@4:07
Exactly what I needed! Thank you!
this is a great video and a wonderful presendation not just due to my lack of intuitive mathematical knowledge
super pumped for your math course!
2:05 i FINALLY understand why is the line touching the circle at one point is called a "tangent"
iirc, implementations usually divide by 2pi first, so using a 0-1 range i think referred to as turns is faster
In first year university calculus we imagined that there are two functions s(x) and c(x) such that the first derivative (slope) of s is given by c and the 1st derivative of c is -s. It is demonstrated that s is sine and c is cosine. In Intel/AMD processors the microprocessor provides all of the primary mathematical functions. This is because it is more efficient to perform the calculations using the underlying hardware directly rather than exploiting the complex instruction set implementations of multiplication and division. This is not an advantage on reduced instruction set computers (RISC) such as ARM or RISC5 so they provide only the basic floating point operations. When you look at the x86 "hardware" instruction that provides sine you will observe that it returns *both* the sine and cosine values for the argument. Although this does permit obtaining the values of tangent and cotangent that is not why this is done. It is an artifact of the use of the Newton-Rapson algorithm to refine an initial approximation together with permitting further restricting the range of values that must be supported to just 0 through pi/4 by observing that sin(x) is cos(pi/2 - x) and cos(x) is sin(pi/2 - x). With modern inexpensive memory it is cheaper to use a table of, for example 256, values of sin and cos or any other transcendental function and then use the Newton-Rapson method to refine the value, especially to get 64 or 128 bit extended floating point values, than to use a polynomial. Each loop through the Newton-Rapson method increases the precision by a number of digits.
All arithmetic operations more complex than multiplication depend intrinsically on gradually refining an approximation, usually by a variant of Newton-Rapson. For example look up videos on how to extract a square root by hand. Why do you divide the remainder by 2 times the current estimate? Because that is the 1st derivative of x^2! Also recall that you all learned how to do arithmetic by a table lookup. 7 times 8 is 56.
This comment kinda brings back memory of this super obscure function in the quake source code I ran into way back, this fast inverse square root that used, aside from some floating point bit wizardry, pretty sure there was a newton iteration in there.
Man, that was excellent! I always wondered how this was done, honestly a little disappointed I didn't figure it out myself.
Great video! What programs did you use to make this video?
My issue with the unit circle explanation is that it's so often framed as "the triangle explanation doesn't make intuitive sense" or some variation thereof, but that just creates (pardon the pun) circular logic:
Lets say we want to figure out the sine and cosine of some angle using nothing but pen and paper. Using the circle we know the radius, the arc length and... now what?
Using the arc length and radius, we could find 1/2 the length of the chord that spans from the point we want to know to it's mirror about the x-axis (to get the y-value) or the y-axis (to get the x-value)... but that requires inverse sine, and how do we find that by hand?
Or say we have the radius, the angle AND the x-value. Cosine = x, but what's sine? How do we know that? Well, see cos = adjacent / hypotenuse and... dangit those are triangle terms.
Okay let's just take it as a given that cos = x. How do we find the y-value? Well if we know x and radius, we could say that r^2 = x^2 + y^2 and solve from there... but that's the Pythagorean formula and it was, surprise surprise, derived from right triangles!
Even the CS solution is either "look it up i guess" or "approximate without calculating it," so, near as I can tell, we're stuck with triangles and the circle just gives a more pleasant visual representation.
All that being said, this was an simple to understand and visually pleasant overview of the underlying trig and how it's enacted in computers. I'm not a game dev, and still I'm considering purchasing your course because it might contain equally enlightening topics told in a simple to understand, yet still concise way
I think with the unit circle and using more lines you can also visualize COT, SEC, and CSC.
If I am not mistaken, the unit hyperbola is used to visualize the HYPERBOLIC TRIG FUNCTIONS, Sinh, Cosh, Tanh, etc
After reading many comments here by different persons, i collected a brief summary about the algorithm CORDIC (not the algorithm itself):
X
The CORDIC algorithm was used to compute Trig functions in hand-held calculators (HP and others), and computers with slow processors and small memory, but is specially useful for implementation in hardware as bit-shift operations and additions and substractions, which is very easy. You can also use CORDIC to compute the absolute value of a vector, and the function atan2() of any pair of x/y .
The Baskhara Approximation formula can be used to compute sin(x), too (but maybe with less precision).
A more modern algorithm than CORDIC ( and vastly superior, they say) uses Chevysev Series (or Approximation).
Taylor series are very slow to converge, so they are not used to compute Trig functions.
Congrats on 100k subscribers :) Have been following your channel for a while now and absolutely love your content. Even the video about NFT minting, people were just bashing everything NFT related. The tech behind it is still interesting so please don't stop making videos about topics you like to learn about even if they are not graphics/game-dev related
Oh and if you are interested in suggestions, I would really like to dive more into physics engines if that's something you wanna do
Looking forward for the "Game Math Explained Simply" course. Your explain thing really well :D
yeah. this makes much clear sense. thanks for sharing.
You can also use half angle formula to get it to small angles for the Taylor series
Very nice visualizations!
I really enjoy watching your videos, what animation software do you use?
I don't use any software, everything is defined mathematically in shaders. The entire video is one long shader.
@@simondev758 That's awesome
That Jigsaw comment. 🤣 I wonder what level of precision he would want for a trig approximation.
"Hello, Reginald. I want to play a game. For the past semester you have neglected your studies of series expansions by relying on Mathematica and Desmos. Inside this cabinet you will find paper and a pencil and three trig problems to solve. You will have to approximate your answers to four decimal places, within an error of..."
You may have just invented the new edu-horror genre!
Very interesting. You make it very easy to understand. Thank you. You forgot to mention the software used to make those trigonometric graphics.
hi, can you make a video on spirals and how they generated? thank you
Thank you for this wonderful explanation!
Am I recognised right that at 6:47 there is a typo in switch parameter? It should be "quadrant" instead of "k"?
This brings back memories' of me being fascinated with the book "Numerical Recipes for (C, Fortran) in the 90's".
i had a conversation with my math teacher about trying to understand what the sin/cos function really is, he had no clue lol
Not somebody with any formal Math or CS education or anything _what-so-ever,_ but I remember reading about the use of "Chebyshev polynomials" for optimization on Stackoverflow, and how the Taylor series approximation has been rejected many times for use in, say, a standard library. Many more tricks were mentioned, and it's just a bunch of tricks about crazy/trivial micro-optimizations.
Thoughts?
Yeah, gave them a quickie mention in the video but didn't wanna stray too far off topic from the basics
Wow, very well done!
Great video as I’m going into calculus and hadn’t quite grasped the relationships and just been working out their inverses & antiderivatives & derivatives. I love a similar videos on csc, sec, and cot if your up for it
0:32 What do you mean with come in handy close to zero times? The unit circle is basically the same thing, only scaled one side to 1. If you use the formula on it you get the same thing. :D
Yeah, they're the same, just presented in different ways. I've always worked with vectors for the most part, so working in the context of the circle and the relationships therein is what I'm used to.
I enjoyed this video, would you consider doing one for other math concepts like matrices?
Definitely! I have a really good idea for some other math ones. I'd also like to get into some more advanced math topic in game development like spherical harmonics.
Besides being a game developer, are you also a mathematician ?