Two things worth adding here: 1) In other resources and in implementations, you'd typically see these formulas in some more compact vectorized form, which carries with it the extra mental burden to parse the Hadamard product and to think through why the transpose of the weight matrix is used, but the underlying substance is all the same. 2) Backpropagation is really one instance of a more general technique called "reverse mode differentiation" to compute derivatives of functions represented in some kind of directed graph form.
You should probably change the thumbnail. The snapshot of the variables with indices (which I didn't know were indices at the time) and subscripts almost deterred me from watching this although it really wasn't that complicated.
I will probably be a TA for a lecture course on Deep Neural Networks again next semester and I will recommend this series to the students (we did it in a very math-heavy way this year and it was a bit too much for some of them, even though its a lecture for master students)
Sirius Black what you could do is download the r package "deepnet" and poke around at the source code. Its written in base r so you can follow it around. This is how I learned, and IMHO the best way to learn.
This series was my first introduction to Machine Learning 3 years ago. I now work full-time as an AIML Scientist, my life is forever changed. Thank you.
how hard was it? I am tryna code my own neural network from scratch, there's so little resources for that it seems. and how do I even make myself unique?
@@envadeh Learn everything, use the feynman technique; if you can't explain how a machine learns to someone who knows nothing about it, keep filling in the gaps. Formal education is great, but honestly more of a waste of time than not. Teach yourself, learn how to learn, and then keep learning. I audited Andrew Ng's Deep Learning Specialization from Coursera, had some formal education, and self taught myself everything I could get my hands on, from theory to application, the underlying math to the practical programming. Understand the importance of data, DATA IS KING. Watch Interviews with industry leaders, understand the big turning points and continued development within the last two decades of AIML (you'll figure out what they are with time). It takes 10,000 hours to become an expert, I'm about 4,500 in, but all it took was a little bit of work every single day. Make learning a habit. Trust yourself, believe in your ability to become who you want to be. "It doesn't matter if your read two research papers in a week, what matters is if you read two research papers a week for a year, now you've read 100 papers" - Andrew Ng (Don't 'read' research papers, watch synopsis! Smarter not harder! There's so much free information, you could probably use a GPT model to teach you what you don't know!) Goodluck, and I believe in you! :)
@@nczioox1116 "Need" is a strong word, it just depends on what kind of work you want to do/who your employer is; people used to go to college because that was the only place you could learn these difficult subjects, but now it's just an archaic way of putting you in debt since you can learn these things online for free UNLESS you want to work for an employer where you need the degree to be recognized. If you are self-motivated and can teach yourself these subjects, seriously consider your options before assuming that spending 4 years of your life and 100k+ is necessary. I have an Electrical Engineering degree, but out of the 40+ classes I had to take for it, only 2 had any sort of impact on my daily job now. It all depends on the context. Goodluck, and I believe in you! :)
@@thomasclark8922 Thank you! I have a mechanical engineering degree. I'm in the process of self teaching myself machine learning concepts and doing some projects. Lots of job postings I've seen in the field seem to require a bachelors or masters in CS, math, or neuroscience. Of course these seem to be for larger companies so maybe smaller companies might take a more holistic approach
Dear Grant, A year ago, I decided I wanted to learn Machine Learning and how to use it to make cool stuff. I was struggling with some of the concepts, so I went to TH-cam and re-discovered this series on your channel. Out of all the courses I've tried and all the hours of other content I've sat through, your videos stand out like a ray of sunshine. I just got my first full-time job as a Machine Learning Engineer, and I can confidently say it would never have happened without this series. Your channel may have affected the course of my life more than almost any other. Thanks for all your hard work!
Congrats on your job. I was wondering, when you finished Andrew Ng’s ML course, what additional steps and how long did you have to take to become a full fledge ML engineer? Thanks in advance
Many guys claim to know. Some guys actually know. But only one guy actually knows and can explain to his grandma as well with very beautiful animations. You are that ONE !!!
Your work of making high levels of math accessible to anyone wishing to learn a variety of new topics is not obvious to me. You succeed to explain everything so clearly, making me want to start learning maths again, reminding me of and introducing me to beautiful aspects of math, and you deserve more than a 'thank you' :)
I just started out with my ML career. This entire series made me feel as if I knew it all along. Thank you Grant I will return to this comment to share my professional progress😊
Hey for all of you getting discouraged because you don’t understand this - that was me last year. I went and taught myself derivatives and came back to try again and suddenly I understand everything. It’s such an amazing feeling to see that kind of work pay off. Don’t give up kiddos
Yup. I tried to watch this series a year ago and make my own neural network which turned out to be disappointing. A semester into college and some passion for calculus is all it took for me to mostly understand this series!
I would say it’s recommended to have read calculus 2 (for partial derivatives and the Jacobian) and linear algebra (for matrix and vector multiplication). Otherwise, just looking up mentioned things is also fine. But it might take time to build up intuition for the math.
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
It has taken me about 3-4 days worth time to understand all of these 4 lectures, lectures which are in total, no longer than 1 hour and 30 minutes. And I feel proud.
I'm attempting to implement it from scratch in C# with no matrix math library or anything so I can get a feel for the nuts and bolts. This is the boss level!
I've worked with AI for 2 years now. I have never seen anyone explain this as succinctly and aptly as you have. This video is legitimate gold. Going to show this to anyone who needs an explanation in future!
My god! A basic machine learning video series that actually makes sense to completely beginners! Subscribed, and waiting in great anticipation for the next one! :D
This is the longest 10 minute video I have ever watched. Literally took me half an hour, but the feeling of the idea behind this completely settling in , makes it totally worth it!
I think it's easier if you take the time to re-write all of it yourself, on a scratchpad, work through writing the formulas etc. Then you can play with these objects in your mind more fluently. It takes a longer time initially but I feel you get more out of it. And that's a good way to begin "translating" into vector algebra by taking "simple" papers on DL (haven't go there myself yet).
@@phil.4688 This, is so important. The math really doesnt click unless you write down where each derivative comes from. And the fact that you need more than one partial derivatives for each layer.
@@phil.4688 Exactly what i had to do. Was getting the concepts, but only really understood once i started taking notes, and perform the calculations by myself (took even more time as had to re-learn derivatives...). The most interesting is that, now that i understood it, i'm even more appreciative the way it's explained in the video.
God Bless You man! I have tried watching many videos about backpropagation but this series made my conceptual understanding and intuition super clear. Thanks a lot. You have no idea how happy I am right now to have understood the concept of backpropagation.
literally thank you. I learned the information that I could not learn at school for 5 weeks in a 10-minute video. The animations of the video are absolutely magnificent. Thank you thank you thank you
Thank you a lot for this series! It has really helped me get into this topic, and changed my life. Your intuitions have been immensely helpful in my efforts to understand backpropagation. I just can't overestimate, how great your channel is!
Hello, I'm greatly impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
this is math at the best and art too at the highest. the grace of the animation, the subtle music, the perfectly paced narration and the wonderful colour scheme! math and art or let's say math is art!
Nicely done video. I knew I learned backpropagation before, but it was hard, and I didn't use it manually ( I used frameworks like TensorFlow which uses computational graphs and backpropagate automatically) so I've forgotten how it actually worked. But this video is a great resource for newcomers to ANNs and people like me that have forgotten the theory behind it all. Thank you.
I really can't believe how well and simply this is explained here by you, I still keep coming back to this exact video even after few years of being in AI to refresh the fundamentals and to this day this is the best resource for this.
I'm taking Machine Learning by Andrew Ng on Coursera right now, and just got stuck on backpropagation. Thank you thank you thank you thank you Grant, you have no idea how incredibly helpful your videos are and how much your channel has inspired me through the years.
Here I am. Same situation. Andrew Ng course and backpropagation is rough. This video in particular really helped to clear things up. Breaking it down to a single neuron is enormously helpful.
Same here!) I was somewhat disappointed when Andrew Ng course just through the formulas at me, so I tried to derive backpropagation myself and got stuck in all the little details. Thankfully, 3b1b rode in like a knight in shining armor and now I am really damn happy))))
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
"So pat yourself on the back! If all of this makes sense, you have now looked deep into the heart of backpropagation, the work horse behind how neural networks learn." felt soooo goooooood
If i get a Job or a Position, just so that I get to share my success story- 80% of my speech will be 'thank you 3Blue1Brown'. You are a blessing! Cheers.
I just finished up calc iii this semester and I have never felt happier with myself for being able to apply my new knowledge than this episode. I also don't think I have ever been more excited to hear calc iii topics being brought up in a field I am trying to teach myself currently. Thank you for making such a simple to understand series!
Months of working with deep neural networks and i still have to come back to this video to re-digest the math. Don't get discouraged if you don't understand it! As 3blue1brown said it, it is very complex and takes time to digest.
I am speechless. This channel undoubtedly contains the best pedagogical scientific material on TH-cam, and possibly in the world. Thanks for making these videos, your skills and passion are unreachable!
I always feel, if u have a mentor who can break complex things into simple stuff so beautifully, even a dumb guy can grasp the concept. Keep doing the good stuff. Admirer++
At time when I just finished my university - I could not imagine that at one chilly Sunday evening, in almost 15 years after the graduation, I will sit with a bottle of beer, watch math videos, and have so much fun! Thank you!
I just watched this video going back and forth several times and I am at a point where I am not sure if it made sense or not, but I still feel like it helped. Thank you so much for this series!
You bring clarity on such potentially complex ideas as Prometheus brought all the light to everyone in the mythology. We are forever grateful for your efforts! You are a math artist and this videos are a masterpieces conveying intuitive understanding of such complex ideas to many people.
I cannot tell you how much I appreciate these videos. I don't have a strong math background (english undergrad) but I'm teaching myself data science. It REALLY helps to have the equations explained rather than just presented and to tie the components of the equation back to intuitions. Thank you thank you thank you.
Grant, I've come back to this series many times over the last five years. Every time I do, I pick up more and more pieces of the puzzle. I think I've finally got it now, but we shall see! Thank you!
*looking at thumbnail oh sh~, Im never going to understand that complex stuff. probably should watch it anyway *ten minutes later whoa! that's actually quite clear now!
I think that’s kinda the point of much of this channel lol. Same experience though! It’s amazing how a TH-cam video can do so much teaching in like 10 minutes
You are some sort of sorcerer. The chain rule was finally elucidated for me in about 20 seconds. So concise and intuitive indeed. That was one thing, for whatever reason, I couldn't get my head around as a younger student. Big props man. Love this channel.
I keep coming back here every time I have doubts about the mechanics of back propagation. I need to visualise concepts to understand, so for me your video so well done. I can't say anything else than thank you.
The point where you addressed the concern that the example you were using was too simple, having only 1 edge, was spot on as you were leading me down this merry garden path. I appreciate how much you watch your own videos and predict where the watcher would mentally say, "but what about..."
These videos are unbelievably well produced. Thank you so much for your effort. You've made this topic incredibly clear and I cannot understate how much I appreciate the amount of effort you put into these. You have incredible talent as a teacher.
Mr. Grant please make videos on CNN and LSTM, the world seriously needs videos of yours on these topics...PLEASEEEEE. Thanks for your astonishing eye-opening hallucinatory mind-altering mind-boggling overwhelming psychedelic staggering stunning wonderful explanation to the world.
thank you SO MUCH for publishing this video! The illustrations, explanation and your calm voice made it so easy to understand! I first hear of backprop yesterday and the professor didn't go into the math of the cost function derivative one step back from the output layer and your explanation really helped me get a better sense of understaning what's happening here. I can not thank you enough - bless you!
Absolutely brilliant explanation! I took a course on deep learning in college, but ended up auditing it in the end because I couldn't grasp the concepts well enough to pass the tests. You just took the entire first unit of the course, which took several weeks, and condensed it into 4 easily digestible videos that anyone can understand!
LOVED these four introductory videos. My brain was so exhausted processing them--lots of pausing--but I feel like I get something conceptually that I knew near nothing about a couple days ago. Great foundation to scaffold the programming and math onto. Thank you! will definitely be watching more of your videos. Looking forward
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
If anyone is interested, I worked through the chain rule for the differential of the cost function w.r.t the weight in the second layer down. Two additional terms are added to make everything cancel as they should. It shows how as you progress down the layers, more partial differentials are added to the equation from all the variables above, making it more unstable and hence more susceptible to the exploding or vanishing gradient problem. dC/dw(L-1) = dz(L-1)/dw(L-1) * da(L-1)/dz(L-1) * dz(L)/da(L-1) * da(L)/dz(L) * dC/da(L)
Nothing more but a deep thank to you for making such amazing video. I truly cannot express how much this whole series had clarified this topic for me. Again, thank you very much.
Grant, I've been thinking about these videos and I have a couple questions.... last time you fed a nonsense, scrambled image through the network, and it confidently identified it as a 5. My question is two fold: 1) Would any value be added to the network by adding a final neuron that indicates "no number" or "not anything identifiable" or "throw this image in the trash can"? Maybe that way the network would sort those trash-type images into a pile for, perhaps, a human to look at. 2) Is there any benefit to training your network to get the right answer, but be less confident about it? You keep mentioning that we want the network to identify a "2" as a "2" and the goal is a final neuron to light up the "2" at 100% (and everything else at 0%). Does this extreme goal perhaps cause really confident misclassifications sometimes (besides the junk image from last video)? Would it be in any way beneficial to train the network to see a "2", but perhaps to only be 90% confident about it? Maybe it would *think* it saw a 2, but it's not totally sure - like a young child learning numbers. (but, being 90% a 2, is still a 2.) Would this help at all with the junk images from the previous question / video? Also, maybe since we train the network to not be so confident all the time, could we then put any image that falls below, say, 75% confidence into another bin for humans to verify? Also, on BB&B you mentioned that it can sometimes feel like too much pressure when people say "yeah, good video, I can't wait for the next one!" Well, I'll instead say, thanks for this series, and I'll patiently await the next video as you take your time to make it a high quality production like you always do - whenever that happens, I'm good. No pressure!
I have no programming background at all, but couldn't you just generate many random images and label them as (0,0,0,0,0,0,0,0,0,0) and then add them to the training data? (instead of adding a new neuron) (i have no idea what this will do but it sounds interesting)
I'm not 3b1b, but I can give you some hopefully give you some answers to your questions, since I'm working with Neural Networks in my PhD: First of all, that confident prediction of a 5 last video was a bit of a trick: It is not actually a completely random image, but was created to generate that specific output. He could have generated an image that looked just as random, but produced any arbitrary output or he could have taken an image that is clearly one number to a human but is caught as another number by the network. This is a (by now) well known flaw with neural networks that you can usually generate an image that produces a desired output but looks very similar to another image (to a human) that produces a sensible output. To your first question, aside from the flaw I described above, if you want to know whether there is a number at all, you usually look at whether any of the outcomes is above a certain threshold and if none is you say 'not anything identifiable'. An additional output neuron would require you to add enough 'not identifiable' example images to the training that it would drown out the other images or otherwise it might not say 'not identifiable' for some random input image. To your second question, I don't think that would be very useful since mosts of what you are doing would be training the network to output 0.9 when it is most confident. Also, the network rarely is completely confident (unlike what the video portrays), so the thresholds you described can be used with the network as it is.
+TalathRhunen Great response, thanks! Also, by the way, the example from the last video was a genuinely random image, not an adversarial example. I'm not saying that happens for all neural nets, of course, but for this simple one that was the emerging behavior.
Stunningly beautiful... The best part of the series (for me, obviously) is that the beauty of this series does NOT make it very easy to understand. No. Each video may need multiple views. But these videos are so beautifully made that you'd want to watch them again and again, not with the frustration of getting your head over a concept but with the thrill of unravelling a mystery... So for creating such excitement in me, thank you.
Great presentation! Very helpful. Although, could you clarify if the indices at 9:30 are consistent? if you sum over j in the 'next' layer, you would have k in the current layer. Instead of w_jk^(l+1), would it be w_ij^(l+1) and sum over i for this next layer? Thanks!
It seems that the second formula (at 9:30) is inconsistent with formulas from Andrew Ng ML course and as of me looks very confusing because of inaccurate use of indexes. Let's simplify the vectorized formulas from the above course and keep using the derivative of the activation function: sigma'(z(L)) in delta(L) as suggested in the video. With index notation we will get for d_C / d_a_j(l): sum_over(q=0 -> n(l+1)-1) [ w_qj(l+1) * sigma'(z_q(l+1)) * d_C / d_a_q(l+1) ]
Thank you very much for your amazing explanations. I'd been buckling down a lot for understanding back propagation. While all the explanations I found failed to make me clearly understand the topic, your explanation just worked wonders! Thanks again for so nice videos!
This NN series was amazing, thank you so much! I've watched a lot of these kinds of videos, read several online articles, and am following a popular online course. This series is the best resource by far, so helpful.
i watched this video like 10 times at least and, Finally! finally! i was able to build my own neural network with back prop, in c# and it actually works! Btw, i found out that if my layers have a lot of neurons the System often over shoting, and deviding the Cost for each neuron by how many neurons, are in that layer help to fix that issue, i mean i can have the same training value for any type of neural network and it wont over shoot at all. but in the video you just said to add the costs for each neuron together rather then getting the average cost for each neuron.... hmm...
@@honeyant3119 well here it is github.com/miko-t/UnityMyNN/blob/master/Assets/MyNet/SimpleBrain.cs it writen in c# and iam not using metrecies just hard codded array's Watch the function Train
@@honeyant3119 it works fine when predicting stuff like, Graph slopes etc... or just the basic stuff, tried to make it learn how to drive a "car" and it fails misserably, but that fail might be just due to the fact i dont know how to calculate real time, cost.
@@mike_o7874 search up reinforcement learning for real-time tasks. Also, driving a car should be really easy. I trained a network to do so using evoltuion and all it took was making a good cost function.
One of the best lectures I have ever heard. Great explanation of NN, cost functions, activation functions etc. Now I understand NN far far better...(P.S. I saw previous videos Part 1, 2,3 as well)
Thank you so much, I love how you explain it. After watching the series 3 times, I know fully understand how it works! And I can convertably make a simple neuralnetwork.
Thanks 3B1B. I'm understanding machine learning mush better, and following your video while note taking was the easiest method for learning. I'm a little confused about what the change in C_0 with respect to change in a_(L-1, k) for the k-th activation in L-1 layer (I just changed to this notation because I feel more comfortable writing like this in one line text). That's 8:40 part of the video I guess. It doesn't make intuitive sense for me as to why you need the summation of impact of a_(L-1, k) on a_(L, 0~n), say without any multiplier or something. Trying to understand the meaning of `dC_0/da_(L-1, k)` I thought of a Neural Network where there are only two layers, input and output layer, and input layer having 1 neuron and output layer having 2 neurons. Does it ever make sense for a_(L-1, k) to be an activation (or neuron?) in an input layer? If so, I think it makes to add the 'impact' all up especially when the weights are all same 'direction' or sign because if so summing them all up would result in greater number, and this would mean changing the input has the biggest impact in this scenario. If not, I'm still confused what `dC_0/da_(L-1, k)` is and why it has the summation.
hello, how are you? I know it's been 3 years since you made your comment. But for me, it's my first few weeks started to learn Deep Learning, and now trying to build Neural Network from scratch, before I try to learn frameworks like Tensorflow, Keras, etc. Do you happen to know now, why do we sum the activation layers? Before watch this video, i thought of square root each of its squared value, like we determine the scalar value of a vector. And i turned out I was wrong after watched this video. I really looking forward for an explanation or any source of information from you, about why do we sum the change of dC/da(L-1). Thank you 😊🙏
This is truly awesome, as pedegogy and as math and as programming and as machine learning. Thank you! ...one comment about layers, key in your presentation is the one-neuron-per-layer, four layers. And key in the whole idea of the greater description of the ratio of cost-to-weight/cost-to-bias analysis, is your L notation (layer) and L - 1 notation. Problem, your right most neuron is output layer or "y" in your notation. So one clean up in the desction is to make some decisions: the right most layer is Y the output (no L value), because C[0]/A[L] equals 2(A[L] - y). So the right most three neurons, from right to left, should be Y (output), then L then L minus one, then all the math works. Yes?
I seriously have a hard time explaining how much I appreciate this video. I am far and away a symbolic thinker, as opposed to a geometric one, and while I love all of your videos and how intuitive you make the concepts, it's sometimes hard for me to think about the geometry. I am much more comfortable working with symbols and that's why I treasure videos like this. Thank you :)
I can't tell you how long I've been trying to visualize all this in my head to get a solid mental picture of backpropagation... Well, I guess I can - it was the duration of a flight from Frankfurt to Seattle (about 9 hours) and it involved one terribly lit backside of an airplane menu and a shitty pencil. I am so grateful for the work you put into animating this algorithm. It has literally brought tears to my eyes and a smile on my face. Thank you.
I remember watching this series 4 years ago, when I was in highschool. It made some sense, I even coded a neural network from it. But now, after Calc 3 in uni, it makes much more sense.
After seeing a few pieces of books, descriptions on the internet about back propagation, with this video I finally reached some kind of enlightenment (especially at about 4:50 into this video). Thank you so much for that! Just as a hobby, I was trying to implement a neural network from scratch in java: plain objects for neurons, connections and layers. I wanted to really visualize how a neural network WORKS. (Visualize either as computer code, but maybe I even want to create some visual for it...) This will certainly help me on my way!
Honestly this channel doesn't deserve a dislike button. It took me days to figure out one video(at the beginning),but the concepts remain still in my head. This channel taught us that maths is not just changing numbers, but its conceptual and intuitive just like science. Grant if you are ever read this, please know that you are one of the very few people that change the world. I just dont have words for you man, great job is an understatement for you. I promise once i earn enough i will contribute to your channel
Thank you so much for making whats very complicated to something slightly less complicated. Spending time watching this videos is the best way i can waste my time ;)
Honestly, this is one of the best (If not the best) channel on Mathematics/Science education I have seen. Intuitive but not oversimplified. Thank you so much that for offering your spectacular work and you help so many people understand these concepts.
Two things worth adding here:
1) In other resources and in implementations, you'd typically see these formulas in some more compact vectorized form, which carries with it the extra mental burden to parse the Hadamard product and to think through why the transpose of the weight matrix is used, but the underlying substance is all the same.
2) Backpropagation is really one instance of a more general technique called "reverse mode differentiation" to compute derivatives of functions represented in some kind of directed graph form.
You should probably change the thumbnail. The snapshot of the variables with indices (which I didn't know were indices at the time) and subscripts almost deterred me from watching this although it really wasn't that complicated.
I will probably be a TA for a lecture course on Deep Neural Networks again next semester and I will recommend this series to the students (we did it in a very math-heavy way this year and it was a bit too much for some of them, even though its a lecture for master students)
I was hoping you'd explain in either of these 2 vids on backprop why the hadamard product and transposing are used, what a waste :(
Amazing video, I think you did a really good job of making it as easy to understand as possible while also not simplifying things too much.
Sirius Black what you could do is download the r package "deepnet" and poke around at the source code. Its written in base r so you can follow it around. This is how I learned, and IMHO the best way to learn.
This series was my first introduction to Machine Learning 3 years ago. I now work full-time as an AIML Scientist, my life is forever changed. Thank you.
how hard was it? I am tryna code my own neural network from scratch, there's so little resources for that it seems. and how do I even make myself unique?
@@envadeh Learn everything, use the feynman technique; if you can't explain how a machine learns to someone who knows nothing about it, keep filling in the gaps. Formal education is great, but honestly more of a waste of time than not. Teach yourself, learn how to learn, and then keep learning.
I audited Andrew Ng's Deep Learning Specialization from Coursera, had some formal education, and self taught myself everything I could get my hands on, from theory to application, the underlying math to the practical programming. Understand the importance of data, DATA IS KING. Watch Interviews with industry leaders, understand the big turning points and continued development within the last two decades of AIML (you'll figure out what they are with time).
It takes 10,000 hours to become an expert, I'm about 4,500 in, but all it took was a little bit of work every single day. Make learning a habit. Trust yourself, believe in your ability to become who you want to be.
"It doesn't matter if your read two research papers in a week, what matters is if you read two research papers a week for a year, now you've read 100 papers" - Andrew Ng
(Don't 'read' research papers, watch synopsis! Smarter not harder! There's so much free information, you could probably use a GPT model to teach you what you don't know!)
Goodluck, and I believe in you! :)
Did you need a CS or math degree to get into the field?
@@nczioox1116 "Need" is a strong word, it just depends on what kind of work you want to do/who your employer is; people used to go to college because that was the only place you could learn these difficult subjects, but now it's just an archaic way of putting you in debt since you can learn these things online for free UNLESS you want to work for an employer where you need the degree to be recognized.
If you are self-motivated and can teach yourself these subjects, seriously consider your options before assuming that spending 4 years of your life and 100k+ is necessary.
I have an Electrical Engineering degree, but out of the 40+ classes I had to take for it, only 2 had any sort of impact on my daily job now. It all depends on the context.
Goodluck, and I believe in you! :)
@@thomasclark8922 Thank you! I have a mechanical engineering degree. I'm in the process of self teaching myself machine learning concepts and doing some projects. Lots of job postings I've seen in the field seem to require a bachelors or masters in CS, math, or neuroscience. Of course these seem to be for larger companies so maybe smaller companies might take a more holistic approach
Dear Grant,
A year ago, I decided I wanted to learn Machine Learning and how to use it to make cool stuff. I was struggling with some of the concepts, so I went to TH-cam and re-discovered this series on your channel.
Out of all the courses I've tried and all the hours of other content I've sat through, your videos stand out like a ray of sunshine. I just got my first full-time job as a Machine Learning Engineer, and I can confidently say it would never have happened without this series.
Your channel may have affected the course of my life more than almost any other. Thanks for all your hard work!
You could say that this channel had the largest nudge to your activation value
@@maruferieldelcarmen9573 The partial derivative of Grant's videos with respect to my career is off the charts!
@@maruferieldelcarmen9573 get out 😂
@@cineblazer also i just saw your comment 11months ago, it's amazing to see your development! keep it up!!!
Congrats on your job. I was wondering, when you finished Andrew Ng’s ML course, what additional steps and how long did you have to take to become a full fledge ML engineer?
Thanks in advance
Many guys claim to know. Some guys actually know. But only one guy actually knows and can explain to his grandma as well with very beautiful animations. You are that ONE !!!
I think my grandma would understand ;D Maybe on a very very high abstract level.
@@Seff2 Python grandma
Your work of making high levels of math accessible to anyone wishing to learn a variety of new topics is not obvious to me. You succeed to explain everything so clearly, making me want to start learning maths again, reminding me of and introducing me to beautiful aspects of math, and you deserve more than a 'thank you' :)
I just started out with my ML career. This entire series made me feel as if I knew it all along. Thank you Grant
I will return to this comment to share my professional progress😊
Now it's time to return to tell your progress
Any update?
we are waiting
“The definition of genius is taking the complex and making it simple.”
- Albert Einstein
You are genius.
I thought he said "You don't understand something well until you can explain it in a simple way"
That's not the definition of genius
@@jean-francoiskener6036 yes, it is a quote that appeared in this youtube channel
"More quotes are attributed to me than I could possibly have said during my entire life." - Albert Einstein
@@Djorgal Did he actually say that?
\
Hey for all of you getting discouraged because you don’t understand this - that was me last year. I went and taught myself derivatives and came back to try again and suddenly I understand everything. It’s such an amazing feeling to see that kind of work pay off. Don’t give up kiddos
Thanks for the nice words 🙂
studied this 3 years ago in college and it all makes sense to me now
Yup. I tried to watch this series a year ago and make my own neural network which turned out to be disappointing. A semester into college and some passion for calculus is all it took for me to mostly understand this series!
Same here man
I seen this video a year ago
But now only i understand fully
Keep commenting
I would say it’s recommended to have read calculus 2 (for partial derivatives and the Jacobian) and linear algebra (for matrix and vector multiplication). Otherwise, just looking up mentioned things is also fine. But it might take time to build up intuition for the math.
It's not that no-one else makes top-notch math/cs videos, it's that this guy makes it CLICK.
hiqwertyhi It rivals even the website BetterExplained. People like this teach me how to teach.
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
It has taken me about 3-4 days worth time to understand all of these 4 lectures, lectures which are in total, no longer than 1 hour and 30 minutes.
And I feel proud.
you should get the t-shirt-merch from 3b1b's description site, the shirt says "pause and ponder"
Same here. Took my time to hear all explanations. This last video is by far more complex than the previous ones, yet still very well explained.
I'm attempting to implement it from scratch in C# with no matrix math library or anything so I can get a feel for the nuts and bolts. This is the boss level!
@@berkebayraktar3556 Yeah, I'd love to once I can get it to train properly! So finicky.
Daniel McKinnon me too! I’m working on the back propagation, this math is hard
Man u deserve a Nobel Prize for teaching Machine Learning with this simplicity.
That is the reason for learning the math! To understand such a beautiful things! That is awesome! Thank's a lot!!!
Math on his own is also the reason for learning math.
Интересно повстречать тебя тут. :)
and when I asked my math teacher, that person told me you need this to pass the test. that didn't make a lot of sense back then
@@vvii3250 Ага, чувствую себя как в своеобразном мини-клубе, где собралась небольшая компашка и тусуется))
@@krenciak Ыыыыыы
If there is an award for educational video series on advanced scientific matters. Please give this award to 3b1b. Love it!
I've worked with AI for 2 years now. I have never seen anyone explain this as succinctly and aptly as you have. This video is legitimate gold. Going to show this to anyone who needs an explanation in future!
Rewatching this over and over and over again... that really does help with understanding it.
It builds connections in the brain
I came here after Andrew Ng's week 5 in coursera and you blew my mind
Oh my god, me too
here too :D
yeah bcz of corona..bw i am doing it for free
same bro
Same bruh
My god! A basic machine learning video series that actually makes sense to completely beginners!
Subscribed, and waiting in great anticipation for the next one! :D
This is the longest 10 minute video I have ever watched. Literally took me half an hour, but the feeling of the idea behind this completely settling in , makes it totally worth it!
I think it's easier if you take the time to re-write all of it yourself, on a scratchpad, work through writing the formulas etc. Then you can play with these objects in your mind more fluently. It takes a longer time initially but I feel you get more out of it. And that's a good way to begin "translating" into vector algebra by taking "simple" papers on DL (haven't go there myself yet).
@@phil.4688 This, is so important. The math really doesnt click unless you write down where each derivative comes from. And the fact that you need more than one partial derivatives for each layer.
@@phil.4688 Exactly what i had to do. Was getting the concepts, but only really understood once i started taking notes, and perform the calculations by myself (took even more time as had to re-learn derivatives...). The most interesting is that, now that i understood it, i'm even more appreciative the way it's explained in the video.
Some memorisation is required.
The way to understanding is easily recalling the different pieces and how they fit together.
I hear you. I rewatched videos 1 and 2 in this series earlier, and will be rewatching videos 3 and 4 later.
youre a deity Grant
Haha, why hello Bill. Nice to find you on 3B1B's channel :D
wut
wut
wut
wut
This series explained all of this to me 100x better than the courses I paid for. You are a genius, keep up the great work !
It is just absolutely crazy how well this guy teaches
This series is insanely good. As a teacher, I feel like Salieri watching Mozart play and being like "It's so beautiful, how is he so good!"
As a former mathematician I feel the same way :).
I like this analogy.
This is the first time I have ever liked a comment bc I could not agree more.
@Stanisław Galas I don't get why we want the devirative of c in respect to w^L. can you explain pls? It isn't a division right?
Too many notes, I do not understand!!
This is the best and clearest explanation in all BP course I could find ! 沒有之一！
没有之一，haha
Are you going to use this knowledge to help develop the social credit system?
kang Chih Lun 哈哈哈哈哈
Good! 沒有之一！
you are right! 没有之一，哈哈哈
I just unsubscribed to this channel, so I can have the pleasure of subscribing again.
I just unliked your comment...
So I can have the pleasure of liking it again
i just liked and then unliked ur comment bc i realized it was at 69
@@ifusubtomepewdiepiewillgiv1569 nice
"Not gonna lie, you got me in the first half"
@@ifusubtomepewdiepiewillgiv1569 .
God Bless You man! I have tried watching many videos about backpropagation but this series made my conceptual understanding and intuition super clear. Thanks a lot. You have no idea how happy I am right now to have understood the concept of backpropagation.
literally thank you. I learned the information that I could not learn at school for 5 weeks in a 10-minute video. The animations of the video are absolutely magnificent. Thank you thank you thank you
this is easily the best channel in youtube today! once I get a job i will more than glad to support you!
Exactly what i was thinking!
It has been two years. Have you supported yet??
Did you get a job ?
Two years ago, could you get AI job?
@@fitokay I think AI won and got his job.
Thank you a lot for this series! It has really helped me get into this topic, and changed my life. Your intuitions have been immensely helpful in my efforts to understand backpropagation. I just can't overestimate, how great your channel is!
Hello, I'm greatly impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
this is math at the best and art too at the highest. the grace of the animation, the subtle music, the perfectly paced narration and the wonderful colour scheme! math and art or let's say math is art!
This saved my life and made so much sense! Why can't more teachers be ACTUAL teachers like you instead of just assuming you know everything?!
Nicely done video.
I knew I learned backpropagation before, but it was hard, and I didn't use it manually ( I used frameworks like TensorFlow which uses computational graphs and backpropagate automatically) so I've forgotten how it actually worked.
But this video is a great resource for newcomers to ANNs and people like me that have forgotten the theory behind it all. Thank you.
This is how 21st teaching should look like. It feels like your work should be made a "human right". Thank you.
No human has the right to another human's labor. That's called slavery.
The quality of this education is top-tier. I absolutely am speechless that you make it freely accessible. Thank you so much!
never in my life have I come across a calculus video explained so beautifully! in awe with this intuition
I really can't believe how well and simply this is explained here by you, I still keep coming back to this exact video even after few years of being in AI to refresh the fundamentals and to this day this is the best resource for this.
Amazing man..... I say 3gold1platinum
More like 3plutonium1antimatter
Yeah, cause he nukes the brains of laymen like myself.
@@ishwarkarthik327 It's not.
Neural networks have layers, ogres have layers
Shrek is an AI confirmed
Goddamn nice reference would like it a thousand times over if I could
Shrek is our AI overlord
My AI overlord has decided they're both onions.
~onions have layers too~
Big. Brain.
I'm taking Machine Learning by Andrew Ng on Coursera right now, and just got stuck on backpropagation. Thank you thank you thank you thank you Grant, you have no idea how incredibly helpful your videos are and how much your channel has inspired me through the years.
I was in the same position 2 years back . This video does clarify the topic - tremendously
Here I am. Same situation. Andrew Ng course and backpropagation is rough. This video in particular really helped to clear things up. Breaking it down to a single neuron is enormously helpful.
Same here!) I was somewhat disappointed when Andrew Ng course just through the formulas at me, so I tried to derive backpropagation myself and got stuck in all the little details. Thankfully, 3b1b rode in like a knight in shining armor and now I am really damn happy))))
I think the wheel gets repeated with every new comer !
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
"So pat yourself on the back! If all of this makes sense, you have now looked deep into the heart of backpropagation, the work horse behind how neural networks learn." felt soooo goooooood
If i get a Job or a Position, just so that I get to share my success story- 80% of my speech will be 'thank you 3Blue1Brown'.
You are a blessing! Cheers.
Thank you for starting your indices at 0.
Awesome channel
I just finished up calc iii this semester and I have never felt happier with myself for being able to apply my new knowledge than this episode. I also don't think I have ever been more excited to hear calc iii topics being brought up in a field I am trying to teach myself currently. Thank you for making such a simple to understand series!
Months of working with deep neural networks and i still have to come back to this video to re-digest the math. Don't get discouraged if you don't understand it! As 3blue1brown said it, it is very complex and takes time to digest.
I am speechless. This channel undoubtedly contains the best pedagogical scientific material on TH-cam, and possibly in the world. Thanks for making these videos, your skills and passion are unreachable!
I always feel, if u have a mentor who can break complex things into simple stuff so beautifully, even a dumb guy can grasp the concept. Keep doing the good stuff. Admirer++
Thanks a lot for creating such a fantastic content! Anticipating to see more videos about AI, ML, Deep Learning!
At time when I just finished my university - I could not imagine that at one chilly Sunday evening, in almost 15 years after the graduation, I will sit with a bottle of beer, watch math videos, and have so much fun! Thank you!
I just watched this video going back and forth several times and I am at a point where I am not sure if it made sense or not, but I still feel like it helped. Thank you so much for this series!
You bring clarity on such potentially complex ideas as Prometheus brought all the light to everyone in the mythology. We are forever grateful for your efforts! You are a math artist and this videos are a masterpieces conveying intuitive understanding of such complex ideas to many people.
I cannot tell you how much I appreciate these videos. I don't have a strong math background (english undergrad) but I'm teaching myself data science. It REALLY helps to have the equations explained rather than just presented and to tie the components of the equation back to intuitions. Thank you thank you thank you.
Grant, I've come back to this series many times over the last five years. Every time I do, I pick up more and more pieces of the puzzle. I think I've finally got it now, but we shall see! Thank you!
*looking at thumbnail
oh sh~, Im never going to understand that complex stuff. probably should watch it anyway
*ten minutes later
whoa! that's actually quite clear now!
same for me.
Тоже учишь нейросети?
просто люблю математику и познавательные видео на эту тему) особенно нейросети
I think that’s kinda the point of much of this channel lol. Same experience though! It’s amazing how a TH-cam video can do so much teaching in like 10 minutes
I had the same reaction. Beautifully broken down into calculus us mere mortals can understand!
You are some sort of sorcerer. The chain rule was finally elucidated for me in about 20 seconds.
So concise and intuitive indeed.
That was one thing, for whatever reason, I couldn't get my head around as a younger student.
Big props man. Love this channel.
I keep coming back here every time I have doubts about the mechanics of back propagation. I need to visualise concepts to understand, so for me your video so well done. I can't say anything else than thank you.
Awesome material. :) Please make a video about convolutional neural networks, too.
The point where you addressed the concern that the example you were using was too simple, having only 1 edge, was spot on as you were leading me down this merry garden path. I appreciate how much you watch your own videos and predict where the watcher would mentally say, "but what about..."
0;090505452 0145
These videos are unbelievably well produced. Thank you so much for your effort. You've made this topic incredibly clear and I cannot understate how much I appreciate the amount of effort you put into these. You have incredible talent as a teacher.
Mr. Grant please make videos on CNN and LSTM, the world seriously needs videos of yours on these topics...PLEASEEEEE.
Thanks for your
astonishing
eye-opening
hallucinatory
mind-altering
mind-boggling
overwhelming
psychedelic
staggering
stunning
wonderful explanation to the world.
thank you SO MUCH for publishing this video! The illustrations, explanation and your calm voice made it so easy to understand! I first hear of backprop yesterday and the professor didn't go into the math of the cost function derivative one step back from the output layer and your explanation really helped me get a better sense of understaning what's happening here. I can not thank you enough - bless you!
Literally the only youtube channel, that makes studying 2 hours of math, go by in a blink of an eye
Absolutely brilliant explanation! I took a course on deep learning in college, but ended up auditing it in the end because I couldn't grasp the concepts well enough to pass the tests. You just took the entire first unit of the course, which took several weeks, and condensed it into 4 easily digestible videos that anyone can understand!
Thanks lord for the info
अद्वितीय!!!!
You explain things a LOTS better than any teacher out there, well done man
LOVED these four introductory videos. My brain was so exhausted processing them--lots of pausing--but I feel like I get something conceptually that I knew near nothing about a couple days ago. Great foundation to scaffold the programming and math onto.
Thank you! will definitely be watching more of your videos. Looking forward
Thank you so much! I have understood more math from this channel than from all teachers I have had in high school or university in total.
Hello, I'm impressed by the way he explained this topic too, but I'm looking for more such great quality resources, youtube channels, books on deep learning, and also math and comp science in general, what do you recommend?
If anyone is interested, I worked through the chain rule for the differential of the cost function w.r.t the weight in the second layer down. Two additional terms are added to make everything cancel as they should. It shows how as you progress down the layers, more partial differentials are added to the equation from all the variables above, making it more unstable and hence more susceptible to the exploding or vanishing gradient problem.
dC/dw(L-1) = dz(L-1)/dw(L-1) * da(L-1)/dz(L-1) * dz(L)/da(L-1) * da(L)/dz(L) * dC/da(L)
what's the vanishing gradient problem?
i'm looking for this in the video. appreciate if you could share your work
Netflix can show these rather than some other questionable material.
Nitin they would probably make it about the gender/race of different numbers, or draw some number in a way that fans of that number don't like.
A new phrase would emerge, "Backprop and chill"!
netflix show what people want . Don't blame them , blame people around you,your friends , family and yOU .
then it won't be free(unaccessible to who can't afford) and netflix doesn't care and get their hands on whatever gets them money
bdw it would be great if they could make education addictive. lol that would be really great, Imagine haha
Nothing more but a deep thank to you for making such amazing video. I truly cannot express how much this whole series had clarified this topic for me. Again, thank you very much.
This is it. Finally it clicked. After countless hours of trying to understand this, I finally somewhat understand neural networks
What an art of math, Jesus Christ
Grant, I've been thinking about these videos and I have a couple questions.... last time you fed a nonsense, scrambled image through the network, and it confidently identified it as a 5. My question is two fold:
1) Would any value be added to the network by adding a final neuron that indicates "no number" or "not anything identifiable" or "throw this image in the trash can"? Maybe that way the network would sort those trash-type images into a pile for, perhaps, a human to look at.
2) Is there any benefit to training your network to get the right answer, but be less confident about it? You keep mentioning that we want the network to identify a "2" as a "2" and the goal is a final neuron to light up the "2" at 100% (and everything else at 0%). Does this extreme goal perhaps cause really confident misclassifications sometimes (besides the junk image from last video)? Would it be in any way beneficial to train the network to see a "2", but perhaps to only be 90% confident about it? Maybe it would *think* it saw a 2, but it's not totally sure - like a young child learning numbers. (but, being 90% a 2, is still a 2.) Would this help at all with the junk images from the previous question / video? Also, maybe since we train the network to not be so confident all the time, could we then put any image that falls below, say, 75% confidence into another bin for humans to verify?
Also, on BB&B you mentioned that it can sometimes feel like too much pressure when people say "yeah, good video, I can't wait for the next one!" Well, I'll instead say, thanks for this series, and I'll patiently await the next video as you take your time to make it a high quality production like you always do - whenever that happens, I'm good. No pressure!
+poundcake2000 You should give both ideas a try. Active learning!
I have no programming background at all, but couldn't you just generate many random images and label them as (0,0,0,0,0,0,0,0,0,0) and then add them to the training data? (instead of adding a new neuron)
(i have no idea what this will do but it sounds interesting)
I was wondering the same thing. Training it to identify random noise as random noise. Great question! Lemme know if you find out.
I'm not 3b1b, but I can give you some hopefully give you some answers to your questions, since I'm working with Neural Networks in my PhD:
First of all, that confident prediction of a 5 last video was a bit of a trick: It is not actually a completely random image, but was created to generate that specific output. He could have generated an image that looked just as random, but produced any arbitrary output or he could have taken an image that is clearly one number to a human but is caught as another number by the network. This is a (by now) well known flaw with neural networks that you can usually generate an image that produces a desired output but looks very similar to another image (to a human) that produces a sensible output.
To your first question, aside from the flaw I described above, if you want to know whether there is a number at all, you usually look at whether any of the outcomes is above a certain threshold and if none is you say 'not anything identifiable'. An additional output neuron would require you to add enough 'not identifiable' example images to the training that it would drown out the other images or otherwise it might not say 'not identifiable' for some random input image.
To your second question, I don't think that would be very useful since mosts of what you are doing would be training the network to output 0.9 when it is most confident. Also, the network rarely is completely confident (unlike what the video portrays), so the thresholds you described can be used with the network as it is.
+TalathRhunen Great response, thanks! Also, by the way, the example from the last video was a genuinely random image, not an adversarial example. I'm not saying that happens for all neural nets, of course, but for this simple one that was the emerging behavior.
9:28 sums up the whole thing
Thanks, I've been learning about NN since the 90s and this is the best explanation of the deep math.
With your explanation, all these partial derivatives finally make sense. Thank you!
That's the end? WTF
Stunningly beautiful...
The best part of the series (for me, obviously) is that the beauty of this series does NOT make it very easy to understand.
No. Each video may need multiple views. But these videos are so beautifully made that you'd want to watch them again and again, not with the frustration of getting your head over a concept but with the thrill of unravelling a mystery...
So for creating such excitement in me, thank you.
Great presentation! Very helpful. Although, could you clarify if the indices at 9:30 are consistent? if you sum over j in the 'next' layer, you would have k in the current layer. Instead of w_jk^(l+1), would it be w_ij^(l+1) and sum over i for this next layer? Thanks!
It seems that the second formula (at 9:30) is inconsistent with formulas from Andrew Ng ML course and as of me looks very confusing because of inaccurate use of indexes. Let's simplify the vectorized formulas from the above course and keep using the derivative of the activation function: sigma'(z(L)) in delta(L) as suggested in the video. With index notation we will get for d_C / d_a_j(l): sum_over(q=0 -> n(l+1)-1) [ w_qj(l+1) * sigma'(z_q(l+1)) * d_C / d_a_q(l+1) ]
Thank you very much for your amazing explanations. I'd been buckling down a lot for understanding back propagation. While all the explanations I found failed to make me clearly understand the topic, your explanation just worked wonders! Thanks again for so nice videos!
This NN series was amazing, thank you so much! I've watched a lot of these kinds of videos, read several online articles, and am following a popular online course. This series is the best resource by far, so helpful.
i watched this video like 10 times at least and, Finally! finally! i was able to build my own neural network with back prop, in c#
and it actually works!
Btw, i found out that if my layers have a lot of neurons the System often over shoting, and deviding the Cost for each neuron by how many neurons,
are in that layer help to fix that issue, i mean i can have the same training value for any type of neural network and it wont over shoot at all.
but in the video you just said to add the costs for each neuron together rather then getting the average cost for each neuron....
hmm...
can you show your backprop code? What are your parameters and what does the cost get to? I can't get mine to go below 1.
@@honeyant3119 well here it is
github.com/miko-t/UnityMyNN/blob/master/Assets/MyNet/SimpleBrain.cs
it writen in c# and iam not using metrecies just hard codded array's
Watch the function Train
@@honeyant3119 it works fine when predicting stuff like, Graph slopes etc... or just the basic stuff, tried to make it learn how to drive a "car" and it fails misserably, but that fail might be just due to the fact i dont know how to calculate real time, cost.
@@mike_o7874 search up reinforcement learning for real-time tasks. Also, driving a car should be really easy. I trained a network to do so using evoltuion and all it took was making a good cost function.
@@honeyant3119 genetic algorithems are easy yeah i already did it that way just wanna try to use back prop for it because people do it.
One of the best lectures I have ever heard. Great explanation of NN, cost functions, activation functions etc. Now I understand NN far far better...(P.S. I saw previous videos Part 1, 2,3 as well)
I'm watching the videos of this channel first and it just blow my mind. You've made a complex matter really simple.You're genius 😍😍
Thank you so much, I love how you explain it. After watching the series 3 times, I know fully understand how it works! And I can convertably make a simple neuralnetwork.
Thanks 3B1B. I'm understanding machine learning mush better, and following your video while note taking was the easiest method for learning.
I'm a little confused about what the change in C_0 with respect to change in a_(L-1, k) for the k-th activation in L-1 layer (I just changed to this notation because I feel more comfortable writing like this in one line text). That's 8:40 part of the video I guess. It doesn't make intuitive sense for me as to why you need the summation of impact of a_(L-1, k) on a_(L, 0~n), say without any multiplier or something.
Trying to understand the meaning of `dC_0/da_(L-1, k)` I thought of a Neural Network where there are only two layers, input and output layer, and input layer having 1 neuron and output layer having 2 neurons.
Does it ever make sense for a_(L-1, k) to be an activation (or neuron?) in an input layer? If so, I think it makes to add the 'impact' all up especially when the weights are all same 'direction' or sign because if so summing them all up would result in greater number, and this would mean changing the input has the biggest impact in this scenario.
If not, I'm still confused what `dC_0/da_(L-1, k)` is and why it has the summation.
hello, how are you? I know it's been 3 years since you made your comment. But for me, it's my first few weeks started to learn Deep Learning, and now trying to build Neural Network from scratch, before I try to learn frameworks like Tensorflow, Keras, etc.
Do you happen to know now, why do we sum the activation layers? Before watch this video, i thought of square root each of its squared value, like we determine the scalar value of a vector. And i turned out I was wrong after watched this video. I really looking forward for an explanation or any source of information from you, about why do we sum the change of dC/da(L-1).
Thank you 😊🙏
This is truly awesome, as pedegogy and as math and as programming and as machine learning. Thank you! ...one comment about layers, key in your presentation is the one-neuron-per-layer, four layers. And key in the whole idea of the greater description of the ratio of cost-to-weight/cost-to-bias analysis, is your L notation (layer) and L - 1 notation. Problem, your right most neuron is output layer or "y" in your notation. So one clean up in the desction is to make some decisions: the right most layer is Y the output (no L value), because C[0]/A[L] equals 2(A[L] - y). So the right most three neurons, from right to left, should be Y (output), then L then L minus one, then all the math works. Yes?
I seriously have a hard time explaining how much I appreciate this video. I am far and away a symbolic thinker, as opposed to a geometric one, and while I love all of your videos and how intuitive you make the concepts, it's sometimes hard for me to think about the geometry. I am much more comfortable working with symbols and that's why I treasure videos like this. Thank you :)
Amount of effort you guys put on this video is insane. Thank you for teaching me who studying to be AI engineer😊.
Wakarimashta ^_^ This is my favorite channel. Watching your videos doesn't even feel like learning. Its like relaxing productively.
As an hobbyist programmer, i can't thank you enough! Once i finish my studies i will donate to you. :)
Thank you so much, Grant. I finally learned back prop, and I have become a patron. I wish I could do more.
I can't tell you how long I've been trying to visualize all this in my head to get a solid mental picture of backpropagation... Well, I guess I can - it was the duration of a flight from Frankfurt to Seattle (about 9 hours) and it involved one terribly lit backside of an airplane menu and a shitty pencil. I am so grateful for the work you put into animating this algorithm. It has literally brought tears to my eyes and a smile on my face. Thank you.
I remember watching this series 4 years ago, when I was in highschool. It made some sense, I even coded a neural network from it. But now, after Calc 3 in uni, it makes much more sense.
It took me 2 whole years to finally fully understand this but its worth all the wait
After seeing a few pieces of books, descriptions on the internet about back propagation, with this video I finally reached some kind of enlightenment (especially at about 4:50 into this video). Thank you so much for that!
Just as a hobby, I was trying to implement a neural network from scratch in java: plain objects for neurons, connections and layers. I wanted to really visualize how a neural network WORKS. (Visualize either as computer code, but maybe I even want to create some visual for it...) This will certainly help me on my way!
The amount of help you are providing is nothing short of amazing.
Honestly this channel doesn't deserve a dislike button. It took me days to figure out one video(at the beginning),but the concepts remain still in my head. This channel taught us that maths is not just changing numbers, but its conceptual and intuitive just like science. Grant if you are ever read this, please know that you are one of the very few people that change the world. I just dont have words for you man, great job is an understatement for you. I promise once i earn enough i will contribute to your channel
You explained this in a way that was unbelievably easy to understand, THANK YOU!!!
Thank you so much for making whats very complicated to something slightly less complicated. Spending time watching this videos is the best way i can waste my time ;)
Honestly, this is one of the best (If not the best) channel on Mathematics/Science education I have seen. Intuitive but not oversimplified. Thank you so much that for offering your spectacular work and you help so many people understand these concepts.