Gradient descent, how neural networks learn | Chapter 2, Deep learning

แชร์
ฝัง
  • เผยแพร่เมื่อ 10 พ.ค. 2024
  • Enjoy these videos? Consider sharing one or two.
    Help fund future projects: / 3blue1brown
    Special thanks to these supporters: 3b1b.co/nn2-thanks
    Written/interactive form of this series: www.3blue1brown.com/topics/ne...
    This video was supported by Amplify Partners.
    For any early-stage ML startup founders, Amplify Partners would love to hear from you via 3blue1brown@amplifypartners.com
    To learn more, I highly recommend the book by Michael Nielsen
    neuralnetworksanddeeplearning....
    The book walks through the code behind the example in these videos, which you can find here:
    github.com/mnielsen/neural-ne...
    MNIST database:
    yann.lecun.com/exdb/mnist/
    Also check out Chris Olah's blog:
    colah.github.io/
    His post on Neural networks and topology is particular beautiful, but honestly all of the stuff there is great.
    And if you like that, you'll love the publications at distill:
    distill.pub/
    For more videos, Welch Labs also has some great series on machine learning:
    • Learning To See [Part ...
    • Neural Networks Demyst...
    "But I've already voraciously consumed Nielsen's, Olah's and Welch's works", I hear you say. Well well, look at you then. That being the case, I might recommend that you continue on with the book "Deep Learning" by Goodfellow, Bengio, and Courville.
    Thanks to Lisha Li (@lishali88) for her contributions at the end, and for letting me pick her brain so much about the material. Here are the articles she referenced at the end:
    arxiv.org/abs/1611.03530
    arxiv.org/abs/1706.05394
    arxiv.org/abs/1412.0233
    Music by Vincent Rubinetti:
    vincerubinetti.bandcamp.com/a...
    Thanks to these viewers for their contributions to translations
    Hebrew: Omer Tuchfeld
    Italian: @teobucci
    -------------------
    Video timeline
    0:00 - Introduction
    0:30 - Recap
    1:49 - Using training data
    3:01 - Cost functions
    6:55 - Gradient descent
    11:18 - More on gradient vectors
    12:19 - Gradient descent recap
    13:01 - Analyzing the network
    16:37 - Learning more
    17:38 - Lisha Li interview
    19:58 - Closing thoughts
    ------------------
    3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with TH-cam, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
    If you are new to this channel and want to see more, a good place to start is this playlist: 3b1b.co/recommended
    Various social media stuffs:
    Website: www.3blue1brown.com
    Twitter: / 3blue1brown
    Patreon: / 3blue1brown
    Facebook: / 3blue1brown
    Reddit: / 3blue1brown

ความคิดเห็น • 3K

  • @3blue1brown
    @3blue1brown  6 ปีที่แล้ว +2671

    Part 3 will be on backpropagation. I had originally planned to include it here, but the more I wanted to dig into a proper walk-through for what it's really doing, the more deserving it became of its own video. Stay tuned!

    • @HeyItsSahilSoni
      @HeyItsSahilSoni 6 ปีที่แล้ว +10

      Can you provide some link to the training set?
      I'm quite new and I'm trying to learn this "Hello World" of NN,

    • @pinguin1009
      @pinguin1009 6 ปีที่แล้ว +3

      Did you consider to do a part about phase functioned neural networks? Would be awesome!

    • @nyroysa
      @nyroysa 6 ปีที่แล้ว +1

      As the part progresses,
      we're getting closer to seeing that "lena.jpg" picture

    • @akhileshgangwar394
      @akhileshgangwar394 6 ปีที่แล้ว +10

      You are doing very good job , lots of hard work behind this video , i salute your hard work thanks

    • @mynameisZhenyaArt_
      @mynameisZhenyaArt_ 6 ปีที่แล้ว +8

      So have you decided to do more of these videos? There is a line with CNNs and LSTMs in video series...

  • @melkerper
    @melkerper 4 ปีที่แล้ว +7717

    Dissapointed you did not animate a 13000-dimensional graph. Would make things easier

    • @havewissmart9602
      @havewissmart9602 4 ปีที่แล้ว +142

      No.... No it would not....

    • @mrwalter1049
      @mrwalter1049 4 ปีที่แล้ว +410

      A 2-dimensional projection of a 13000-dimensional graph would probably look like a pile of garbage.

    • @cchulinn
      @cchulinn 4 ปีที่แล้ว +323

      If 3Blue1Brown cannot animate a 13000-dimensional graph, then noone can.

    • @tehbonehead
      @tehbonehead 4 ปีที่แล้ว +106

      @@mrwalter1049 You're not thinking fourth-dimensionally!!

    • @mrwalter1049
      @mrwalter1049 4 ปีที่แล้ว +29

      @@tehbonehead No-one is. We're trapped in three dimensions. That's why you could never imagine what a 4-dimensional cube looks like.
      Making a 4-dimensional projection of a 13000-dimensional object isn't significantly better than 3 dimensions.
      If you meant to be humorous I hope someone gets a chuckle, because I didn't. Then your effort won't be in vain.
      Have a nice day 🙂

  • @snookerbg
    @snookerbg 6 ปีที่แล้ว +2220

    One of youtube's highest quality content channels! Chapeau

    • @Dom-nn1kg
      @Dom-nn1kg 6 ปีที่แล้ว

      Kosio Varbenov +

    • @yoavtamir7707
      @yoavtamir7707 6 ปีที่แล้ว +2

      True!

    • @JD-jl4yy
      @JD-jl4yy 6 ปีที่แล้ว +4

      'chapeau'

    • @achillesarmstrong9639
      @achillesarmstrong9639 6 ปีที่แล้ว +1

      agree

    • @tubbyszabo6586
      @tubbyszabo6586 5 ปีที่แล้ว +6

      Why you write "chapeau"? Hat in French. What does hat in that sentence do?????Be honest tho

  • @DubstepCherry
    @DubstepCherry 3 ปีที่แล้ว +1594

    I'm an IT student, and we have an Assignment on exactly this topic. We even have to use the MNIST data set. I have to say, this is absolutely lifesaving and I can not thank you enough Grant. What you do here is something that only a handful of people on this planet can do, explain and visualize rather complicated topics beautifully and simple. So from me and A LOT of students all around the globe, thank you so so much

    • @vsiegel
      @vsiegel 2 ปีที่แล้ว +36

      Yes, it is just extremely good, in an objective way. He is brilliant at it, and spends a lot of time on each video. If there is an explanation of something by 3blue1brown, you will not find anything explaining it nearly as good.

    • @johndough510
      @johndough510 2 ปีที่แล้ว +3

      @@vsiegel bro you guys are so much smarter than i am im jealous

    • @vsiegel
      @vsiegel 2 ปีที่แล้ว +17

      @@johndough510 If you are thinking about how smart you are, you are probably smarter than you think. No worries.

    • @johndough510
      @johndough510 2 ปีที่แล้ว +5

      @@vsiegel thanks for being so cool about it man, hope you have a good one

    • @alfredoaguilar2076
      @alfredoaguilar2076 ปีที่แล้ว

      @@johndough510 I'm kill

  • @plekkchand
    @plekkchand 3 ปีที่แล้ว +647

    Unlike most teachers of subjects like this, this gentleman seems to be genuinely concerned that his audience understands him, and he makes a concerted and highly successful effort to convey the ideas in a cogent, digestible and stimulating form.

    • @hamidbluri3135
      @hamidbluri3135 2 ปีที่แล้ว +5

      TOTALLY agreed

    • @redflipper992
      @redflipper992 ปีที่แล้ว +1

      concerted with whom? I don't think you understand how to use that word.

    • @werwinn
      @werwinn ปีที่แล้ว

      he is a true proffesor!

    • @macchiato_1881
      @macchiato_1881 ปีที่แล้ว

      @@redflipper992 I don't think you understand what concerted means. Stop trying to act smart and think you're better than everyone here. Be humble. You are irrelevant in the big picture.

    • @JeffCaplan313
      @JeffCaplan313 9 หลายเดือนก่อน

      ​@@redflipper992I read that as "concerned"

  • @Shrooblord
    @Shrooblord 6 ปีที่แล้ว +2142

    I'm only 12 minutes into this video right now, but I just wanted to say how much I appreciate the time and spacing you give to explaining a concept. You add pauses, you repeat things with slightly different wording, and you give examples and zoom in and out, linking to relevant thought processes that might help trigger an "a-ha" moment in the viewer. Many of these "hooks" actually make me understand concepts I've had trouble grasping in Maths, all because of your videos and the way you choose to explain things. So thanks! You're helping me a lot to become a smarter person. :)

    • @Dom-nn1kg
      @Dom-nn1kg 6 ปีที่แล้ว +4

      Shrooblord +

    • @luke7503
      @luke7503 6 ปีที่แล้ว +2

      yes.

    • @Appscaptain
      @Appscaptain 5 ปีที่แล้ว +1

      Totally agree!

    • @atulct
      @atulct 5 ปีที่แล้ว

      Absolutely agree

    • @CaerelsJan
      @CaerelsJan 5 ปีที่แล้ว

      Couldn't agree more

  • @seC00kiel0rd
    @seC00kiel0rd 6 ปีที่แล้ว +3174

    My math career is over. Once I learned about gradient descent, it was all downhill from there.

    • @rlf4160
      @rlf4160 6 ปีที่แล้ว +263

      I had a similar fate, except mine went negatively uphill.

    • @yepyep266
      @yepyep266 5 ปีที่แล้ว +185

      just remember there are people in an even lower minima than you are.

    • @jomen112
      @jomen112 5 ปีที่แล้ว +110

      Yea, but making random choices makes you eventually reach the bottom.

    • @thetinfoiltricorn7797
      @thetinfoiltricorn7797 5 ปีที่แล้ว +30

      It's all planar vectors from here.

    • @nateschultz8973
      @nateschultz8973 5 ปีที่แล้ว +35

      You just need to take a few steps back and turn your life around.

  • @JockyJazz
    @JockyJazz 3 ปีที่แล้ว +193

    3:38 you missed the chance of using the meme *"AI: I've found an output, but at what cost?"*

  • @colonelmustard7078
    @colonelmustard7078 11 หลายเดือนก่อน +160

    Not only the videos themselves are great on this channel but the lists of the supporting materials are amazing too! Drives me down a breathtaking rabbit hole every time! Thank you!

  • @Nyhilo
    @Nyhilo 6 ปีที่แล้ว +334

    After watching your first video, I ended up drawing a "mock" neural network up on paper that would work on a 3x3 grid (after all what else are you supposed to do during a boring lecture class?). It was supposed to recognize boxes, x's, sevens, simple shapes, and I defined the 7 or so neurons that I thought it might need by hand. I did all the weighted sums and sigmoid functions on paper with calculator in hand. It took maybe an hour and a half to get everything straight but once I did, it worked. It guessed with fairly good accuracy that the little seven I "inputted" was a little seven. All that excitement because of your video.
    Later that evening and the next one, I tried to program the same function taking PNGs as inputs and definitions of the neurons and it honestly was only a little more rewarding. But now that I see what the hidden neurons *actually* look like, I only want to learn so much more. I expected the patterns to be messy, but I was really surprised to see that it really does almost look like just noise.
    Thank you for making these videos. I find myself suddenly motivated to go back to calculus class tomorrow and continue our less on gradients. There's just so much out there to learn and it's educators like you that are making it easier for curious individuals like me to get there.

    • @3blue1brown
      @3blue1brown  6 ปีที่แล้ว +107

      That's so cool, thanks for sharing! I didn't expect anyone to actually go an play with it by hand, but simplifying down to a 3x3 grid seems really smart. Stay curious!

    • @Dom-nn1kg
      @Dom-nn1kg 6 ปีที่แล้ว

      Nyhilo +

    • @Dom-nn1kg
      @Dom-nn1kg 6 ปีที่แล้ว

      3Blue1Brown +

    • @jayeshsawant6734
      @jayeshsawant6734 6 ปีที่แล้ว +5

      Were you able to do all of that by watching this video series alone? Please can you add other resources you referred? Thanks!

    • @michaelhesterberg702
      @michaelhesterberg702 6 ปีที่แล้ว

      .....go anD play... AND AN+D

  • @hangilkim245
    @hangilkim245 4 ปีที่แล้ว +111

    "But we can do better! Growth mindset!" at 5:18 .... a wholesome intellectual i love to see it

  • @kraneclaims
    @kraneclaims 3 ปีที่แล้ว +231

    I just sat through a 3 day ML accelerator class and you series did a far better job at explaining them with 4 twenty minute videos. Well done mate. Really appreciate it. Thank you

    • @NoobJang
      @NoobJang ปีที่แล้ว +9

      this youtuber is the best in maths and engineering in general
      i have never been so astounded for how easy learning machine learning can be, without having to take in bunch of complex topics that doesnt add to the discussion.
      Like most of the courses try to make you understand various different complex topics and by the time you finished it, you will ahve forgotten mostly about machine learning. Why dont you just explain the catch for each concept then allow us learn it in depth afterwards
      like these channels only explaining the concepts with both ease of learning and depths are the best.

  • @paulo1680
    @paulo1680 ปีที่แล้ว +60

    i have no words to describe how thankful i am. thank you so much for such great content.

  • @bikkikumarsha
    @bikkikumarsha 6 ปีที่แล้ว +112

    You are changing the world, shaping humanity. I wish you and your team, happy and peaceful life. This is a noble profession, god bless you guys.

  • @TheJeSuzZ
    @TheJeSuzZ 3 ปีที่แล้ว

    Rarely seen such well explained videos that break down a complex topic into components that are so easy to understand. Perfect speed and focus on core aspects. Plus summaries and references to previously discussed correlations. Very good examples and animations, which pick up the viewer without/with little knowledge where it has to be. THANK YOU

  • @imad_uddin
    @imad_uddin 2 ปีที่แล้ว +13

    Cant believe you explained this so easily. I thought it would take me ages to wrap my head around what neural networks basically are. This is truly amazing explanation!

  • @Skydmig
    @Skydmig 6 ปีที่แล้ว +44

    That end comment with Lisha Li really points out how important it is to put a lot of effort into gathering and creating good and structured data sets. I know it's cliché to state "garbage in, garbage out", but these findings put very precise context and weight to this particular issue.

    • @atlas7425
      @atlas7425 6 ปีที่แล้ว +3

      Haha, "weight".....get it?

    • @theespatier4456
      @theespatier4456 6 ปีที่แล้ว +1

      StiffWood True. This also becomes ethically important in medical applications of AI, where poor input can create racist AI and the like.

  • @cody._.--._.--.
    @cody._.--._.--. 4 ปีที่แล้ว +272

    I wish someone would have introduced this to me at a young age back in the 90s. I had no idea neural network have existed for so long

    • @lopezb
      @lopezb 4 ปีที่แล้ว +43

      Now it's easier to explain. He couldn't have made a video like this back then, both because TH-cam didn't exist, and all the relevant stuff would be in technical papers...

    • @Djorgal
      @Djorgal 4 ปีที่แล้ว +23

      @@lopezb Also, it was a really niche field that didn't show that much promise.

    • @damienivan8946
      @damienivan8946 4 ปีที่แล้ว +7

      Also, from my understanding, modern neural networks are very different from the one in the 90s

    • @bubblelyte401
      @bubblelyte401 3 ปีที่แล้ว

      It's a college graduate course.

    • @georgalem3310
      @georgalem3310 3 ปีที่แล้ว

      In the 90s NN fell into disfavor.

  • @Shubhi021
    @Shubhi021 ปีที่แล้ว +10

    This video is truly a work of art. The animations are mesmerizing. Thank you for all your work, Grant!

  • @-beee-
    @-beee- 9 หลายเดือนก่อน

    Keep coming back to this series and sharing it with so many people. This whole channel is truly a gift. Thank you so much for making these!

  • @musthavechannel5262
    @musthavechannel5262 6 ปีที่แล้ว +478

    "I'm more of a multiple choice guy" LOL

  • @obsidianblade4228
    @obsidianblade4228 6 ปีที่แล้ว +1486

    Did anybody else feel bad for the network after he called the output utter trash?😢

    • @sarahmchugh4169
      @sarahmchugh4169 4 ปีที่แล้ว +138

      I know, especially with those sad computer eyes. Tragic

    • @MarioRodriguez-or9fn
      @MarioRodriguez-or9fn 4 ปีที่แล้ว +61

      Yes, specially when he called it bad computer :(

    • @DavidLee8981
      @DavidLee8981 4 ปีที่แล้ว +7

      we are all utter trash for future robots

    • @theshermantanker7043
      @theshermantanker7043 4 ปีที่แล้ว +46

      Bruh there's literally Reinforcement Learning where the Network is tortured by the researchers when it gets a wrong answer and the torture continues until it gets the right answer lol

    • @David5005ful
      @David5005ful 4 ปีที่แล้ว +1

      Lmao.

  • @jasonzhang6534
    @jasonzhang6534 3 หลายเดือนก่อน +6

    my professor has explained this in 3 lectures for about 6-7 hours. 3B1B explained it in 30 mins and it is much more clearer. I can now visualize and understand the what/why/how behind the basic deep learning algorithms. Really appreciate it!!!

  • @Heisenberg355
    @Heisenberg355 ปีที่แล้ว +1

    This man is a living legend. I really sincerely believe he's one of the best "explainers" for many complex mathematical topics. I found your channel because of linear algebra, and now I'm relieved whenever I search for a topic and see one of your videos. You truly are the master of your league

  • @tonraqkorr230
    @tonraqkorr230 5 ปีที่แล้ว +732

    We need AI to recognise what the doctors write

    • @frankchen4229
      @frankchen4229 3 ปีที่แล้ว +53

      whoever designs the algorithm and engineers the software deserves a nobel peace prize

    • @flyinglack
      @flyinglack 2 ปีที่แล้ว

      @@frankchen4229 LOL

    • @johnbarbuto5387
      @johnbarbuto5387 2 ปีที่แล้ว +4

      Who writes any more??? That horse left the barn a long time ago. Besides, we are no longer doctors. Courtesy of insurance companies we are "providers". (The same strategy of devalued identities has long been used by invading armies to anonymize those being conquered, an apropos metaphor.)

    • @MrWite1
      @MrWite1 2 ปีที่แล้ว +6

      @@johnbarbuto5387 why so mad

    • @centerfield6339
      @centerfield6339 2 ปีที่แล้ว

      @@johnbarbuto5387 not courtesy of insurance companies; courtesy of the fact that healthcare needs to be paid for. State systems are also payer systems.

  • @svilen12345
    @svilen12345 5 ปีที่แล้ว +121

    How confident I feel in my current knowledge of neural networks:
    15:41 - *"Uh...I'm really more of a multiple choice kinda guy."*

    • @natchu96
      @natchu96 3 ปีที่แล้ว +2

      The neural networks themselves generally feel the same, so at least we won't be alone in that sentiment.
      Assuming thinking rocks and metal count as good companionship at any rate.

  • @darshita1270
    @darshita1270 2 ปีที่แล้ว +17

    Math courses in my college are basically trash compared to your videos , finally now I understand how math is being applied in computer science . Thank you so much for teaching in such an illustrative way .

  • @mohithkanala8228
    @mohithkanala8228 4 ปีที่แล้ว +3

    Thank you soo much.. 3Blue1Brown! For this video and this platform.. that has made learning of such complexity.. much more easier. Even for people, like beginners, who are not from the specific background. You guys, make such topics, so fun to learn. I wish you guys all the best for you channel, and wish you get much more appreciation for the beautiful change.. you bring to the minds of this generation.

  • @blunderbus2695
    @blunderbus2695 6 ปีที่แล้ว +767

    "It's actually just calculus."
    "Even worse!"
    i'm dead

    • @fitokay
      @fitokay 4 ปีที่แล้ว +5

      Actually, AI just lie to people of the world

    • @fitokay
      @fitokay 4 ปีที่แล้ว +1

      so far

    • @the.abhiram.r
      @the.abhiram.r 3 ปีที่แล้ว +2

      calculus is the easiest form of math

    • @ahmedezat1353
      @ahmedezat1353 2 ปีที่แล้ว

      @@the.abhiram.r
      I wish you are joking

  • @superj1e2z6
    @superj1e2z6 6 ปีที่แล้ว +769

    Watching 3b1b
    Step 3b. Drop Everything
    Step 1b. Watch religiously.

    • @jonasvanderschaaf
      @jonasvanderschaaf 6 ปีที่แล้ว +23

      oh the accuracy of this comment

    • @spiderforrest7816
      @spiderforrest7816 6 ปีที่แล้ว +6

      My god I relate

    • @Petch85
      @Petch85 6 ปีที่แล้ว +31

      for me it is.
      step 3b: make sure you are ready. you need to be 100% focused.
      step 1b: Watch it critically, be sure not to strengthen your miss believes. If it seems simpel and obvious I am probably misunderstanding it.

    • @fossilfighters101
      @fossilfighters101 6 ปีที่แล้ว +1

      +

    • @Cosine_Wave
      @Cosine_Wave 6 ปีที่แล้ว +9

      counting level: Parker

  • @VincentKun
    @VincentKun 2 ปีที่แล้ว +1

    I saw this video when i know nothing and i had a lots of intuitions, I'm rewatching after studied a lot more and I'm still learning a lot. You're a great teacher

  • @andinosa
    @andinosa 3 ปีที่แล้ว

    I want to thank you for the great effort that you put in simplifying such complex topics.
    Universities really lag behind in terms of making concepts crystal clear to student like you do. Evern MIT and other fancy universities which post their lectures online.
    Keep up the great work!!

  • @souvikroy7570
    @souvikroy7570 4 ปีที่แล้ว +79

    Hands down, I have never seen anyone explain mathematics so beautifully the way he does. Kudos!

  • @sweepy84
    @sweepy84 4 ปีที่แล้ว +328

    You sir deserve a noble, or an oscar...what an incredibly effective method of teaching. thank you so very much!!!
    NO! BAD COMPUTER! made me crack up! lol

    • @shakhaoathossain5032
      @shakhaoathossain5032 2 ปีที่แล้ว +5

      A balon di or too

    • @sohambhattacharjee951
      @sohambhattacharjee951 2 ปีที่แล้ว +1

      @@shakhaoathossain5032 XD good one.

    • @Phoenix-nh9kt
      @Phoenix-nh9kt ปีที่แล้ว +3

      @@shakhaoathossain5032 add a grammy in there too hahaha

    • @ChemEDan
      @ChemEDan ปีที่แล้ว

      The fact it was recorded digitally meant he said that to a real computer.😭
      AND SO DID YOU!!! 😠

  • @nityarajan9323
    @nityarajan9323 2 ปีที่แล้ว +1

    First of all, thank you infinitely for how beautifully and elegantly you have taught this, as with everything. Oftentimes my classes in uni bore me with factdumping, but you never fail to help me fall in love with maths and cs again. Second, i didn't know you did the KA course on multivariate calc! Yet another time you saved my life!!

  • @bishalthapaliya4069
    @bishalthapaliya4069 3 ปีที่แล้ว +13

    Probably, even a 5 year old would master deep learning when taught in this way. What a video man ! Awesomeeeeeeeee

  • @GAment_11
    @GAment_11 5 ปีที่แล้ว +8

    When I watch your videos, all I want to do is keep going. Thanks for motivating me, as well as others, with your amazing content. I really appreciate it.

  • @hikaruyoroi
    @hikaruyoroi 6 ปีที่แล้ว +33

    I love you so much. I'm taking multivariate calculus and I'm doing some neural network work right now, and none of my teachers have the passion nor the capability to teach as well as you. You help me keep my passion for learning alive

    • @ssa3101
      @ssa3101 5 ปีที่แล้ว

      Incompetence galore.

  • @skintaker1949
    @skintaker1949 4 ปีที่แล้ว +1906

    So uhhhh, did you just say that this was the "Hello World!" of neural networking.....

    • @GabrielCarvv
      @GabrielCarvv 4 ปีที่แล้ว +71

      ​@Winston Mcgee "p r e t t y m u c h i t"

    • @hassanakhtar7874
      @hassanakhtar7874 4 ปีที่แล้ว +153

      Yeah its scary to think but many programmers can easily code stuff like this as a simple AI tool. Just a bunch of linear algebra and statistics. It gets more interesting when you try to do more than minimizing a cost function and really make it 'learn'. Even after I say all this, it took me quite a bit to fully digest the "hello world" haha.

    • @rudigerbrightheart7304
      @rudigerbrightheart7304 4 ปีที่แล้ว +35

      Well, the data are the hello world, because it is the first image set that people take to test or learn about an algorithm.

    • @junkailiao
      @junkailiao 4 ปีที่แล้ว +18

      Because you don't need all those knowledge to build a network that can read digits. It's easy with Keras even my grandmother can do it

    • @domizianostingi9504
      @domizianostingi9504 4 ปีที่แล้ว +13

      Yes it is, because the dataset is veerry clean and CNN through Keras is very easy to implement, though you need to have huge background in math and code-writing (I'm a statistician so I have a little bit of both) :)

  • @gersonribeirogoulart9895
    @gersonribeirogoulart9895 ปีที่แล้ว

    When something is amazing, it will look like
    with your work. Even your bg voice is totally understandable, legit and direct

  • @ajnelson1431
    @ajnelson1431 6 ปีที่แล้ว +201

    3:40 "NO! Bad computer!"

    • @stydras3380
      @stydras3380 6 ปีที่แล้ว

      AJ Nelson Bad boy!

    • @NF30
      @NF30 6 ปีที่แล้ว +41

      I felt so sorry for the computer...

    • @johnchessant3012
      @johnchessant3012 6 ปีที่แล้ว +17

      "To say that more mathematically..."

    • @Shockszzbyyous
      @Shockszzbyyous 6 ปีที่แล้ว +1

      i heard eric cartman say it.

    • @Dom-nn1kg
      @Dom-nn1kg 6 ปีที่แล้ว

      +

  • @andrasiani
    @andrasiani 6 ปีที่แล้ว +26

    how can anyone dislike these videos? very detailed, accurate explanations and cool animations. Keep up the good work!!

  • @imtanuki4106
    @imtanuki4106 3 หลายเดือนก่อน

    Possibly one of the best mini-courses on ML anywhere. Clearly explained concepts, beautiful post-production. kudos

  • @danielamurphy8560
    @danielamurphy8560 ปีที่แล้ว +6

    I'm doing my Masters in applied Econ right now and we briefly went over Neural Networks in my advanced econometrics class. Some of the terminology was a bit different and I felt like I could understand it decently in office hours with my professor, but this was still a great resource to solidify my understanding of the concept. (Also we looked at the MNIST dataset in class too) :D

  • @genoir-itsmusicart9169
    @genoir-itsmusicart9169 6 ปีที่แล้ว +16

    This is mindblowingly interesting and extremely well explained. Thank you!

  • @abdenn
    @abdenn 6 ปีที่แล้ว +3

    The best explanation so far. Viewed over 50 videos. No one could explained in simple terms like you did. Thank you.

  • @rob651
    @rob651 ปีที่แล้ว

    I've watched many videos and done some reading on how neural networks work (learn), but I couldn't find a satisfactory explanation until I watched this video. Your examples, analogies, visuals... were just perfect. Thank you so much.

  • @sidiyababah3519
    @sidiyababah3519 3 ปีที่แล้ว

    Hi,
    I just started learning machine algorithms including neural networks and I find what you are publishing on this channel extremely useful, perfectly explained and quite helpful and I thank you for that from the bottom of my heart 🙏🏻
    Continue and may God bless you all

  • @shwetamayekar1863
    @shwetamayekar1863 5 ปีที่แล้ว +11

    Love the eye/ pi animations! :) Gets me smiling amidst all the complexities of Neural Networks 😲

  • @olesyabondar4826
    @olesyabondar4826 4 ปีที่แล้ว +6

    The graphics of this video is absolutely stunning! Thank you for your work ♡

  • @pebre79
    @pebre79 2 ปีที่แล้ว +1

    This series is a masterpiece of education. All the time and effort shows. Kudos!

  • @EmptyGlass99
    @EmptyGlass99 2 ปีที่แล้ว

    Easily one of the best videos I've seen. To explain such a complex concept so clearly is such an amazing skill. Thank you.

  • @mirlamontano6640
    @mirlamontano6640 5 ปีที่แล้ว +3

    amazing videos! I just discovered them, and I'm so hooked into this, all this algebra and math I've tried learning, starts to make sense to me.
    thanks, and keep on the good work!

  • @yuandongzhong
    @yuandongzhong 6 ปีที่แล้ว +3

    This series is the best neural network explanation I have ever encountered as a beginner!

  • @super266
    @super266 2 ปีที่แล้ว +2

    You are the best science teacher I have every seen. If anyone upstairs is serious about our education system they should use your videos as baseline for how to teach properly; you never use a term that wasn't clearly defined prior, you use analogies perfectly, and you tie new technical info back to the original concept thereby making sense how the new info fits together in the larger picture. If my high-school and college teachers were like you I would have done infinitely better at school.

  • @swealf-nonofficial
    @swealf-nonofficial 4 หลายเดือนก่อน +1

    You do such a great job at explaining complex topics in a "simple" way, great job

  • @cocopapagalul
    @cocopapagalul 6 ปีที่แล้ว +3

    Your videos are priceless. Thank you very much for your work and also a big thank you to your Patreon supporters.

  • @geregeorge1589
    @geregeorge1589 6 ปีที่แล้ว +261

    At the 16 minute mark, I got sucker punched. After having gone through this and the previous video on machine learning and just loving how an art student like myself is enjoying math such as this and feeling like I'm making some progress..... You tell me that this is all stuff that was figured out in the 80s and I'm like...... Oh Come On! Lol!

    • @apuapustaja2047
      @apuapustaja2047 5 ปีที่แล้ว +88

      Honestly, the 80s is actually very recent compared to other stuff. In math undergrad I was learning concepts from the 1800s lmao

    • @SimberLayek
      @SimberLayek 5 ปีที่แล้ว +16

      @@apuapustaja2047 yup! Math is older than all of us... it's our discoveries that are "new"~

    • @DiegoGonzalez-vn3qx
      @DiegoGonzalez-vn3qx 5 ปีที่แล้ว +44

      Honestly, don’t feel discouraged. General Relativity was formulated almost a century ago, but that doesn’t mean it is easier to understand.

    • @SimberLayek
      @SimberLayek 5 ปีที่แล้ว +2

      @Dark Aether some definitely could say that~

    • @DiegoGonzalez-vn3qx
      @DiegoGonzalez-vn3qx 5 ปีที่แล้ว +18

      @Dark Aether What do you even mean by that? Right now, we are living in a moment in which scientific knowledge is being acquired at the fastest rate we have ever seen. The number of active scientists right now, as you might expect, is the largest in history. Now, if you are talking about "raw" intelligence... well, I'm pretty sure evolving into creatures with a noticeable higher intelligence is going to take a long, long, long time.

  • @theflaminglionhotlionfox2140
    @theflaminglionhotlionfox2140 3 ปีที่แล้ว +103

    Me in part 1: Ah I think I'm starting to understand this whole thing.
    Me in part 2: Nevermind...

    • @saicharansigiri2964
      @saicharansigiri2964 3 ปีที่แล้ว +3

      excatly

    • @osmanyasar9602
      @osmanyasar9602 3 ปีที่แล้ว +6

      Once you learn more math it will be meaningful. I guess if you dont understand this video then something is missing in your calculus and/or linear algebra

  • @josephfrancis.j9576
    @josephfrancis.j9576 ปีที่แล้ว

    Thank you very much for your in sights, every second of your video is worth watching without skipping. Your way of explaining the concepts are very clear and easy to grasp, it starts a fire to learn more on the topic. Thank you once again for your efforts. Lookin forward to more of your contents

  • @HaouasLeDocteur
    @HaouasLeDocteur 6 ปีที่แล้ว +52

    WOO BEEN WAITING FOR THIS

  • @Maffoo
    @Maffoo 6 ปีที่แล้ว +4

    This series is fantastic and just the right level of being complex but understandable. Thanks!

  • @PunmasterSTP
    @PunmasterSTP ปีที่แล้ว +1

    This lecture series (and channel overall) is phenomenal! I tutor some students in various math and science subjects, and I have to start recommending these videos more.

  • @akashshrestha01
    @akashshrestha01 2 ปีที่แล้ว

    This is a gem to practically understand all the topics we learn at school and see the beauty of maths

  • @Tri_3st
    @Tri_3st 6 ปีที่แล้ว +4

    Hi 3B1B, as a technical physics student, beeing interested into this topic for quite a while now, and also enjoying your content for quit a while, i really wanna thank you for not only going into this topic particulary, but also for educating a relatively large audience with your informative videos and improving the interest into mathematical sciences for a lot of people including me, which is pretty important in my opinion! Keep it up!!

  • @leonelgouveia7573
    @leonelgouveia7573 6 ปีที่แล้ว +3

    Thank you for making these videos. They are truly amazing because anyone can learn something from them. All the way from people who are currently studying in this domain who have the mathematical toolbox to truly grasp every moment to people who are just curious and generally end up learning the intuitive basics ! Eagerly waiting for the next video :)

  • @flavio4923
    @flavio4923 2 ปีที่แล้ว +1

    I've never studied calculus but I'm learning how to use NN in Python and I'm reading a book about it and have watched some videos, so I have the rough idea of how it works. This video showed up on my recommended page while I was studying, and the explanation made some grey areas seem more clear now! And the crazy thing is a neural network helped me find this video to understand more about neural networks! and it doesn't know what it's doing, it's just minimizing some hidden cost function!

  • @somag6810
    @somag6810 4 หลายเดือนก่อน

    "Our growth mindset is reflected when we think always if we can do better!" You are always awesome. Thanks for all the informative videos that imparts a lot of fundamental knowledge to people like me.

  • @notbobbobby
    @notbobbobby 6 ปีที่แล้ว +3

    You wouldn't believe what my prof showed to us in class today! Only a day after this video was released. You're videos are incredible. Thank you! :)

  • @Iextrimator
    @Iextrimator 6 ปีที่แล้ว +4

    Absolutely love your videos! I'm trying to show this video to my friends who doesn't know English so well, and I decided to make subtitles. Hope you approve them, I really want to spread word about your work.

  • @frenchycman
    @frenchycman 5 หลายเดือนก่อน

    The visualizations are so great. Appreciate the thoughtfulness of these videos

  • @RacingMachine
    @RacingMachine ปีที่แล้ว +1

    Wow, what a brilliant job you do in your videos, i can barely imagine the hardwork and efforts necessary to summarize all this deep knowledge and create meaningful animations, congrats and pleas keep it up!!!!

  • @pseudo_goose
    @pseudo_goose 4 ปีที่แล้ว +24

    Some thoughts on the results:
    1. 14:01 The weights for the first layer _seem_ to be meaningless patterns when viewed individually, but combined, they do encode some kind of sophisticated pattern detection. That particular pattern detection isn't uniquely specified or constrained by this particular set of weights on the first layer; rather, there are infinitely many ways that the pattern detection scheme can be encoded in the weights of this single layer.
    These infinite other solutions can be thought of as the set of matrices that are row-equivalent to the 16x700ish matrix where each row is the set of weights for each of the neurons on this layer. You can rewrite each of the rows as a linear combination of the set of current rows, while possibly still preserving the behavior of the whole NN by performing a related transformation to the weights of the next layer. In this way, you can rewrite the patterns of the first layer to try and find an arrangement that tells you something about the reasoning. Row reduction in particular might produce interesting results!
    2. 15:10 I think I understand the reason why your NN produces a confident result - it's because it was never trained to understand what a number _doesn't_ look like. All of the training data, from what I can tell, is numbers associated with 100% confident outputs. You'd want to train it on illegible handwriting, noise, whatever you expect to feed it later, with a result vector that can be interpreted as 0% confidence, by having small equal weights, having all weights to zero, or maybe an additional neuron that the NN uses to report "no number".

    • @watchm4ker
      @watchm4ker ปีที่แล้ว +1

      2 is a painfully easy mistake to make, because it requires the human assembling the programming data to think outside the box.

    • @josboersema1352
      @josboersema1352 11 หลายเดือนก่อน

      Quite hard to read your comment, but it seems that we have the same idea: the neural network _is_ detecting smaller elements like "edges and loops" (as video author puts it), assuming those pictures 14:01 are of the actual results. The next layer then starts combining these elements, and it seems that if you stare at it long enough you can almost start guessing what it might be doing, like combining row 1 column 2 = strong + row 3 column 1 = strong + row 3 column 4 = strong + row 2 column 4 = weak + row 1 column 4 = weak, you might be going toward an 8 on those counts, and depending on some other combination of pattern strengths it might be a 6 or 9 if there is absence of signal upper/right or lower/left given by some of those patterns which are matched with the input. This is almost certainly not accurate as an example, but it seems to be the theme of how it works. 16:05 _"... picking up on edges and ... not at all what the network is doing."_ This statement in the video seems wrong.
      P.S. If this first part above is true, then the neural network might be capable of drawing a 5 (15:39). You just have to extract that answer in the way that it is in there, which is a bit more involved that following it's normal operation for which it is built. If you look into what combination of patterns from the first layer output, in what strengths, leads to a number (5 for example), than you could superimpose those patterns unto each other, and that would be what this neural network thinks is that number. It shouldn't be too hard to write a function to the already trained network, to draw this out.

  • @jamesluc007
    @jamesluc007 6 ปีที่แล้ว +9

    You explained in less than 4 minutes something that took me several days to understand from other sources. You are awesome!

  • @dsmogor
    @dsmogor 6 วันที่ผ่านมา

    I think what puts this material apart from the competition is the authors intuition of the focal points where the audience might loose the plot. Then he takes a patient and systematic turn to reiterate what have been learned so far to reinforce the basics to decrease the cognitive leap needed to grasp the next step. This ability is in my experience pretty unique.

  • @mtrifiro
    @mtrifiro 2 หลายเดือนก่อน

    I am so happy I discovered this today. I ignored all (well, most) of the math, and I still came away with a pretty solid understanding of how it works. Your explanations are ridiculously clear; you have a gift.

  • @jacoblund8289
    @jacoblund8289 6 ปีที่แล้ว +6

    I love how he has people like Desmos and Markus Persson supporting him on patreon

  • @nourddinesofiir3525
    @nourddinesofiir3525 6 ปีที่แล้ว +27

    Thanks for the part 2, I was waiting for it impatiently.

  • @muhammadbadar6089
    @muhammadbadar6089 2 ปีที่แล้ว

    man the animations are super helpful it's amazing how much effort was put into this

  • @Alex-to8tg
    @Alex-to8tg 2 ปีที่แล้ว

    You have the gift of teaching things that you know ! Not all teacher have this!

  • @williamwilliams1000
    @williamwilliams1000 6 ปีที่แล้ว +3

    Most brilliant educator I've ever had the please of seeing work. Thank you, you give me some semblance of hope for the human race.

  • @minerawesome28
    @minerawesome28 6 ปีที่แล้ว +6

    I was looking forward to this video all week.

  • @andysavage2361
    @andysavage2361 ปีที่แล้ว +1

    Your video is outstanding. I'm truly happy to have found your channel. I love the teaching method and constant visual examples.

  • @JT-si6bl
    @JT-si6bl 3 ปีที่แล้ว

    I'd like to applaud 3Blue1Brown for these chapters and the work. Bravo, you achieved eloquent concise examples and 'splainin. Bravo (y). I'm learning!

  • @angelrodriguez3247
    @angelrodriguez3247 6 ปีที่แล้ว +3

    You're an amazing explainer of amazing things and this particular thing you explained is particularly amazing!

  • @fpham8004
    @fpham8004 5 ปีที่แล้ว +2

    Wow, incredible! You really do know how to explain complex things the simple way.

  • @jamesleetrigg
    @jamesleetrigg ปีที่แล้ว

    Thanks sounds simple when you explain it but when I’ve looked at books all I see was formulas now I kind of understand what’s going on. Thanks you sir!

  • @rachadlakis1
    @rachadlakis1 ปีที่แล้ว

    Thank you for the extraordinary effort to present this in a simple and elegant and beautiful way. Thanks a lot.

  • @siddheshmisale3904
    @siddheshmisale3904 4 ปีที่แล้ว +20

    Would just take a moment here to appreciate the sheer brilliance of Grant on this series. I would not have reached a decent level of NN w/o these explanations and so would so many other people. Single best series on NN / Math out there in general.

  • @jabug_1144
    @jabug_1144 4 ปีที่แล้ว +7

    Once I graduate and start working, I’m gonna send you the money I owe you for watching all these videos. I’m doing BSEE for control systems so hopefully it works out.

  • @tobiasferrara936
    @tobiasferrara936 ปีที่แล้ว

    You explain things very well and very patiently. Thank you.

  • @wlidster
    @wlidster 3 ปีที่แล้ว

    The production effort of your work ---- amazing

  • @Treegrower
    @Treegrower 6 ปีที่แล้ว +148

    @ 3:39
    Wow... I didn't realize 3B1B likes to bully neural networks. That was ruthless.

    • @Brian.001
      @Brian.001 4 ปีที่แล้ว

      Yes, it's a jungle in there.

    • @oskarjung6738
      @oskarjung6738 4 ปีที่แล้ว

      @- RedBlazerFlame -
      ' Oversimplified' reference

    • @theshermantanker7043
      @theshermantanker7043 4 ปีที่แล้ว

      There's a training method called Reinforcement Learning where you literally torture the Network when it gets the wrong output lol

  • @shaylempert9994
    @shaylempert9994 6 ปีที่แล้ว +20

    Pause and ponder?! Every 10 seconds I stop for a minute of thinking! And on all of your videos!
    This time I had a time I thought for like half an hour.

  • @samurock100
    @samurock100 2 ปีที่แล้ว

    This is the best lesson on the basics of NN. Awesome content and delivery

  • @korandlasaipraneeth2816
    @korandlasaipraneeth2816 3 ปีที่แล้ว

    Love you for this sort of fully clearcut explaination with logic

  • @rubyjohn
    @rubyjohn 6 ปีที่แล้ว +5

    BEST VISUAL THERAPY IN MY LIFE

  • @mariafanmyself
    @mariafanmyself 4 ปีที่แล้ว +3

    Fantastic series! Doing a PhD in neural networks and I had never seen them so well explained! Thank you!

    • @Dman82499
      @Dman82499 4 ปีที่แล้ว +1

      I’m curious to know why you’d want a PhD in only neural networks and not some broader major that covers neural networks in it such as like a data analytics or CSE. I didn’t think they had that much content in them, unless you’re doing more biological imitations

  • @ilducedimas
    @ilducedimas ปีที่แล้ว

    Your work is much much appreciated my good sir. We can't thank you enough.

  • @santircastillo
    @santircastillo 2 ปีที่แล้ว

    This is the first video I found about AI that actually explains the how, instead of just dumbing everything down and leaving just as confused as before watching. Thank you!