I've been trying to study neural networks for the last couple of months, and this video series is being surprisingly helpful. Most other more 'formal' courses are much harder to keep up. Your videos are indeed somewhat more loose, but I could finally understand how to code a neural network, so thank you!
I'm creating a neural network, but I got stuck at some point of the backpropagation implementation and I ended up here watching your videos. You're so great at explaining! I'm glad everything looks so similar to my code so far. Hope my answer is in the next video, cause I've been watching those for 3 and a half years so far lol
Dear Dan, I worked through Tariq Rashid's book based on you recommendation(using Jupytor notebooks and python), now I'm working through this in Javascript. You are a great teacher!
I was late on my AI class yesterday because of work - Your video is the best video that explained about neural network, and the coding part actually made me understand it even better. Thank you! I am sure there are thousands of people that find your video helpful and very informative!
0:33 😂, you are the ideal teacher that we require but we do not deserve. Keep going sir, understanding the subject is so easy when theres teaching with a happy, excited face.
You are just the best. I have been taking courses on ML and DL, as I am trying to transition from ChemEng to Data Science, but didnt understand them fundamentally. Omg, you are making me feel part of computer science community, now that I understand it.
Merci Beaucoup !!!!!!! You're doing great. Everything is so simple with you aside. So much energy and will to teach. I can perfectly understand what you're explaining althougth I'm french and I don't even know what JavaScript is (I'm a python beginner). It's now 1 am and I've been watching you're neural network lessons since like 10 am (yes from the previous day). Thank you so much coding train !!!
This is the theme in my research in fermentation's reactions. That is awasome that my a ideia is not so away of the road by applying this tecnic in de excel/vba code. I hope that you keep going this nice work. Spread the knowledge is so amazing thing to do. Thanks for sharing!
I love this channel so much ♥️ I finally feel like I understand Math! I might actually watch all of this channel! (But with my pace it’ll probably take me years lol)
Side note: convinced my school department to let take an independent study as a tech elective and just follow your videos and code a neural network so thank you
The Coding Train, Northern Illinois University! Studying IE on my last semester!!!! Still needed a tech elective and my curriculum isn’t the most updated, (really manufacturing focused) so I wrote up this proposal saying I’m more interested in big data and machine learning so now I’m doing research with my professor and learning together!
This was a great tutorial! It was a great combination of your quirky teaching style and being quick and effective! Keep it up! :) Also, you should call the 'train' function 'codingTrain'
7:50 isn't that going to add the bias matrix to the weights matrix in a element by element fashion resulting in adding the bias to every element so later when you do the sum it will have the bias added in 3 times instead of 1?
I just want to let you know that I am one of your biggest rainbowed fan :D it's so interesting and I always wanted to learn such a thing. For this purpose, I watched the 3blue1brown videos and then TADA I saw you uploaded a video about neuralnetworks
Hi, Firstly this is an awesome channel and it is seriously useful. I am struggling with one thing at the moment, how does the matrix product function work when there is only one column for the inputs? As I see it, to get h1 it loops over the columns of the weights and the rows of the inputs....but then when it moves to h2, it moves to the next row of weights and loops over its columns....but there is no second column to loop over the rows for the inputs??
I have the opposite problem. i'm mainly a javascript programmer, so when I try to use processing or another language, i get somewhat confused of why i have to declare what the variable type is and what the function will return, and what type of variable is the input , because i want to be able to either import a instanceof the neuralnetwork or just the input nodes which is easy in javascript.
he wrote a function, toArray() { let arr = []; for (let i = 0; i < this.rows; i++) { for (let j = 0; j < this.cols; j++) { arr.push(this.data[i][j]); } } return arr; } that converted the output to an array. Also return output.toArray(); means that it is does the function toArray() and returns the output, and then print(output) in setup is the same as console.log(output);
Take in account easy changeability of the activation function (maybe even for each layer). After you are done you should try other (like ELU) activation function and compare Cost Plots :)
16:15 I don't know if you're still reading comments on this video, but you should make an iteration song for your soundboard for when you're typing "for" loops XD
Im trying to build a nural network and im a little confused. Dose each layer need a seperate bias input value? because i thought a bias input would be enough? Also why are you making entierly seperate bias matrix's? how do they fit in? maybe i need to watch this again...
I can imagine that you'll treat about decision criterion of the class with the output value (threshold?) in the following videos. But now, I'm asking myself: how we adjust of W in feedforward NN??
So when multiplying 2 matrix's, i thought the collumns of a had to be the same as the rows for b? So why does multiplying the input with the weights work? Because to my understanding, the weights are 2x2 and the inputs are 2x1 in the example you went through. I've been following along and get errors at this stage. Could you explain how the multiply function works? Im struggling to grasp it. thank you :)
Why did you randomize the whole bias_o (and bias_h) vector, when the bias for each weight matrix between two layers should be only with only one particular value?
i made a version of matrix library in ES5. everything works fine except when i call the feedforward it returns NaN . i cross checked and found if i call the matrix multiplication from the nn library it results a matrix of NaN. could you guide?
Hello, could you post a known output for a given input including all the intermediate weights and biases so I can verify my code is producing the same results, thank you.
This is an excellent idea. I'm not able to do this right now but you can see my code in progress here for the time being (drive.google.com/open?id=1BFim9qORvxFxUqiLrS-g-spbautXtADd). Github link forthcoming.
Using a sigmoid function for output 'Y' will restrict the output to range [0, 1]. Will it be better not to use the sigmoid for calculation of Y (i.e. the last layer)?
I would appreciate it if you started with a test. Then in the test you can specify your expectations (like the XOR function) and then backfill the code.
I'm a bit confussed about the Map() function. Isn't there already another function called the same that maps one interval to another? Won't this cause problems by having the same name? Sorry for my english, not my mother tongue.
It's all in these two places: github.com/CodingTrain/website/tree/master/Courses/natureofcode/10.18-toy_neural_network github.com/CodingTrain/Toy-Neural-Network-JS/
im having some issues, my output is always between 0.71... and 0.98... its never one and never less than 0.71.... i have tested with thousands of inputs
So the inputs are [1,0] the output is [0.59310]? I'm sorry but what's the idea of feedforward if you don't have a tarjet? the neural network reacts to that inputs and gives you a "whatever" number? it's unidireccional, there's not learning nothing
Excuse me ! I noticed that p5 is needs you to pay to get it, i used to work with p5 earlier and it was free to download So we can't get it for free anymore !? Please i'm waiting for your answer, Thank you!!
Hi TheWarLord, I could download it here without making a donation. After the file is downloaded, the p5js website indeed redirects me to a page with instructions as to how you can support the project, but the download is working here. Maybe there's something happening with your browser?
Hey ,Rodrigo Ângelo; yes yes i found the problem , i had an add blocker, it was blocking the installation as well, i just disabled it and it worked fine, thank you anyway !
This is very convenient when the author also points out the kind of lunatic we deal with when we hit pause on a regular basis. this is the real Dan, this picture right here. one should not be fooled by this calm, collect, well trimmed facade before us here. he is showing us the real Dan here, the one trying to escape, all the time, bleeps and bloops occasionally. i'm sure it would be quite a spectacle to let him loose in a data center while he thinks he's alone and it all comes out. Unintelligible bleeps and bloops, a roller chair, spinning in circles, madness, and no English . probably this face the entire time
wow, this is shockingly more obvious than brown blue. this i very interesting with dot products from a matrix with a couple nodes. freakishly obvious lol hopefully i'm not speaking to soon. the way that just "clicked", felt a needful impulse say something. super wow.
Once, there was also this video - Neural Network + XOR operator: th-cam.com/video/KkwX7FkLfug/w-d-xo.html I really liked it, will you make it public again?
Slice returns a shallow copy, so if you have a 2D array like here, you'll end up with references to the internal arrays, so changing them will alter the matrix data. Which I think you want to avoid.
I've been trying to study neural networks for the last couple of months, and this video series is being surprisingly helpful. Most other more 'formal' courses are much harder to keep up. Your videos are indeed somewhat more loose, but I could finally understand how to code a neural network, so thank you!
This is so nice to hear! It's really hard, I'm trying my best!
I'm creating a neural network, but I got stuck at some point of the backpropagation implementation and I ended up here watching your videos. You're so great at explaining! I'm glad everything looks so similar to my code so far. Hope my answer is in the next video, cause I've been watching those for 3 and a half years so far lol
Doing this in Vanilla Java while reading the book you mention ... so satisfying when it starts working!!!
How'd you get on with the java version?
Dear Dan, I worked through Tariq Rashid's book based on you recommendation(using Jupytor notebooks and python), now I'm working through this in Javascript. You are a great teacher!
I was late on my AI class yesterday because of work - Your video is the best video that explained about neural network, and the coding part actually made me understand it even better. Thank you! I am sure there are thousands of people that find your video helpful and very informative!
0:33 😂, you are the ideal teacher that we require but we do not deserve. Keep going sir, understanding the subject is so easy when theres teaching with a happy, excited face.
Will you plz tell me language name of algorithm and software in which it is written
You are just the best. I have been taking courses on ML and DL, as I am trying to transition from ChemEng to Data Science, but didnt understand them fundamentally. Omg, you are making me feel part of computer science community, now that I understand it.
How do you make everything so simple? you are great!
Merci Beaucoup !!!!!!! You're doing great. Everything is so simple with you aside. So much energy and will to teach. I can perfectly understand what you're explaining althougth I'm french and I don't even know what JavaScript is (I'm a python beginner). It's now 1 am and I've been watching you're neural network lessons since like 10 am (yes from the previous day). Thank you so much coding train !!!
Merci!
Presented in so interesting and simplified way ... God bless you
This is the theme in my research in fermentation's reactions. That is awasome that my a ideia is not so away of the road by applying this tecnic in de excel/vba code. I hope that you keep going this nice work. Spread the knowledge is so amazing thing to do. Thanks for sharing!
I love this channel so much ♥️
I finally feel like I understand Math! I might actually watch all of this channel! (But with my pace it’ll probably take me years lol)
Side note: convinced my school department to let take an independent study as a tech elective and just follow your videos and code a neural network so thank you
Cool! Where do you go to school?
The Coding Train, Northern Illinois University! Studying IE on my last semester!!!! Still needed a tech elective and my curriculum isn’t the most updated, (really manufacturing focused) so I wrote up this proposal saying I’m more interested in big data and machine learning so now I’m doing research with my professor and learning together!
This was a great tutorial! It was a great combination of your quirky teaching style and being quick and effective! Keep it up! :)
Also, you should call the 'train' function 'codingTrain'
Hah!
Man you are doing amazing work. I love your teaching style.
You’re the best lecturer ever
Cara, você é muito bom! Sigo seu canal em sempre aprendo mais! Obrigado!
This series is dope. Thank you for sharing.
7:50 isn't that going to add the bias matrix to the weights matrix in a element by element fashion resulting in adding the bias to every element so later when you do the sum it will have the bias added in 3 times instead of 1?
i was gonna ask the same
as always perfect, keep on going Prof.
seriously I'm so sad I can only like this once!!
I love you Dan I love you Man...You are Awesome!!
]
Excellent explanation
I watch through all the series
this guy is crazy and i like it :)
I just want to let you know that I am one of your biggest rainbowed fan :D it's so interesting and I always wanted to learn such a thing. For this purpose, I watched the 3blue1brown videos and then TADA I saw you uploaded a video about neuralnetworks
Thank you!
Amazing! After spending like 9 hours researching this stuff, your videos are the most helpful ones!
This "hello" in first second looks like "I'm sorry this series is so long already" :D
Amazing , Thanks a lot
Hi, Firstly this is an awesome channel and it is seriously useful. I am struggling with one thing at the moment, how does the matrix product function work when there is only one column for the inputs? As I see it, to get h1 it loops over the columns of the weights and the rows of the inputs....but then when it moves to h2, it moves to the next row of weights and loops over its columns....but there is no second column to loop over the rows for the inputs??
Amazing series!
YOU ARE GREAT.....FANTASTIC
As a primarily C-based language developer, the lack of datatype declarations for variables in JavaScript always makes me a little anxious.
I have the opposite problem. i'm mainly a javascript programmer, so when I try to use processing or another language, i get somewhat confused of why i have to declare what the variable type is and what the function will return, and what type of variable is the input , because i want to be able to either import a instanceof the neuralnetwork or just the input nodes which is easy in javascript.
Cannot thank you enough Dan.
You can make an array of weights and biases, and use that array system in the algorithms.
Can you or someone else please show me where you code the nn.js file?
in this video? i dont understand your question
@@samuelgunter where is the source code of this video?
I love your work and jesture keep it up man
How do you get paid for your work?
glad to see you got your shit together
Thank you , you owe me
16:39 why output isn’t a matrix ? When did it become just a number
he wrote a function,
toArray() {
let arr = [];
for (let i = 0; i < this.rows; i++) {
for (let j = 0; j < this.cols; j++) {
arr.push(this.data[i][j]);
}
}
return arr;
}
that converted the output to an array. Also
return output.toArray();
means that it is does the function toArray() and returns the output, and then print(output) in setup is the same as console.log(output);
Thanks for Excellent Tutorials!.I want to know how many neuron or nodes should be inside of hidden layer?? waiting for your ans... Thanks
This video is unlisted. You might want to publish it and put it in the playlist..
I think he normally uploads a whole live-session's-worth of episodes in one go, but only publishes them one-per-day?
no, i dont think so, I think he forgot to publish this one.
Is that better now?
Phil Boswell hahaha yeah
How did you find this video 3 days ago (at the moment of this comment's writing) when it was unlisted?
Take in account easy changeability of the activation function (maybe even for each layer). After you are done you should try other (like ELU) activation function and compare Cost Plots :)
16:15 I don't know if you're still reading comments on this video, but you should make an iteration song for your soundboard for when you're typing "for" loops XD
does the bias weight have to be random too?
I love this video!
You are amazing
So the Perceptron class, that you did before in one of your previous videos, you will not use it here?
I'm starting over in order to use the matrix math, but the same ideas should map?
thanks for videos.It is verry super)
Awesome.
Awesome!!
Could you make a video where you create a neural network using R programming?
Im trying to build a nural network and im a little confused. Dose each layer need a seperate bias input value? because i thought a bias input would be enough?
Also why are you making entierly seperate bias matrix's? how do they fit in? maybe i need to watch this again...
Is the complete code for this video available anywhere?
I found this, which appears to be close enough: github.com/CodingTrain/website/tree/master/Courses/natureofcode/10.18-toy_neural_network/lib
I can imagine that you'll treat about decision criterion of the class with the output value (threshold?) in the following videos. But now, I'm asking myself: how we adjust of W in feedforward NN??
Did i miss something? Shouldn't inputs be summed after multiplied? Where is the sum step?
Oh, forgot that he is doing dot product inside multiply method.
So when multiplying 2 matrix's, i thought the collumns of a had to be the same as the rows for b? So why does multiplying the input with the weights work? Because to my understanding, the weights are 2x2 and the inputs are 2x1 in the example you went through. I've been following along and get errors at this stage. Could you explain how the multiply function works? Im struggling to grasp it. thank you :)
I recommend you join our Discord where you can ask code related questions to the community : discord.gg/hPuGy2g
- The Coding Train Team
Why did you randomize the whole bias_o (and bias_h) vector, when the bias for each weight matrix between two layers should be only with only one particular value?
I've converted the code from Github into Java, but the 'dot' function does not work
i made a version of matrix library in ES5. everything works fine except when i call the feedforward it returns NaN . i cross checked and found if i call the matrix multiplication from the nn library it results a matrix of NaN. could you guide?
I have doubts about bias, its value is 1, or is it random? Great video!
Bias -1 is an input too of that neuron, so is affected by a weight too, W*X-Bias*(weight)=0, that weight change when the neuron is learnig
Hello, could you post a known output for a given input including all the intermediate weights and biases so I can verify my code is producing the same results, thank you.
This is an excellent idea. I'm not able to do this right now but you can see my code in progress here for the time being (drive.google.com/open?id=1BFim9qORvxFxUqiLrS-g-spbautXtADd). Github link forthcoming.
Using a sigmoid function for output 'Y' will restrict the output to range [0, 1]. Will it be better not to use the sigmoid for calculation of Y (i.e. the last layer)?
👍
I might have an idea for future videos. Train weights using Genetic algorithms since you've done some work about GA in the past. Wha do you think?
Yes I am planning to do this for sure!!
unnikked I have two videos where I explore this exact concept
NEAT 😂
unnikked The authors of the paper came up with that 😌
I know, pun intended :)
I would appreciate it if you started with a test. Then in the test you can specify your expectations (like the XOR function) and then backfill the code.
does anyone know why you multiply weight* inputs instead of inputs* weights?
I'm a bit confussed about the Map() function. Isn't there already another function called the same that maps one interval to another? Won't this cause problems by having the same name? Sorry for my english, not my mother tongue.
Will you plz tell me language of this algorithm
In which language this program is written... And software.?
Java script so far. (ES6 version)
16:07 Just kind of sums up Dan doesn't it :D
it would be nice that fromarray function be a constructor
What language are you using?
This is all JavaScript.
thank youu!!!!!
Thank you for that great work. But i can't find the source code.
It's all in these two places:
github.com/CodingTrain/website/tree/master/Courses/natureofcode/10.18-toy_neural_network
github.com/CodingTrain/Toy-Neural-Network-JS/
Great video, bro. Hold back on the Adderall tho. Still, I learned a lot. Thanks.
You can also normalize your inputs for better results.
After all that I feel like I should get a 0.5931076632680795 tattoo haha what the heck.
im having some issues, my output is always between 0.71... and 0.98... its never one and never less than 0.71.... i have tested with thousands of inputs
Will you plz tell me language using in algorithm and software name in which algorithm written
18:07 loooooooool
So the inputs are [1,0] the output is [0.59310]? I'm sorry but what's the idea of feedforward if you don't have a tarjet? the neural network reacts to that inputs and gives you a "whatever" number? it's unidireccional, there's not learning nothing
this is just doing the math for the outputs, back propagation is where it trains to get the desired output
I mean seriously, who the hell disliked???
Excuse me ! I noticed that p5 is needs you to pay to get it, i used to work with p5 earlier and it was free to download
So we can't get it for free anymore !? Please i'm waiting for your answer, Thank you!!
p5 is a free, open-source JavaScript library supported by donations. See: p5js.org. For more processingfoundation.org
The Coding Train but jt asks me every time to make a donation befor I could install it ,! Please can you help !?
Hi TheWarLord, I could download it here without making a donation. After the file is downloaded, the p5js website indeed redirects me to a page with instructions as to how you can support the project, but the download is working here. Maybe there's something happening with your browser?
Hey ,Rodrigo Ângelo; yes yes i found the problem , i had an add blocker, it was blocking the installation as well, i just disabled it and it worked fine, thank you anyway !
I thing we need weights for biases too, don't we?
how to do this in python??
In which language this code is written?
16:14
friendly! 😄
When do I get to be with someone ?
I love you
This is very convenient when the author also points out the kind of lunatic we deal with when we hit pause on a regular basis. this is the real Dan, this picture right here. one should not be fooled by this calm, collect, well trimmed facade before us here. he is showing us the real Dan here, the one trying to escape, all the time, bleeps and bloops occasionally. i'm sure it would be quite a spectacle to let him loose in a data center while he thinks he's alone and it all comes out. Unintelligible bleeps and bloops, a roller chair, spinning in circles, madness, and no English . probably this face the entire time
Adaboost Ensemble Learning pls sir.. :)
wow, this is shockingly more obvious than brown blue. this i very interesting with dot products from a matrix with a couple nodes. freakishly obvious lol hopefully i'm not speaking to soon. the way that just "clicked", felt a needful impulse say something. super wow.
Om shanti k
Once, there was also this video - Neural Network + XOR operator:
th-cam.com/video/KkwX7FkLfug/w-d-xo.html
I really liked it, will you make it public again?
Depois falam que professor brasileiro que parece meio louco olha esse do usa ai brincadeira kķkkkkkkkkkkkk.
doo doo doo
You really hate simplicity...
toArray() { return this.data.slice() }
Slice returns a shallow copy, so if you have a 2D array like here, you'll end up with references to the internal arrays, so changing them will alter the matrix data. Which I think you want to avoid.
Thank you, I didn't know that
Hi Goofy
Please marry me
When do I get to be with someone ?
Else