Its reaallly nice but it would be a very nice addition to include variational autoencoders and Generative adversial networks as well :). Maybe they can be helpful to many struggling with class imbalance during classification
Hey Patrick a really informative and concise video! Thoroughly enjoyed it :DD Just a small correction at 12:51, you used the word dimension while explaining the Normalize transform, whereas the two attributes are just the mean and standard deviation of the resultant normalized data.
It should be noted that the performance difference between Linear and CNN as shown here comes from the chosen compression factor. Linear chose 12 Byte per image, CNN chose 256 Byte per image, where an original image is 784 Byte. So, the CNN code does not compress enough, less than PNG actually! You need two more linear layers to compress 64 down to 16 and then 4.
Hey Patrick I used your exact code to train the CNN based autoencoder but couldn't get it to converge without Batch Normalization, after adding BatchNorm2d after every ReLU it works fine, but without it, it doesn't, tried different values for lr from 1e-2 to 1e-5. I was training on MNIST dataset only. the loss becomes NaN or ranges between 0.10 to 0.09.
Great video! Could you provide the same walkthrough for a variational autoencoder? Or point point me to a good walkthrough on the theory and implementation of a variational autoencoder?
hi, I have a question: if we pass the image as input of the model, it will put the weights to zero and then the output will be exactly the same of the input image. So, why the image is given as input of the model? It doesn't make sense to me. Could yu explain this to me?
I don't understand a little bit the sintaxis. Why do you define the method 'forward' but never call it explicitly ? Maybe the line "recon = model(img)" is where you are using it, but I didn't know that it could be done like this. I would had written "recon = model.forward(img)", is it the same ?
Great animations my suggestion is to add in more animations not only in theory but also in the working of the code . Just my suggestion but great video thanks for Ur teaching.
you used recons and img as input for loss function, however if we want to train my model and test it we should use "recon" and "labels" as an input for loss function. but the labels are 3D, how we can do that?
Since, AutoEncoder is an unsupervised technique, so, recons and img are used as input to loss function. But, in semi-supervised or supervised methods, yo got labels, so we use them against the predicted values in loss function.
If you normalize the input image which is also the label, the values will be between -1 to +1 but your output since passed through sigmoid will be between 0 and 1. How will you decrease loss for pixels that are between -1 to 0 as your predictions will never be less than 0?
Let me know if you enjoyed the new animations in the beginning and want to see this more in the future :)
Hi Patrick,
Any plans to have a series on NLP state of the art models?
They're great
Love the animations and want more. Learning pytorch and following your tutorials.
For python 3.11+, pytorch 2.3+ change the dataiter.next() to next(dataiter)
Thanks man
big thanks to you. i cannot imagine how could i learn my dl course without your tutorial. Your work is the best in youtube so far!
This channel is so underrated.Please upload tutorials about Django
I love all your PyTorch videos, please do more :D
Thanks! I will :)
at 22:17 when calculating the shape of the conv output, it should be 128*128*1 => 64*64 * 16 and the rest should also be different accordingly
Its reaallly nice but it would be a very nice addition to include variational autoencoders and Generative adversial networks as well :). Maybe they can be helpful to many struggling with class imbalance during classification
Great suggestion!
Nice, was waiting for this :)
Hope you enjoyed it!
Yet another Pytorch video🔥
Can u help to implement pytorch with django
man you are fast :D
Can u upload Ur GUI chat bot code in GitHub I tried code along approach I was able to learn the concept but I got some few bugs . Can u upload it
I added the code here: github.com/python-engineer/python-fun
@@patloeber thanks
Hey Patrick a really informative and concise video! Thoroughly enjoyed it :DD Just a small correction at 12:51, you used the word dimension while explaining the Normalize transform, whereas the two attributes are just the mean and standard deviation of the resultant normalized data.
thanks for the hint!
Thank you so much for clear presentation of Autoencoder!
glad you like it!
It should be noted that the performance difference between Linear and CNN as shown here comes from the chosen compression factor. Linear chose 12 Byte per image, CNN chose 256 Byte per image, where an original image is 784 Byte. So, the CNN code does not compress enough, less than PNG actually! You need two more linear layers to compress 64 down to 16 and then 4.
Is it wrong to choose 4 feature maps in the last CNN layer, without adding the 2 linear layers?
This channel is really good, I learned PyTorch from this channel. Guys I assure you subscribe to this channel.
Thanks so much:) appreciate the nice words
Hey Patrick I used your exact code to train the CNN based autoencoder but couldn't get it to converge without Batch Normalization, after adding BatchNorm2d after every ReLU it works fine, but without it, it doesn't, tried different values for lr from 1e-2 to 1e-5. I was training on MNIST dataset only. the loss becomes NaN or ranges between 0.10 to 0.09.
which one is the link that explains how you make the pytorch classes please?
TopNotch explanation! Thx.
Thank you for your nice tutorials please do the same for a non-image data. I'm curious to see CNN auto-encoders with non-image data.
Thank you so much , you explained it really good.
Great video! Could you provide the same walkthrough for a variational autoencoder? Or point point me to a good walkthrough on the theory and implementation of a variational autoencoder?
But how do we leverage the low dimensional embedding given that they represent the PCA?
Please can you do for a network intrusion detection
Nice explanation
is it possible to use sentences as input data?
How you found that the image size was decreased from 28 to 14?
Very clear, thank you very much
hi, I have a question: if we pass the image as input of the model, it will put the weights to zero and then the output will be exactly the same of the input image. So, why the image is given as input of the model? It doesn't make sense to me. Could yu explain this to me?
Very cool. Thank you.
I don't understand a little bit the sintaxis. Why do you define the method 'forward' but never call it explicitly ? Maybe the line "recon = model(img)" is where you are using it, but I didn't know that it could be done like this. I would had written "recon = model.forward(img)", is it the same ?
Thanks ❤
I tried to rerun your code in the part of Autoencoder CNN, but then I got Loss = nan in each epoch. Do you know what is wrong?
for _ in range(100):
print("Thank you")
Great animations my suggestion is to add in more animations not only in theory but also in the working of the code . Just my suggestion but great video thanks for Ur teaching.
thanks! definitely a good idea
Do you have anything specific for face images, this won't work on face images.
Bro always waiting for your pyt🔥rch video ....🤙🏼🤙🏼🤙🏼
🙌
why there is no require_grad there?
you used recons and img as input for loss function, however if we want to train my model and test it we should use "recon" and "labels" as an input for loss function. but the labels are 3D, how we can do that?
Since, AutoEncoder is an unsupervised technique, so, recons and img are used as input to loss function.
But, in semi-supervised or supervised methods, yo got labels, so we use them against the predicted values in loss function.
Great work!! Thanks :))
If you normalize the input image which is also the label, the values will be between -1 to +1 but your output since passed through sigmoid will be between 0 and 1. How will you decrease loss for pixels that are between -1 to 0 as your predictions will never be less than 0?
you need to change sigmoid function at the end to tanh, to output will also be betweet -1 to 1
hello, nice videos you have. looking forward new videos on paper review and implementations.
Beautifully explained!! thank you!
Can you please show how to work with variational autoencoders and applications such as Image segmentation.
will look into this!
@@patloeber Thank you. Because for me it's hard to program it.
Can u make a video on DNCNN
thank you!
You're welcome!
module 'torch.nn' has no attribute 'ReLu'
Is anyone else getting the same error
its ReLU
@@theupsider Thanks a lot!
your tutorials are amazing! Thank you so much... Could you please make a video for nn.embedding as well?
I'll have a look at it
youre the best
thanks!
Could you please make GANs, VAE , transformers and BERT in pytorch
Very nice but need more
Am I the only one encountering nan values during training this ?
Next video is on GAN
dude my CNN autoencoder is doing worse than the linear autoencoder, lmao
me too
the ouputs show strange padding artefacts as well
"Jew-Pie-Tar notebook"