Thanks for this, I just recently completed the paper reading and I will go towards implementation tomorrow and your video will give it a boost. Thanks again.
@@yizhouchen652 Thank you! Feel free to create an issue on GitHub github.com/jankrepl/mildlyoverfitted/issues and I will try to help there:) Ideally, include the code that lead to the problem.
Many thanks for this great content! I'm a grad student in computational neuroscience and machine learning, and this was very helpful for taking a deep dive into DINO. I really enjoy your style of writing and explaining code. I'd find it helpful at times if you take a few more seconds when you put the modules that you implemented together. For example, I stopped the video quite a few times and jumped to earlier sections to understand what a particular projection or module was doing. Or what dimensionality it expects/returns. Most of the time you provide this as a comment, but I found myself to stop and playback the video a couple of times. I've watched the ViT video right before, which I also very much enjoyed, but similarly I was jumping back and forth in the video quite a bit to remind myself what was going on (especially when you implement the forward methods). In most videos that cover pytorch/ML stuff, I find myself skipping forward because the pace is too slow. in your videos it's quite the opposite. But that might be just me. Another comment that I had about this particular video: I found the training script with the arg-parser a bit hard to read on the first go. And the point where you instantiated the student/teacher networks is almost at the very end of the video, and I carried some confusion on how exactly they look like, up towards the end of the video. But those are all minor points. I really appreciate this great content. Hope you'll keep it up. All of your videos are on my watchlist for the next few days. If I may suggest some topics of interest: I'd find contrastive learning approaches interesting (CLIP and the like), or generative models (from AEs to VAEs to GANs to all the fancy modern GANs), and interpretability/feature visualization techniques. Thanks, and greetings from Germany!
First of all, thank you for such a long comment! Really appreciate it:) I absolutely agree that my videos are not necessarily easy to follow and require the viewer to be very alert and ideally even familiar with the paper already. When I record them I know the code by heart almost and I probably see it very differently than someone who sees it for the first time. I am trying to improve with each new video (e.g. I started creating custom diagrams in recent videos) incorporating feedback I got from the viewers. So again, thank you for your message! IMO the best way to watch my videos is to 1) Read the paper 2) Go through the official implementation and try to run it 3) Check out my implementation (I always release it on GitHub) + watch the video! Thank you for the great suggestions:)!!
This is the best video i have ever seen about DINO i think , for me , the most impressive part is the implementation part. Many guys may explain those code clearly but they don't teach people how to inplement . Anyway i hope there will be more explanation with inplementation Thank you a lot
Thank you very much! I was exactly looking for this!! I subscribed to your channel immediately! I just have one question, is it possible to train a GPT-2 model on my notebook that has a Nvidia gtx 965M? I would love to train my own GPT-2 in my native language (portuguese)
Glad you liked the video! Regarding your question, just give it a try and you will see how things go. It really depends on how big your dataset is. I would definitely recommend checking this for some existing finetuning scripts: github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling
I really like this video and the channel, impressive, thank you, can I ask you if I have a different dataset containing about 100k images which is better to use this pertained model or build the same model from scratch trained on my own dataset? please answer me I have limited access to online GPU thanks again
Thank you. IMO just try to use publicly available pretrained models first. And if you find their performance to be not great for your use case only then go for training from scratch.
![[Full] 4 ต่อ 4 Celebrity EP.922 | 10 พ.ย. 67 | one31](http://i.ytimg.com/vi/zmV5HRIxIBI/mqdefault.jpg)
Thank you for sharing this, I was actually looking for results of DINO on smaller compute/data so this is so helpful
this is extremely useful; not just for dino implementation but for deeper underatanding of pytorch.
Happy to hear that:)
Thanks for this, I just recently completed the paper reading and I will go towards implementation tomorrow and your video will give it a boost. Thanks again.
Hope you will find it helpful:)
This is good stuff. You the real MVP.
Hehe:) Thank you!
You are the teacher model, and I'm student model in dino.
Thanks you♥︎
hahaha:) Thank you!
Amazing video, so underrated channel.
Thanks for the kind words!
See you again! I really appreciate your time. It certainly boosts my research progress!!!!!!!
Btw, when I try to run it, it say: RuntimeError: Unknown model (vit_deit_small_patch16_224)
@@yizhouchen652 Thank you! Feel free to create an issue on GitHub github.com/jankrepl/mildlyoverfitted/issues and I will try to help there:) Ideally, include the code that lead to the problem.
Many thanks for this great content! I'm a grad student in computational neuroscience and machine learning, and this was very helpful for taking a deep dive into DINO. I really enjoy your style of writing and explaining code. I'd find it helpful at times if you take a few more seconds when you put the modules that you implemented together. For example, I stopped the video quite a few times and jumped to earlier sections to understand what a particular projection or module was doing. Or what dimensionality it expects/returns. Most of the time you provide this as a comment, but I found myself to stop and playback the video a couple of times. I've watched the ViT video right before, which I also very much enjoyed, but similarly I was jumping back and forth in the video quite a bit to remind myself what was going on (especially when you implement the forward methods). In most videos that cover pytorch/ML stuff, I find myself skipping forward because the pace is too slow. in your videos it's quite the opposite. But that might be just me.
Another comment that I had about this particular video: I found the training script with the arg-parser a bit hard to read on the first go. And the point where you instantiated the student/teacher networks is almost at the very end of the video, and I carried some confusion on how exactly they look like, up towards the end of the video.
But those are all minor points. I really appreciate this great content. Hope you'll keep it up. All of your videos are on my watchlist for the next few days.
If I may suggest some topics of interest: I'd find contrastive learning approaches interesting (CLIP and the like), or generative models (from AEs to VAEs to GANs to all the fancy modern GANs), and interpretability/feature visualization techniques.
Thanks, and greetings from Germany!
First of all, thank you for such a long comment! Really appreciate it:)
I absolutely agree that my videos are not necessarily easy to follow and require the viewer to be very alert and ideally even familiar with the paper already. When I record them I know the code by heart almost and I probably see it very differently than someone who sees it for the first time. I am trying to improve with each new video (e.g. I started creating custom diagrams in recent videos) incorporating feedback I got from the viewers. So again, thank you for your message!
IMO the best way to watch my videos is to
1) Read the paper
2) Go through the official implementation and try to run it
3) Check out my implementation (I always release it on GitHub) + watch the video!
Thank you for the great suggestions:)!!
This is the best video i have ever seen about DINO i think , for me , the most impressive part is the implementation part. Many guys may explain those code clearly but they don't teach people how to inplement . Anyway i hope there will be more explanation with inplementation Thank you a lot
Glad you liked it!
Thank you for sharing
Cool content. Really enjoyed it.
Glad to hear that!
Nice video, keep going man :)
Thank you very much! I was exactly looking for this!! I subscribed to your channel immediately!
I just have one question, is it possible to train a GPT-2 model on my notebook that has a Nvidia gtx 965M? I would love to train my own GPT-2 in my native language (portuguese)
Glad you liked the video! Regarding your question, just give it a try and you will see how things go. It really depends on how big your dataset is. I would definitely recommend checking this for some existing finetuning scripts: github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling
I really like this video and the channel, impressive, thank you, can I ask you if I have a different dataset containing about 100k images which is better to use this pertained model or build the same model from scratch trained on my own dataset? please answer me I have limited access to online GPU
thanks again
Thank you. IMO just try to use publicly available pretrained models first. And if you find their performance to be not great for your use case only then go for training from scratch.
is there any way of re SSL a pretrained DINO?
thank you for the videos...i am new to this... i want to know what is the software to be used and how many gpu's required to run.... please reply
may I know your font and color scheme of your vim? Thanks
I use this plugin:
github.com/morhetz/gruvbox
yoo can someone help me out with this, have to make a project on this. I am really new to this and have no idea whats going on
Sorry to hear that and good luck!
is this code for classification images !!
Not sure what you mean, but DINO is a self-supervised algorithm:) Not a supervised one (e.g. classification)
@@mildlyoverfitted i want use dino for classification task how!!