Implement BERT From Scratch - PyTorch

[@nhatnguyenviet8083]

Great, I hope that will be more "from Scratch" like this. Thanks very much!

[@uygarkurtai]

there will be! thank you :)

[@goktankurnaz]

Great video with great explanations!

[@uygarkurtai]

thank youu :)

[@islevkurt1532]

This is great for me👍🏻
Thank you🙏🏻

[@uygarkurtai]

Thank you :)

[@learntestenglish]

I want to learn about this topic but haven't been able find good resources. This is great for me. Thanks 🙏🙏

[@uygarkurtai]

thank you :)

[@user-wr4yl7tx3w]

this is really helpful. thanks!

[@uygarkurtai]

Thank you :)

[@Ece-kx6qk]

Çok faydalı bir video olmuş teşekkürler

[@thomasfathy4769]

you are genius brooo 🤩🤩🤩

[@uygarkurtai]

Thank you!

[@cartoonsondemand_]

please make a video on implementing gpt2 using pytorch

[@uygarkurtai]

I'll try to. Thank you for recommendation!

[@wilfredomartel7781]

😊 does this new implementation include flash attention?

[@uygarkurtai]

Hey, this is just the vanilla implementation. However I believe you can enable flash attention while loading the model. I suggest you check huggingface documentation.

[@TheAnistop007]

Hello ! great video thanks ! Is there a code where is model is trained ? like with loss and optimizer ?

[@uygarkurtai]

Hi thank you :) I only did the model for this. Maybe I do the full training in the future. If you want to try however I can give a tip. Train a BERT as usual (from huggingface maybe). Instead of importing it from a library, use this model. It should work.