Tokenization in Natural Language Processing

[@skillbits5941]

Make sure to see this video on stems and lemmas!
th-cam.com/video/7klD678xYxE/w-d-xo.html

[@skillbits5941]

timestamps:
0:10 what tokenization is
0:30 why tokenization is important
0:44 how to apply tokenization in python
0:50 NLTK library
1:16 sentence tokenization
1:28 word tokenization

[@MaybelleThrone]

Thanks for the interesting content! 😍 Just a small off-topic question: 😅 I only have these words 🤔. (behave today finger ski upon boy assault summer exhaust beauty stereo over). I don't know what they are. What should I do with them? 🤷‍♀️