It's a bit unclear to me how the Mamba architecture works recurrently when looking at the architecture in 5.30. What is the input here? the whole sequence or individual tokens? Surely it'd have to be the whole sequence for Mamba to build a representation recurrently. But then it seems strange to have a skip connection on the whole sequence. I think I've missed something.
Hi, thanks for your comment. I mentioned that delta discretizes the input as the word sequence into tokens, ..., and the fact that, at every step of the hidden state update, it takes into account the previous hidden state and the 'current input word'. I try to make an update on this, maybe reviewing the entire article if I can. Please do let me know if you are interested in any particular topic for a video.
Great video! Keep making them!
Thanks! Will do!
Thanks for this video, keep up the good work.
Thanks for watching!
Very informative looking forward for the in depth video on vision mamba or vmamba
Thanks for watching and for your suggestion. Stay tuned :)
It's a bit unclear to me how the Mamba architecture works recurrently when looking at the architecture in 5.30. What is the input here? the whole sequence or individual tokens? Surely it'd have to be the whole sequence for Mamba to build a representation recurrently. But then it seems strange to have a skip connection on the whole sequence. I think I've missed something.
Hi, thanks for your comment. I mentioned that delta discretizes the input as the word sequence into tokens, ..., and the fact that, at every step of the hidden state update, it takes into account the previous hidden state and the 'current input word'. I try to make an update on this, maybe reviewing the entire article if I can. Please do let me know if you are interested in any particular topic for a video.
I liked your style and your funny personality
Thanks for watching, I love your comment too :)
a great video. next video, may be you can explain the details about selective mechanisms in code
Great suggestion! Thanks for watching :)