I like the low level stuff. I attempt to read these papers and your grasp and explanations give me confidence that I can decode them too. Almost always they’re built on top of other work. I liked when you distilled that history out in stylegan session.
@@TheAIEpiphany . I once came across some python code I cloned on GitHub that could take a PDF and create multi quiz questions based off any content. Maybe I could help you one day and have you nut out the answer. You remember that sort of stuff in physics class where the teacher makes things clear eliminating nonsense and elucidating correct answer.
keep it up with the detailed explanations. For those who are interested in the low level stuff can just skip the detailed parts, win for both. Thank you.
Great work and explanation. Probably you have noticed but VQVAE is a bit rough to train since it’s not differentiable. In parallel there is GumbleSoft which is differentiable and therefore easier to train, wav2vec v2 use that. It might be interesting to cover that next :) cheers
15:56 I thought it is arbitrary at first but later realized it is just balancing between loss terms, namely L_{rec} and L_{GAN}. If gradients of L_{GAN} is big, then less weight on L_{GAN} and vice versa
Thanks again, a masterpiece like the VQ-VAE one. But it would be great if you also add the code part like in the VQ-VAE part, perhaps even more detailed one. Thanks aloooot again!
i feel like you lost me on the semantic segmentation --> image generation step; you say that the semantic token vector from the semantic VQGAN is appended to the front of the CLS token and then the token vector of...the output VQGAN? and then this 2N+1 length vector is input, and the output is a length N vector? how is this possible, aren't transformers necessarily the same dimensional input and output?
It's always interesting to me how a bit of constrained resources can produce very intelligent, next-gen results instead of just pumping up the model with weights and using crazy amounts of compute 😂
The adversarial loss - i think the explanation is wrong You said the discriminator tries to maximize it, however, you have just shown that it tries to minimize is (the term becomes 0 if D(x) is 1 and D(\hatX) is 0). So the discriminator tries to minimize it (and because its a loss function it makes sense), and the generator tries to do the opposite, maximize it, to fool the discriminator. So I think you mis-labeled the objective: L_GAN we try to minimize (minimize loss) in order to train the discriminator.
Answer : I find in depth explanation very very useful. 🙂 you could also explain codes here. But great work. Thanks. 👍🏻🙂 Could you please also review paper „A Disentangling Invertible Interpretation Network for Explaining Latent Representations“ from same author. It would be great. Thank you. 🙂
Thanks for your awesome explain about this paper, I want to ask a question, How does the transformer use the characteristics of the transformer to achieve autoregressive prediction?
What do you get combining DeepMind's VQ-VAE, GANs, perceptual loss, and OpenAI's GPT-2, and CLIP? Well, I dunno but the results are awesome haha!
TH-cam should have recommended me this channel a year ago. What a quality content! Keep it up :D
Hahah misterious are the paths of the YT algorithm. 😅
I like the low level stuff. I attempt to read these papers and your grasp and explanations give me confidence that I can decode them too. Almost always they’re built on top of other work. I liked when you distilled that history out in stylegan session.
Thanks, It's fairly a complex tradeoff to decide when to stop digging into more nitty-gritty details. 😅 I am still figuring it out
@@TheAIEpiphany . I once came across some python code I cloned on GitHub that could take a PDF and create multi quiz questions based off any content. Maybe I could help you one day and have you nut out the answer. You remember that sort of stuff in physics class where the teacher makes things clear eliminating nonsense and elucidating correct answer.
In-depth explanations are game changer. Keep doing the same. Great work!!
The in-depth explanation is pretty useful. Thank you so much.
keep it up with the detailed explanations. For those who are interested in the low level stuff can just skip the detailed parts, win for both. Thank you.
I think you cover both the high-level explanation and details fairly well :) Keep it up, please.
Great work and explanation. Probably you have noticed but VQVAE is a bit rough to train since it’s not differentiable. In parallel there is GumbleSoft which is differentiable and therefore easier to train, wav2vec v2 use that. It might be interesting to cover that next :) cheers
15:56 I thought it is arbitrary at first but later realized it is just balancing between loss terms, namely L_{rec} and L_{GAN}. If gradients of L_{GAN} is big, then less weight on L_{GAN} and vice versa
So great! Love the explanation of the loss functions.
this is a master pieceee.
Great Job! The condition part is super useful. The paper is confusing there.
U are on fire 🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥.
This video was much needed for me, Thank you so much.
I am just getting started 😂 awesome!
Thanks again, a masterpiece like the VQ-VAE one. But it would be great if you also add the code part like in the VQ-VAE part, perhaps even more detailed one.
Thanks aloooot again!
i feel like you lost me on the semantic segmentation --> image generation step; you say that the semantic token vector from the semantic VQGAN is appended to the front of the CLS token and then the token vector of...the output VQGAN? and then this 2N+1 length vector is input, and the output is a length N vector? how is this possible, aren't transformers necessarily the same dimensional input and output?
It's always interesting to me how a bit of constrained resources can produce very intelligent, next-gen results instead of just pumping up the model with weights and using crazy amounts of compute 😂
thank you sooo much
You're welcome man!
how did you do it can you share with me , thank you
The adversarial loss - i think the explanation is wrong
You said the discriminator tries to maximize it, however, you have just shown that it tries to minimize is (the term becomes 0 if D(x) is 1 and D(\hatX) is 0). So the discriminator tries to minimize it (and because its a loss function it makes sense), and the generator tries to do the opposite, maximize it, to fool the discriminator.
So I think you mis-labeled the objective: L_GAN we try to minimize (minimize loss) in order to train the discriminator.
Answer : I find in depth explanation very very useful. 🙂 you could also explain codes here. But great work. Thanks. 👍🏻🙂
Could you please also review paper „A Disentangling Invertible Interpretation Network for Explaining Latent Representations“ from same author. It would be great. Thank you. 🙂
Thanks for your awesome explain about this paper, I want to ask a question, How does the transformer use the characteristics of the transformer to achieve autoregressive prediction?
Some segments in the video are stamped not adjacent to each other
What exactly do you mean by that?
Rodriguez Barbara Martinez Jose Johnson Larry
Robinson Sandra Hall Melissa Lee David
Robinson Cynthia Young Gary Thompson Kevin
how did you do it can you share with me , thank you