Focal Transformer: Focal Self-attention for Local-Global Interactions in Vision Transformers

แชร์
ฝัง
  • เผยแพร่เมื่อ 28 ก.ย. 2024

ความคิดเห็น • 21

  • @TheAIEpiphany
    @TheAIEpiphany  3 ปีที่แล้ว +5

    Transformers are all you need

  • @sakuragi9570
    @sakuragi9570 3 ปีที่แล้ว +1

    Swin and Focal reminds me of local feature extraction (keypoints and descriptor) like SIFT, SURF, or ORB. This research is fascinating! Great video and explanation

    • @TheAIEpiphany
      @TheAIEpiphany  3 ปีที่แล้ว +1

      Well in a way yes there is a connection. Those are super shallow extractors though, thanks!

    • @sakuragi9570
      @sakuragi9570 3 ปีที่แล้ว

      @@TheAIEpiphany Hmm I think we should start reminiscing early extractor algorithm and try to transform it into a layer of neural network. What do you think?

  • @varunsai9736
    @varunsai9736 3 ปีที่แล้ว +1

    Love your videos amazing content
    Can you do video on Coberl and pondernet if possible plz

  • @Zantorc
    @Zantorc 3 ปีที่แล้ว +1

    Good explanation thanks. Be careful with positioning though - you're sometimes clipping the right hand part of the paper.

    • @TheAIEpiphany
      @TheAIEpiphany  3 ปีที่แล้ว +1

      Thanks! Yup, thanks, I'm experimenting I screwed it up this time haha - hopefully it didn't ruin the content/flow of information

  • @444haluk
    @444haluk 3 ปีที่แล้ว +4

    Like every phone site in 2014 obsessing on iPhone: "Is this new person the new Yannic-killer?" :D

    • @TheAIEpiphany
      @TheAIEpiphany  3 ปีที่แล้ว +2

      😂 lol. Nah I don't want to compare to anybody there are many good guys/gals out there - I am just doing my fair share.

    • @bertchristiaens6355
      @bertchristiaens6355 3 ปีที่แล้ว +2

      I watch both of you, it’s nice to get interesting insights from different persons and sources :D

    • @TheAIEpiphany
      @TheAIEpiphany  3 ปีที่แล้ว

      @@bertchristiaens6355 That makes a lot of sense haha.

  • @НиколайНовичков-е1э
    @НиколайНовичков-е1э 2 ปีที่แล้ว

    Thank you!

  • @MrMIB983
    @MrMIB983 3 ปีที่แล้ว +3

    StyleGAN2-ADA please

  • @rafaelortizferegrino8894
    @rafaelortizferegrino8894 2 ปีที่แล้ว

    Hi, I'm working with this model, and I notice that in the Tiny, small, and base model, the stages 2 and 3 for the S[w, r] for l= 1 doesn't make sense if we consider de shape of 28x28 and 14x14 respectively. Do U know what's going on there?

  • @ameynaik2743
    @ameynaik2743 2 ปีที่แล้ว

    Is it better than swin transformer v2?

  • @johnpope1473
    @johnpope1473 3 ปีที่แล้ว +1

    I think your gameplan could be - make enough money to quit your day job. it would require going to wider audience - or getting more corporate sponsors. you can do this.

    • @TheAIEpiphany
      @TheAIEpiphany  3 ปีที่แล้ว

      Haha thanks man! There are actually some updates on my side I'll share them on Twitter/LI soon.

  • @kimanthony1667
    @kimanthony1667 2 ปีที่แล้ว +1

    Great video!! Thx for review!