ComfyUI With Florence 2 Vision LLM - This Is Not Just A Segmentation Model

แชร์
ฝัง
  • เผยแพร่เมื่อ 23 ก.ค. 2024
  • ComfyUI With Florence 2 Vision LLM
    In this video, I delve into a new LLM - Florence 2, an extraordinary vision foundation model developed by Microsoft. Join me as I discuss its features, demonstrate its capabilities, and guide you through the installation process.
    Florence 2: An Image-to-Text Prompt Large Language Model
    Florence 2 is trained with the massive FLD-5B dataset, making it one of the most accurate and detailed text generation models for images. In this video, I'll showcase two custom nodes that connect to Florence 2: the KJ version and the Spacepxl version. These custom nodes enable segmentations, image captioning, and object detection.
    Explainer About Florence-2 : • Florence-2 And Deepsee...
    Workflows In This Tutorial : / 106792381
    ComfyUI-Florence-2
    huggingface.co/microsoft/Flor...
    arxiv.org/abs/2311.06242
    github.com/spacepxl/ComfyUI-F...
    github.com/kijai/ComfyUI-Flor...
    Installing Florence 2 Custom Nodes in ComfyUI
    Before we dive into the demonstrations, we need to install the Florence 2 custom nodes. Don't worry, I'll guide you through the process step by step. Just head to the ComfyUI manager, search for the custom nodes, click install, and wait for the downloads to complete. Once installed, you'll have the powerful Florence 2 custom nodes at your fingertips.
    Exploring the KJ Version of Custom Nodes
    Let's start our journey by testing the KJ custom nodes. With just two simple custom nodes, you'll be able to perform segmentations, captionings, and bounding boxes. I'll demonstrate their usage using an example image, providing you with a clear understanding of how these custom nodes enhance your workflow.
    Unleash the Power of the Spacepxl Version
    Next, we'll explore the Spacepxl version of the Florence 2 custom nodes. These custom nodes offer even more features and functions, allowing you to create diverse and versatile workflows. With separate custom nodes for each capability, you'll have the flexibility to incorporate Florence 2 seamlessly into your ComfyUI projects.
    The Time is Now: Experience the Future of AI Image Generation
    Florence 2 harnesses the potential of large language models, providing accurate and detailed text descriptions for any element in an AI image. Witness the magic of caption-to-phrase grounding, region captions, and object detection as Florence 2 effortlessly combines text and visuals. Prepare to be amazed by the future of AI image and video generation!
    If You Like tutorial like this, You Can Support Our Work In Patreon:
    / aifuturetech
    Discord : / discord
  • วิทยาศาสตร์และเทคโนโลยี

ความคิดเห็น • 45

  • @swannschilling474
    @swannschilling474 29 วันที่ผ่านมา

    Great tutorial! Thanks!!😊

  • @crazyleafdesignweb
    @crazyleafdesignweb 29 วันที่ผ่านมา +1

    that is great stuff! We are getting out of Stable Diffusion with more alternatives.

    • @TheFutureThinker
      @TheFutureThinker  29 วันที่ผ่านมา +2

      There's more 😉 well, I think many users are going to keep SD1.5 and SDXL, others from Stupidity AI throw to bin. 🤭

  • @Rico-nj3vl
    @Rico-nj3vl 29 วันที่ผ่านมา +1

    Very nice !

  • @reaperhammer
    @reaperhammer 29 วันที่ผ่านมา +3

    It will be interesting to see how you integrate this into other workflows as you suggested

    • @TheFutureThinker
      @TheFutureThinker  29 วันที่ผ่านมา +2

      Yes I will make another video about that. It should be interesting.

    • @Ai-dl2ut
      @Ai-dl2ut 29 วันที่ผ่านมา

      @@TheFutureThinker Can't Wait :)

    • @TheCinefotografiando
      @TheCinefotografiando 29 วันที่ผ่านมา

      I have found myself watching your videos more and more

  • @x3gxu
    @x3gxu 29 วันที่ผ่านมา +4

    Hi. How did you make the talking head in the corner? Looks pretty good.
    It's not wav2lip. V-express? Hallo? I wasn't able to achieve results like this, so I'm very interested. Can you point me in the right direction?

    • @pabloapiolazza4353
      @pabloapiolazza4353 28 วันที่ผ่านมา

      Also interested!

    • @TheFutureThinker
      @TheFutureThinker  19 วันที่ผ่านมา

      This is a short videos done for you guys : th-cam.com/video/EcAMY-sZ9mY/w-d-xo.html
      :) hope this help

  • @vitalis
    @vitalis 28 วันที่ผ่านมา

    So cool

  • @SageGoatKing
    @SageGoatKing 28 วันที่ผ่านมา

    I don't miss the right click menu bar at all since getting the sidebar where I can pin my favorite nodes, etc.

    • @TheFutureThinker
      @TheFutureThinker  28 วันที่ผ่านมา

      Normally, I use search. I don't have favourite nodes. Cause I use too many

  • @RamonGuthrie
    @RamonGuthrie 21 วันที่ผ่านมา

    Hey can you prompt the Florence2 model on what parts of an image you want to describe?
    Example Describe the background only or Describe the person in detail only!
    or are there better vision models for this?

    • @TheFutureThinker
      @TheFutureThinker  21 วันที่ผ่านมา

      Yes this vision model can be segment the background and then do captioning

  • @kalakala4803
    @kalakala4803 29 วันที่ผ่านมา +2

    nice! I just check the Florence-2 LLM video you did. this AI model looks promising. Can you integrate this with AnimateDiff V2V?

  • @promptaganda
    @promptaganda 24 วันที่ผ่านมา

    using spacepxl node, i am getting strange polygons for all images i try to run region to segmentation on. The captioning is working correct. any ideas?

    • @TheFutureThinker
      @TheFutureThinker  19 วันที่ผ่านมา

      This might possibly happen, when the node have no condition logic to specific conditions you are setting.
      Hopefully the developer will do a new update and add something to handle this.

    • @davidsmith-lv4kq
      @davidsmith-lv4kq 19 วันที่ผ่านมา

      @@TheFutureThinker do u mean the prompt must match a word in the caption or region suggestions? i seemed to try alot of different images and regions and could never get it to do anything other than make the polygons

  • @context_eidolon_music
    @context_eidolon_music 29 วันที่ผ่านมา

    Holy crap!

  • @patagonia4kvideodrone91
    @patagonia4kvideodrone91 27 วันที่ผ่านมา

    There are other nodes, I don't remember the name now, but what do you say detect me such a thing, and it generates the automatic mask, (but not square) but with its real contour,

    • @TheFutureThinker
      @TheFutureThinker  27 วันที่ผ่านมา

      Segment Anything

    • @promptaganda
      @promptaganda 24 วันที่ผ่านมา

      @@TheFutureThinker every time ive ever tried to add a prompt to a segment anything mode it makes zero mask ....... any suggestions?

    • @TheFutureThinker
      @TheFutureThinker  24 วันที่ผ่านมา

      @@promptaganda what is your setting?

  • @jairuskersey8311
    @jairuskersey8311 29 วันที่ผ่านมา

    Nice vid. Can you also make a tutorial on how you made the talking avatar in this video? Thanks~

    • @TheFutureThinker
      @TheFutureThinker  29 วันที่ผ่านมา

      just use Hedra, very easy website no need tutorial :) I believe you can do it

  • @nkofr
    @nkofr 29 วันที่ผ่านมา +1

    what's the use of the 'finetuned' versions?

    • @TheFutureThinker
      @TheFutureThinker  29 วันที่ผ่านมา +1

      Finetuned model on a collection of downstream tasks

    • @nkofr
      @nkofr 29 วันที่ผ่านมา

      @@TheFutureThinker excuse me but what does that mean?

    • @riggitywrckd4325
      @riggitywrckd4325 23 วันที่ผ่านมา +1

      It means that if you have a dataset of stuff you want it to learn you can train those ideas and it will learn to spot them like it does in this one.

  • @triojakeson116
    @triojakeson116 28 วันที่ผ่านมา +1

    Hey bro i wanted to ask about the kling ai video, can u tell me how much time it will take for u to get accepted after getting into waitlist, cz am already in waitlist for one day, just wanna make sure, if u know pls reply thanks?

    • @TheFutureThinker
      @TheFutureThinker  27 วันที่ผ่านมา

      Depends, some got it few days in waiting.

    • @triojakeson116
      @triojakeson116 27 วันที่ผ่านมา

      ​@@TheFutureThinkeralso bro am from india, and here all chinese apps are banned, so i had to use an American vpn and a fake chinese number, do u think they will accept my waitlist request if they see all this, will they check all this info 😢

    • @TheFutureThinker
      @TheFutureThinker  27 วันที่ผ่านมา +1

      @@triojakeson116 sorry to hear that. This is more about the company policy, I have no comment.

    • @triojakeson116
      @triojakeson116 27 วันที่ผ่านมา

      @@TheFutureThinker ok bro i will update if something happens 😭

    • @TheFutureThinker
      @TheFutureThinker  27 วันที่ผ่านมา

      @@triojakeson116 but wish you good luck , i see other people are getting access now. So hopefully it will be okay for ya

  • @bilalalam1
    @bilalalam1 29 วันที่ผ่านมา

    Hi , I would like to use Florence 2 with LM studio

    • @TheFutureThinker
      @TheFutureThinker  25 วันที่ผ่านมา

      No sure, but I use OpenWebUi x Ollama

  • @robd7724
    @robd7724 16 วันที่ผ่านมา +1

    I'm sorry but the AI voice you are using sounds awful and the video is very creepy.