so glad i sub'ed a few weeks ago. i think you provide the best explanations of what's going on vs other in the AI space. even tho my 3060ti can't handle the flux workflows, i (as a know nothing) was able to adapt your FLUX, and FLUX -> SVD workflows in to something that worked with SDXL. i'm really looking forward to seeing what else you do, and learn from it.
Thanks so much! Glad its helping. I have some awesome stuff coming for you guys and when I get the new unified sampler done, you can use it with sdxl too so you wont have to adapt the workflow at all!
@@flowstateeng THANK YOU! i have one request. if it isn't too much trouble, if not forget it. for the image output, and svd output, please keep the options for lossless images like webp, png, tiff or what ever. personally, i'm a lot more comfortable/capable editing images in capture one along with topaz then recompiling in davinci studio vs doing all that in davinci. why don't i just do it all in davinci? i'm getting there, still learning, but i've been using capture one for years. :)
also have you tried the Flux NF4 model? it only runs at about 10-12gb vram for me. should work on a 3060ti and provides identical results as the Flux Dev model. worth a try! the model is here: huggingface.co/lllyasviel/flux1-dev-bnb-nf4/blob/main/flux1-dev-bnb-nf4.safetensors and you save it in your checkpoints folder. my unified model loader will load it up for you
You will need to change your security level configuration in ComfyUI, to use the Install via Git URL feature, it's in the ComfyUI-Manager package under config.ini . Ask perplexity about the settings.
Actually I think the security is set to weak automatically in the updated Comfy, but just in case I updated the description to include a link to a video where I cover how to change the settings. There were other issues with the installation in a fresh Comfy though, that I wouldn't have caught if you hadn't mentioned this and caused me to go check it in a fresh Comfy! So thanks for that!
audio is great! i wonder if it would be possible to have one node combine all of the prompt/quality enhancer nodes, such as clipattentionmultiply, perturbed attention guidence, etc. one node for maximum quality!
yeah its definitely possible. lmk which ones you want. my plant is to build all of that into the LLM node. i also has some automated prompt engineering to build into that which im really excited to get done. for example, if you want a realistic photo, or a animated photo, or a video thumbnail, or if you want to emphasize text in the output, etc, you can just select from a dropdown list and the node will tell the llm what to do, instead of spending time hand engineering prompt to get the right type of image. that will save me personally a ton of time with thumbnails. its always a struggle to get great thumbnails with the right placement and emphasis on text, without sacrificing quality. would be nice to just give a few words on what i want, how i want the text, and let it do the rest
@@flowstateeng you have great ideas! to add onto the LLM prompt engineering, tag styled prompting options would be cool to see for stable diffusion users (1girl, pink dress, smiling, etc). i appreciate that the nodes you create are supporting both flux and SD. a sort of more random but controllable system could be cool, but maybe a bit too niche, like if you have a prompt, but want the LLM to randomize an aspect of it, like hairstyle, angle, etc. its like SD's wildcard feature, but automated. another very very niche idea is a node for more streamlined model and lora merging, potentially combining the load loras stack, checkpoint stack, clip merge, merge by block, and save checkpoint nodes. im very big into model merging, and its a bit annoying to keep track of like 7 nodes and each parameter, not to mention merging in general could use a user friendly overhaul. not even sure if this one is possible, or really even useful for 95% of people, but its an idea
i can add in prompt feature randomization! thats a good idea, ill add it to the list! yeah im starting on the unified sampler now and i was thinking of adding the loras, ip adapters, and control nets into that, or maybe itll be its own separate node. depends on how cluttered it gets. either way, if it is a separate node, itll have all the loaders, preprocessors, etc all in one node. now that ive typed that out, i think yeah a separate node will be good but one node should do for all the lora, ip, and cn stuff. ill include stacking and chaining in it too
yep, its on the list! i am doing the SVD nodes right now (see the latest patreon post) and then im doing the unified flux/sd sampler, and then im doing a node to contain all of the functionality for loras, control nets, and ip adapters. that will include options for in/out painting, canny/depth/hed/pose control nets, etc... i want to get an SVD video out tonight though, so im focusing solely on that for today
yes, please do lol! i have a spare 12 core xeon cpu lying around that i want to use to do a giveaway xeon/rtx pc build when i hit a certain sub goal but i need to figure that out and announce it
Dude is redesigning comfy the way it should have been to start with. Cheers! FYI, I am hating the way model, specifically FLUX, load nodes don't look in the same/correct folder(s) for the model.
yeah i had that same thought. would be nice to have it all in the same place, instead of having separate directories for specific custom nodes. thats one thing i am gong to do with my unified loader asap, give it the ability to load all models from the same place. all loras in one place, all control nets in one place... i got a lot of cool stuff coming. thanks for the kudos!
whats coming is gonna blow your mind btw lol. vid in about an hour ish. just scrolling through comments and saw this so figured id mention it real quick. its gonna be wild.
you make a folder in your ComfyUI/models called llm_gguf, so ComfyUI/models/llm_gguf and you put all your .gguf LLM models there. I covered it in my video covering Searge installation, so I didn't think to cover it here. My Searge video is here: th-cam.com/video/46on642Wk3g/w-d-xo.htmlsi=Z_pX4BDC6QsiYmyN you dont need the Searge node but if you install the LLM like I show in that video, my node will work for you
The FlowStateUnfiedModelLoader still comes up red no matter how I have installed everything. I see it in the "H:\ComfyUI_windows_portable\ComfyUI\custom_nodes\FlowState-Nodes" folder but its like it not even in there. I have updated everything. Anyone know how I can fix it?
FlowState LLM doesn't work for me. One thing to note, I am on Linux. All of my models and outputs are on another drive. Everything else is fine. Why I use Linux? Windows is too much of a resource hog. My render times are noticeably faster in Linux. And Ollama works fine in the Linux environment. Could be because of the Python3?
oh yeah this is only tested on windows. I will get around to linux testing at some point. i do like linux better for a lot of things but with all the adobe software, i need windows for now. comfy has its own python installation internally if youre using the comfy portable version. could be worth a try. there is also a way to have your native ollama loaded into comfy so i will take a look at integrating that option. today i'm joining Flux and SVD in holy matrimony with my new FVD (not SVD) node. but i will circle back to the LLM node in a couple days.
just curious, do you get any errors in terminal? does the FS LLM node load for you at all? if so, are your LLMs listed in the dropdown? lmk what your terminal says when you try to run the node
Trying to give this a go as it looks interesting Due to security setings, I git cloned via command line into the custom nodes folder into an freshly updated Comfy [on windows 11] The Custom Nodes folder now contains a Flow State folder However, I'm not seeing the Flow State nodes when I drop in the car image? I can see the essential and image picker nodes Any one got any ideas?
Do you have red nodes or just no nodes showing? Can you see the nodes if you type "FlowState" in the search bar? Have you checked your "ComfyUI/user/default/workflows" for a FlowState folder and/or a "FS - T2I.json" file? I'm pushing a new commit right now to include a workflows folder. If you save that somewhere in your "ComfyUI/user/default/workflows" folder and still don't see the nodes when you open that workflow, lmk. BTW, in case you don't know, if you put separate folders in your "ComfyUI/user/default/workflows" folder, it creates collapsible menus in the Comfy workflows UI, which keeps it from being cluttered with a long list of workflows. That's why I save my workflows in "ComfyUI/user/default/workflows/FlowState" or "ComfyUI/user/default/workflows/Testing" or whatever other folder names you want in there. github.com/flowstateeng/FlowState-Nodes/tree/main/workflows
@@flowstateeng Thanks for responding, I had red nodes showing, none of the FlowState nodes installed, the FlowState folder was not in the add nodes list otherwise I'd have just selected them from there, and the cmd console was showing FlowState Nodes not installed. Sorted now though. I was getting multiple errors, including llama-cpp, which it was failing to download for some reason. After multiple restarts it downloaded what was required and the nodes came up as expected. Phew! I've got a several versions of the flux models and clips organised into separate directories and familar with selecting them, or other models, I too am a bit of an OCD freak. Would be neat if your model loader also had a row to install flux.gguf models to test out Thanks for all the work you've put in. I've subscribed, so good luck with the channel.
gonna add the model installation into the llm node too. i was thinking it would be nice to have something like that but your idea solidified it for me, thanks!
@@flowstateeng it much better now, personally I'd still tweak it, but it ain't that bad now, but you better ask some audio guy! BTW, was actually looking for videos on how to write Comfy UI nodes - so these last videos of yours I'll definitely watch.
I might be insane but I like noodle workflows. But I do think this is cool.
yeah i like a few. i dont want it to be completely like a form like in forge but i want the noodles to flow neatly from start to end.
so glad i sub'ed a few weeks ago. i think you provide the best explanations of what's going on vs other in the AI space. even tho my 3060ti can't handle the flux workflows, i (as a know nothing) was able to adapt your FLUX, and FLUX -> SVD workflows in to something that worked with SDXL. i'm really looking forward to seeing what else you do, and learn from it.
Thanks so much! Glad its helping. I have some awesome stuff coming for you guys and when I get the new unified sampler done, you can use it with sdxl too so you wont have to adapt the workflow at all!
@@flowstateeng THANK YOU! i have one request. if it isn't too much trouble, if not forget it. for the image output, and svd output, please keep the options for lossless images like webp, png, tiff or what ever. personally, i'm a lot more comfortable/capable editing images in capture one along with topaz then recompiling in davinci studio vs doing all that in davinci. why don't i just do it all in davinci? i'm getting there, still learning, but i've been using capture one for years. :)
also have you tried the Flux NF4 model? it only runs at about 10-12gb vram for me. should work on a 3060ti and provides identical results as the Flux Dev model. worth a try!
the model is here: huggingface.co/lllyasviel/flux1-dev-bnb-nf4/blob/main/flux1-dev-bnb-nf4.safetensors
and you save it in your checkpoints folder. my unified model loader will load it up for you
@@flowstateeng i have that one already. forge is just moody for what ever reason. so i kinda gave up on using it, and stuck with SDXL. 🤷♂
You will need to change your security level configuration in ComfyUI, to use the Install via Git URL feature, it's in the ComfyUI-Manager package under config.ini . Ask perplexity about the settings.
good catch, i didnt think to mention that! Thank you!
Actually I think the security is set to weak automatically in the updated Comfy, but just in case I updated the description to include a link to a video where I cover how to change the settings. There were other issues with the installation in a fresh Comfy though, that I wouldn't have caught if you hadn't mentioned this and caused me to go check it in a fresh Comfy! So thanks for that!
audio is great! i wonder if it would be possible to have one node combine all of the prompt/quality enhancer nodes, such as clipattentionmultiply, perturbed attention guidence, etc. one node for maximum quality!
yeah its definitely possible. lmk which ones you want. my plant is to build all of that into the LLM node. i also has some automated prompt engineering to build into that which im really excited to get done. for example, if you want a realistic photo, or a animated photo, or a video thumbnail, or if you want to emphasize text in the output, etc, you can just select from a dropdown list and the node will tell the llm what to do, instead of spending time hand engineering prompt to get the right type of image. that will save me personally a ton of time with thumbnails. its always a struggle to get great thumbnails with the right placement and emphasis on text, without sacrificing quality. would be nice to just give a few words on what i want, how i want the text, and let it do the rest
...but if you have good ideas on other features for prompting, lmk and ill look at building that in as well
@@flowstateeng you have great ideas! to add onto the LLM prompt engineering, tag styled prompting options would be cool to see for stable diffusion users (1girl, pink dress, smiling, etc). i appreciate that the nodes you create are supporting both flux and SD. a sort of more random but controllable system could be cool, but maybe a bit too niche, like if you have a prompt, but want the LLM to randomize an aspect of it, like hairstyle, angle, etc. its like SD's wildcard feature, but automated.
another very very niche idea is a node for more streamlined model and lora merging, potentially combining the load loras stack, checkpoint stack, clip merge, merge by block, and save checkpoint nodes. im very big into model merging, and its a bit annoying to keep track of like 7 nodes and each parameter, not to mention merging in general could use a user friendly overhaul. not even sure if this one is possible, or really even useful for 95% of people, but its an idea
i can add in prompt feature randomization! thats a good idea, ill add it to the list! yeah im starting on the unified sampler now and i was thinking of adding the loras, ip adapters, and control nets into that, or maybe itll be its own separate node. depends on how cluttered it gets. either way, if it is a separate node, itll have all the loaders, preprocessors, etc all in one node. now that ive typed that out, i think yeah a separate node will be good but one node should do for all the lora, ip, and cn stuff. ill include stacking and chaining in it too
Any change you could take a look at inpainting combined with these new nodes?
yep, its on the list! i am doing the SVD nodes right now (see the latest patreon post) and then im doing the unified flux/sd sampler, and then im doing a node to contain all of the functionality for loras, control nets, and ip adapters. that will include options for in/out painting, canny/depth/hed/pose control nets, etc... i want to get an SVD video out tonight though, so im focusing solely on that for today
@@flowstateeng Nice! Thanks :)
Atleast Give this channel 100k subs
yes, please do lol! i have a spare 12 core xeon cpu lying around that i want to use to do a giveaway xeon/rtx pc build when i hit a certain sub goal but i need to figure that out and announce it
How to use CnNet with this Custom node? There is only one conditioning slot but ConNet node has two output slots, positive and negative.
which control net are you trying to use?
Dude is redesigning comfy the way it should have been to start with. Cheers!
FYI, I am hating the way model, specifically FLUX, load nodes don't look in the same/correct folder(s) for the model.
yeah i had that same thought. would be nice to have it all in the same place, instead of having separate directories for specific custom nodes. thats one thing i am gong to do with my unified loader asap, give it the ability to load all models from the same place. all loras in one place, all control nets in one place... i got a lot of cool stuff coming. thanks for the kudos!
whats coming is gonna blow your mind btw lol. vid in about an hour ish. just scrolling through comments and saw this so figured id mention it real quick. its gonna be wild.
where do u put the llm model? it says no model available.
you make a folder in your ComfyUI/models called llm_gguf, so ComfyUI/models/llm_gguf and you put all your .gguf LLM models there. I covered it in my video covering Searge installation, so I didn't think to cover it here. My Searge video is here: th-cam.com/video/46on642Wk3g/w-d-xo.htmlsi=Z_pX4BDC6QsiYmyN
you dont need the Searge node but if you install the LLM like I show in that video, my node will work for you
The FlowStateUnfiedModelLoader still comes up red no matter how I have installed everything. I see it in the "H:\ComfyUI_windows_portable\ComfyUI\custom_nodes\FlowState-Nodes" folder but its like it not even in there. I have updated everything. Anyone know how I can fix it?
FlowState LLM doesn't work for me. One thing to note, I am on Linux. All of my models and outputs are on another drive. Everything else is fine. Why I use Linux? Windows is too much of a resource hog. My render times are noticeably faster in Linux. And Ollama works fine in the Linux environment. Could be because of the Python3?
oh yeah this is only tested on windows. I will get around to linux testing at some point. i do like linux better for a lot of things but with all the adobe software, i need windows for now. comfy has its own python installation internally if youre using the comfy portable version. could be worth a try. there is also a way to have your native ollama loaded into comfy so i will take a look at integrating that option. today i'm joining Flux and SVD in holy matrimony with my new FVD (not SVD) node. but i will circle back to the LLM node in a couple days.
just curious, do you get any errors in terminal? does the FS LLM node load for you at all? if so, are your LLMs listed in the dropdown? lmk what your terminal says when you try to run the node
Is possible to do outpainting?
i dont have a node for that at the moment but there are nodes for that. you can use them with my workflow as you would in your existing workflow
Trying to give this a go as it looks interesting
Due to security setings, I git cloned via command line into the custom nodes folder into an freshly updated Comfy [on windows 11]
The Custom Nodes folder now contains a Flow State folder
However, I'm not seeing the Flow State nodes when I drop in the car image? I can see the essential and image picker nodes
Any one got any ideas?
Do you have red nodes or just no nodes showing? Can you see the nodes if you type "FlowState" in the search bar? Have you checked your "ComfyUI/user/default/workflows" for a FlowState folder and/or a "FS - T2I.json" file?
I'm pushing a new commit right now to include a workflows folder. If you save that somewhere in your "ComfyUI/user/default/workflows" folder and still don't see the nodes when you open that workflow, lmk.
BTW, in case you don't know, if you put separate folders in your "ComfyUI/user/default/workflows" folder, it creates collapsible menus in the Comfy workflows UI, which keeps it from being cluttered with a long list of workflows. That's why I save my workflows in "ComfyUI/user/default/workflows/FlowState" or "ComfyUI/user/default/workflows/Testing" or whatever other folder names you want in there.
github.com/flowstateeng/FlowState-Nodes/tree/main/workflows
@@flowstateeng Thanks for responding, I had red nodes showing, none of the FlowState nodes installed, the FlowState folder was not in the add nodes list otherwise I'd have just selected them from there, and the cmd console was showing FlowState Nodes not installed. Sorted now though. I was getting multiple errors, including llama-cpp, which it was failing to download for some reason. After multiple restarts it downloaded what was required and the nodes came up as expected. Phew!
I've got a several versions of the flux models and clips organised into separate directories and familar with selecting them, or other models, I too am a bit of an OCD freak.
Would be neat if your model loader also had a row to install flux.gguf models to test out
Thanks for all the work you've put in. I've subscribed, so good luck with the channel.
glad you got it working! lmk if you have any questions. model installation in the node is a good idea, ill add it to the todo list!
gonna add the model installation into the llm node too. i was thinking it would be nice to have something like that but your idea solidified it for me, thanks!
👌
took me a while on the thumbnail. couldnt decide what to do but figure an upgraded version of the last car is appropriate. the audio is good?
@@flowstateeng it much better now, personally I'd still tweak it, but it ain't that bad now, but you better ask some audio guy! BTW, was actually looking for videos on how to write Comfy UI nodes - so these last videos of yours I'll definitely watch.
Thanks man, lmk if you have any questions on building the nodes.