วีดีโอ

Using first frame conditioning is an option- you set useFirstFrame to True and use the last frame of your generated video as your image in the next generation. You can easily use the last frame as an image by uploading the generated video and, assuming it's 32 frames, you can skip the first 31 frames in the video upload node. For resolution_x=512, I will advise that the initial image should be 512x768. 32 frames seem to be the ideal for most of these video generation tools, but I have gotten good results with 48 frames using Animate-x and 120 frames with UniAnimate-long, and I could probably go higher. A lot depends on the seed, image and video quality & resolution.

@@Isi-uT I agree, but last frame could be blurry and each next step would be video quality degradation. If I would upscale each 32th frame then the video would blink a little every upscaled image. Maybe Framer could help somehow. I try to think maybe there are more options and can not imagine how.

Considering the issues you raised, the alternative is to run as many frames as possible, probably skipping some frames and using video interpolation to later make up, and also using faceswap or restoration to ensure consistency in the face. I sometimes have to manually edit some of the frames in my generated video.

@@Isi-uT Thank you for you advise.I will try restoration and interpolation. Maybe resampling technique

You're welcome.

Yes, all the tests I have done also suggest that to be the case. I think the main animate-x repo only showed inferences with 512. I included 768 in the comfyui version because I assumed that it will also work since it was built on unianimate and others. But it seems the animate-x model may have been trained on only 512 resolution.

I have already made a tutorial for installing unianimate which can also be followed to install animate-x as they were both implemented in the same github repository. You can watch it here: th-cam.com/video/NFnhELV4bG0/w-d-xo.html You can also read the written installation guide from any of the following sites: penioj.blogspot.com/2024/12/windows-installation-guide-for.html medium.com/@isinsemail/windows-installation-guide-for-unianimate-and-animate-x-in-comfyui-d69cc2e6b5d1

@Isi-uT thanks

You're welcome.

You can install the nodes by searching for ComfyUI-Animation_Nodes_and_Workflows with the comfyui manager. After installation, you will find the workflows in Comfyui/custom_nodes/Comfyui-Animation_Nodes_and_Workflows/animationWorkflows/img2img.

I've not tried MimicMotion and StableAnimator because their memory requirements look too high for my system. I will probably make a comparison video when I get a new one.

Tất cả đều gần giống nhau vẫn hay bị lỗi tay, chân, mặt, quần áo

You can see a possible solution here: github.com/Isi-dev/ComfyUI-UniAnimate-W/issues/23

@@Isi-uT thank you for your reply

You're welcome

ComfyUI \custom_nodes\ComfyUI-UniAnimate-W-main\checkpoints”

Found it

Thanks. I had similar observations. Animate-x sometimes has issues following the pose but tends to maintain the character's appearance better than unianimate. There were other pose related functions in the main animate-x repo which I have not yet implemented in the comfyui nodes because they would lead to much higher RAM & VRAM usage and I don't yet know if they would make a significant difference in the pose. I will decide if to implement them after I test the main repo.

@@Isi-uT Thanks and looking forward!! i use the same Pig monster image in animateX page example. then put into my comfy animateX. the generated video/ repose don't have the same pose following as shown in their demo page, I suspect more is needed to reproduce the animateX original result

Thanks for the info. I have not yet made comparisons with the videos on the demo page. I will do so soon and modify the code where necessary.

It would be amazing if platforms like Leonardo integrated something similar for wider accessibility. Hopefully, more cloud-based options become available.

For a 512x768 output resolution, 32 frames take around 8minutes.

I would've loved to, but my system does not meet the memory requirements.

I think it depends on your VRAM.

@@Isi-uT Are you in banodoco? I have a 4090, I'm going to try it, I want to try that output with other wf. In the company I work we tried muse a while ago but it failed a lot

@@mrboofy No I'm just hearing of banodoco. Your system is perfect for all the workflows.

Thanks for your comment. AIFSH/ComfyUI-UniAnimate is another comfyui repository on Unianimate and it's advisable that you install only one, that's why the warning comes up when you want to install or update the nodes via the comfyui Manager. If you haven't installed AIFSH/ComfyUI-UniAnimate, then you can go ahead and install or update the 'unianimate nodes for comfyUI.'

@@Isi-uT Thanks for your swift reply. I don't have AIFSH/ComfyUI-UniAnimate installed but the ComfyUI Manager still shows a conflict. Should I ignore it? Also, v2-1_512-ema-pruned.ckpt marked as a "pickle" not safe?

Yes you can ignore it. I see it every time I want to update the uniAnimate nodes. You are right about the v2-1_512-ema-pruned.ckpt not being safe. I took the risk of using it that way, but I guess there should be a way to turn it into a safetensors format before using it.

I will have to do more tests to give an accurate response. I'm currently away from my system. I'll respond within 4 days.

@@Isi-uT I'm not in a huge rush, please take your time. This is currently a hobby for me, so I won't demand anything unreasonable.

No problem. Cheers!

It works great for half-body portraits when there’s noticeable movement in the limbs or changes in body position in the driving video. It’s less effective for idle animations with minimal motions, such as subtle breathing.

Echo-Mimic v2 + sapience . All that magic should be in one place

Thank you so much for your comment. Currently, I don’t have comprehensive documentation available, but I’ll be working on creating proper documentation in the nearest future. I will briefly describe the nodes here: The 'Align & Generate poses for UniAnimate' node This node takes in an image and a reference video and outputs the dwpose (a skeletal representation) of the person in the image, and frames of dwpose images representing the pose sequence of the person in the reference video. The 'Animate image with UniAnimate' node This node receives three inputs (the image, and the two outputs from the 'Align & Generate poses for UniAnimate' node) and outputs image frames which show an animation of the input image when connected to a 'video combine' node (the documentation for the 'Load Video' and 'Video Combine' nodes can be found in this github repository: github.com/Kosinkadink/ComfyUI-VideoHelperSuite ) -> The 'steps' variable determines the number of iterations used during the diffusion process to generate an image from noise. It influences the quality, detail, and speed of the generation. Higher steps result in more detailed image frames but at the cost of longer generation time. Several tests have shown that 30 steps is ideal, but I have gotten good results with 20 steps, depending on the input image and reference video. -> The 'useFirstframe' variable is used to decide if the first frame of the target pose sequence (from video) should be used to calculate the scale coefficient for aligning the pose sequence with the reference image (when set to True), or if a random frame from the target pose sequence should be used to calculate the scale coefficient for aligning the pose sequence with the reference image (when set to False). You can set it to 'False' so that a random frame is used for the calculation in cases where the first frame of the reference video does not include the entire face and full-body pose (hands and feet) which are required for more accurate estimations and better video generation results. -> The 'frame_interval' variable is used to determine how many frames to skip or use from the input pose sequence. If there are 30 frames and this variable is set to 1, then every frame will be used. If set to 2, then every second frame will be selected from the input pose sequence and therefore only 15 frames will be used. Higher frame intervals will result in faster video generation but less smooth animation. -> The 'max_frames' variable determines the total number of frames that will be used for the generation. If the video has 120 frames and the max_frames is set to 32, then only 32 frames will be generated. -> The 'resolution_x' variable determines the width and height of each output frame. There are only two options (512 & 768). Selecting 512 will result in a 512X768 video, while 768 will result in a 768X1216 video which takes more time to generate and requires more GPU, depending on the number of frames). If you only want to change the pose of an image using another image as reference, then 'useFirstFrame' should be false, 'frame_interval' should be 1, and 'max_frames' should be 1. I later added two more nodes which I introduced in this video: th-cam.com/video/Ne-DSBhfg8A/w-d-xo.html These node work directly with the input images and videos and therefore do not need the 'Align & Generate poses for UniAnimate' node. The 'Repose image with UniAnimate' node is for changing the pose of an image to that of a reference image, while the 'Animate image with UniAnimate_Long' node is for generating longer videos that maintain the properties of the input image much better than the 'Animate image with UniAnimate' node. I will briefly describe the significant parameters in the two nodes that are absent from the previous nodes. -> The 'dontAlignPose' variable is used to determine if any frame of the target pose sequence (from video or reference image) should be used to calculate the scale coefficient for aligning the pose sequence or reference image with the main image. If set to true, then no alignment calculation will be done and the main image (the image you want to repose or animate) has to be manually aligned with the image in the video or reference image before uploading them. Some poses that are difficult to align via calculation by the nodes can be better transferred by adjusting the main image such that the height of the person in the main image is the same as the height of the person in the reference video or image. You can always try your generations twice by changing the setting of this variable to see which one is more desirable. -> The 'context_size' variable controls how many frames are processed together or the number of frames in a context window (actual set of frames being processed together at any given time). Larger context sizes allow the model to consider more temporal information, improving coherence and consistency between frames. -> The 'context_stride' variable determines the spacing between context windows or the number of frames between the starting points of consecutive context windows. A stride equal to the context size means no frames are reused between windows. A smaller stride enables overlapping context windows, which can enhance temporal consistency but increases computational cost. -> The 'context_overlap' variable determines how much of a context window overlaps with the next window. Higher values enhance temporal consistency but increases computational cost. The context_overlap = context_size - context_stride. So if context_size = 16 and context_stride = 11, the context_overlap is 5 frames. The context_overlap might look unnecessary since it can be derived from the previous two variables, but I think there was a criteria for determining which to use in the code. I can't remember the criteria.

@@Isi-uT That was a huge help. Thank you for the information. Great work!

Thank you.

Я не понимаю вашего комментария. Пожалуйста, не могли бы вы уточнить.

This looks either like a corrupted file issue or the models were not properly extracted from the zip file download into the ComfyUI-UniAnimate-W/checkpoints folder. You can check your ComfyUI-UniAnimate-W/checkpoints folder and see if you can find the 'open_clip_pytorch_model.bin' model. It is around 3.85GB in size. If the size is less, then you will have to download it again. Also ensure other models are present and fully downloaded: unianimate_16f_32f_non_ema_223000.pth ~ 5.5GB v2-1_512-ema-pruned.ckpt ~ 5.1GB yolox_l.onnx ~ 211.6MB dw-ll_ucoco_384.onnx ~ 131.2MB If you downloaded all the models in a zip file, then ensure the models are first extracted before placing them in the ComfyUI-UniAnimate-W/checkpoints folder. Please let me know the outcome.

@@Isi-uT Yes. I suspected that as well. I have downloaded them from the link you provided several times now. Could the source files be corrupted?

@@Isi-uT You were correct. It was the 'open_clip_pytorch_model.bin' file. I downloaded it manually and it got past that step. Working on the next error now.

Thanks for the feedback.

Thank you :) I am not familiar with OCIO OpenColorIO. I will look into it and see if it can be implemented in comfyui. I will give you a reply as soon as I fully understand the requirements.

I think it's possible to import OCIO in a node and change from one color space to another in comfyui. Since there are nodes to upload and save in EXR format, all that is needed is to create a node that receives the image tensor from the load EXR node, transforms the image to another color space depending on user input, and outputs the resulting image as a tensor to the Save EXR node. You can try implementing a code similar to what is shown below. I generated it using chatGPT and I have not tested it, but it looks reasonable. First install OCIO in your environment (e.g. pip install opencolorio), then code: import torch import OpenColorIO as ocio class TransformColorSpace: @classmethod def INPUT_TYPES(s): return { "required": { "image": ("IMAGE",), "config_filepath": ("STRING", {"default": "/path/to/config.ocio"}), "input_space": ("STRING", {"default": "sRGB"}), "output_space": ("STRING", {"default": "ACEScg"}), } } CATEGORY = "color" RETURN_TYPES = ("IMAGE",) RETURN_NAMES = ("transformed_image",) FUNCTION = "transform" def transform(self, image, config_filepath, input_space, output_space): # Load the specified OCIO configuration config = ocio.Config.CreateFromFile(config_filepath) ocio.SetCurrentConfig(config) # Create a processor for the input and output spaces processor = config.getProcessor(input_space, output_space) # Convert image to NumPy for processing numpy_image = image.squeeze().numpy() transformed_image = processor.applyRGB(numpy_image) # Convert back to Torch tensor transformed_image_tensor = torch.unsqueeze(torch.from_numpy(transformed_image), 0) return (transformed_image_tensor,) You can modify the code to provide selection options for the input and output space: Input color space (e.g., sRGB, ACES2065-1, Rec.709). Output color space (e.g., ACEScg, DCI-P3, Linear) You can browse or refer to the OCIO documentation for how to get the required config.ocio file. All the best.

​@@Isi-uT thank you! :) Im working on fix now but I did get further :) ...I at least manage to implement Ocio into comfy :)

@@michalgonda7301 Thanks. Glad to hear you made progress.

You can check if 'libtorchaudio.pyd' exists in your folder: 'F:\ComfyUI_Ai\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torchaudio\lib\' If not, then uninstall and reinstall torchaudio in your python_embeded environment using your CLI as shown below: F:\ComfyUI_Ai\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded>python.exe -m pip install torchaudio --force-reinstall If you find 'libtorchaudio.pyd', then it might be a compatibility issue. Check your torch, torchaudio, and cuda versions and use the following link to check if they are compatible: pytorch.org/get-started/previous-versions/ If they are not, then install a torchaudio version that is compatible with your torch and cuda versions based on the information in the above link. Please let me know the outcome.

@@Isi-uT Just seeing your message now. Thank you for taking the time to answer. I checked the path and it is there. I will check if they are compatible. I will let you know if it works. Thanks!

You're welcome. Looking forward to the outcome.

Thank you. I used a comfyUI workflow to convert a video to this comic-style animation.

Thank you.

Totally agree.

Thank you.

It seems the torch & torchaudio in your comfyui environment are not compatible. You can check the versions of both libraries and confirm if they are compatible by visiting this pytorch site: pytorch.org/get-started/previous-versions/ Also note that this project was tested successfully with pytorch versions 2.0.1 & 2.3.1 with compatible torchvision and torchaudio libraries. I don't know if other versions work well.

You can get them from this github repository: github.com/Isi-dev/ComfyUI-Animation_Nodes_and_Workflows

Thanks!

I did, but it looked better with the bg.

Thank you.

@@Isi-uT Buddy....nothing appear at video combine node..... any suggestion ??

You can check your CLI for any error.

Thank you.

I thought I had updated the code to handle this error. Did you install the nodes within the last 2days?

If you did not install the nodes within the last 2days, you can try updating the nodes.

UniAnimate Nodes for ComfyUI

Yes it can. I have done a 360 before,which was great for guys, but long-haired ladies tend to lose part of their back hair. As for bending, it's great for male characters, but it doesn't handle boobs well. I will upload my tests soon.

somehow it works when creating new workflow and not using ther one provided...both video and image..and fast! great

Good to know it now works. The img2lineart makes more computational demands than the other nodes when the deep_clean_up option is greater than zero. But setting it to zero results in lots of noise in the output. The alternative is to reduce the value of the details which, depending on the input image, could make most of the lines disappear.

I really can't say if it is better in terms of output quality because I haven't tested mimic motion. I actually came across the paper on mimic motion before hearing of unianimate, but the VRAM requirements of mimic motion put me off. I don't know if any improvement has been made in that area since then. Based on what I have read, unianimate can animate images faster and can handle far longer video frames than mimic motion.

I suggest you watch the following video by Mickmumpitz since I haven't done much related to your questions. He shared some workflows which might help you get started: th-cam.com/video/mEn3CYU7s_A/w-d-xo.html

You can get the workflows from this github repository: github.com/Isi-dev/ComfyUI-UniAnimate-W You will find two json files at the root of the repository: UniAnimateImg2Vid.json & uniAnimateReposeImg.json You can also find the workflows for the new nodes in the newWorkflows folder.

I have not yet used comfyui on the cloud, but based on what I have read & heard, it is generally true that you can do the things you mentioned. You can do as you like with models in colab since it uses your Google drive for storage, although I don't think you can run comfyui on colab with a free account. Other cloud services like comfy.icu & runComfy.com provide lots of models, workflows, and technical assistance that appear to make it easier to run comfyui than doing so locally. I don't know if there's provision to upload and modify models with these other platforms.

You should be able to render a video up to 20 sec if you have a high VRAM. The highest I have done is 4 sec. Someone mentioned rendering up to 370 frames which is a little above 12 sec for a 30fps video. The video quality depends on the inputs and the seed. The team behind the original project suggested using a seed of 7 or 11 in their project page. You have to keep experimenting with different seeds, and upscaling vids and images to find out what works best.

Thanks It was helpful. I'll try and response you when I get desire result

I really don't know because I haven't tried it. All I did was create a comfyui wrapper for the original project, so I don't know much about the unified model.

Please can you post the full error message let's see if we can resolve it.

Interesting! I never considered that. I think you should be able to use a Webcam node, although I'm not familiar with any.

@@Isi-uT I would give a try to see if would work

The xformers requirement makes the installation quite difficult and it took me sometime to get it working. Can you check for any error in your CLI?

@@Isi-uT Actually when i installed unianimate Custom node it reindstalled Pytorch to other version, and my comfy ui was not working,then i deleted the Unianimate Custom node and i reinsatlled the pytorch version: 2.3.1+cu121 ,then comfy started working BUT now i am Not sure to install the custom node again as i fear it will mess up my pytorch again,can you help plz..Thanks for replying.

@@ParthKakarwar-b7j it`s my problem too

The pytorch version in the requirements.txt file in the Unianimate custom node is 2.3.1 which is the same as what you currently have, so I am quite surprised that it would install another pytorch version. The only other thing is to ensure that the xformers version is 0.0.27.

An alternative is to have another comfyUI installed for unianimate to avoid dependency conflicts with other custom nodes. That's what I usually do for new custom nodes with requirements that conflict with the ones I already have.

Your setup looks okay except the VRAM. I have only tested it with 12GB VRAM and sometimes my system struggles depending on the input.

@@Isi-uT Can't i use Reposer atleast,with 4gb vram...? and when i install Unianimaate custom node it reinsatll the pytorch to new and my comfy UI dont work can you help me with this.Thanks for your reply.

The repose workflow might work. Considering your high RAM, I guess your shared VRAM is quite high. If setting up the node is interfering with the normal operation of your current Comfyui, then I suggest you install a separate Comfyui for unianimate so that you can focus on troubleshooting a specific installation. Sometimes I spend a whole day trying to make a workflow that worked in comfyui a month ago work again. The dependency conflicts among custom nodes and those introduced by updates is a serious issue.

Thank you. I will keep seeking.

The new workflows are : image2VidLong.json & reposeImgNew.json You can find both workflows in the newWorkflows folder in this github repository: github.com/Isi-dev/ComfyUI-UniAnimate-W

There's an implementation by someone else on Google Colab, although I haven't tried it. You can see this github repository: github.com/camenduru/UniAnimate-jupyter

I haven't come across this error. Please can you show the full error let me know the part of the code that threw it, and also the sizes of the picture and video so that I can try to reproduce the error.

@@Isi-uT Sure. I've used a variety of resolution videos as the input. All portrait orientation. I started with 720 X 1080 and I resized it down to 660x512 to see if that would help, but it didn't. I also tried different sized images, so that's not it. Here's what the CLI says... 2024-09-14 19:33:07,823 - dw pose extraction - INFO - All frames have been processed. 32 Ready for inference. Running UniAnimate inference on gpu (0) Loaded ViT-H-14 model config. Loading pretrained ViT-H-14 weights (X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\checkpoints/open_clip_pytorch_model.bin). X:\AI\ComfyUI3\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\functional.py:5504: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen ative\transformers\cuda\sdp_utils.cpp:455.) attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal) Restored from X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\checkpoints/v2-1_512-ema-pruned.ckpt Load model from (X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\checkpoints/unianimate_16f_32f_non_ema_223000.pth) with status (<All keys matched successfully>) Avoiding DistributedDataParallel to reduce memory usage Seed: 30 end_frame is (32) Number of frames to denoise: 32 0%| | 0/25 [00:00<?, ?it/s] !!! Exception during processing !!! The size of tensor a (64) must match the size of tensor b (96) at non-singleton dimension 4 Traceback (most recent call last): File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\uniAnimate_Inference.py", line 128, in process frames = inference_unianimate_long_entrance(seed, steps, useFirstFrame, image, refPose, pose_sequence, frame_interval, context_size, context_stride, context_overlap, max_frames, resolution, cfg_update=cfg_update.cfg_dict) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\tools\inferences\inference_unianimate_long_entrance.py", line 76, in inference_unianimate_long_entrance return worker(0, seed, steps, useFirstFrame, reference_image, refPose, pose_sequence, frame_interval, context_size, context_stride, context_overlap, max_frames, resolution, cfg, cfg_update) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\tools\inferences\inference_unianimate_long_entrance.py", line 467, in worker video_data = diffusion.ddim_sample_loop( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\tools\modules\diffusions\diffusion_ddim.py", line 825, in ddim_sample_loop xt, _ = self.ddim_sample(xt, t, model, model_kwargs, clamp, percentile, condition_fn, guide_scale, ddim_timesteps, eta, context_size=context_size, context_stride=context_stride, context_overlap=context_overlap, context_batch_size=context_batch_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\tools\modules\diffusions\diffusion_ddim.py", line 787, in ddim_sample _, _, _, x0 = self.p_mean_variance(xt, t, model, model_kwargs, clamp, percentile, guide_scale, context_size, context_stride, context_overlap, context_batch_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\tools\modules\diffusions\diffusion_ddim.py", line 719, in p_mean_variance y_out = model(latent_model_input, self._scale_timesteps(t).repeat(bs_context), **model_kwargs_new[0]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\modules\module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch n\modules\module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "X:\AI\ComfyUI3\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-UniAnimate-W\tools\modules\unet\unet_unianimate.py", line 522, in forward concat = concat + misc_dropout(dwpose) ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~ RuntimeError: The size of tensor a (64) must match the size of tensor b (96) at non-singleton dimension 4 Prompt executed in 41.08 seconds

I see, thanks. I will look into it.

The UNET model was getting the default resolution from the config file which could sometimes be different from the resolution used by the noise. I have updated the code to prevent the error. All you need to do is to add: cfg.resolution = resolution at line 240 in tools/inferences/inference_unianimate_long_entrance.py at line 234 in tools/inferences/inference_unianimate_entrance.py Please let me know how it goes. As for the VRAM requirement, I can't think of any other way to reduce it. The inference initially required at least 22GB of VRAM to run, but was reduced to around 10GB by transferring the clip_embedder and autoencoder computations to CPU. The only advantage I know for using the long version is to maintain consistency of appearance of the output.

@@Isi-uT YES! That worked perfectly and the results are awesome lol! Am now able to create MUCH longer videos. Just made one with 370 frames, and maybe I can do longer! Thanks so much for your nodes, your help and your hard work :)

Yes, you can upload any video and the movement will be transferred to the picture, but I cannot guarantee that it will be very professional. Sometimes, extra editing might be needed. Please note that this implementation is for the Windows OS. You can watch a video on the installation here: th-cam.com/video/NFnhELV4bG0/w-d-xo.html Or you can install the custom nodes with the ComfyUI Manager by searching for: ComfyUI-UniAnimate-W You can download the required models (about 14GB) from huggingface.co/camenduru/unianimate/tree/main and place them in '\custom_nodes\ComfyUI-UniAnimate-W-main\checkpoints' folder. In case you haven't done so, you can download comfyUI from this link: www.comfy.org/