Tutorial 1/3 - New Body Tracking Plugin for TouchDesigner - No Kinect Needed
ฝัง
- เผยแพร่เมื่อ 8 มิ.ย. 2024
- Hey! In this tutorial, we'll go over my new body-tracking plugin for TouchDesigner. It runs off the webcam and works on both Mac and PC without any setup. In this video I'll show you an overview of the component and how to start using it in your TouchDesigner Projects. I've refined this plugin based on a number of motion tracking projects I've done with the Azure Kinect and built this to have the setup process simple and affordable. I'm really looking forward to seeing what you create with this!
The component supports BlazePose(x, y, z positions) and MoveNet(x, y positions).
Link to the Project File is on my Patreon: / tblankensmith
Link to OBS for Website: obsproject.com/
0:00 Intro and Overview
2:20 Project Requirements & Component Overview
4:34 Selecting a Webcam
5:15 Selecting an ML model
9:13 Example 1 Moving a Circle with Your Hands
12:36 Example 2 Mapping Hand Velocity to Circle Size
14:24 Example 3 Mapping Distance Between Hands to Circle Size
16:50 Example 4 Instancing all the Position Data for Rendering
18:53 Example 5 Passing your Own Video in for Body Tracking
![Metallic Chaos in Touchdesigner - Using Feedback Blur and Displace TOP [Super Fast Tutorial Series ]](/img/n.gif)
I was *just* looking through the laste MoveNet project I built using your WebSockets tutorial to see how I could better embed it and switch out the tools for others (MediaPipe JS instead of TFJS, and face landmark tracking rather than poses). This is amazing, thank you!
You bring so much to this community. Thank you for sharing this
Thanks! that means so much :)
Amazing stuff. Not sure I will ever needed but very nice. Bravo!
this is such a nice tool, thank you soo much. looks like a gamechanger for me
You're an icon! Thank you!
I cannot wait to start! Pretty experienced with motion tracking and real time technologies but never tried TD before so I'm suuuuuper excited! Are the download files / all the beginner download files available in basic or another tier?
This is so awesome! I'm %100 gonna be hitting up your patreon for the project files, this is exactly the kind of stuff I've been trying to learn more about. I would love to implement this into a project. How many people is this able to track at once?
so good!
amazing!
absolute goat
OMG Mac users thank you !!
WOWWWWWWWW thankyou !!!!!
Hi Torin, do you think it's somehow possible to directly feed video data
(from touchdesigner) into the container without the hassle of going around OBS Studio? That would be a huge benefit! Besides that, what a great tool you've made! We've been searching for ages to find a directly integrated tracking solution in Touch Designer. Thank you so much for sharing this! Your tutorials are fantastic and it's really enjoyable to see what's possible in TD these days.😊
hey thanks so much! Since the body tracking is running in the browser all you need is some way to send your video into a virtual camera. It looks like there might be an app for syphon that makes a virtual webacam, but it's built on top of OBS. There should also be a virtual webcam app for Spout depending on what OS you're running on.
Hi Torin. Thanks a lot for this tutorial! Is it possible to create a hand reactive pixel distortion of any image with your plugin? similar to a watercolor/liquid movement
Cheers
never used obs with this before setup would be helpful.
Hi I just downloaded the file. But TouchDesigner cannot use my FaceTime camera on my Mac, and the body tracking window is always white and showing nothing. Can you help me to fix this problem ?
May I ask why the MediaPipe is not displayed and is black when I use the built-in camera in the Mac version?😭
What about feeding in multiple synchronized videos of the same scene(person) from different camera angles?
You could potentially stick together a video, pass in the video through OBS and enable MoveNet Multi pose
Hi, this is so cool! Is there any way to use video rather live camera?
Yeah if you're on windows you can you Spout Cam. If you're on Mac you can use Syphon Out to OBS and OBS will make a virtual webcam
Hi Torin! I was wondering if you have tried this on multiple people at once? How accurate is the body tracking?
There's MoveNet Multi Pose, but it's honestly pretty jittery and the player poses swap around pretty often. The Blaze pose for a single person works really well though and is surprisingly accurate
Hi Torin, I'm slowly getting to understand more about the plugin. I tried following this tutorial with the Media Pipe plugin, everything worked really great, just needed to adjust the lagging for easing but what I can't seem to find on that one is the lagging features in the recent version of the plugin. Does it contain the lagging features? if so do they have another name or where can I find it?
hey, yeah you can use a Lag CHOP to smooth out the motion. Alternatively, you can use a Filter CHOP, but turn down the filter width
thank you!@@blankensmithing
Thanks for this video. I have a dream of creating a nighttime projection mapped busking show with body tracked projection. If you know of any links or hardware/software ideas that would help me make my dream become reality, I greatly appreciate it.
Hi, thanks for a great tutorial! I am using windows and am new to OBS and when I launch the software I don't see any source for Spout or for Syphon, do you now how to get these to show up? Thanks a lot ;)
hey any chance you're on windows?
If you are here's how to install the plugin: docs.offworld.live/#/obs-spout-plugin/README
Hey! Thanks a lot. :) indeed I’m using windows. I will try it out. And just one question, if you’re using a Mac would you know if you need to install the plugin too?
@@jaimealyss I think it comes pre-installed on Mac
@@blankensmithing thanks so much! it's working great :)
Hello Torin! I tried your file on my Mac M1 but I can't get a video return in the main component. It seems that your cameras are still inside and don't take into account the one on my computer. I'm not sure what I'm saying, maybe it's something else haha.
Also, some options in the "Custom" tab are greyed out: Network adress, Websocket Port, Movenet Port and the MoveNet Camera Resolution is yellow. I don't know why, but if this helps you to find out what's going on.
@@louisfievet9341 Hey! I turned off those options in this component to prevent people from tampering with them. My original camera options will show up on the component until the website has loaded correctly and pulled your camera options. You can hit the "Reset" button on the high-level body tracking component and the website will take a moment to re-load the website. Also, I can pass along the private discord to you through patreon if you need any help getting this started
how can I add own scanned drawings to touchdesigner and be able to move them with MediaPipe'?
you can bring images in through a MovieFileIn TOP. And you can use the translate TOP to move them around. Use the CHOP Data for any position and attack it to the translation
cannot find project on your patreon
www.patreon.com/posts/new-body-plugin-86666663
hey can you guide me i installed the TD but its only running in bg how do i fix this
hey, I'm gonna need some more info on what you're trying to do and what's going wrong. What do you mean TD is running in the background?
@@blankensmithing thank you so much for the reply
Im using a hp pavilion gaming laptop with ryzen 5 4600h and 1650ti with 16gb ram
I downloaded and installed the full free version of touchdesigner but when im trying to launch ít nothing appears and when i open taskmanager and scroll down i can see touchdesigner running but nothing is showing.. i even downloaded thé earlier version but same problem.. help me please
@@Harshverma-hv7ck Hey I think you might have better luck reaching out to Derivative's support team: derivative.ca/contact-us
Have you tried updating your graphics drivers?
@@blankensmithing yes I tried updating too and no difference maybe its just à common problem ive seen online many laptop users are getting this problem btw thank you for your support
how did you create the last one?
Which one?
3/3 I guess @@blankensmithing It's the one I was hoping to use in a project!