Good day! Do you have a way to add the hand(fingers) to your recent full body tracking project? I am working on a Sign Language 3D avatar that can do the signs and the hand part specifically the fingers are badly needed. Can you reply when you have even just a bit of free time? We have a deadline to submit our prototype work and it is near. Thank you so much!
Hi! Is there a way to add some smoothening to the tracking output? It appears very jittery... Also, can it track the users position on the area? For example if the user is on the right side of the room or the left side of the room.
Hi, thanks for reaching out. Currently there is a basic form of smoothing implimented, a more robust smoothing could be implimented through unity. I would say that you should make sure your webcam input is high quality (this is usually the problem): make sure your full body is visible (head to toe), lighting is good, and area is free of clutter. I found doing these things helped greatly with improving tracking quality (see my demo video). Tracking the players position as being on the right/left is indeed possible by reading the landmark position in screen coordinates in python. I will look into implimenting this at somepoint after I am done the next video. Thanks for your support :)
Hi, could you submit an issue on github including your operating system, python version, unity version, and reproduction steps (or just message me if not familar with github). Thanks If anyone else has this problem please let me know
I got into trobble asign the avatar .. i have got some error i could not understand... Help me pls Assertion failed on expression: '!CompareApproximately(aScalar, 0.0F)' UnityEngine.Quaternion:Lerp (UnityEngine.Quaternion,UnityEngine.Quaternion,single) CalibrationData:Tick (UnityEngine.Quaternion,single) (at Assets/Scripts/CalibrationData.cs:25) Avatar:Update () (at Assets/Scripts/Avatar.cs:188) Assertion failed on expression: '!CompareApproximately(aScalar, 0.0F)' UnityEngine.Quaternion:Lerp (UnityEngine.Quaternion,UnityEngine.Quaternion,single) CalibrationData:Tick (UnityEngine.Quaternion,single) (at Assets/Scripts/CalibrationData.cs:25) Avatar:Update () (at Assets/Scripts/Avatar.cs:189) Assertion failed on expression: '!CompareApproximately(aScalar, 0.0F)' UnityEngine.Quaternion:Lerp (UnityEngine.Quaternion,UnityEngine.Quaternion,single) CalibrationData:Tick (UnityEngine.Quaternion,single) (at Assets/Scripts/CalibrationData.cs:25) Avatar:Update () (at Assets/Scripts/Avatar.cs:190)
hello, I can't install the pip, it said " 'pip' is not recognized as an internal or external command, operable program or batch file." what should I do :(
Hello, make sure to follow the install instructions in the video exactly. You likely missed the option to add to path. discuss.python.org/t/why-does-my-python-not-come-with-pip-installed/44960/2
Hi, I am busy at the moment, but will consider this in the future (reply if anyone else needs this too). In the meanwhile you can email me if you have specific difficulties in following the github steps.
Hi, yes this should be possible to some extent. I have been busy recently which is why content production has slowed down. I will be exploring this after my next video. Stay tuned! :)
Hello, Unity itself is crossplatform - so it should be possible. However you would need to do some work to replace the Python side into something integrated in Unity.
Yes, this is a little bit outdated. The new scene is called Calibration Scene. Besides that everything is the same. Feel free to delete the rollercosters and anything else that is unneeded.
@@onedayapp3534 Yes, the newer version is designed to be calibrationless. If you want to recalibrate/calibrate a new avatar reference: th-cam.com/video/DFHDnALoiQE/w-d-xo.htmlsi=6ItGTgB5_Uh7_48b I will perhaps re-add the old scene if this is causing confusion. Let me know
Hi, thanks - you can change the avatar and the same rig is NOT required. The only requirement is that you import the avatar as a humanoid in the unity import settings. All the models in the video are from different unity asset store publishers (different rigs).
Hi, at present it doesn't support SteamVR but it is something that could in the future. I've also thought similar, so we will see with time. Thanks for watching
Does this work on mobile? I'm trying to find a universal real time body tracking solution for android and iOS since the only other one is MARS which is iOS based.
This particular project unfortunately doesnt support mobile standalone. In theory some mobile + pc combination would be possible however. I will perhaps investigate this more in the future, but it is not currently a major priority. Thanks for watching
Hi, thanks for your comment. I will test mac soon, my understanding is that there might be some issues if you are not running windows. What are the errors you are getting? I will work on a fix.
@@ganthefan Thanks for the reply, the main.py keeps crashing. I googled and read something that said that Mac throws this error when UI interactions are not in the main thread? I don't know for sure...
@@IntaeHwang One thing you can quickly try is to disable debug mode in the settings.py. Otherwise I will need to do some further testing. Also feel free to join discord/email if you need more urgent support (I will fix it, however currently quite busy with exams).
@@ganthefan Hi I've hit the same wall, using Mac os M1 , getting the same error crash when launching main.py via python Pycharm. I believe disabling debug mode can be problematic? Any help with this will be super helpful, thanks
Hey, the problem is related to opencv not supporting multithreading on mac. Try out this user solution (untested, but reportedly works): github.com/ganeshsar/UnityPythonMediaPipeAvatar/issues/6
Hi, I dont know about making such a specific tutorial since others might not find much use in it but here's some general guidance: Ghost: probably would be a 3d model that moves around. Multiplayer: you can try using Unity Netcode for Gameobjects.
@@Kskospo99 For swapping the avatar follow the instructions posted: github.com/ganeshsar/UnityPythonMediaPipeAvatar The inability for the avatar to turn is a limitation of the approach I was using. I am working on upgrading these solutions to include motion around the area and more. During that upgraded release I'll also add a more detailed guide on swapping characters. Email if you are still having trouble: ganthefan101@gmail.com
@@super_robi_bros Nice, glad you found a solution. I tried to install from scratch and found that I needed to use python 3.10 for it to find the package.
Latest version, plug and play: th-cam.com/video/XfL8cgQN2aw/w-d-xo.htmlsi=h1lhDME5_uYStliG
Great work again!
Good day! Do you have a way to add the hand(fingers) to your recent full body tracking project? I am working on a Sign Language 3D avatar that can do the signs and the hand part specifically the fingers are badly needed. Can you reply when you have even just a bit of free time? We have a deadline to submit our prototype work and it is near. Thank you so much!
I gave up, can't run the py media.py part
Hi! Is there a way to add some smoothening to the tracking output? It appears very jittery... Also, can it track the users position on the area? For example if the user is on the right side of the room or the left side of the room.
Hi, thanks for reaching out. Currently there is a basic form of smoothing implimented, a more robust smoothing could be implimented through unity. I would say that you should make sure your webcam input is high quality (this is usually the problem): make sure your full body is visible (head to toe), lighting is good, and area is free of clutter. I found doing these things helped greatly with improving tracking quality (see my demo video).
Tracking the players position as being on the right/left is indeed possible by reading the landmark position in screen coordinates in python. I will look into implimenting this at somepoint after I am done the next video. Thanks for your support :)
Position tracking would be awesome!! Can't wait!
both projects getting error FormatException: Input string was not in a correct format.
Hi, could you submit an issue on github including your operating system, python version, unity version, and reproduction steps (or just message me if not familar with github). Thanks
If anyone else has this problem please let me know
Hi, i love your project and how easy you explain it. Is ist possible to transfer this to a other 3d person?
Yes, see th-cam.com/video/DFHDnALoiQE/w-d-xo.htmlsi=mPwhbb0FpH0u31xP
For an example.
I got into trobble asign the avatar .. i have got some error i could not understand... Help me pls
Assertion failed on expression: '!CompareApproximately(aScalar, 0.0F)'
UnityEngine.Quaternion:Lerp (UnityEngine.Quaternion,UnityEngine.Quaternion,single)
CalibrationData:Tick (UnityEngine.Quaternion,single) (at Assets/Scripts/CalibrationData.cs:25)
Avatar:Update () (at Assets/Scripts/Avatar.cs:188)
Assertion failed on expression: '!CompareApproximately(aScalar, 0.0F)'
UnityEngine.Quaternion:Lerp (UnityEngine.Quaternion,UnityEngine.Quaternion,single)
CalibrationData:Tick (UnityEngine.Quaternion,single) (at Assets/Scripts/CalibrationData.cs:25)
Avatar:Update () (at Assets/Scripts/Avatar.cs:189)
Assertion failed on expression: '!CompareApproximately(aScalar, 0.0F)'
UnityEngine.Quaternion:Lerp (UnityEngine.Quaternion,UnityEngine.Quaternion,single)
CalibrationData:Tick (UnityEngine.Quaternion,single) (at Assets/Scripts/CalibrationData.cs:25)
Avatar:Update () (at Assets/Scripts/Avatar.cs:190)
Is everything behaving as expected otherwise?
hello, I can't install the pip, it said " 'pip' is not recognized as an internal or external command,
operable program or batch file." what should I do :(
Hello, make sure to follow the install instructions in the video exactly. You likely missed the option to add to path.
discuss.python.org/t/why-does-my-python-not-come-with-pip-installed/44960/2
@@ganthefan ahh my bad. I added the path manually and it works now, thanks! :)
Hello
ganthefan can you please make tutorial on changing AVATAR , It wil really help to do my project.
Hi, I am busy at the moment, but will consider this in the future (reply if anyone else needs this too).
In the meanwhile you can email me if you have specific difficulties in following the github steps.
Hi, is it possible if we can move the 3D model, I mean when we walk the 3D model also moves, it doesn't stay in the middle.
Hi, yes this should be possible to some extent. I have been busy recently which is why content production has slowed down. I will be exploring this after my next video. Stay tuned! :)
Can this be built to iOS?
Hello, Unity itself is crossplatform - so it should be possible. However you would need to do some work to replace the Python side into something integrated in Unity.
The unity project from github doesnt look like youser, it has roller coasters
Yes, this is a little bit outdated. The new scene is called Calibration Scene. Besides that everything is the same.
Feel free to delete the rollercosters and anything else that is unneeded.
@@ganthefan but the new one doesnt show any text on calibration steps
@@onedayapp3534 Yes, the newer version is designed to be calibrationless. If you want to recalibrate/calibrate a new avatar reference: th-cam.com/video/DFHDnALoiQE/w-d-xo.htmlsi=6ItGTgB5_Uh7_48b
I will perhaps re-add the old scene if this is causing confusion. Let me know
just wanted to ask can we change the avatar and if yes does it need to have the same rig? btw great work
Hi, thanks - you can change the avatar and the same rig is NOT required. The only requirement is that you import the avatar as a humanoid in the unity import settings.
All the models in the video are from different unity asset store publishers (different rigs).
thank bro@@ganthefan
Hello, a question can connect with Steam VR?
Hi, at present it doesn't support SteamVR but it is something that could in the future. I've also thought similar, so we will see with time. Thanks for watching
Does this work on mobile? I'm trying to find a universal real time body tracking solution for android and iOS since the only other one is MARS which is iOS based.
This particular project unfortunately doesnt support mobile standalone. In theory some mobile + pc combination would be possible however. I will perhaps investigate this more in the future, but it is not currently a major priority. Thanks for watching
Hi thanks for the great video. Does this work on mac too? I am having trouble running the python code...
Hi, thanks for your comment. I will test mac soon, my understanding is that there might be some issues if you are not running windows. What are the errors you are getting? I will work on a fix.
@@ganthefan Thanks for the reply, the main.py keeps crashing. I googled and read something that said that Mac throws this error when UI interactions are not in the main thread? I don't know for sure...
@@IntaeHwang One thing you can quickly try is to disable debug mode in the settings.py. Otherwise I will need to do some further testing. Also feel free to join discord/email if you need more urgent support (I will fix it, however currently quite busy with exams).
@@ganthefan Hi I've hit the same wall, using Mac os M1 , getting the same error crash when launching main.py via python Pycharm. I believe disabling debug mode can be problematic? Any help with this will be super helpful, thanks
Hey, the problem is related to opencv not supporting multithreading on mac. Try out this user solution (untested, but reportedly works): github.com/ganeshsar/UnityPythonMediaPipeAvatar/issues/6
i want to make 2 leg in to 1 leg as like ghost(because of movement) and i want to make it multiplayer(cheep VR game). can you make a tutorial for that
Hi, I dont know about making such a specific tutorial since others might not find much use in it but here's some general guidance:
Ghost: probably would be a 3d model that moves around.
Multiplayer: you can try using Unity Netcode for Gameobjects.
@@ganthefan please make a tutorial how to change the character. and fix the issue - when i turn around the avatar not turn around parfrctly. please
@@Kskospo99 For swapping the avatar follow the instructions posted: github.com/ganeshsar/UnityPythonMediaPipeAvatar
The inability for the avatar to turn is a limitation of the approach I was using. I am working on upgrading these solutions to include motion around the area and more. During that upgraded release I'll also add a more detailed guide on swapping characters.
Email if you are still having trouble: ganthefan101@gmail.com
i cant install in using pip install mediapipe
What error are you getting?
I used to get that it cannot fibd the version but i installed it with thr whl file and it works really well
@@super_robi_bros Nice, glad you found a solution. I tried to install from scratch and found that I needed to use python 3.10 for it to find the package.
😎