why dont you have more subs/views? youre amazing at explaining your code and actually seem passionate unlike most coding tutorial creators. keep going!
You are AWESOME!!! Your tutorials are easy to follow and I love how you break down the concepts as well. As a point of feedback, I would love if you could mention some possible applications for noobs. Thanks again, Subscriber!
@@NicholasRenotte If you have a patreon, buy me coffee.. or any sort of donation links I will personally donate 100 dollars if you cover how to make a snapchat like camera experience in terms of putting filters on ones face.
Heya @@haibertbarfian6343, I have a Patreon but no need to donate for this particular use case. Got it on the list already, what types of filters are you thinking of? Background removal, adding sunglasses etc?
@@NicholasRenotte You are a SAINT MY FRIEND!!!! I was thinking of a snapchat like experience where you can put sunglasses, and all sorts of fun filters on a face using react native. Many people have joined the RN community and we need to show the prowess that RN has obtained over the years.. if snapchat filters are possible in RN.. almost anything is! I know that currently there are SDKs that allow this functionality on WEB.. but no one has done it on RN for mobile apps... this is going to be bananas.
Canva has a delay in relation to the video component. Hide the video and add the camera image to the canva, along with the points on the face. Congratulations on the video!
This is an amazing video, what could make it even better, is possible applications for the landmarks, for example, detecting certain dots, and based on it's movements, detect micro-expressions, that could be interesting
like video filter,is it possible to change image instead of drawing line ? in here only to show us how to detect face! to change line to image ! what is best way ? if y know , plz tell me know ㅠㅠ
This is amazing! There is no video out there showing an actual implementation of face filters using react native. Do you think you would be able to pull this off? I believe that is a highly coveted topic and there are absolutely 0 videos on it out there.
@@NicholasRenotte github.com/haibert/graphics-project I want to share my attempt at this with you incase it helps or gives you any ideas.. I have used the expo-face-detector to get a 2D X Y coordinate of the nose and map that to the rotation of this 3D box I have generated on top of the camera using the libraries mentioned below. It is by no means near what it needs to be.. because I only have 2D data using expo face detector... but with the face landmarks you should be able to do much more interesting things. I'M SO STOKED TO SEE YOUR VIDEO! "expo-gl": "~10.2.0", "expo-graphics": "^2.0.0", "expo-three": "5.4.0", (the package versions REALLY matter) github.com/haibert/graphics-project 🥴
The key points for me was to switch to video.videoWidth instead of video.offsetWidth & second one is resetting canvas width and height for each loop. Thanks for the simple learning
Not too sure about pushing out to blender but you could definitely export the x,y and z coords for use elsewhere? I've seen it used alongside three.js for 3d viz @KriGeta.
Great work Nicholas. But, there is a typo on line 2673 of utilities.js: 'TRIANGULATION[i * 3 + 2],u' doesn't need the 'u', so guessing a typo. Really enjoyed though - thank you.
@@NicholasRenotte Yeah! I would love to be able to work with the detected landmark points and trigger certain areas (mouth, eyes). Something like gaze recognition or emotion detection.
You could apply an object detection model over the top of the keypoint predictions or simply train based on the images to detect if your head is turned @Paritosh Singh.
Thanks so much @Haesun Shim, I sure am! I've done a Posenet basic video but have a few others planned with Tensorflow.JS as well: th-cam.com/video/PyxsziqatFE/w-d-xo.html
Hi Nick! I just completed your Media Pipe face mesh video and now i am here... one question is coming into my mind is ... can i build the same react app and integrate it with the MediaPipe one ?? please let me know if this is possible then i will try that integrating media pipe with this react app (I don't have much depth knowledge in react.js ). Also I feel media pipe is better than tf face mesh. Please let me know what are your answers/thoughts on this :)
Heya @Abhishek, yes you can definitely use medipipe on React in place of TF.js. I believe they're actually based on the same model so I think it's all about using what's easiest. I haven't tested out the js version yet but it's definitely on the list!
Is there a way to choose either to display or not the predictions based on a "confidence" level of the predicition? I mean, when I get off camera the model keeps trying to predict if there are any faces and draws over the objects in front of the camera.
Your tutorial is very good and I understand it completely. Thank you so much for sharing. How to use landmark coordinate point to generate animation avatar? In other words, how to use landmark coordinates to compare with virtual components such as eyebrows, and then select a matching eyebrow for image assembly;
Awesome work honestly you deserve much more recognition :D One question though when I cloned it and tried to run it it keeps crashing before the detection with an Unhandled Rejection (Error): method must be bilinear or nearest, but was undefined error :) thanks for your help
Ah, baby steps, the channel is growing thanks to all the love from you guys 😍!! Can you share your code? Where exactly is the error happening in the trace?
@@NicholasRenotte I just cloned the code from your repo and tried to start the react app :) (I installed all the prerequisites). Thank you fro your help
I love it the way you explained........ i have a query - ow do I overlay a 3D object using the TensorFlow "face-landmarks-detection" ML model? trying to overlay 3d object on face like glasses but unable to render glasses on facemesh. We also looked into mediapipe but did not find appropriate solution for java script (web).
@@NicholasRenotte Thanks for this video. I'm new the space but I also don't understand the triangulation data. It's hardcoded data. Why is there hard coded triangle data in there?
Hello! Thanks for the tutorial! What is your recommendation for the next steps to create a filter; such as distortion/displacement of the face or particles that attach to the mesh? I'm familiar with doing these things with a Kinect camera in Touchdesigner, but working in the browser is new territory for me. Thanks!
You could apply an viz overlay from the JS canvas inside of utilities.js! Where and how you adjust the canvas would be dependent on what points you want to attach the filter onto or from. Want me to make a video on this in the future?
Amazing Content man, spent a day trying to figure this out till I came across your video! Lastly, i was trying to use this model to use pre recorded videos. Do u have any resource/tips I could look into to change the input from webcam to pre recorded videos?
Heya @Tejas, does it have to be with .Js or are you comfy with Python as well? TBH it'll probs be easier to do with Python if you're wanting to work with pre-recorded vids. LMK I can chase it up for you!
Hey @@NicholasRenotte! Thank you for your response! I’m comfortable with Python but not sure how I would use it to implement this for pre recorded videos! And Yes! Can you please look into this/guide me?? Thanks a lot!!
@@tejaschoudhary3882 will add it to the list of videos. You should be able to do it with Media Pipe: google.github.io/mediapipe/solutions/face_mesh.html
@@NicholasRenotte Just had a look at it! The facemesh is implementable on Python but Iris tracking (separately on Media Pipe but included in the latest version of tensor flow) is unavailable on Python yet.
how did you get those triangulation metrics? did you calculate them or are those standard? I am completely new to this, I was able to follow all along till this point...
Great video Nicholas! I want to link this model to a react native based app. ( Hoping to determine whether a driver is drowsy or not) Can you refer me a direction to look for on how to do it? I'm kinda new for these environments. Thanks.
Hi this project was very interesting I only have one doubt as to how are u getting the TRIANGULATION metric data since I can't wrap my head around it if you can help or explain
awesome content dude. Can you please create a Video showing how you would test any ML modules whether it's the UI or the API level. I'm a test automation engineer and i'm trying to practice and get more exposure around AI and ML testing but i've struggled to find much content on youtube and i'm sure there are thousands of QA professionals feeling the same way while trying to learn the ins and outs of AI.
Hello! I loved your video, it was very helpful and easy to understand. Sorry for the inconvenience but would there be some way to recognize the face of a specific person and allow only this person to log in? Like in the phone?? Thank you very much for everything
Hi Nicholas, Do you have a plan to add an anomaly detection tutorial using ML ? This is not the same as single class model as I understand. Could they be the same ?
Got it planned, they can be treated differently. In fact scikit-learn has alternate algorithms for it: scikit-learn.org/stable/modules/outlier_detection.html
You're awesome man!👍👍👍 Instead of using create-react-app, can I use react native instead for this code to work? And how do I add my face mask image on the detected area of my face?
Sure can, but I think the webcam module is slightly different if you're doing it on RN. Working on the overlay code now, will share once I've got it up and running @Benjamin!
the new version of tensorflowjs is different and i am fully confused now! it doesn't know the model and i don't know how and from where it's requesting to download about 4MB models for each request!! so setinterval can not be the best practice for me! is there anyone here that can help me?!
Thank you so mush Nicholas, this is a great content, so much valuable. I have just one confusion that why the Triangulation matrix is used and how did it formed ?. Again thank you so much.
Loved the video it was awesome - do you know how you might remove the flickering of the mask? I'm trying to expand on this project to make a snapchat-like filter application and the flickering gets in the way.
Heya @Jacob, I haven't had a chance to fix it yet, I believe it's because the drawing is staged in two parts the dots and the lines. I'm working on a fix in the coming weeks though!
Awesome tutorial! btw, can you use the landmarks to get the actual measurement of your face? I'm trying to get the actual measurements of the eyes and nose something like that. its for my school project. TIA
Hi Nicolas, you are great, I have a big ask for your, can you help me to create a good app by React and Face Detection functionality in your video for my graduated work in university
Hi there... I am already working to annotate Nose manually without having any course but I wanted to know the job prospects... can you provide any info about it?? That would be a life saver
@@NicholasRenotte Thank you for your reply...I am in a profile where we annotate facial features and I am specifically for Nose annotation and its expression...so wanted to know how to proceed further
I asked a question about this in your other video - but now that I looked at your latest push to your repo. Does the use of useEffect and RequestAnimationFrame prevent some of "lag" the previous implementation had? I used that with blaze pose in my code and it seems to have fixed any sense of lag which is great. I would love to know why this works if this is what fixed that issue. Thanks again for the content.
this is an amazing tutorial i just had to open my laptop and try it for myself! thank you so much. i have a question, i followed your tutorial, however the package @tensorflow-models/facemesh is no longer available. do you have any suggestions on how to resolve this issue? once again, thank you!
Heya @Putri, looks like the model name has changed from facemesh to face-landmarks-detection (github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection). Was looking into it over the last couple of days. There's three quick changes required to get it to work (plus I've updated the code here: github.com/nicknochnack/FacialLandmarkDetection): 1. Install the new package // OLD Installation npm install @tensorflow-models/facemesh // NEW Installation npm install @tensorflow-models/face-landmarks-detection 2. Change the import // OLD Import import * as facemesh from "@tensorflow-models/facemesh"; // NEW Import import * as facemesh from "@tensorflow-models/face-landmarks-detection"; 3. Change the load // OLD Neural Network Load const net = await facemesh.load({ inputResolution: { width: 640, height: 480 }, scale: 0.8, }); // NEW Neural Network Load const net = await facemesh.load(facemesh.SupportedPackages.mediapipeFacemesh); 4. Last but not least update the detection function // OLD Detection const face = await net.estimateFaces(video); // NEW Detection const face = await net.estimateFaces({input:video});
awesome tutorial👍🏼 btw, do we have to use react webcam? how do we make it for phone user? I'm a junior student in IT department. I just figured out how to make phone app with unity (using C#).. and wonder how can I combine it with ai face detection
Heya @Rhena thanks so much! Nope you don't! I just found it easy to work with. Oooooh Unity is something I'm super interested in. Have you checked out Unity ML?
I'm trying to make an app where the user can try different shades of lipstick virtually.. I've found the annotations where it separates the lips out into 4 arrays, 3 of 11 and one of 10. Where I'm getting confused is how did you find the triangulation to draw the whole face mask? How would I map out drawing a texture for just the lips? Any advice or links to other tutorials would be greatly appreciated. It just doesn't seem to be well documented or I'm missing a keyword in my searches.
Yeah, I'm with you here, there isn't much out there on it. This isn't quite what you're looking for but it's a start: livecodestream.dev/post/detecting-face-features-and-applying-filters-with-javascript/ I've got some stuff planned for filters and textures planned!
Heya @Rubik, that's weird, it shouldn't be that slow. There might be some residual flickering but it shouldn't be running at 1FPS. Few things to check: 1. If you run the template code, is it still running slow? github.com/nicknochnack/FacialLandmarkDetection 2. Are there any differences between your app and the template? 3. How frequently are you seeing detections in the console? Is it more than one per second?
@@NicholasRenotte thanks for the feedback!! Apparently it is an issue with Chrome. On Firefox it is much faster, speed is like in your video. On Chrome however it is almost 10 times slower, the console logs correlate with what is being displayed on the screen (I am using your code from GitHub, no changes whatsoever) I think it may be a hardware acceleration issue, but the GPU usage is the same in both browsers, weird.
Which one is better py or javascript I am from the earth observation domain and I want to work on spatial using images for advanced applications in the Earth observation domain
Hi Nick, any idea how to implement custom Face Landmark Detection. example: only for lips, so i didn't need to track other landmark. maybe can u make video for make this custom models :D :thumsup:
Great video! You’re really great at explaining explaining everything but is it possible you could help me out? Because I’m getting stuck at the Load Facemesh part. After coding everything It’s throwing an error saying “load” is not a function…
I haven't even started watching and can already tell you deserve more views.
🙏 thanks so much @Rav Ada! Getting there!
Oh yes!
One of the best Facemesh tutorials I've seen so far online. Keep up the good work.
YESSS! Thanks so much @Thang!
why dont you have more subs/views? youre amazing at explaining your code and actually seem passionate unlike most coding tutorial creators. keep going!
Thanks soo much, I've only been going at it since May so we're only just getting started! But plenty more to come!!!
Best 30 minutes spent in my life
🙏🙏🙏 thanks so much @Alfredo!
WHOA YOU WILL RECEIVED ONE MILLIONS OF VIEWS THAT AMAZING CONTENT RIGHT HERE
Hell yeah, that's the goal, thanks so much 😊!!
Absolutely great tutorial...And believe me, you will get dozens of subscriber soon..
All the Best!
Heya @Sufyan Khan, thanks a million! More tutorials to come 👨💻!
As a backend dev, it wasn't difficult to understand the front-end part.
I like your teaching style. 👍
Thanks a bunch! So glad you enjoyed it.
You deserve more subs dude. I'm on your team now
Hell yeah!! Welcome to the team @Nico Leiva!
You are AWESOME!!! Your tutorials are easy to follow and I love how you break down the concepts as well.
As a point of feedback, I would love if you could mention some possible applications for noobs.
Thanks again,
Subscriber!
Solid suggestion, thanks @Tony!
Best Face Landmark Detection + Coding Tutorial that I've seen!
Thanks so much @Timothy! So glad you enjoyed it. Same Timothy from LinkedIn?
@@NicholasRenotte Yes! :) It's me. I just sent you another epic video idea.
@@TagenigmaLLC yesss, alrighty will check it out!
best tutorial ever
Thanks so much @mustafa! Definitely, in the works!
There is a lot to learn from you! You rock
I would love to see a continuation of this project. Maybe create a filters of your own? Rabbit ears, whiskers, etc...
Yah, working on it as we speak. Trying to get the hang of 3d rendering for it!
@@NicholasRenotte If you have a patreon, buy me coffee.. or any sort of donation links I will personally donate 100 dollars if you cover how to make a snapchat like camera experience in terms of putting filters on ones face.
Heya @@haibertbarfian6343, I have a Patreon but no need to donate for this particular use case. Got it on the list already, what types of filters are you thinking of? Background removal, adding sunglasses etc?
@@NicholasRenotte You are a SAINT MY FRIEND!!!! I was thinking of a snapchat like experience where you can put sunglasses, and all sorts of fun filters on a face using react native. Many people have joined the RN community and we need to show the prowess that RN has obtained over the years.. if snapchat filters are possible in RN.. almost anything is! I know that currently there are SDKs that allow this functionality on WEB.. but no one has done it on RN for mobile apps... this is going to be bananas.
@@haibertbarfian6343 hell yesss! Definitely, I'm going to get to work on it soon!
I was wondering if you can make a separate video on who to map the detected face landmarks on a 2D/ 3D avatar
Working on it @Sarina, definitely planned!
Love from India.... Hoping one day I can also figure it out by myself...... Ur videos are awesome..... Explained well..
Awesome! Let me know if you need anything along the way :)!
Canva has a delay in relation to the video component. Hide the video and add the camera image to the canva, along with the points on the face. Congratulations on the video!
Awesome suggestion @Diego!
This is an amazing video, what could make it even better, is possible applications for the landmarks, for example, detecting certain dots, and based on it's movements, detect micro-expressions, that could be interesting
like video filter,is it possible to change image instead of drawing line ? in here only to show us how to detect face! to change line to image ! what is best way ? if y know , plz tell me know ㅠㅠ
Thanks a lot man! looking for more quality content like these
Ayyy, cheers! Thanks so much!
I try your GitHub repo and build in 1 command " npm i " and it's working so amazing :)
Awesome!! Glad you got it running so easily, YESSS!
Looking forward to converting this to Angular!
Yess, let me know when you do, would love to take a look at the code @Libby!
This is amazing! There is no video out there showing an actual implementation of face filters using react native. Do you think you would be able to pull this off? I believe that is a highly coveted topic and there are absolutely 0 videos on it out there.
😉 stay tuned!
@@NicholasRenotte github.com/haibert/graphics-project I want to share my attempt at this with you incase it helps or gives you any ideas.. I have used the expo-face-detector to get a 2D X Y coordinate of the nose and map that to the rotation of this 3D box I have generated on top of the camera using the libraries mentioned below. It is by no means near what it needs to be.. because I only have 2D data using expo face detector... but with the face landmarks you should be able to do much more interesting things. I'M SO STOKED TO SEE YOUR VIDEO!
"expo-gl": "~10.2.0",
"expo-graphics": "^2.0.0",
"expo-three": "5.4.0", (the package versions REALLY matter)
github.com/haibert/graphics-project
🥴
This was super helpful thank you very much
The key points for me was to switch to video.videoWidth instead of video.offsetWidth & second one is resetting canvas width and height for each loop. Thanks for the simple learning
Anytime! Awesom suggestions.
I encounter error
Is there any way to export this data into 3D software like blender?
Not too sure about pushing out to blender but you could definitely export the x,y and z coords for use elsewhere? I've seen it used alongside three.js for 3d viz @KriGeta.
@@NicholasRenotte sounds intresting
Bro thanks very much for all these tutorials.
Great content man, thank you !!!
Great work Nicholas. But, there is a typo on line 2673 of utilities.js: 'TRIANGULATION[i * 3 + 2],u' doesn't need the 'u', so guessing a typo. Really enjoyed though - thank you.
Thanks so much @David, ah i'll update the repo, I think I might have thrown it in just before committing to GH.
Found your channel today! Great content!
🙏 Thanks so much @Yona, anything you'd like to see?
@@NicholasRenotte Yeah! I would love to be able to work with the detected landmark points and trigger certain areas (mouth, eyes). Something like gaze recognition or emotion detection.
@@yonakislinger6301 awesome stuff, was just planning iris detection today!
@@NicholasRenotte Amazing man! Really cool what and especially how you create!
@@yonakislinger6301 thanks so much, honestly appreciate it a ton 🙏!
Can you advise on how to detect for head turns using this model
You could apply an object detection model over the top of the keypoint predictions or simply train based on the images to detect if your head is turned @Paritosh Singh.
Look up optical flow models (Lucas-Kanade is a good one)
@@Luqmaan131 interesting, will take a look!
Awesome! Great tutorial. Are you planning for more tutorials using Posenet and React JS? (Using Teachable Machine)
Thanks so much @Haesun Shim, I sure am! I've done a Posenet basic video but have a few others planned with Tensorflow.JS as well: th-cam.com/video/PyxsziqatFE/w-d-xo.html
Thank you friend for sharing the teachings learned here will be of great help for my projects
Thanks so much for your effort!
Plenty more to come @Panda! 🙏
Hi Nick! I just completed your Media Pipe face mesh video and now i am here... one question is coming into my mind is ... can i build the same react app and integrate it with the MediaPipe one ?? please let me know if this is possible then i will try that integrating media pipe with this react app (I don't have much depth knowledge in react.js ). Also I feel media pipe is better than tf face mesh. Please let me know what are your answers/thoughts on this :)
Heya @Abhishek, yes you can definitely use medipipe on React in place of TF.js. I believe they're actually based on the same model so I think it's all about using what's easiest. I haven't tested out the js version yet but it's definitely on the list!
Is there a way to choose either to display or not the predictions based on a "confidence" level of the predicition? I mean, when I get off camera the model keeps trying to predict if there are any faces and draws over the objects in front of the camera.
how can i calculate the distance between eyes in cm
Haven't dug into it that far, I'd assume you could use a separate depth model and use some trig
i am facing error while installing tensoflow and facemesh. can anyone help
Your tutorial is very good and I understand it completely. Thank you so much for sharing.
How to use landmark coordinate point to generate animation avatar?
In other words, how to use landmark coordinates to compare with virtual components such as eyebrows, and then select a matching eyebrow for image assembly;
Working on this atm :) will share a tutorial once I'm done!
@@NicholasRenotte Is the tutorial out yet?
Can you try with external usb connected camera plz?
Can you make something about body detection? I wanna make a virtual try on
hey Nicholas! this is so cool! do you think I can use this on a tiktok clone I already built using dart?
19:51- 20:39 DONE but draw point can't show .. how to fix it ?
No error but why draw point didnt show in my webcam
Heya! Did you get anything when you console logged out the output e.g. console.log(faces)?
@@NicholasRenotte yesss, when i check inspect element , i see that console log work detect my face .
@@NicholasRenotte OKAAYY THANKS, oh god i forget ".video" in --> cons videoWidth = webcamRef.current.videoWidth;
@@m.imamfaruqbudiman2019 wait...so you got it working?!?
Awesome work honestly you deserve much more recognition :D One question though when I cloned it and tried to run it it keeps crashing before the detection with an Unhandled Rejection (Error): method must be bilinear or nearest, but was undefined error :) thanks for your help
Ah, baby steps, the channel is growing thanks to all the love from you guys 😍!! Can you share your code? Where exactly is the error happening in the trace?
@@NicholasRenotte I just cloned the code from your repo and tried to start the react app :) (I installed all the prerequisites). Thank you fro your help
@@alaeddineyacoub6225 you cloned it directly and it's still crashing? Just checking is your webcam activating when you start the app?
Dope tutorial, keep it going :)
Thanks so much @Diego Caumont! A bunch more coming. 😄
Thạnk you very much. Your video is wonderful!
I love it the way you explained........ i have a query - ow do I overlay a 3D object using the TensorFlow "face-landmarks-detection" ML model?
trying to overlay 3d object on face like glasses but unable to render glasses on facemesh. We also looked into mediapipe but did not find appropriate solution for java script (web).
Any response will be highly appreciated.
Heya @Sachin, I'll be working on this over the next few weeks! Stay tuned!
@@NicholasRenotte thanks...I will be eagerly waiting for this.
@@NicholasRenotte any updates on it?
@@romanpulich16 yah, about 50% of the way in the code. Had to knock off some other stuff first.
@@NicholasRenotte can't wait for it to come
Hi, first of all, great job explaining. But i have a question, where do you get the triangulation data? Keep it up!
The landmark map? I got it from the official TF Model repository. Need a link?
@@NicholasRenotte Thanks for this video. I'm new the space but I also don't understand the triangulation data. It's hardcoded data. Why is there hard coded triangle data in there?
Hello! Thanks for the tutorial! What is your recommendation for the next steps to create a filter; such as distortion/displacement of the face or particles that attach to the mesh? I'm familiar with doing these things with a Kinect camera in Touchdesigner, but working in the browser is new territory for me. Thanks!
You could apply an viz overlay from the JS canvas inside of utilities.js! Where and how you adjust the canvas would be dependent on what points you want to attach the filter onto or from. Want me to make a video on this in the future?
@@NicholasRenotte yes!
@@nixcampbell5399 done, added to the list!
@@NicholasRenotte kindly,,,
@@NicholasRenotte Can you please share the link
Amazing thanks for such great videos, Can you please tell me how to just extract eyes coordinates
Amazing Content man, spent a day trying to figure this out till I came across your video! Lastly, i was trying to use this model to use pre recorded videos. Do u have any resource/tips I could look into to change the input from webcam to pre recorded videos?
Heya @Tejas, does it have to be with .Js or are you comfy with Python as well? TBH it'll probs be easier to do with Python if you're wanting to work with pre-recorded vids. LMK I can chase it up for you!
Hey @@NicholasRenotte! Thank you for your response! I’m comfortable with Python but not sure how I would use it to implement this for pre recorded videos! And Yes! Can you please look into this/guide me?? Thanks a lot!!
@@tejaschoudhary3882 will add it to the list of videos. You should be able to do it with Media Pipe: google.github.io/mediapipe/solutions/face_mesh.html
@@NicholasRenotte Just had a look at it! The facemesh is implementable on Python but Iris tracking (separately on Media Pipe but included in the latest version of tensor flow) is unavailable on Python yet.
@@tejaschoudhary3882 ah got it!
how did you get those triangulation metrics? did you calculate them or are those standard? I am completely new to this, I was able to follow all along till this point...
No problems, great question!! They're standard triangulation metrics for that specific facemesh model.
Good tutorial! ❤️👍🏻
Thanks so much @Tennyson 🙏🙏!
Excelent tutorial as usual. Do you have a tip about how to implement liveness in face detection?
are you able to implement
Amazing tutorials!
Thanks so much @Jerry LEE 🙌 !
Great video Nicholas!
I want to link this model to a react native based app. ( Hoping to determine whether a driver is drowsy or not) Can you refer me a direction to look for on how to do it? I'm kinda new for these environments. Thanks.
Heya @Chris, awesome use case. I'm actually working on a RN solution as we speak, will let you know as soon as it's out!
@@NicholasRenotte Okay. looking forward to it. Cheers!
Hi this project was very interesting I only have one doubt as to how are u getting the TRIANGULATION metric data since I can't wrap my head around it if you can help or explain
Great video Nicholas. However, I want to plot only the landmarks of the eyes and nose. Do you know their landmark id?
Oooh, I don't have them fully mapped out unfortunately. I think best way to do it is just try mapping out a few at a time.
Thanks for sharing!
wow sir, i love the Beginning ||||
awesome content dude. Can you please create a Video showing how you would test any ML modules whether it's the UI or the API level. I'm a test automation engineer and i'm trying to practice and get more exposure around AI and ML testing but i've struggled to find much content on youtube and i'm sure there are thousands of QA professionals feeling the same way while trying to learn the ins and outs of AI.
nice, thanks for your videos, i will learn
i from indonesia
Really awesome video.
Thanks so much @Vijay!
Hey, great video! couldn't stop myself from implementing this! React-webcam is not working on localhost, do you have a way out? 🤔
Hmmm, any errors?
Do u have same playlist with Angular?
Hello! I loved your video, it was very helpful and easy to understand. Sorry for the inconvenience but would there be some way to recognize the face of a specific person and allow only this person to log in? Like in the phone?? Thank you very much for everything
i am curious if you could make a facial recognition using this face mesh?? hmm just like loading or storing dataset as 3d???
Great Tutorial! Can you please come up with react-native example of same tensor flow model using expo-gl?
Ya, in the works @Yash, will let you know once it's out!
@@NicholasRenotte Thanks! Appreaciated
@@yashthakkar3453 anytime!!
Thank you to make me gradueated
how can we use this to make a blink detector/ face verification/ yawn detector
Hi Nicholas, Do you have a plan to add an anomaly detection tutorial using ML ? This is not the same as single class model as I understand. Could they be the same ?
Got it planned, they can be treated differently. In fact scikit-learn has alternate algorithms for it: scikit-learn.org/stable/modules/outlier_detection.html
@@NicholasRenotte Great...! W
ill play with it until you upload your tutorial.
@@PUBUDUCG awesome, let me know how you go!
You're awesome man!👍👍👍
Instead of using create-react-app, can I use react native instead for this code to work? And how do I add my face mask image on the detected area of my face?
Sure can, but I think the webcam module is slightly different if you're doing it on RN. Working on the overlay code now, will share once I've got it up and running @Benjamin!
@@NicholasRenotte Yes, I think instead of webcam, you'll use RNCamera. i will be waiting for the overlay video. Thanks for your kind response.
@@benjaminikwuagwu6469 brilliant! Will let you know as soon as it's out!
@@NicholasRenotte I really appreciate :)
@@benjaminikwuagwu6469 anytime!
can you make a face recognition program, please?
Yep! Siamese networks for recognition is coming up!
the new version of tensorflowjs is different and i am fully confused now! it doesn't know the model and i don't know how and from where it's requesting to download about 4MB models for each request!! so setinterval can not be the best practice for me! is there anyone here that can help me?!
did you find the solution if yes help me plz
How can I train ...facial landmark datapoints ... for some pattens ...
Could use a separate ML model to perform classification. Similar to what we do in this video: th-cam.com/video/We1uB79Ci-w/w-d-xo.html
Thank you so mush Nicholas, this is a great content, so much valuable. I have just one confusion that why the Triangulation matrix is used and how did it formed ?. Again thank you so much.
Thanks for sharing bro, btw how to get blink eye, node, turn head etc (gesture face)
thanks before
Loved the video it was awesome - do you know how you might remove the flickering of the mask? I'm trying to expand on this project to make a snapchat-like filter application and the flickering gets in the way.
Heya @Jacob, I haven't had a chance to fix it yet, I believe it's because the drawing is staged in two parts the dots and the lines. I'm working on a fix in the coming weeks though!
@@NicholasRenotte Thank you for the help!
@@jacobpiirsalu2862 anytime, will let you know once it's updated!!
Awesome tutorial! btw, can you use the landmarks to get the actual measurement of your face? I'm trying to get the actual measurements of the eyes and nose something like that. its for my school project. TIA
Ya, I believe you could, will add it to the upcoming tutorials @Randy!
@@NicholasRenotte alright I will watch out for that, thanks for the response! thanks in advance for the tutorial
@@randycook6802 anytime, let me know if there's anything else you'd like to see. I'm always updating the video list!
@@NicholasRenotte i think you need a special webcam for that with 2 cameras
Sir is this same tutorial available for TensorFlow.js with React ?
great stuff man!
Thanks so much @scott!
Hi Nicolas, you are great, I have a big ask for your, can you help me to create a good app by React and Face Detection functionality in your video for my graduated work in university
Have you tried loading 3d objects on canvas with it?
Actually got some 3D rendering stuff planned!
Do you have an idea how to convert coordinates from Tensorflow to coordinates for 3d canvas to position 3D model according to them?
Heya @Roman, the coordinates you get back are actually 3d coordinates already!
@@NicholasRenotte Yeah but if I set those coordinates to the model position, it goes out of the canvas borders.
@@romanpulich16 ah, got it, you would need to leverage a 3D canvas. Want me to make a video on it?
@@NicholasRenotte Yeah would be awesome!
@@romanpulich16 got it, added to the list!
Hi there... I am already working to annotate Nose manually without having any course but I wanted to know the job prospects... can you provide any info about it?? That would be a life saver
Job prospects for data science?
@@NicholasRenotte Thank you for your reply...I am in a profile where we annotate facial features and I am specifically for Nose annotation and its expression...so wanted to know how to proceed further
@@Resfeber123 oh, I would be delving deeper and learning how to build models. Tons of opportunity in the DS space but it is competitive.
Hey nicholas, is it possible to do that but with python tensorflow?
And also opencv?
Sure can, check out MediaPipe! I've got a series on it coming soon @Bernardo.
@@NicholasRenotte you should have more subs
@@bernardoolisan1010 the YT algo works in mysterious ways! Either way, I'm still putting out videos, I really enjoying making them for you guys!
Is it possible to use this for facial expression recognition, with data point used for data science /machine learning?
I can detect which face it is based on these points? Like save them in a database an use for face detection
I asked a question about this in your other video - but now that I looked at your latest push to your repo. Does the use of useEffect and RequestAnimationFrame prevent some of "lag" the previous implementation had? I used that with blaze pose in my code and it seems to have fixed any sense of lag which is great. I would love to know why this works if this is what fixed that issue. Thanks again for the content.
I was wondering the same thing
Hi, can you help me with text detection in an image using tensorflow js or opencv js
this is an amazing tutorial i just had to open my laptop and try it for myself! thank you so much. i have a question, i followed your tutorial, however the package @tensorflow-models/facemesh is no longer available. do you have any suggestions on how to resolve this issue? once again, thank you!
Heya @Putri, looks like the model name has changed from facemesh to face-landmarks-detection (github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection). Was looking into it over the last couple of days. There's three quick changes required to get it to work (plus I've updated the code here: github.com/nicknochnack/FacialLandmarkDetection):
1. Install the new package
// OLD Installation
npm install @tensorflow-models/facemesh
// NEW Installation
npm install @tensorflow-models/face-landmarks-detection
2. Change the import
// OLD Import
import * as facemesh from "@tensorflow-models/facemesh";
// NEW Import
import * as facemesh from "@tensorflow-models/face-landmarks-detection";
3. Change the load
// OLD Neural Network Load
const net = await facemesh.load({
inputResolution: { width: 640, height: 480 },
scale: 0.8,
});
// NEW Neural Network Load
const net = await facemesh.load(facemesh.SupportedPackages.mediapipeFacemesh);
4. Last but not least update the detection function
// OLD Detection
const face = await net.estimateFaces(video);
// NEW Detection
const face = await net.estimateFaces({input:video});
@@NicholasRenotte Thanks a lot for this update!
awesome tutorial👍🏼 btw, do we have to use react webcam? how do we make it for phone user? I'm a junior student in IT department. I just figured out how to make phone app with unity (using C#).. and wonder how can I combine it with ai face detection
Heya @Rhena thanks so much! Nope you don't! I just found it easy to work with. Oooooh Unity is something I'm super interested in. Have you checked out Unity ML?
How To Convert Your Photo To Avatar Though Tensorflow Face Landmark Detector?
I'm trying to make an app where the user can try different shades of lipstick virtually.. I've found the annotations where it separates the lips out into 4 arrays, 3 of 11 and one of 10. Where I'm getting confused is how did you find the triangulation to draw the whole face mask? How would I map out drawing a texture for just the lips? Any advice or links to other tutorials would be greatly appreciated. It just doesn't seem to be well documented or I'm missing a keyword in my searches.
Yeah, I'm with you here, there isn't much out there on it. This isn't quite what you're looking for but it's a start: livecodestream.dev/post/detecting-face-features-and-applying-filters-with-javascript/ I've got some stuff planned for filters and textures planned!
Great vid!
Any idea why my face detection is slow as hell? Like it is detecting at 1 frame per second, and my pc is pretty high end
Thanks!
Heya @Rubik, that's weird, it shouldn't be that slow. There might be some residual flickering but it shouldn't be running at 1FPS. Few things to check:
1. If you run the template code, is it still running slow? github.com/nicknochnack/FacialLandmarkDetection
2. Are there any differences between your app and the template?
3. How frequently are you seeing detections in the console? Is it more than one per second?
@@NicholasRenotte thanks for the feedback!!
Apparently it is an issue with Chrome. On Firefox it is much faster, speed is like in your video. On Chrome however it is almost 10 times slower, the console logs correlate with what is being displayed on the screen (I am using your code from GitHub, no changes whatsoever)
I think it may be a hardware acceleration issue, but the GPU usage is the same in both browsers, weird.
@@TheGluv4e super weird! I'm running on Chrome in the video as well.
Which one is better py or javascript I am from the earth observation domain and I want to work on spatial using images for advanced applications in the Earth observation domain
Hey! Thank you for a great tutorial. Is there a way to save the facemash network alone as png or something we can use later on?
Yep, probably easier to do it with Python and cv2.imwrite th-cam.com/video/pG4sUNDOZFg/w-d-xo.html
@@NicholasRenotte Awesome. Thank you for reply. 😊
@@fassoyangce1075 anytime!
Hi Nick, any idea how to implement custom Face Landmark Detection. example: only for lips, so i didn't need to track other landmark.
maybe can u make video for make this custom models :D :thumsup:
Heya @Andi, you can filter out what you show by isolating the keypoints. Although it might be easier to train a custom segmentation model for this!
Okai. Thank you Nick. Could you refer me to video how to make custom model for lips segmentation?🙏
How can i training model , can you explain using python and open CV
Great video! You’re really great at explaining explaining everything but is it possible you could help me out? Because I’m getting stuck at the Load Facemesh part. After coding everything It’s throwing an error saying “load” is not a function…