I like what you're doing with the video editing. Having the robot and the data visible at the same time is great for having the less technologically inclined understand what's going on.
The pixy cam is a great addition. I don't know enough about it to know limitations, facial recognition and all that, but, since this robot will be interacting with the public at cons, I imagined the pixy recognizing an iron man mask. Whoever wanted to interact with the ultron would put on an special iron man mask the pixy is programmed to recognize. directional speech perception/recognition and then associating that speech with the mask wearer is another host of challenges, but I could imagine those interacting with the robot really enjoying role playing as iron man talking to ultron.
MrCBroz that would be cool. it might also be good to add different masks that cause different reactions the wearer. for example a different voice line taken from the movie would be played depending on what colour the mask is as all the avengers have fairly distinct colours
Depending on how complex the colour configuration can be, a simplified pixelated image of the mask might be enough to get the real thing recognised by the camera (just with the positions of the different colours)
Great work James. Have you thought about possibly integrating a sound board so that when someone pushes an arm or something, Ultron would say something like, "Please don't do that." Just a thought. Keep up the great work.
I think that someone would be able to splice a speech pattern together from most of James Spader shows and movies. Don't know how hard it would be, but it would be awesome.
I've been passively watching every third video or so of this. It's getting so incredible! It will be a great day when he posts the video of this robot finished! Can't wait!
Nice to see his reaction to your poking him. It's cool because the code can be expanded upon, such as: if (a certain amount of pokes within a certain duration){ gets annoyed and reacts in another way}. It's the kind of coding I love because AI coding is very revealing about the systems our own brain uses to make decisions.
The camera interface makes it trivial to make some rudimentary stereo capability functions. Like only track objects that are close by which you can determine if the X on both objects with similar width and height are apart by some number. Or.. perhaps it's creepy enough as it is!
Looking good :) Will be cool to see it in puppet mode. Lifelike animations are tough to script, but being able to capture gestures via the imu suit should help immensely!
for the tracking - you could print out little circles with the avengers heads (set it to the specific colour sets) then he follows specific avengers And you can have them in a priority order
for the holographic control thing, you could use something like clear plastic tubing with a pot at one end and an led at the other, so when you hold the 'hologram' and twist it, it would move the pots slightly. there would have to be something that held the 'hologram' in position though.
you should definitely have 2 cams in either eye holes, make them look the other direction slightly so you are seeing as much horizontal space as you can to increase the size it can track things, and maybe make a part of the code when it looses track of something it will look into the direction it was heading and then head back or maybe look around some more.
this got really creepy really fast with the object tracking! properly unsettling at this point, i love it! :D what a relief it's got no legs so it cannot actually *go* on a rampage. :)
James been too long since I caught up with this project, wow it's really taking shape. The sensors seem great and the brain is a great idea. I'll be back (sorry bad pun) for more. ta.
That camera is awesome! You could train it to track a certain color of t-shirt and whenever a person at a convention walks by with that same color of shirt...Ultron will creepily stare at them! xD
James, just as a suggestion, you could set the default lighting to blue, when he's happy it turns yellow, but when he gets annoyed/hungry/sad etc. it turns red. Just my thoughts.
please just make him look at you and have him say "stop pushing me around for a second,will you?" if you push on his joints to often in a given timespan ^^
and build a sensor into the external brain,so when someone tries to touch it,all of ultrons LEDs turn red,he turns towards the brain and shouts: "don't you dare touch my brain!"
Am i the only one that would of rather it have an Ultron drone style head? Don't get me wrong, you did amazing with the head prints, just my personal preference. You're doing great mate.
You should probably use the lowest part of someone to recognize people instead of the biggest, since some people are bigger and some are smaller. For example, a kid who is very close would seem to be further away than an adult who's a bit further away. Though, this only works if the ground is flat. Anyway, nice progress!
What about some sort of IR camera in addition to the Pixy Cam? If Ultron could track warm objects of a certain size, that would let it detect humans that walk up to it with less complexity than face tracking. I could picture it sitting in a booth at a convention, moving its head back and forth, apparently making eye contact with passers by.
instead of having 10 var with a number at the end, I would use lists. then if you have to affect abc1 = def1 for each, you can use a for loop. for i = 0 i < list.lenght i++ do abc[i] = def[i] Just my thoughts as a programmer ;)
This is a bit of an insane idea but if you did redesign the head to have the camera in the eye, and got the new firmware you should have two so it could have peripheral vision. Idk if the software could work like that, but i thought id throw the idea at you, I mean that's sorta what comments are for right?
EZ Robot works great for any type of robot. My humanoid robot Dave uses an EZ Robot board to control all of his servos and a camera for face tracking. EZ Robot has amazing voice recognition and speech. My robot Dave understands so many commands that I don't need to touch my computer mouse or keyboard to control him.
It's not too hard to implement actual face recognition. I did something earlier this year, which used facial recognition to simulate The Machine from Person of Interest, in Python with opencv. You could run it on a Pi or one of the other mini Linux boxes you mentioned.
for the ai brain viz, try a projector through a mist or fog. use pulses and orbits, or a laser at crumpled reflectic mylar or aluminum, through some particle in the air...great worx!
Mm, in regards to the light colors, will it be screen-accurate red at all times, changing based on sensor input, or will it be changing according to "mood"?
wow, i just happened to find your channel mentioned in the comments on a video i was watching. this is amazing!!! my major is robotics engineering. currently Im doind my prereqs though so i havent gotten to do anything with robotics just yet. might i ask how you learned all this? did you teach yourself or go through university??
Would be interesting to add some sort of weapon (nerfgun?) to one of the arms that can be put to "sentry mode" in which it tracks a color-coded target...
I must say, just wow, i watch all of your builds from beginning and i am a bit jealous, that i don't have the time and money to do something like that. I wish i can meet you and talk over this Project and make some brain storming :3 But iam from germany, so this will never happen xD You have all my respect for this project and hope you get some invites from big companies that want to hire you for this awesomeness :)
ooh wodner if you can put in a second pixy cam and give Ultron binocular vision:D and maybe cannibalize a kinect for its depth sensor so it has full 3d vision:D and on a tangent i just popped in from watch a documentory about robots filmed in 1999 and how Asimo was the pinnical of robotics due to its ability to walk on two egs and the video after this one was James' updat eon his gonk droid. technology has advanced so much that what took a corperation millions of dollars to research and develope can now be knocked out by a vlogger with a budget around $1000 at best:D what a time to be alive.
Does the camera have some library for pre-trained facial recognition? If not, do you reckon it'd be hard to hack in some facial recognition? It would be awesome if it could track the faces nearest to it, and look around the room by default as people walk past or stare at him.
Very cool james! Ultron is becoming one of my favorites. Do you think the pixy cam could track a face by looking for 2 white objects of the same size and close together(eyes?) Probably not worth the effort if they are coming out with their own firmware for it though.
I see a problem that might happen, You might of addressed it when you shook Ultron but if the whole bot is shook and the sensors freak out wouldn't it make it look like Ultron would be having a seizure? and just not stop shaking? or do you have It set up so it won't do that? P.S. great work so far! i live in the U.S. and i hope to see your work some day in the flesh.
While the pixy is a very good option because of its usability and simplicity, that's also it's downfall due to its limits, as you said it's rudimentary. Personally I would use openCV in Java, C or in Python if you wanted to use it for face tracking like you said. In saying this I understand your hesitation in writing software.
You could use something like google tts(text to speech) to make him talk! (I don't think that would be possible with arduinos but it would still be fun)
Sketch It D.I.Y Just like Tony Stark , before he create ultron , he got a lot of suits too , and he is avenger, so it doesn't matter did he got a suit .
Sketch It D.I.Y And tony already learned from the mistake after create Ultron , he let Ultron find the best way to protect the earth by "itself" . but it think destroy humanity is the way to peace . Now he knew let the AI have a self-awareness maybe dangerous . And now XRobots try to create new Ultron , we don't know what will XRobots do to Ultron , =) , And it names ULTRON , it dosent seem good for The Avengers XD. Its just a joke , dont have to too serious.
Once this is fully completed (As in, as much as possible), are you planning on making this fully autonomous? Or have it controlled by a user? And will you be adjusting it to minimising the lag/input time between the user's movement and the robot's movement? Honestly mate, it already looks bloody amazing. And slightly creepy.
Please tell me that you're going to put in a color recognition pattern so that Ultron will prioritize giving Ironman cosplayers some added guff at events.
Have you thought of doing a div nights at Freddy's SpringBonnie suit? It supposedly can go from a animatronic to a wearable suit. And I would love to see one
just an idea, but you should look into batch files and .vbs files. they're super easy to learn and they also can make great interfaces. you could also make him speak with one and code in what you want him to say. then it uses the computer's narrator voice to say it. if you go online and look up "how to make a batch file speak" and click on the one on instructables you can just copy and paste from there. the only thing is on Windows 7 it uses a female voice but pretty much everything else is a male voice (XP, W8, W9, W10, etc.) if you're interested please tell me and I'll put some code that I've made that I think can work for Ultron. I'm not sure how to make it work in tangent with the robot, but I think with your arduino skill you could easily figure it out. Good luck on the project either way!
I like what you're doing with the video editing. Having the robot and the data visible at the same time is great for having the less technologically inclined understand what's going on.
The pixy cam is a great addition. I don't know enough about it to know limitations, facial recognition and all that, but, since this robot will be interacting with the public at cons, I imagined the pixy recognizing an iron man mask. Whoever wanted to interact with the ultron would put on an special iron man mask the pixy is programmed to recognize.
directional speech perception/recognition and then associating that speech with the mask wearer is another host of challenges, but I could imagine those interacting with the robot really enjoying role playing as iron man talking to ultron.
MrCBroz that would be cool. it might also be good to add different masks that cause different reactions the wearer. for example a different voice line taken from the movie would be played depending on what colour the mask is as all the avengers have fairly distinct colours
Also a simple hat would work
Depending on how complex the colour configuration can be, a simplified pixelated image of the mask might be enough to get the real thing recognised by the camera (just with the positions of the different colours)
REDESIGN OF HEAD CONFIRMED!
Me and my OCD are so happy!
Hey, James maybe u can try and check some Lego mindstorms sensors out for the vision because it is quite easy to program
MrBaronmoll my ocd and I*
Great work James. Have you thought about possibly integrating a sound board so that when someone pushes an arm or something, Ultron would say something like, "Please don't do that." Just a thought. Keep up the great work.
yes my lord... yes my lord... STOP POKING ME!
I think that someone would be able to splice a speech pattern together from most of James Spader shows and movies. Don't know how hard it would be, but it would be awesome.
It would only have to run through a distortion code to give it a machine sound. Just an idea.
Joshua Kendrick MATRIX Creator has an 8 mic array and onboard FPGA... people are doing some amazing things with it...
Amazing! this project is really starting to progress quickly and I love where it is going! Keep up the great work, as always, James.
I find the camera vision so exciting! I didn't even expect making the head follow objects would work that well.
I've been passively watching every third video or so of this. It's getting so incredible! It will be a great day when he posts the video of this robot finished! Can't wait!
Incredible work James, this is such an exciting project!
Nice to see his reaction to your poking him. It's cool because the code can be expanded upon, such as: if (a certain amount of pokes within a certain duration){ gets annoyed and reacts in another way}.
It's the kind of coding I love because AI coding is very revealing about the systems our own brain uses to make decisions.
Holy McAwesome! This was a really interesting episode. Ultron's interactive feedback is going to be SO COOL.
The camera interface makes it trivial to make some rudimentary stereo capability functions. Like only track objects that are close by which you can determine if the X on both objects with similar width and height are apart by some number.
Or.. perhaps it's creepy enough as it is!
I think making a separate interactive "brain" is a brilliant idea for a social robot! Keep up the good work James! :)
Awesome stuff! Thanks for taking time to walk through the code. Really impressed with the Pixie cam.
James - you are a genius. Thanks for sharing these videos!
You got my subscribe. I just hope this project isn't the beginning of the end!
I remember the first pixy kickstarter, probably going to get one when I have a use for it.
Looking good :)
Will be cool to see it in puppet mode. Lifelike animations are tough to script, but being able to capture gestures via the imu suit should help immensely!
for the tracking - you could print out little circles with the avengers heads (set it to the specific colour sets) then he follows specific avengers And you can have them in a priority order
for the holographic control thing, you could use something like clear plastic tubing with a pot at one end and an led at the other, so when you hold the 'hologram' and twist it, it would move the pots slightly. there would have to be something that held the 'hologram' in position though.
you should definitely have 2 cams in either eye holes, make them look the other direction slightly so you are seeing as much horizontal space as you can to increase the size it can track things, and maybe make a part of the code when it looses track of something it will look into the direction it was heading and then head back or maybe look around some more.
This is getting scary! Glad that you didn't make it with legs :P
Awesome job James!!!
wow i love the head tracking with the colors
Brilliant and fun! this is my favorite project yet!
this got really creepy really fast with the object tracking! properly unsettling at this point, i love it! :D what a relief it's got no legs so it cannot actually *go* on a rampage. :)
James been too long since I caught up with this project, wow it's really taking shape. The sensors seem great and the brain is a great idea. I'll be back (sorry bad pun) for more. ta.
That camera is awesome! You could train it to track a certain color of t-shirt and whenever a person at a convention walks by with that same color of shirt...Ultron will creepily stare at them! xD
James, just as a suggestion, you could set the default lighting to blue, when he's happy it turns yellow, but when he gets annoyed/hungry/sad etc. it turns red.
Just my thoughts.
You have a good idea to collect data to your 🤖 and your mini harddrive looks awesome
please just make him look at you and have him say "stop pushing me around for a second,will you?" if you push on his joints to often in a given timespan ^^
and build a sensor into the external brain,so when someone tries to touch it,all of ultrons LEDs turn red,he turns towards the brain and shouts: "don't you dare touch my brain!"
Das Nutzkonto Maybe have his voice get darker with that action too.
Yes, yes, yes, this needs to be a thing.
It lights in his head flicker when you were talking about it recognizing faces
Am i the only one that would of rather it have an Ultron drone style head? Don't get me wrong, you did amazing with the head prints, just my personal preference. You're doing great mate.
You should probably use the lowest part of someone to recognize people instead of the biggest, since some people are bigger and some are smaller. For example, a kid who is very close would seem to be further away than an adult who's a bit further away. Though, this only works if the ground is flat. Anyway, nice progress!
If you want to use openCV, there is a visual programming interface called GRIP that is extremely easy to use
I would like to formally welcome our new robot overlord. Hail Ultron!!
This is a big project. I am impressed at your ambition and skills. I am still trying to figure out arduino code writting.
There are no strings on me. Only bluetooth connections.
just a typing error in 12:18 --> "sla5 = read6" should be "sla6" i guess
What about some sort of IR camera in addition to the Pixy Cam? If Ultron could track warm objects of a certain size, that would let it detect humans that walk up to it with less complexity than face tracking. I could picture it sitting in a booth at a convention, moving its head back and forth, apparently making eye contact with passers by.
21:30 Ultron blinked.
Neat.
instead of having 10 var with a number at the end, I would use lists. then if you have to affect abc1 = def1 for each, you can use a for loop. for i = 0 i < list.lenght i++ do abc[i] = def[i] Just my thoughts as a programmer ;)
This is very cool! Was just thinking you should build a General Grievous.
This is a bit of an insane idea but if you did redesign the head to have the camera in the eye, and got the new firmware you should have two so it could have peripheral vision. Idk if the software could work like that, but i thought id throw the idea at you, I mean that's sorta what comments are for right?
I think you would be interested in ez-robot.
I use it in my astromech. adding their camera has face, color, and object tracking.
EZ Robot works great for any type of robot.
My humanoid robot Dave uses an EZ Robot board to control all of his servos and a camera for face tracking.
EZ Robot has amazing voice recognition and speech. My robot Dave understands so many commands that I don't need to touch my computer mouse or keyboard to control him.
Nice, James! :D
Any plans to use the tracking in the gonk droid? THAT will be the external sensory queue needed to keep him walking straight :)
10:47 who knew defeating ultron was a simple case of pulling on a USB wire. That would've made the film a lot shorter
YOU HAVE DOOOOMED US ALLLLLLLLLLLLLLLLLLLLLLLLL!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
It's not too hard to implement actual face recognition. I did something earlier this year, which used facial recognition to simulate The Machine from Person of Interest, in Python with opencv. You could run it on a Pi or one of the other mini Linux boxes you mentioned.
i have been wanting to build a robot like this for years and have no idea where to start
It looks amazing!
That is so cool
This is very cool indeed! I think I wowed like 3 times.
for the ai brain viz, try a projector through a mist or fog. use pulses and orbits, or a laser at crumpled reflectic mylar or aluminum, through some particle in the air...great worx!
I love this series
Mm, in regards to the light colors, will it be screen-accurate red at all times, changing based on sensor input, or will it be changing according to "mood"?
wow, i just happened to find your channel mentioned in the comments on a video i was watching. this is amazing!!!
my major is robotics engineering. currently Im doind my prereqs though so i havent gotten to do anything with robotics just yet.
might i ask how you learned all this? did you teach yourself or go through university??
also, subbed!
Would be interesting to add some sort of weapon (nerfgun?) to one of the arms that can be put to "sentry mode" in which it tracks a color-coded target...
I must say, just wow, i watch all of your builds from beginning and i am a bit jealous, that i don't have the time and money to do something like that. I wish i can meet you and talk over this Project and make some brain storming :3
But iam from germany, so this will never happen xD
You have all my respect for this project and hope you get some invites from big companies that want to hire you for this awesomeness :)
i like your videos, keep up the good work man
ooh wodner if you can put in a second pixy cam and give Ultron binocular vision:D and maybe cannibalize a kinect for its depth sensor so it has full 3d vision:D
and on a tangent i just popped in from watch a documentory about robots filmed in 1999 and how Asimo was the pinnical of robotics due to its ability to walk on two egs and the video after this one was James' updat eon his gonk droid.
technology has advanced so much that what took a corperation millions of dollars to research and develope can now be knocked out by a vlogger with a budget around $1000 at best:D what a time to be alive.
That is so cool!
A robot that can sense a disturbance in the force...
You shouldn't give Ultron Vision, image what he could do in an infinity stone
Does the camera have some library for pre-trained facial recognition?
If not, do you reckon it'd be hard to hack in some facial recognition?
It would be awesome if it could track the faces nearest to it, and look around the room by default as people walk past or stare at him.
That's embarrassing, saw that part after asking.
Anyway, great work!
I like seeing Ultron move his head around and react!
Very cool james! Ultron is becoming one of my favorites.
Do you think the pixy cam could track a face by looking for 2 white objects of the same size and close together(eyes?) Probably not worth the effort if they are coming out with their own firmware for it though.
Epic episode man!
opencv and basic face recognition would be better for interaction with people
20:21 "so many strings that pull me down"..
What do you think how many hours have you spent on building ultron?
Will you paint Ultron at all? Make him look a bit more metallic or dirty?
I think you need to see "3D Printer Calibration - Marble Machine X #35" to sort out your 3D print quality.
man, that's so cool! awesome
incroyable men tu fait rever le monde entier !!!!
Wooooo, that's insane man
wow you are really amazing :D
I see a problem that might happen, You might of addressed it when you shook Ultron but if the whole bot is shook and the sensors freak out wouldn't it make it look like Ultron would be having a seizure? and just not stop shaking? or do you have It set up so it won't do that?
P.S. great work so far! i live in the U.S. and i hope to see your work some day in the flesh.
you should do 6 not 2 statements so uptron will look toward the joint/s that are bumped and not just in the general direction
when it sees the sticky notes its like a kid watching someone with food
While the pixy is a very good option because of its usability and simplicity, that's also it's downfall due to its limits, as you said it's rudimentary. Personally I would use openCV in Java, C or in Python if you wanted to use it for face tracking like you said. In saying this I understand your hesitation in writing software.
You could use something like google tts(text to speech) to make him talk! (I don't think that would be possible with arduinos but it would still be fun)
The Avengers will stop you doing this
Sketch It D.I.Y Just like Tony Stark , before he create ultron , he got a lot of suits too , and he is avenger, so it doesn't matter did he got a suit .
Sketch It D.I.Y And tony already learned from the mistake after create Ultron , he let Ultron find the best way to protect the earth by "itself" . but it think destroy humanity is the way to peace . Now he knew let the AI have a self-awareness maybe dangerous . And now XRobots try to create new Ultron , we don't know what will XRobots do to Ultron , =) , And it names ULTRON , it dosent seem good for The Avengers XD. Its just a joke , dont have to too serious.
that is incredible!!!
that will be amazing
GENIUS!!!!
So yea i was going to say are u going to pin a colored block to each visitor so it would track them, but if u can lock on to a face your good to go
You're catching up to Honda Asimo faster than Asimo is being developed. So, good on you. Bad on Honda.
if the camera were to be a heat vision camera,you could track relatively warm objects(like humans) without the need of that facial recognition
Screw the sound board. When you push on Ultron he should do some quick tai chi and put you in a choke hold! XD
Once this is fully completed (As in, as much as possible), are you planning on making this fully autonomous? Or have it controlled by a user? And will you be adjusting it to minimising the lag/input time between the user's movement and the robot's movement? Honestly mate, it already looks bloody amazing. And slightly creepy.
TheBritishBrit It will decide on its own whether it will mimic the user's motions or react to them, such and pushing back when shoved.
KingOfYou115 Yeah, I had presumed that's what the sensors in the joints were for, amongst other things?
Looking good
Will you replace the plastic with Vibranium? If so I may need to make a call to the Avengers. ;)
I think Shapeways are bringing out Vibranium filament.
Please tell me that you're going to put in a color recognition pattern so that Ultron will prioritize giving Ironman cosplayers some added guff at events.
Hi James nice Part I m really Bad in Programming and I wanted to ask you how you learned it?
If you want to track a face, something free like Facetrack Noir could be made to work...
Have you thought of doing a div nights at Freddy's SpringBonnie suit? It supposedly can go from a animatronic to a wearable suit. And I would love to see one
just an idea, but you should look into batch files and .vbs files. they're super easy to learn and they also can make great interfaces. you could also make him speak with one and code in what you want him to say. then it uses the computer's narrator voice to say it. if you go online and look up "how to make a batch file speak" and click on the one on instructables you can just copy and paste from there. the only thing is on Windows 7 it uses a female voice but pretty much everything else is a male voice (XP, W8, W9, W10, etc.) if you're interested please tell me and I'll put some code that I've made that I think can work for Ultron. I'm not sure how to make it work in tangent with the robot, but I think with your arduino skill you could easily figure it out. Good luck on the project either way!
You should program iron man's suit colour combo such that when anyone in an iron man mask come over to ultron he gets really angry
What about tracking skin tone? Also, if you deliberately blur the camera signal, it might help average our a head into a skin tone sphere.
it's aliiiiiiiiiive! :0
the future is here already
the coding and stuff goes right over my head it's all very impressive though
Can you give a rough estimation of how much this project costs? Just curious.