Thumbs Up! The Mechanics of TeslaBot's Thumb--Plus Much More about Optimus! With Scott Walter
ฝัง
- เผยแพร่เมื่อ 3 ต.ค. 2022
- Another video of Optimus, Tesla's robot's hand, and this one shows the INSIDE of the hand (the palm area). Dr. Scott Walter takes his robotics expertise and breaks down exactly what cool stuff Tesla is doing with the Teslabot Thumb, which is the most critical joint in the hand. We also get into why AI and computer vision are so critical to making a mass-produced robot work!
If you like Scott's appearances, check out his TH-cam channel: • Video
**To become part of our Patreon team, help support the channel, and get awesome perks, check out our Patreon site here: / drknowitallknows . Thanks for your support!
**Want some awesome Dr. Know-it-all merch, including the AI STUDENT DRIVER Bumper Sticker? Check out our awesome Merch store: drknowitall.itemorder.com/sale
For a limited time, use the code "Knows2021" to get 20% off your entire order!
**Lift pucks for your Tesla: amzn.to/3vJF3iB
**Emergency tire fill and repair kit: amzn.to/3vMkL8d
**If you're interested in a CO2 Monitor, you can check it out here: amzn.to/3PsQRh2
**Check out Artimatic: www.artimatic.io
**For 30% off the amazing OneCarStereo instrument display for Tesla Model 3/Y, use this link: bit.ly/3vd7NRv
**Get 20% off an awesome Ekster wallet and/or wallet locator(!), check out my affiliate link here: shop.ekster.com/DrKnowitall
**Want to get in on WeBull's awesome Crypto and stock fun? Check out this link, and get started trading stock and Crypto!
a.webull.com/i/DrKnow-it-allK...
**If you are looking to purchase a new Tesla solar roof, solar tiles or PowerWall, just click this link to get up to $500 off! www.tesla.com/referral/john11286. Thank you!
**You can help support this channel with one click! We have an Amazon Affiliate link in several countries. If you click the link for your country, anything you buy from Amazon in the next several hours gives us a small commission, and costs you nothing. Thank you!
* USA: amzn.to/39n5mPH
* Germany: amzn.to/2XbdxJi
* United Kingdom: amzn.to/3hGlzTR
* France: amzn.to/2KRAwXh
* Spain: amzn.to/3hJYYFV
**What do we use to shoot our videos?
-Sony alpha a7 III: amzn.to/3czV2XJ
--and lens: amzn.to/3aujOqE
-Feelworld portable field monitor: amzn.to/38yf2ah
-Neewer compact desk tripod: amzn.to/3l8yrUk
-Glidegear teleprompter: amzn.to/3rJeFkP
-Neewer dimmable LED lights: amzn.to/3qAg3oF
-Rode Wireless Go II Lavalier microphones: amzn.to/3eC9jUZ
-Rode NTK Studio Microphone: amzn.to/3vjMomW
-Focusrite Scarlette 2i2 audio interface: amzn.to/3l8vqDu
-Studio soundproofing tiles: amzn.to/3rFUtQU
-Sony MDR-7506 Professional Headphones: amzn.to/2OoDdBd
-Apple M1 MacBook Pro: amzn.to/3wPYV1D
-Docking Station for MacBook: amzn.to/3yIhc1S
-Philips Brilliance 4K Docking Monitor: amzn.to/3xwSKAb
-Sabrent 8TB SSD drive: amzn.to/3rhSxQM
-DJI Mavic Mini Drone: amzn.to/2OnHCEw
-GoPro Hero 9 Black action camera: amzn.to/3vgVMrH
-Suction car mount for camera: amzn.to/3tcUfRK
-Extender Rod for car mount camera: amzn.to/3wHQXsw
**Here are a few products we've found really fun and/or useful:
* NeoCharge Dryer/EV charger splitter: amzn.to/39UcKWx
* A totally fun Tesla coil toy--awesome for the whole family: amzn.to/30rvzbI
* The Humane Interface, by Jef Raskin: amzn.to/3ajM7YT
* AI Superpowers, by Kai-Fu Lee: amzn.to/3dgRSs3
* A camping mattress for your Tesla model S/3/X/Y: amzn.to/3chSRrQ
*And of course the amazing "Elon Musk: Tesla, SpaceX and the Quest for a Fantastic Future": amzn.to/36hqdmS
**Music by Zenlee. Check out his amazing music on instagram -@zenlee_music
or TH-cam - / @zenlee_music
Tesla Stock: TSLA
**JOWUA:
*For an awesome wireless charging phone holder mount, check out: bit.ly/3tRMSQR
use the Promotion Code: DRKNOW5
*Tesla USB Hub: bit.ly/2PXHPit
use the Promotion Code: DRKNOW5
*Powerful wireless iPhone/Android charger for your Tesla Model 3 or Model Y, check this link: bit.ly/3b5fHSW
use the Promotion Code: DRKNOW5
**EVANNEX
Check out the Evannex web site: evannex.com/
If you use my discount code, KnowsEVs, you get $10 off any order over $100!
**For business inquiries, please email me here: DrKnowItAllKnows@gmail.com
Twitter: / drknowitall16
Also on Twitter: @Tesla_UnPR: / tesla_un
Instagram: @drknowitallknows
**Want some outdoorsy videos? Check out Whole Nuts and Donuts: / @wholenutsanddonuts5741
Thumbs Up! The Machanics of TeslaBot's Thumb--Plus Much more about Optimus! With Scott Walter - วิทยาศาสตร์และเทคโนโลยี
It is so remarkable that the team came up with this hand in such a short time.
What nobody has mentioned... is they showed the robot using a WATERING CAN.
* think about the off-center grip point that is very narrow and the wrist torque to hold it level..
* think about the sloshing of water as it walks and turns that would make the can torque and rotate in the hand.
* think about the changing center of gravity as it tips the can and the arm gimbal to pour smoothly
* think about the changing mass as water pours out.
If bot can use a watering-can with accuracy and dispense controlled amounts of water this is actually a VERY VERY impressive demo.
Very true. They may use the feedback from the change in torque to imply the amount of water dispensed. So much they could do with software and ML.
My question is: why they should?
They aim to robot to do specific works, not party tricks, your dishwasher can't walks, do it make it less useful? That this robot have humanoid form, don't mean it must do all thing humans do. Look at video by "Francisco Galarza" with title "Tesla bot is an industrial machine | And why it WILL work', because he give this interesting perspective in better words when I could.
@@m_sedziwoj I think it's an extremely valid point that robot demos should be things humans don't want to do (factory work) .. if you can do that for 20k you're a better product than the 2 million dollar robot that can dance or play soccer.
The water can was empty.
@@paintedpony2935 for now
12:50 Did it miss on the OK sign? That really depends on how you make an OK, finger inside or fingertips touching, The way the Bot is doing it is the way I have always made it. Looking at the internet, I see it both ways. So, it would depend on who was training it.
I personally love the design of Optimus. It looks better than I thought.
Thank you Scott! I couldn't figure out how to do stereoscopic vision (useful for depth perception) with a single camera without it having prisms/mirrors. You just use time-lapsed images. Eureka! Thanks for that Scott!
21:29 Although we can look down and see our feet , we typically don't look at our feet when walking, we look before us and our mind keeps a "map" of what we see and navigates from that. In the case of the Bot perhaps an extra camera pointing down could be used for tight areas. (this is also when we tend to look down at our feet)
Bot could lean forward if needed, no?
@@knoworiginality Possibly, but probably not in all situations could the robot lean forward (such as when it is caring something), and not as quickly as having a camera to turn to. Cameras are cheap (they don't need to be AAA class for the non-primary views). And they could be just used when needed, not increasing computing power much overall.
Stereoscopic (two cameras with overlapping fields of view) vision is no longer necessary for environment depth estimation and object simultaneous localization and mapping (SLAM). Single camera monocular image processing systems are able to provide depth information directly from statistical analysis and mathematical interpretation/transformations of input photons. This is how portrait modes work and other visual measurement tools work on Apple, Samsung, Google, Huawei and other smartphone cameras. Plenoptic Light Field cameras offer another way to do the same thing.
Optimus is after my job! I'm a Sign Language Interpreter. We love u too Optimus!! 🤟 The O.K sign is actually the letter F and the number 9 in ASL. How fast can Optimus move those fingers? I won't worry until it can understand ASL and speak. 😜
Love the commentary on the hands. Interesting to see the design restricted to one type of actuator in the hand, and the compromises it requires. The LURE prosthetic arm that I helped design went with all geared drives, and linkages in the fingers for a repeatable and deterministic motion. The thumb has a differential gear arrangement for two degrees of freedom.
I bet they are using some bus comunication for talking to the servos, likely CAN which requires just 2 wires and has lots of built in robustness and its also used already in the Tesla car.
One - additional possible - use of this process at Tesla ... To use lessons learned on how to make appendages (hands/arms/feet/legs) and apply that knowledge to prosthetic replacements and connect it to Neuralink.
Actuators in the palm is also better for maintenance/durability, if a finger gets crushed or bent, just replace the cheap bent part, not bent parts and a expensive actuator
28:35 - I'm a VFX artist and we use "3D camera solves" all the time. You really don't need very much movement at all in order to calculate the parallax of each pix pretty well. Just a small sway to the side or even back and forth would be enough. Wouldn't even have to take a step.
In addition, you can already get depth data from a single camera even in photos taken in smartphone cameras using nothing but vision and algorithms. I'd guess they are using that, plus the parallax data, plus the occupancy network all together.
Without an opposable thumb, you can't pick up a sqrew.
And without that, you can't use a power tool.
But Optimus might still not need to be able to rotate the thumb. There's little need for having the thumb on the same side as the fingers. As you do when gripping a too thick bar or handle.
22:50 what they do is to use autolabel to generate the occupancy network training labels. They have 3 cameras moving through space and time, this generates the world "point cloud" using SLAM to get poses and then basic self supervision for depths mapping to generate the point cloud. Once they have the poses of the robot's center of mass at each time step and the 3D world, they can generate labels for what the neural network should predict in each timestep.
A combination of vision and tension on the motors is probably enough to start with. Removing the touch sensors massively reduces signal complexity and makes models a lot easier to interpret IMHO. Also, as a detection transformer, Panoptic Segmentation (PS) seems like an obvious way to go, assigning both classes (what something is) with ow it is arranged in the image. I can't image the difficulty of stitching video together and then performing PS analysis continuously (lots of compute) but it seems tp be a close approximation to how much of fleshy vision system works (recognition and class assignment). Of course, once something is classes, the semantic properties of that class can then be inferred. Nice.
13:00 about control of pulling in fingers, if you don't want add actuators on each join, make weakest spring at tip and stronger near hand, and add small lock mechanism so if you don't want to tip move but whole finger lock cable so all load go to spring closer to hand, maybe with something allowing to not be 0-1 but allow only some tension so could be move independently
EDIT interesting is that it have only 2 finger joins, not 3 as humans
EDIT 2 16:00 you mention what I said in edit 1 :D
That's the t-shirt I want, Tesla bot throwing the Shaka 🤙🏼
Thanks John Thanks Scott
If you look at BumbleCee’s head you can see an articulation point and actuator that lets the face tilt down.
Great overview guys
Way too much guess work.
Thank you for your curiosity 🎉
52VDC would be available from POE sourced network switches (POE range 44 to 57 VDC).
Too cool!
The thumb over the finger made with the ok sign is dramatically stronger in grip strength, ask any high level rock climber, or watch spinning bar hang challenge videos
They Should Program it to do Sign Language as a Demo.
I was thinking about the motor feedback for propreoception and it seems that Optimus would be able to pick up an object and tell you it’s weight. Also it could probably just look at an object, recognize it and estimate it’s weight before attempting to pick it up.
Well, I'm doing the ok sign exactly like Tesla Bot.
11:11 you may study the inside of a yamaha clavinova.
Thanks For Posting This. I love your #tesla channel
People should check out the hand designed by Will Cogley. Lots of videos here on TH-cam.
I could not help but wonder why two bright individuals focused on the thumb not meeting the tip of the forefinger when it was inconsequential for functionality. Happily, you both came around to that realization. Amusing detail of a very good video.
Isn’t it fairly important for a pinching motion though? Will it be able to do things like pick up paper if it can’t put its thumb and index finger together?
@@ZeroRelevance Good question. I just did and it was easy. Try it yourself.
At 1:53 when they show the servo you can see the wire count of 6. It could be a bldc, y-u-v, +5, gnd, tach/commutation. Or A dc motor with mot+-, a ,b, +5vdc, gnd. And yea, for sure, they are going to be monitoring current for a force measurement. 48vdc is a nominal 14 cell lipo voltage but it can charge up to ~52 volts. Maxon makes similar motors.
with some good engineering, you could send data over power. That could keep down the wires too. I know power is notoriously noisey, but with good protocol it could be done, imho....
I may be wrong but I think they are doing that in the Model Y to reduce the number of wires.
31:50 about data transfer, I would not be surprise if they use same as on cars which is better for hard environments, but who knows.
52v CAN-bus for communication?
Are Optimus 2 hands still cable actuated? Looks like it might be the same design but with the addition of pressure sensors, but can't tell for sure. If they aren't the same would love to see an updated analysis.
I would argue there is no international standard or requirement for the OK sign to have fingertip and thumbtip touching. If you do an online image search for ok sign you'll see people doing both this and the one Teslabot uses.
If you put a finger from your other hand though the hole and try to use it to pull towards the connecting point, you will notice that the Teslabot method is much stronger (at least for human hands!)
Tesla + Google in discussions... GPT-3 and GPT-4 making rapid progress on speach-memory development. Next year's Tesla AI day will see Optimus following instructions... "come here"... "pick up the water can"... "place the water can under the tap"... "turn on the tap"... etc.
You mentioned how 3 cameras may be used to create stereovision. However, in a connected, constrained factory environment, isn't it possible that Optimus can be just one of many edge appendages in a large integrated system processed by dojo? Meaning that all the sensing/ computing doesn't have to be done by Optimus. But Dojo can integrate/ process videos from all the Optimus robots along with the videos coming from wired cameras in the factory to create real-time, superhuman sensory output?
I do the ok sign exactly the way the robot does.
So for a 48v pack, the operating voltages will be 42-56vdc. So this should be able to share components with any 48v system, although they should be called 42v systems as it's the minimal vdc and 42 is the answer.
The joints need to meet all the attributes of what a human hand can do, or more. Other features that can help save power, is locking joints. I.e. when a human holds a 1 or 2 hand held object, it takes up much energy at long periods. Lock joints, feature 🔋 saving.
I think you forgetting depth estimation NN, which are doing it on one image, not need for stereoskop vision. And Tesla did use them before for FSD.
Deducing 3D by moving the camera in space works well for walking. But if the bot has to stay still at a place and work on fitting screws, nuts, bolts etc then the bot has to keep 'bobbing' to get the 3D visualization.
Also, the front camera and side cameras don't seem to be overlapping all viewable areas. There seems to be a strip of view 'straight ahead' that is visible to the single front camera...thus depriving the 3D view of object right in front of the bot.
I also question the prudency of building a prototype with fewer cameras. They should be fitting as many cameras as possible, capturing data from as many angles as possible; and then processing several combinations of datasets to determine the minimal number of cameras and their respective positions for optimal functionality of the bot. The production model could have a pared down number of cameras!
Some humans due to being cross eyed at birth don’t develop parallax based 3-D vision yet they can still pick up a screwdriver and a screw and line it up
I just thought again about the motor current as feedback for control. I don’t think that will work because of the arrangement of the work gears, the feedback force from the linear actuators would be pushing perpendicular to the axis of motor rotation? Seems like it would need an encoder on the screw drive the measure absolute linear displacement.
To me its seems that the fingers overlap with the O.K sign since fingers are the same length. So its a best fit scenario.
The OK symbol is exactly how I do it. Tip to tip is one method
No ability to give the “Live Long and Prosper” sign.
link for the hand video?
Typo in title 'Mechanics'. 👍
Power and data seems perfectly reasonable to keep hand easily exchangeable and allow.for.feedback from sensors so you must have at least 20 things to monitor and control. Probably a tad cleverer than I2c and pico 16unit servo controller.
No reason to leave out that rear facing fish eye camera. Training will have rooms or places with known dimensions for reference.
The reason it has springs to extend the fingers is because we humans don't need strength when opening the fingers, only closing them to grip stuff.
Or maybe the engineer does an ok gesture in that way. When I do an ok gesture my thumb overlaps
Thanks for watching☝️Get-in-touch on whats' @pp for more F i nancial Insights
Are all the fingers having the same part number (exactly same)?
What about the thumb?
Thanks for watching☝️Get-in-touch on whats' @pp for more F i nancial Insights
4:34 In terms of motion, i have yet to see something new. What can´t be seen if the hand has some sort of feedback.
🤗👍 VERY VALUABLE ..THANK YOU BOTH 😁😎💚💚💚
how does the bot judge distances, it it were to pick up a wine glass for instance?
Do humans have direct actuators in their hands?
yes, muscles, maybe not main strength, but they are still there.
I’m not sure that Teslabot needs to look down to walk because it can remember any obstacles from videos it stitches together.
After it steps on the first baby I think that would end.
Something always can get under your feet in last moment, so for safety it should.
Thanks for watching☝️Get-in-touch on whats' @pp for more F i nancial Insights
The hand will be better with finger spread ability
I think the bot can be put to work as a greeter in department stores before Christmas.
Would self checkout kiosks be better for that?
@@TheSpartan3669 It was just intended as a way to counter the FUD, by demonstrating to the public the Tesla bot is real and not a dog-and-pony show. In a few months, IMO, it should be far more capable then it is now, so it could be up to the task. Imagine all the media coverage....another good story to pound down the stock further ;)
@@craigruchman7007 Given Elon/Tesla's track record for predictions, especially those concerning FSD, uncertainty and doubt about the claim Optimus (which under most circumstances must handle situations that are different and much more complex than FSD) will be ready anytime soon isn't unreasonable. We'll see though
Machanics?
A play on Ex Machina. Or the Mach Number. Or a typo. Or Occam's Razor: spelling mistake.
Tesla did not try to recreate the human hand. The original model looks like a hand drawing model like Scott's human drawing model. Human thumb opposition and ABDUCTION muscles are in the palm. Human finger flexion movements are via tendons to muscles in the forearm. Using first principles, Tesla moved the flexion actuators to the palm and replaced extension actuators with springs. Proprioception is built into all actuators. Touch and temperature sensors are in the gloves and foot plates because they wear out and need to be replaced easily. They are the reason why the hand did not crush the mango and the leg model did not crush the egg. Limb and finger calibration with vision is done on command. Individual movements are stored as behaviors (like turning right or left in the car). Complex movements are assembled from these basic behaviors.
Communication is almost certain to be done using one of the industrial control I/O networks. This would require three wires, large black and white for power and small red or blue for serial communication.
Tesla fixed the head to simplify the design. Having an independently movable head would profoundly increase design and computational complexity.
There are multiple cameras. I have not found an explanation where the cameras are, but I would include near and far 3d stereo cameras for distance and left and right and rear view cameras for full 360. Eventually, I might expand to full 360 distance stereo for use in a complex work environment because the bot cannot rotate its head.
Do you have any idea if it would be hard to teach it to typewrite?
Awwwe, ... no steno or typing pool full of Optimis!
Thanks for watching☝️Get-in-touch on whats' @pp for more F i nancial Insights
Hope it's water proof, or that'll quickly be a robot with grubby hands...
What you are discussing about the high frequency points, tesla calls it NeRF system but I think they said it's not yet ready for high relayable data. I think they have other ways to make a 3d picture. I think there are some machine learning alcorithm that can make a 3d picture out of a 2D picture. On the street it's easier because there you have things like the line that is getting thinner how further it gets away on a picture but has in reality always the same thickness. That's probably how the alcorithm works that he can tell how far away things are. By telling how the things are getting smaller in distance and calculate out of it the distance. 🤷But this is just my guess. Or maybe he has still two cameras in the front to also see things far away. And he can use the normal Tesla hardware and algorithm
Robot is doing it correctly. it's not tip to tip, that's for smokin joints. make the OK sign and you will see for your self.
Biologically inspired design, produced by somebody who has apparently never seen or used a human hand
To me all those black spotsnin the fingers are all pressure sensitive pads, that can be used to mesure the fingers touching ... etc
@Tesla ; Teach the robot to learn tasks just by watching someone perform that same task, like teaching a kid. Then start selling the robot to the public and people will train the robot with tasks + over the air updates. Just like FSD😉
My thoughts exactly. Programming this bot is going to be much easier than people think. With the Tesla's vision suite, the bot will observe, listen and do.
The future iterations will likely include head movement. The evolution of the Tesla Bot won’t likely stop.
In order to form the "OK" hand signal, I have to capture the tip of my index finger with the first pad of my thumb in order to straighten my middle finger.. Try it 👌... I think they were showing off how "human like" the mechanisms are 🤔
How does this compare to the Terminator hand?
QUESTION: Can you just do a BRIEF bottom-line "take-aways" video of AI DAY ⁉ JUST hitting the high-points for us mere mortals ❤
The palm doesn't curl so how good of a golf grip will it have? Will it be able to hold and swing a baseball bat?
This hand seems pretty simplistic to me. Compare it to the hands of Ameca, the robot from Engineered Arts of the UK. True, that hand doesn't have an articulated thumb, but the finger movements are very life-like compared to the Tesla hand. Most of the vids available showing the Ameca hand seem to be at least a year old. I suspect that they would have added thumb articulation by now. Also, the Ameca hand's appearance is waaaay more life-like than the Tesla hand. Further, the motion of the Ameca arms, shoulders and torso are very far ahead of what Tesla is doing. Don't get me wrong. I'm a huge fan of Musk, Tesla, and Spacex. I just don't think this iteration of their robot is as far ahead as the fan-boys seem to think it is. Boston Dynamics and Engineered Arts are still way ahead of Tesla in some areas. Sure, Tesla is pouring TONS of money and talent into robotic development and they'll probably catch up to and surpass these much less generously funded efforts. If other teams had funding and resources similar to Tesla, I'm sure they'd hold their own.
A $43,000 profit sent to my portfolio each week, Mrs. Angela cole carr is amazing.
That's awesome. I dealt with crypto last year on Robinhood, tried some index but didn't take it out so I lost it by the end. Any consistent strategies?
Trading with Angela's strategy is nothing new to me, I opened a trading at $5K, and I've been able to scale to $18K in two weeks
I got Angela's recommendations in December 2021 and started following her lead and made $23,450 in a week. Lost my job in January 2022 and right now I'm back on top again and ready to go with her guidance.
@@elizabethpalmer9546 Amazing!!! found Mrs Angela Cole ones 2 months ago here also!!! This is so validating hearing you also got her recommendations!!! Yes cheers to be part of her coaching program.
Crypto Trading remains a priority, I'm proudly a huge beneficiary of Angela Cole Carr services. About a month ago sold most of my stocks at a lost and took what I have and reinvested and those stocks were green today. I gained over $30k.
I'm done sitting tight for the award advance since i acquire$23,000 every 12 days of my investment
I heard a lot of investing with Mrs Clarissa and how good she is, please how safe are the profit?
I started with just 4,000 dollars now earning upto 17,000 dollars WEEKLY
The crypto market is highly profitable with an expert broker just like CLARISSA I got recommended to her and since then my financial life has been a success.
I'm sorry, but why should a robot have fingers that look like our fingers?
Why not tentacles like an octopus? 8 of them, and they grab any shape much larger than what a human can grab.
Why not replaceable hands to match the task at hand? Like special devices to handle 800F steel?
Or for working in liquid nitrogen?
or picking up chunks of ice
or underwater to work on scraping barnacles off a boat while it is moving at 30 knots.
or for a thousand other activities where 8 fingers and 2 thumbs are just stupid.
Like I said - Elon is recreating 1980's robots that are not state of the art in 2022.
No imagination at all.
Knows it all... except how to spell "mechanics" :)