I remember one of your earlier videos where Carol followed you in her car and tried to guess if you were driving or if FSD was driving. Would love to see an updated version of that! But no cheating this time... lol
I'm impressed how brave you were with this video. That tire could seriously damage your car and you really waited to give it all the chances in the world. Edit: lol I commented this before the last test!
@@AIDRIVR So it works for slow moiving car and slow moving target in the open on a clear sunny day! LOL Lets see you test it with your kid running at night with no street lights and rain! Theres a reason ti will never work! SONAR is not suitable for this!
ROFL!!! “Good, it deserved that…” Thank you AiDriver and Carol! I’m glad Carol didn’t fall when rolling the wheel. ;) Carol is brave! Incredibly Awesome video
This is awesome. As a tesla driver I often find myself wondering what kinds of objects the car will/will not notice and plan around, and this gives me a much better sense: some things, better with people, but not super reliable in any of the cases -- at least for the current release. Thanks for doing this.
I would expect the boxes and tire lying flat get interpreted as pot holes or road variations instead of debris. As best I can tell, FSD is not yet modeling low lying features and imperfections in the road surface. That probably a very hard problem.
@@MusicalMemeology No, a radar world not help. From a radar's perspective, the unmoving objects are the same as the road. That's why the early versions of Autoilot didn't prevent collisions with stopped vehicles on a freeway or trailers moving horizontally across a road. Vehicular millimeter wave radars only track movement in the direction of travel. Anything not moving is the same as the road. My Corolla has a similar radar. In testing it's lane keeping assist, it doesn't see things like concrete barriers, which are stationary. All current vehicle radars have this limitation. Vision is a de facto better solution.
@@MusicalMemeology As Ed said, radar has serious issues with stationary objects or objects moving horizontally in front of the vehicle. If you've ever driven a Tesla or other vehicle using radar, you'll notice how very late it brakes for stopped vehicles and how it doesn't care in the slightest about road debris. By comparison, Tesla's vision approach is vastly superior at this task.
@@MusicalMemeology Also, keep in mind, we're seeing the results of Tesla building and testing this software. It has lots of room for improvement and many are expected to come before it's ready for general release. Road debris probably needs more attention from the auto labeling system during training. But the resources to handle that are still being built, namely Dojo. I would expect reactions to potholes and random debris to improve after Dojo has been online for a while.
@@Tanstaaflitis Not at all accurate. LiDAR can see objects, moving or not, very well. Software has to be intelligent enough to identify them, but the information is there and more easily interpreted than visualization.
I had the issue of FSD avoiding puddles in the road. It kept changing lanes to avoid them. These puddles were bigger than potholes and a human would know they could safely be driven through (not a flooded road, just puddles in some lanes)
@@phyzygy these were obviously not potholes. They caused the car to weave from lane to lane in moderate traffic. Cars in front were passing thru them without dropping so the AI should have deleted the difference.
This is your funniest video so far! :). Keep up the good work and congrats to your fantastic wife. The support of your loved ones is important for reaching the next level! ❤
On the 2nd attempt (StarLink box sideways), the box upclose looks exactly like the rest of the shadows on the ground. I think as it got closer it "forgot" there was something there (AFAIK current FSD has no object permanence) and just saw it as shadow.
@@superkikim4916 FSD beta doesn't use the radar even on cars that have it. I don't know if radar sees cardboard or not, it definitely doesn't see some materials. Radar is mainly for metal (other cars). For this case vision is definitely better.
its amazing how a video like this can make me laugh more than a tv-series episode designed by a large number of professional comedians to make me laugh.
Deke - It seems like it is taking the "Training" reasonably seriously! 🤔🧐 in watching the Repeats, it also seemed to me it was figuring out that it was not good to hit it, so the next try, moved over to avoid the Obstacle! Cats & Squirrels? Low Height "Objects" - But, the are not always "Moving!" 😳🤔🧐
My theory in the beginnin: As soon as the car can't see the object anymore it also changes paths. But then the flat wheel should've also not been picked up. Another test with a cardboard box pulled by a string across the road would be really nice. Then you can "safely" slam into it instread of a wheel. As always a very interesting video and I hope FSD will come to Europe soon!
It's not a height problem but a memory problem, everything above 15cm is recognized as an object by the cameras but as soon as the object is outside the camera angle, the software forgets that there was an object and decides to drive over it.
That's what I'm thinking as well, as soon as it leaves the camera vision it leaves the neural networks understanding of the world. The car happily reroutes to a straight forward line through the object because as far as it is concerned the object no longer exists.
Thanks to your wife for assisting in the testing. Tesla FSD Beta handles moving objects much bettie as the algorithms are focused on moving objects in the road. It is the stationary ones the technology struggles with. 1) What will FSD do when there is a massive pothole in the road? 2) What will FSD do when the object in the road blends into the road surface? 3) What will FSD do if a person wearing black had a green light on their head? I have seen people investing time to discover how they can trick FSD into seeing something and making wrong decisions based on what it 'sees'. Great work.
I think when the AI sees an upright car wheel in an orientation that is perpendicular to the direction the AI is driving it thinks it’s part of a car backing out into the street which is why is doesn’t want to go around and instead chooses to wait.
It's like you and a million other folks are working for TESLA for free. Brilliantly designed system to collect data and user generated content at the same time saving massive amounts of money on marketing, advertising, data collection, research and development, quality content creation (good job guys and gals for producing such a good video) and super valuable consumer feedback. Buying some stock Monday as it seams to be the bargain of the century.
Really nice benchmark. Maybe to time consuming, but if you manage to do this for all new releases or every other, it would surely show us when they release those avoidance updates.
Thank you for doing this! It's super, super interesting. I've been riding in Cruise AVs a lot in SF (so these scenarios you're testing for actually happen sometimes lol especially the truck ones), and the main difference I see is that Cruise seems to react much quicker and more confidently in situations like obstacles and pedestrians jumping into the street and further ahead with stationary obstacles than FSD does. I wonder if it has something to do with the power of the computers on board? The whole trunk of a Cruise is full of computers and you can hear the fans blowing away. I think when Tesla figures this out they'll be the leader because their system is more flexible, but as of now Cruise and Waymo seem to have quicker and more confident decision making.
It doesn't seem to be temporal at all. As soon as the box is not visualized for one frame, the route planner instantly routes through where the box was. As humans we of course know that box is still there whether we see it or not. Boxes don't just vanish. I'm sure they're working on this - the box should not be visualized for many frames before the car assumes it blew away or whatever
It's a balancing act. You might falsely detect an obstacle from far away. If you remember things too long, as you get close, you might no longer see it but still remember you thought you saw something earlier. Of course, a human would either decide they can obviously see the road is clear as they get closer, or they would gently decelerate because they're not sure. "Gently decelerate" is generally the best response, but the issue might be that FSD gets false positives too often, and it would be constantly slowing and unsure of itself. So I guess if they can make the obstacle detection more accurate, they can turn up the temporal aspects so it remembers for longer.
Your wife is a real trooper! If that had been me and my wife, she would have said, "Yeah, no way. How about I drive and you walk out in front of the car?"
Do you think it would make a difference if it had a camera on the front bumper to help detect lower objects? IMO: I think the closer you get it the car does not see the object since the cameras are on top. As you can see it makes corrections the further you are but as soon as you can close to it FSD like kind of forget the object is there.
i doubt it. its judging the object as it approaches and it definitely knows its there. id say the difference is in what its detecting it as. it might even take into account the drivers eyesight. IE. FSD beta watches your eyes, if it knows your looking forward and can see it and you are ignoring it, then FSD can probably ignore it since you would take over if it was something it shouldnt hit. This isnt something they would want to rely on in the final version, but for training,... you never know.
Very nice video! But I think you missed the brief moment at 2:22 where FSD did visualize the cardboard. I hope this system gets more control in the future and isn't overridden at the last second. Having gotten rid of the ultrasonic sensors, Tesla seems to want to develop the occupancy network much further. After that, the test should be a little more positive, since the close range is probably the problem.
I think the false positive prevention threshold is the reason it simply doesnt react. Its safer to let the beta team annotate some data and swirve manually, rather than the car slamming on the brakes one too many times. If this is the case, the system should become much better over time.
This. They have to balance between making the drive safe and comfortable. Most probably on the roads there would be many more false positives then not recognised real cases. In other words, there are fewer cases of having a box in the middle, than having a kid running through the road, or a motorbike braking, etc, etc.
@@SzabolcsSzekacs the vast majority of difficulties in software come from crossing the 99.999% threshold. Tesla can release a unified stack with parking and madmax california level merging right now, but it'd only work like 95% of the time and be offensively unsafe. capability != ability
Makes sense actually, from far away the top of the box would likely appear above the road, making it a clear 3D object. As the car approaches the angle between the camera, the box, and the ground grows. This perspective probably causes the car to think that these are just road markings. Putting a camera as low as possible in the front of the vehicle would help resolve this.
I'm hoping Tesla has this kind of testing internally as well and that it's high up on the list of things to fix... Elon said many time the prime directive is "Do not crash" so I'm expecting much better results.
@@ericwiese7479 It's a very nice technology. Just compare what it's aware of to what humans are comprehending. This could make the real time decisions way more complex then what humans would consider in the moment. If I stop hard then I will cause a 5 car pileup behind me since 2 of the cars are not likely to have auto braking and none have decent following distance. If I slow to 10mph I'm 87% likely not to cause permanent damage to the adult human running in front of me, And 93% likely to avoid the 5 care pileup behind me. With a 40% chance of loosing a 5 million dollar lawsuit for not stopping...
@@cifey definitely in its infancy. The real world data the FSD beta testers are supplying Tesla’s Dojo supercomputer will help a lot. Plenty of data points
it's not really a threshold that they set, the issue is the cameras get to see the short objects from far away but the closer it gets, the less they can see it because the cameras are sitting on top of the windshield, close range blind spots pretty much, if they kept the radar they had in their older cars, all of these issues would be much easier to deal with
Your nuts to brave these tests at such high speed. I would recommend 15 or slower. But hey no accidents so we are good! I can’t wait for my FSD computer to be installed.
Your channel literally gave me interest into tesla's autopilot, something that I didn't really care about before That just shows the quality of the content you create, so thank you!
If you have a small not too dangerous object on the road the safest way might be to hit it rather than swerving or braking hard. It might not have chosen the most optimal path, but arguably the safest. Of course a lot depends on the particular situation.
I don't think it's learning so much as choosing a different route each time. When it's planning its going through it's options many times each frame. Eventually it settles on a plan and executes that, so I think your seeing it choose a poor plan the first time and then the second time, with the situation slightly different, it chose a better plan.
I thought the same. I didn't think it did any sort of learning whilst driving (just executed the model it already had) but the tests did seem to indicate it might be remembering that the first path was a mistake.
@@clonkex It appears that way but I don't think that's how the neural net works. Its an illusion brought about by minute differences in each attempt forcing a different planning outcome. They train at home base then deploy to all. That way they can retain consistency across the fleet. The FSD computer architecture doesn't even have much on board data storage that I'm aware of. The memory would be flooded with data too quickly to be useful for a purpose like learning on the fly. If he had attempted this then gone home and received a new update awhile later then attempted again that's where the learning could occur and you may see actual improvement in understanding and executing the maneuver.
Great video, very insightful! I think it makes sense to have a vertical threshold. When the car is far away the uncertainty about the height is likely higher so it starts planning for it but then as it sees it's under some vertical threshold it ignores it. As they improve the occupancy network they should be able to lower that threshold without introducing false positives that could lead to "phantom braking".
Good job on that video to both of you :) I have yet to let the car handle things when there are crossways involved, here in norway pedestrians have the right of way if there are crossways without traffic lights, and I have yet to see that FSD accknowleges that.
Excellent assessment and real world testing w/ FSD and reaction times. What’s also interesting, is that FSD seems to be LEARNING as you move through similar iterations of obstacles. I could be wrong here, but I really think the R&D engineers purposely have the BETA algorithms feed data to the Tesla servers when multiple mimic obstacles are presented within under minutes of each other. Even so within a 24 hr period. This tells me that Tesla and Elon has a learning software that gets BETTER at dodging obstacles on that GPS side street. Could it know the obstacles iterations within the GPS locations?! Perhaps. But clearly the computer and software IS LEARNING! Thanks again for your assessment. Thank you for sharing with consumers. I’m here for technology and science to help make the world a more safer place, all the while enjoying creature comforts. Big oil industries days are limited - as more and more consumers are introduced to EV technology. EV and AI taxi 🚕 cars are coming. Lastly, I have two FSD vehicles in my arsenal. I build up my confidence utilizing FSD on non surface streets all the time. From my subdivision/community mailbox, to my home. FSD is brilliant in knowing how to switch lanes- AFTER I PURPOSELY park next to mailbox, in the opposite incoming lane. I start up the FSD-SUMMON feature (without anyone in the car), and every single time- it pulls out from the opposite lane, and merge to the correct lane, and travels to my house-(while I walk along side monitoring) with neighbors mouth wide open! They simply can’t believe this technology is here. One neighbor tried to call the police and threaten me that I shouldn’t trust the technology because the car could accelerate into their house!!?? 😂 😂. I assured them- that it would not! They still recorded me, and sent the video to the local sheriffs office. I’m still awaiting my citation. Anyway, thanks for educating the public in EV, AI, and FSD TESLA cars. Tesla is far more advanced than any other car/technology company when it comes to transportation.
I heard that tesla wants to erase ultrasonic sensors called parking sensors. Some low and small obstacles are below the hood of the car, so the cameras do not cover it.
If the car is in motion, it should be able to see such objects over time and model their behavior. When parked, I expect an updated Sentry Mode can model the changing world around the vehicle to prevent accidents when returned to driving condition.
It seems like obstacles that it can’t see after a certain distance it ignores. If you look at what it hit (or got close to) they were mostly objects that barely were tall enough to hit the bumper of the car.
Great test! Looking forward to more targeted testing like this. I suspect the detection system might be dialed to not be overly sensitive in regards to potential objects in the road since the occupancy network is new and false positives would lead to a lot of phantom braking. I expect them to tweak the sensitivity over time as the system gets more confident in its detection.
In the cardboard one where the Tesla ran over it the first and then went around it the second time. It could be possible that the Tesla is some how learning about it and fixing it’s mistakes.
Very interesting videos and love the work both you and your wife are doing. One thing I did notice, her hair was visible when using the window shade surface so I am uncertain of some of those tests. Attempt with the window shade covering some wood/foam that isn't exposed whatsoever that you may create an easy lean using some thin wood making triangles. A right triangle will hold it perpendicular to the floor that it may be too easy for it to fall like test 2.5 microphone arm box when it was standing up. So you may want the triangles side that touches the window shade support to be slightly at an angle inward so the object does not fall. As you said, despite it just being a box, it may still have things inside it. And I do wonder if height is important. Say something like, "given object taller than 1 meter, drive around." That being said, alternative to this test. Do these tests with light hanging objects with one of those clear almost invisible string. Just go to a branch and heighten the object little by little and see if the objects(at least cardboard) is treated the same.
It's a pretty normal occurrence for a piece of cardboard to blow onto a highway, in those cases, it's actually important that the car to keep driving and go over it. So I'm not sure whether it's doing anything wrong here. I guess it depends on whether it can actually tell it's a box. I'd love to see a test done with a short object that isn't a cardboard box (or is, but has been made to look like something else)
It's not recommended to drive over cardboard. You never know what's inside. When you can see it from a distance, the best response is to go around, changing lanes if necessary and safe. In this case the car had ample time to safely go around, but it kept ignoring it once it got close.
Jesus , I'm not McGuyver myself but just put something small and heavy (like a couple of rocks that you may be able to find at the side of the road) at the bottom of the mic arm box and voila' ! it won't fall.
Great video so far. I am 3:27 seconds in. Idk if u do this later in the video but if u don’t next time car u try with 2 cars u behind the other car and we can see how fast Tesla reacts to it. Cuz 75% of the time we are trying to avoid a accident/debris
Very nice demonstration of how the FSD beta fails in special situations... personally i find this very dissapointing and my trust in FSD really decreased. Humans are still much faster in reaction and would brake very quick in uncertain traffic situations. On the other hand when humans get distracted they don´t see anything that could be dangerous , so it´s nice to have a backup system which has it´s eyes always on the road ... let´s hope that there will be an updated version of the FSD that also recognizes small objects on roads and avoids hitting them. Just imagine that there was little rocks on the road ( which could have been fallen down from a truck or downhill by landslide or similar things ) ... that would hit the car and destroy the complete front ... OMG. Thank you very much for both of you, you have a wonderful assistant, she did perfect work there ! And i just loughed out loud when you mentioned that you have a life insurance policy for her ( Minute 7:24 ) That humor is brilliant !
I have had an Insta 360 X2 for about a year now and absolutely love it. It’s my absolute favourite camera and I highly recommend it. And no I am not associated with Insta 360 except is a very satisfied customer
@@NewCastleIndiana because it takes 360° photo, and I can compose the final picture Long after I’ve taken the photo and get just the results I want. When there are many different things happening around me I can take a 360° movie of everything and then pick out the individual pictures later. It’s waterproof and this summer On vacation in Jamaica I got lots of great family pictures of all of us playing in the ocean. It has much higher resolution and takes much sharper pictures than my old 360° camera.
I think what we're seeing is that while far away, there's nothing urgent, and it just plans for the object. But when it gets close, the level of uncertainty about what it is seeing contributes to uncertainty about what that thing might do, so a much more urgent/cautious adjustment is made, mainly by braking. Hard to cause an accident by stopping or slowing to a crawl (tailgating car behind being the exception, and even then its the tailgater's fault). We may also be seeing the Collision Avoidance System kicking in there and overriding FSD beta, as it might override a human driver. With FSD beta active, it may have the same CAS active as with human driver, but I'd bet at a lower threshold to kick in. Or something roughly equivalent to the CAS may be folded into the FSD beta perception/planning stack, overriding other priorities when it deems appropriate.
The whole thing proved how inteligent it is. It didn't stop for some cardboard cause it literally knew its cardboard. It can detect hollow objects. See the tire (layed) was same height as cardboard and it detected that it can go thru cardboard but not thru tire. That's a good thing cause imagine if it did full emergency braking for plastic bag on the road. It's kinda crazy how smart it is. And for the fast tire welllll it would stop eventually.
It's so cool that you are doing this real world testing which I'm sure Tesla is doing but just like the guy driving streets in NYC new things come up that no one ever thought about. I hope Tesla is monitoring your channel!
If you watch the pathing @4:35 it looks like it's detecting the fence at the end of the road as an object. During the AI Day it showed the cars network "predicting" cars that may be coming (even if it doesn't see something) and as it would have to swerve into the oncoming traffic lane. I wonder if that's why it's showing the right move, but failing to make the move.
It would be interesting to see how far away fsd can recognize cars in the passing lane on highways. Here where I live people can legally drive 120mph+ so it should be able to detect it quite early to not cut them off and cause an accident. Would be awesome if you could test that aswell
The starlink box is all but camouflaged. Camouflaged objects are difficult to handle with vision only systems. Systems seems to be much better with objects it is quite familiar with like cars and people.
This is my 69th video. That is all
Nice!
Nice
Don't show too much love to the number 69, you weirdos:)
Nice!
Nice
Just right amount of love shown for Carol!
you stole my comment
you stole my wife
you stole my wife`s comment
@@Adrianoid15 I'll also have this man's dead comment.
@@q7end That man stole his wife!
Honestly one of the best fsd testing videos so far. Thanks a lot for this great insight.
It’s not FSD grow up wtf!!!!
I agree completely, this obstacle avoidance testing of FSD is fascinating and important. Compare with other some other systems when convenient.
I remember one of your earlier videos where Carol followed you in her car and tried to guess if you were driving or if FSD was driving. Would love to see an updated version of that! But no cheating this time... lol
I saw that episode 2 we need those episode every time there is a major update
yeah that sounds fun
I would need an "aggressive af" setting on FSD to fool my wife into thinking it was me driving LOL
@@superhero6785 hahaha relatable af
Please repeat this test after a major update.
Kudos for the well thought out safety plan of getting the life insurance.
pffffff how many more updates? lol
@@MaticTheProto until it's fixed. It's a beta after all
I'm impressed how brave you were with this video. That tire could seriously damage your car and you really waited to give it all the chances in the world.
Edit: lol I commented this before the last test!
Great video, your FSD content is really next level, keep it up!
Thank you, appreciate it!
@@AIDRIVR So it works for slow moiving car and slow moving target in the open on a clear sunny day! LOL
Lets see you test it with your kid running at night with no street lights and rain!
Theres a reason ti will never work! SONAR is not suitable for this!
salut barosane, vrei si tu Tesla?
Giveaway cu o Tesla când?
Thank you for all you do showing the good and bad of self driving. It’s a much needed and well done visualization for the public.
The transitions between the two perspectives look amazing!!!
Smoooooth
That duck sound on the falling object was just 🤣 love the comedic elements you put in there in a subtle way. Interesting vid!
Its was on F key u know🤣
ROFL!!! “Good, it deserved that…” Thank you AiDriver and Carol! I’m glad Carol didn’t fall when rolling the wheel. ;) Carol is brave! Incredibly Awesome video
This is awesome. As a tesla driver I often find myself wondering what kinds of objects the car will/will not notice and plan around, and this gives me a much better sense: some things, better with people, but not super reliable in any of the cases -- at least for the current release. Thanks for doing this.
Good job Carol, ok that’s enough love.
I would expect the boxes and tire lying flat get interpreted as pot holes or road variations instead of debris. As best I can tell, FSD is not yet modeling low lying features and imperfections in the road surface. That probably a very hard problem.
Stuff like that I really think they need radar.
@@MusicalMemeology No, a radar world not help. From a radar's perspective, the unmoving objects are the same as the road. That's why the early versions of Autoilot didn't prevent collisions with stopped vehicles on a freeway or trailers moving horizontally across a road. Vehicular millimeter wave radars only track movement in the direction of travel. Anything not moving is the same as the road.
My Corolla has a similar radar. In testing it's lane keeping assist, it doesn't see things like concrete barriers, which are stationary. All current vehicle radars have this limitation. Vision is a de facto better solution.
@@MusicalMemeology As Ed said, radar has serious issues with stationary objects or objects moving horizontally in front of the vehicle.
If you've ever driven a Tesla or other vehicle using radar, you'll notice how very late it brakes for stopped vehicles and how it doesn't care in the slightest about road debris.
By comparison, Tesla's vision approach is vastly superior at this task.
@@MusicalMemeology Also, keep in mind, we're seeing the results of Tesla building and testing this software. It has lots of room for improvement and many are expected to come before it's ready for general release.
Road debris probably needs more attention from the auto labeling system during training. But the resources to handle that are still being built, namely Dojo. I would expect reactions to potholes and random debris to improve after Dojo has been online for a while.
@@Tanstaaflitis Not at all accurate. LiDAR can see objects, moving or not, very well. Software has to be intelligent enough to identify them, but the information is there and more easily interpreted than visualization.
Thanks to both you and CAROL! For an enthusiast like me, I really appreciate the science and the benchmark!
7:31 💀
Thanks for letting me know the limitations of FSD. Very valuable, keep it up.
Thank you so much Deping!
bro when u got carol to walk across the street with an obstructed obstacle LOL
props to u for getting her to do that somehow lmaoooo
A little scared for brave Carol! She's a trooper! Glad all went according to plan. Video very insightful.
I had the issue of FSD avoiding puddles in the road. It kept changing lanes to avoid them. These puddles were bigger than potholes and a human would know they could safely be driven through (not a flooded road, just puddles in some lanes)
Except some puddles are potholes filled with water! Ouch!😢
@@phyzygy these were obviously not potholes. They caused the car to weave from lane to lane in moderate traffic. Cars in front were passing thru them without dropping so the AI should have deleted the difference.
@@russadams3008 Maybe it did delete the difference, in which case there's no longer a difference. Because it was deleted.
This is your funniest video so far! :). Keep up the good work and congrats to your fantastic wife. The support of your loved ones is important for reaching the next level! ❤
more of these avoidance videos with every major beta version pls! Also, sending reasonable love to Carol
Lots of love for Carol for participating in unnecessarily dangerous experiments. I can't believe they let you test an FSD this bad
On the 2nd attempt (StarLink box sideways), the box upclose looks exactly like the rest of the shadows on the ground. I think as it got closer it "forgot" there was something there (AFAIK current FSD has no object permanence) and just saw it as shadow.
I wonder if it would become sentient if you parked it in front of a mirror.
I wonder if the radar would have made a difference. I believe this car has no radar. They removed it from the design.
@@superkikim4916 FSD beta doesn't use the radar even on cars that have it. I don't know if radar sees cardboard or not, it definitely doesn't see some materials. Radar is mainly for metal (other cars). For this case vision is definitely better.
@@kalmard radar would have known it can hit it. but its a waste of resources for the system. as elon said they are better off improving it in general.
its amazing how a video like this can make me laugh more than a tv-series episode designed by a large number of professional comedians to make me laugh.
I've experienced Teslas completely missing small animals like raccoons. I think you're completely correct about the minimum height of objects thing.
This is exactly what I've been looking for to see the capabilities. Also, it looks like it is actually learning. Great job Carol.
Deke - It seems like it is taking the "Training" reasonably seriously! 🤔🧐
in watching the Repeats, it also seemed to me it was figuring out that it was not good to hit it, so the next try, moved over to avoid the Obstacle!
Cats & Squirrels? Low Height "Objects" - But, the are not always "Moving!" 😳🤔🧐
"I just got her a nice insurance policy" LMAO 🤣
Programmer: Human > object
Tesla: MOVING TALL TINGS > RANDOM OBJECT
She did a good job. 🤙🏽🤙🏽🤙🏽
It’s always a humbling feeling seeing a husband and wife working together..
Much respect.
My theory in the beginnin: As soon as the car can't see the object anymore it also changes paths. But then the flat wheel should've also not been picked up.
Another test with a cardboard box pulled by a string across the road would be really nice. Then you can "safely" slam into it instread of a wheel. As always a very interesting video and I hope FSD will come to Europe soon!
It's not a height problem but a memory problem, everything above 15cm is recognized as an object by the cameras but as soon as the object is outside the camera angle, the software forgets that there was an object and decides to drive over it.
That's what I'm thinking as well, as soon as it leaves the camera vision it leaves the neural networks understanding of the world. The car happily reroutes to a straight forward line through the object because as far as it is concerned the object no longer exists.
Thanks to your wife for assisting in the testing. Tesla FSD Beta handles moving objects much bettie as the algorithms are focused on moving objects in the road. It is the stationary ones the technology struggles with.
1) What will FSD do when there is a massive pothole in the road?
2) What will FSD do when the object in the road blends into the road surface?
3) What will FSD do if a person wearing black had a green light on their head?
I have seen people investing time to discover how they can trick FSD into seeing something and making wrong decisions based on what it 'sees'. Great work.
my wife does not allow me to show too much love to your wife :) thanks a lot for this experiment to you and your wife!
Cool test. The corner cases keep coming. Wheel recognized as a car. Makes me remember the Bicycle strapped to a car case
I think when the AI sees an upright car wheel in an orientation that is perpendicular to the direction the AI is driving it thinks it’s part of a car backing out into the street which is why is doesn’t want to go around and instead chooses to wait.
It's like you and a million other folks are working for TESLA for free. Brilliantly designed system to collect data and user generated content at the same time saving massive amounts of money on marketing, advertising, data collection, research and development, quality content creation (good job guys and gals for producing such a good video) and super valuable consumer feedback. Buying some stock Monday as it seams to be the bargain of the century.
Really nice benchmark. Maybe to time consuming, but if you manage to do this for all new releases or every other, it would surely show us when they release those avoidance updates.
Thank you for doing this! It's super, super interesting. I've been riding in Cruise AVs a lot in SF (so these scenarios you're testing for actually happen sometimes lol especially the truck ones), and the main difference I see is that Cruise seems to react much quicker and more confidently in situations like obstacles and pedestrians jumping into the street and further ahead with stationary obstacles than FSD does. I wonder if it has something to do with the power of the computers on board? The whole trunk of a Cruise is full of computers and you can hear the fans blowing away. I think when Tesla figures this out they'll be the leader because their system is more flexible, but as of now Cruise and Waymo seem to have quicker and more confident decision making.
It doesn't seem to be temporal at all. As soon as the box is not visualized for one frame, the route planner instantly routes through where the box was. As humans we of course know that box is still there whether we see it or not. Boxes don't just vanish. I'm sure they're working on this - the box should not be visualized for many frames before the car assumes it blew away or whatever
Yeah seems like the system needs object permanence and probably some sort of short term memory.
It's a balancing act. You might falsely detect an obstacle from far away. If you remember things too long, as you get close, you might no longer see it but still remember you thought you saw something earlier. Of course, a human would either decide they can obviously see the road is clear as they get closer, or they would gently decelerate because they're not sure. "Gently decelerate" is generally the best response, but the issue might be that FSD gets false positives too often, and it would be constantly slowing and unsure of itself. So I guess if they can make the obstacle detection more accurate, they can turn up the temporal aspects so it remembers for longer.
Your wife is a real trooper! If that had been me and my wife, she would have said, "Yeah, no way. How about I drive and you walk out in front of the car?"
Do you think it would make a difference if it had a camera on the front bumper to help detect lower objects?
IMO: I think the closer you get it the car does not see the object since the cameras are on top. As you can see it makes corrections the further you are but as soon as you can close to it FSD like kind of forget the object is there.
i doubt it. its judging the object as it approaches and it definitely knows its there. id say the difference is in what its detecting it as. it might even take into account the drivers eyesight. IE. FSD beta watches your eyes, if it knows your looking forward and can see it and you are ignoring it, then FSD can probably ignore it since you would take over if it was something it shouldnt hit. This isnt something they would want to rely on in the final version, but for training,... you never know.
Thank you Carol! Excellent collaboration.
Very nice video!
But I think you missed the brief moment at 2:22 where FSD did visualize the cardboard.
I hope this system gets more control in the future and isn't overridden at the last second. Having gotten rid of the ultrasonic sensors, Tesla seems to want to develop the occupancy network much further. After that, the test should be a little more positive, since the close range is probably the problem.
The "really nice insurance policy" comment made me LOL
I think the false positive prevention threshold is the reason it simply doesnt react. Its safer to let the beta team annotate some data and swirve manually, rather than the car slamming on the brakes one too many times. If this is the case, the system should become much better over time.
This. They have to balance between making the drive safe and comfortable. Most probably on the roads there would be many more false positives then not recognised real cases. In other words, there are fewer cases of having a box in the middle, than having a kid running through the road, or a motorbike braking, etc, etc.
@@SzabolcsSzekacs the vast majority of difficulties in software come from crossing the 99.999% threshold. Tesla can release a unified stack with parking and madmax california level merging right now, but it'd only work like 95% of the time and be offensively unsafe.
capability != ability
"Breaks"?
@@bobjohnson1587 o snap!
Dude it smashed directly into the boxes
Makes sense actually, from far away the top of the box would likely appear above the road, making it a clear 3D object. As the car approaches the angle between the camera, the box, and the ground grows. This perspective probably causes the car to think that these are just road markings. Putting a camera as low as possible in the front of the vehicle would help resolve this.
did you press the report button for the tyre and starlink box?
Great video! We appreciate the wifey putting her life on the line!
I'm hoping Tesla has this kind of testing internally as well and that it's high up on the list of things to fix... Elon said many time the prime directive is "Do not crash" so I'm expecting much better results.
This whole FSD is still in its infancy, It will get better. Ten years ago I wouldn’t have guessed that we are as far as we are with autonomy
@@ericwiese7479
It's a very nice technology.
Just compare what it's aware of to what humans are comprehending.
This could make the real time decisions way more complex then what humans would consider in the moment.
If I stop hard then I will cause a 5 car pileup behind me since 2 of the cars are not likely to have auto braking and none have decent following distance.
If I slow to 10mph I'm 87% likely not to cause permanent damage to the adult human running in front of me,
And 93% likely to avoid the 5 care pileup behind me.
With a 40% chance of loosing a 5 million dollar lawsuit for not stopping...
@@cifey definitely in its infancy. The real world data the FSD beta testers are supplying Tesla’s Dojo supercomputer will help a lot. Plenty of data points
it's not really a threshold that they set, the issue is the cameras get to see the short objects from far away but the closer it gets, the less they can see it because the cameras are sitting on top of the windshield, close range blind spots pretty much, if they kept the radar they had in their older cars, all of these issues would be much easier to deal with
Your nuts to brave these tests at such high speed. I would recommend 15 or slower. But hey no accidents so we are good! I can’t wait for my FSD computer to be installed.
Your channel literally gave me interest into tesla's autopilot, something that I didn't really care about before
That just shows the quality of the content you create, so thank you!
If you have a small not too dangerous object on the road the safest way might be to hit it rather than swerving or braking hard. It might not have chosen the most optimal path, but arguably the safest. Of course a lot depends on the particular situation.
Given there was no traffic, and high visibility, it definitely chose poorly.
It’s the EGO confidence, that override is what cured the jitter that happens during curves
“Ya Weirdo’s” 😂😂😂
Very brave and patient wife! For sure, mine wouldn't had had the patience 😅😅😅 Thanks to both! 👏👏👏
am I imagining things or does it actually learn and improve after the first (failed) run with those initial objects? So cool!
I don't think it's learning so much as choosing a different route each time. When it's planning its going through it's options many times each frame. Eventually it settles on a plan and executes that, so I think your seeing it choose a poor plan the first time and then the second time, with the situation slightly different, it chose a better plan.
I thought the same. I didn't think it did any sort of learning whilst driving (just executed the model it already had) but the tests did seem to indicate it might be remembering that the first path was a mistake.
@@clonkex It appears that way but I don't think that's how the neural net works. Its an illusion brought about by minute differences in each attempt forcing a different planning outcome.
They train at home base then deploy to all. That way they can retain consistency across the fleet. The FSD computer architecture doesn't even have much on board data storage that I'm aware of. The memory would be flooded with data too quickly to be useful for a purpose like learning on the fly. If he had attempted this then gone home and received a new update awhile later then attempted again that's where the learning could occur and you may see actual improvement in understanding and executing the maneuver.
Great video, very insightful! I think it makes sense to have a vertical threshold. When the car is far away the uncertainty about the height is likely higher so it starts planning for it but then as it sees it's under some vertical threshold it ignores it. As they improve the occupancy network they should be able to lower that threshold without introducing false positives that could lead to "phantom braking".
"not too much love you weirdos" :D yeah, sad this has to be said
Good job on that video to both of you :) I have yet to let the car handle things when there are crossways involved, here in norway pedestrians have the right of way if there are crossways without traffic lights, and I have yet to see that FSD accknowleges that.
OMG AI DRIVER ALIVE
Excellent assessment and real world testing w/ FSD and reaction times. What’s also interesting, is that FSD seems to be LEARNING as you move through similar iterations of obstacles. I could be wrong here, but I really think the R&D engineers purposely have the BETA algorithms feed data to the Tesla servers when multiple mimic obstacles are presented within under minutes of each other. Even so within a 24 hr period. This tells me that Tesla and Elon has a learning software that gets BETTER at dodging obstacles on that GPS side street. Could it know the obstacles iterations within the GPS locations?! Perhaps. But clearly the computer and software IS LEARNING!
Thanks again for your assessment. Thank you for sharing with consumers. I’m here for technology and science to help make the world a more safer place, all the while enjoying creature comforts. Big oil industries days are limited - as more and more consumers are introduced to EV technology.
EV and AI taxi 🚕 cars are coming. Lastly, I have two FSD vehicles in my arsenal. I build up my confidence utilizing FSD on non surface streets all the time. From my subdivision/community mailbox, to my home. FSD is brilliant in knowing how to switch lanes- AFTER I PURPOSELY park next to mailbox, in the opposite incoming lane. I start up the FSD-SUMMON feature (without anyone in the car), and every single time- it pulls out from the opposite lane, and merge to the correct lane, and travels to my house-(while I walk along side monitoring) with neighbors mouth wide open! They simply can’t believe this technology is here.
One neighbor tried to call the police and threaten me that I shouldn’t trust the technology because the car could accelerate into their house!!?? 😂 😂. I assured them- that it would not! They still recorded me, and sent the video to the local sheriffs office. I’m still awaiting my citation. Anyway, thanks for educating the public in EV, AI, and FSD TESLA cars. Tesla is far more advanced than any other car/technology company when it comes to transportation.
I’m ready to run in front of your car, if you pay me a live insurance
Thanks for the video and this kind of testing. Be interesting to see what the improvements you are talking about do when you get to try them.
what does the tesla do if it cannot switch lanes? break to full stop? why does it not have emergency lights on when on full stop?
Digging the opening humor, thanks very much Carol!
I heard that tesla wants to erase ultrasonic sensors called parking sensors. Some low and small obstacles are below the hood of the car, so the cameras do not cover it.
If the car is in motion, it should be able to see such objects over time and model their behavior. When parked, I expect an updated Sentry Mode can model the changing world around the vehicle to prevent accidents when returned to driving condition.
It seems like obstacles that it can’t see after a certain distance it ignores. If you look at what it hit (or got close to) they were mostly objects that barely were tall enough to hit the bumper of the car.
That fact you have a wife engaged enough to help you with this, is amazing. I envy you!
this channel deserves more views, really interesting video. keep up the good work 👍
I had the FSD Beta dodge around a dog walking out into the road yesterday and was totally impressed
Love the camera transitions and sounds
@7:30 Why did I not expect that? 🤣🤣
I loved this video it was super interesting to see how the car handled these objects, especially when somebody runs out into the road!
Great test! Looking forward to more targeted testing like this.
I suspect the detection system might be dialed to not be overly sensitive in regards to potential objects in the road since the occupancy network is new and false positives would lead to a lot of phantom braking. I expect them to tweak the sensitivity over time as the system gets more confident in its detection.
This is the best FSD channel out there. It will blow out soon, I'm sure of it.
In the cardboard one where the Tesla ran over it the first and then went around it the second time. It could be possible that the Tesla is some how learning about it and fixing it’s mistakes.
Very interesting videos and love the work both you and your wife are doing.
One thing I did notice, her hair was visible when using the window shade surface so I am uncertain of some of those tests.
Attempt with the window shade covering some wood/foam that isn't exposed whatsoever that you may create an easy lean using some thin wood making triangles. A right triangle will hold it perpendicular to the floor that it may be too easy for it to fall like test 2.5 microphone arm box when it was standing up. So you may want the triangles side that touches the window shade support to be slightly at an angle inward so the object does not fall.
As you said, despite it just being a box, it may still have things inside it. And I do wonder if height is important. Say something like, "given object taller than 1 meter, drive around."
That being said, alternative to this test. Do these tests with light hanging objects with one of those clear almost invisible string. Just go to a branch and heighten the object little by little and see if the objects(at least cardboard) is treated the same.
It's a pretty normal occurrence for a piece of cardboard to blow onto a highway, in those cases, it's actually important that the car to keep driving and go over it. So I'm not sure whether it's doing anything wrong here. I guess it depends on whether it can actually tell it's a box. I'd love to see a test done with a short object that isn't a cardboard box (or is, but has been made to look like something else)
It's not recommended to drive over cardboard. You never know what's inside. When you can see it from a distance, the best response is to go around, changing lanes if necessary and safe. In this case the car had ample time to safely go around, but it kept ignoring it once it got close.
Best FSD and wife channel on TH-cam.
Jesus , I'm not McGuyver myself but just put something small and heavy (like a couple of rocks that you may be able to find at the side of the road) at the bottom of the mic arm box and voila' ! it won't fall.
Thanks for your work as a test technician Carol!
Ty Carol for throwing dangerouse objects and risking your life for your hubby xD makes good content!
Great video so far.
I am 3:27 seconds in.
Idk if u do this later in the video but if u don’t next time car u try with 2 cars u behind the other car and we can see how fast Tesla reacts to it. Cuz 75% of the time we are trying to avoid a accident/debris
You should try rolling in a basketball or soccer ball next time! Happens often where I live and it's usually followed by a kid chasing it
Very nice demonstration of how the FSD beta fails in special situations... personally i find this very dissapointing and my trust in FSD really decreased.
Humans are still much faster in reaction and would brake very quick in uncertain traffic situations. On the other hand when humans get distracted they don´t see anything that could be dangerous , so it´s nice to have a backup system which has it´s eyes always on the road ... let´s hope that there will be an updated version of the FSD that also recognizes small objects on roads and avoids hitting them. Just imagine that there was little rocks on the road ( which could have been fallen down from a truck or downhill by landslide or similar things ) ... that would hit the car and destroy the complete front ... OMG.
Thank you very much for both of you, you have a wonderful assistant, she did perfect work there !
And i just loughed out loud when you mentioned that you have a life insurance policy for her ( Minute 7:24 ) That humor is brilliant !
I have had an Insta 360 X2 for about a year now and absolutely love it. It’s my absolute favourite camera and I highly recommend it. And no I am not associated with Insta 360 except is a very satisfied customer
What’s good about it?
@@NewCastleIndiana because it takes 360° photo, and I can compose the final picture Long after I’ve taken the photo and get just the results I want.
When there are many different things happening around me I can take a 360° movie of everything and then pick out the individual pictures later. It’s waterproof and this summer On vacation in Jamaica I got lots of great family pictures of all of us playing in the ocean.
It has much higher resolution and takes much sharper pictures than my old 360° camera.
@@Scott4271 OK. I’ve had a handful of 360° cameras and the resolution was really really bad.
@@NewCastleIndiana Yes, I’ve got an old Samsung that was just terrible
“Bought her a good life insurance policy”. ROFLMAO.
Thanks for that - great test. Cheers from Oz. 🇦🇺
I think what we're seeing is that while far away, there's nothing urgent, and it just plans for the object. But when it gets close, the level of uncertainty about what it is seeing contributes to uncertainty about what that thing might do, so a much more urgent/cautious adjustment is made, mainly by braking. Hard to cause an accident by stopping or slowing to a crawl (tailgating car behind being the exception, and even then its the tailgater's fault). We may also be seeing the Collision Avoidance System kicking in there and overriding FSD beta, as it might override a human driver. With FSD beta active, it may have the same CAS active as with human driver, but I'd bet at a lower threshold to kick in. Or something roughly equivalent to the CAS may be folded into the FSD beta perception/planning stack, overriding other priorities when it deems appropriate.
The whole thing proved how inteligent it is. It didn't stop for some cardboard cause it literally knew its cardboard. It can detect hollow objects. See the tire (layed) was same height as cardboard and it detected that it can go thru cardboard but not thru tire. That's a good thing cause imagine if it did full emergency braking for plastic bag on the road. It's kinda crazy how smart it is. And for the fast tire welllll it would stop eventually.
Man I love the Insta360 product line. I have the One R and it is amazing.
It's so cool that you are doing this real world testing which I'm sure Tesla is doing but just like the guy driving streets in NYC new things come up that no one ever thought about. I hope Tesla is monitoring your channel!
Good video. Love that you got her an insurance policy.
If you watch the pathing @4:35 it looks like it's detecting the fence at the end of the road as an object. During the AI Day it showed the cars network "predicting" cars that may be coming (even if it doesn't see something) and as it would have to swerve into the oncoming traffic lane. I wonder if that's why it's showing the right move, but failing to make the move.
It would be interesting to see how far away fsd can recognize cars in the passing lane on highways. Here where I live people can legally drive 120mph+ so it should be able to detect it quite early to not cut them off and cause an accident. Would be awesome if you could test that aswell
Great video. I don't want to see this self driving on the road soon though it's not ready yet.
Just found the channel. "Some love but not too much love you weirdos" lol 😂subscribed. Great work
Love your work Carol!
The starlink box is all but camouflaged. Camouflaged objects are difficult to handle with vision only systems. Systems seems to be much better with objects it is quite familiar with like cars and people.
Great intro with the appearing and disappearing text etc. 🤩
Thanks! :]