ฝัง
- เผยแพร่เมื่อ 7 ส.ค. 2024
- You'll see the Vicon AI Drone's first ever attempt at a brand new track that It's never done before!
Follow up to our AI vs Human Drone Race, in that this video covers the whole process of getting the AI Vicon Drone to race a brand new track for the first time. The whole process was done in under and hour and that includes us designing the brand new track for it and once that was completed, the team could then digitise the track on their PC to begin the Virtual Learning process.
Website: bmsfpv.com
If you have any questions or would like to use content from this video, please ask first. Contact Details in channel about section or use: bmswebyoutube@bitmaster.com.au
00:00 Introduction
00:33 Discussing the performance of AI vs Human Racing
01:12 The New Track Design
02:12 Vicon Drone Recap (From Previous Video)
02:39 The Process of Digitising the Track
03:14 Training AI in Simulation
03:36 Thomas testing the Track
04:05 The Simulation Progress
05:31 Explanation Drone Orientation
06:08 First Real Flight Attempt
08:05 What it would look like if the Vicon system had an FPV Camera
Gear that Thomas Runs on his Race Fleet
- - - - - - - - - - - - - - - - - - - -
Radio Master TX16S
www.radiomasterrc.com
JS-1 Overview
• JS-1 The MultiGP World...
Battery :
Tattu 6S 1300mah V4.0
www.genstattu.com
Current Gear:
Frame - BMSRacing JS-1
impulserc.com/products/bmsrac...
www.team-blacksheep.com/produ...
www.risingsunfpv.com.au/colle...
www.phaserfpv.com.au/products...
www.defiancerc.com/collection...
Motors:
BMS Racing Raptor V2
USA
www.getfpv.com/catalogsearch/...
pyrodrone.com/pages/search-re...
www.racedayquads.com/
reduxair.com/search?q=bms
Australia
www.phaserfpv.com.au/pages/se...
buzzfpv.com.au/index.php?rout...
www.mantisfpv.com.au/t-motor-...
France
www.drone-fpv-racer.com/ - กีฬา
Wow! The FPV feed of the AI drone is crazy! The way it goes through the gates what ever way is fastest is nuts! Thanks BMS Fam!
It's really trippy isn't it 😂 orientation has no relevance when it comes to that flight
@@BMSWEB that's a fascinating way to put that. We puny humans rely on that camera facing forward. The all the AI knows is 4 motors equally spaced apart, so it just points and shoots 🤣
No doubt, my brain was hurting! By the time my brain figured it’s orientation it threw it off again!
Amazing Tech !
Keep up the Great work
@@BMSWEB I wonder if anyone has tried to fly with 360 fpv or placing the fpv camera in the center and pointing it 90 degrees. Maybe it would make this possible in humans.
Chasing you down the street at night screeching "halt citizen!" Because you missed the trash can and it thinks you littered
Not a good future being built
*Beep Boop* "Pick up that can"
wouldn’t work since they’re too loud and people would complain.
That fpv footage at the end is terrifying. Very unsettling, very impressive.
I do a lot with AI, but this is freaking cool to see real world action of it! Congrats, that was definitely a success! Awesome job!
Oh man I've been seeing some of the upgrade work they've done since this video and its even more insane. Super excited for a rematch next year, but I suspect in a controlled environment there won't be much a competition for their creation!
What a time to be alive hey? :D
-Thomas
@@BMSWEB Yes, the bad thing about it is, the military will take advantage of it first and use it... We made many innovative devs, but it was always the military who had in the end the right to say if its for public or not. And drones you can see it in the Ukraine war... are the generation in which we are living... right now. AI at it self is really cool, like a new dimension. I often compare it to LEGO or Fisher-Technik, when you was able to start to build your ideas :-)
WOW! I'm 60 years old and only just starting to get into this whole quad scene. I wish i was 20 again! I've had an Atom camera drone for a few months and love flying it.
I just bought a Boxer and 3 Sims to practice on before i either buy or build my first freestyle quad. My reflexes will never be near you guys, but i can still have fun with it.
If i can get to be 20% as good as you are Thomas, i will be more than happy. That made me smile bigly after that AI run :)
We can't even imagine where this tech will take us in 20-30 years time. You guys are living into a fascinating future. Make it count.
That FPV view at the end is just crazy ridiculous I love it
that shit was going backwards
I know nothing about AI or Drone-Racing but TH-cam thought I should watch this. And Wow! That was f… fast. Keep it up, people like you will make our world a better place.
I'm glad you enjoyed the video! Ha ha drone racing in general is a pretty crazy sport, hard to believe it didn't even exist a few years ago! I'll do my best!
-Thomas
i love that FPV, it really show how the AI does not care what is "front" and "back" like a human, big advantage
It's pretty cool to see it in real life. I'd love to go back through the footage and see if there are any movements that are easy enough to pickup using this method. I don't think there's much but surely there's an advantage for human spec racing there somewhere!
-Thomas
@@BMSWEB the camera see 360°, you could get something similar with multiple fpv cameras + VR headset.. The issue is the latency, keeping them < 100ms on digital is a pain (aka: VERY expensive)
the entire concept of ai race drones blows my mind. phenomenal footage!!!
I remember being curious about what an FPV might look like. This is great.
That gave me chills alone just watching the full send
Amazing! Thank you for sharing this journey!!
Thanks! :D
Commenting just for support! Amazing tech!
Thank You
these are going to be chasing us real soon. thanks a bunch
those things exist and are capable of chasing you for more than 10 years now...
This crazy!!!! 🤯 They done a good job wow! Thank you Thomas for sharing this impressive project that the University of Zuirch has!
This is so sick, thanks for the follow up vid!
Thanks buddy, it always surprises me when you comment on our vids! - Paul Here
damn! its very amazing man! great job for teams
Amazing! Thanks for sharing this!! Best regards from Brasilia, Brazil.
This is beyond interesting to me! Thank you guys!
incredible video and production guys! May it gain many millions of views :)
How cool is that?! Wow!
super cool!
Cool. Very cool. Niiice videorrr guys
Incredible watching the FPV view of the AI drone. It completely has no regard for typical forward motion like us humans need for most of our flights. It'l just fly at whatever angle it wants to get the job done.
Some of the sim guys are pretty insane and starting to fly a little like this in some corners, but to see it in real life is mind bending!
-Thomas
I think human brains are capable of it, it's just a matter of training and forcing yourself to "fly blind". Perhaps not to the extent shown here though, as the mo-cap data allows the drone to "see" the entire course at all times.
That is so awesome.
That was insane.
Wtf, this is completly amazing.. Holly shit :D
Awesome work guys!
Cheers!
-Thomas
super amazing lot of work behind
this is sick
absolutely insane!
That's awesome
Holly shit. That was fuckin nuts. I was not even there, and I felt the adrenaline rush.
Re watching the flight footage is still mind blowing for me. I dream I can fly a quad like that 🤩
-Thomas
So amazing love this pls bring more
We've probably got at least one more video on the AI vs Human race that we can do for sure. It's just been so hard finding the time to edit these videos. We've still got the Aussie Nats, WA State Titles and also Thomas' Japan drone Racing League race which will be so much fun to watch. It was a pretty eventful race that one lol
fantastic
This is insane
INCREDIBLE!!!
I don't think any AI "Pilot" will be capable to beat Tomas. He's way beyond! Can't wait to watch the race! Nice project by the way and thank you for sharing this. Your BMS Fan.
I am not sure - computers have every single advantage over us and once the AI get's enough flights in it will be able to fly at the edge of the drones aerodynamic capabilities without errors and it will be consistent and will or can keep it at that level of performance for as long as the drone has batteries... However I don't think it matters, we as people should just enjoy what we are, what we can do, etc, have fun pit racing, talking with friends, actually racing and working to improve our skills, etc.
The technology is mind boggling, AI is creeping into every single aspect of our lives and one day not too long from now and if we don't blow up the planet before then we will see AI everywhere. Now about job loss to AI, etc... a serious problem, a horrifyingly serious problem really. But I have an idea - it is one that I have had for a year or 2 now - let's ask AI how we can solve that problem.
Also, here, a little off frack but related to this video is the greatest idea ever in all humanity - I would say that we should replace 99% of all human politicians with AI. Think about it, an unbiassed, un-corruptible (don't say hackers... yea that idea is concerning, but let's stick to the core for now), what I mean is it would have no friends to give jobs to, no nepotism, no backroom deals, etc... it can work tirelessly for ever, non stop...
it will be smarter, make better decisions, make decisions that are the best for us, for the whole world. Solve the energy crisis, solve global warming, solve equity in society, solve everything, cure diseases, and who knows what else... and it will beat us at any game that we challenge it to. I for one welcome our new overlords. Cheers?
its like he said, maybe it cant match his peak run, but in terms od consistency it can be better than a human pilot
Ai will beat us all soon
Idk if you saw that end footage but damn, no FPV flyer is gonna beat that. Give the AI a few more iterations and a longer simulation time and it's gonna be unbeatable.
@@rizen3467 No I get it, but the computer is capable of such precision, perfection, ability to repeat exactly what is needed lap after lap... But also, who cares - so what if it is 100 times better then us, maybe we would learn some new ways of flying, who knows, but I do this for fun, and I am terrible and that does bum me out sometimes a bit but generally I just enjoy trying, talking to friends, seeing what I could learn, etc. I came from racing motorcycles for run but I can't really do that anymore and I really miss the tracks and racing and everything that went with them. For me, FPV seems like it is tapping into that same need. Cheers
Awesome
That’s crazy 😮
nice!!
Pretty crazy to see that thing full send maiden on that track 🤯💥
We honestly had no idea what to expect and hence why everyone made sure they were well and truly behind that net 😂
This is badass
Increible!!!
Keep those Cold Fusion vibes coming. 👍
Thanks 😊, more like Mini Fusion :P - Paul Here
Holy Moly!
Finally arrival
Insane stuff but a little bit scary as well lol
i will follow you because I like what you do and want to learn what you can do
FC From Thailand
Dat fpv flight at the end looks like mck Line of sight
INSANE!! The fpv view looks like its flying it LOS
It doesn't use any yaw, which is what creates that insane perspective. I believe they have another version that keeps the camera oriented more forwards with horizon level!
-Thomas
Considering the way it detects the position and orientation of the quad is via cameras mounted around the room, it technically is flying Line-of-Sight....
A nerrr drerrn racing videorrrr. Nice!
Thomas is editing the Japan video right now! I think you'll enjoy that adventure
nice
Is it me or did you use "The vicon drone" at 2:10 again at 2:12 to transition between both scenes?
Also, seeing that drone lift off and just full send it made me audibly gasp, that was amazing.
ha ha yes they were from two different recording days. We learn't a lot about voice over setup and made improvements. We could have re-recorded the middle segment but we have so much other footage to push out atm we decided its better to get the information out rather then linger on it and have the info become obsolete!
The launches are insanity, I don't think I can move the pitch stick quickly enough let alone manage the throttle to pull off a launch similar to that, but I so wanna learn it because it looks so damn cool!
-Thomas
@@BMSWEB Might be able to set up a "launch control" like mode similar to what cars have. Have a separate button set up to turn the motors on full blast for the start, or the reverse. A button that allows you to input full throttle but it not send to the drone until you let go of that button. That way you lose the stick movement input time and a tiny bit of the human latency. No idea how one would code this... or if you would need to code..
Very interesting. Would like to see the same but the turned FPV camera to the moving direction.
Crazy
Just mad!
This is a very interesting approach to for flight control. I really like the idea of tracking markers. It would be interesting to see what kind of other sensors could be used to gather info about the environment and what could potentially be used for a on-the-fly ai-learning flight system.
I worked for Fortem Technologies and we developed a swap-c radar to track drones and shoot them down. I had some ideas that I would have loved to explore, but the path that was rod for the company didn’t really allow for exploration in some of those directions.
I love flying fpv and auto racing, I notice that a lot of the fpv pilots don’t take the optional lines, because humans don’t have reaction times necessary, and I’ve always wanted to know what would happen if AI was used. So it’s interesting seeing this happen and extremely satisfying!!
Well keep in mind humans also fly with one camera so the orientations of the quad is limited, where as this thing can fly in virtually any direction. It's a fascinating approach for sure and I'm sure this sort of tech will allow for some amazing breakthroughs that can then flow on to their Vision Based Drone which realistically what's going to be more practical in the real world. Irrespective of everyones different takes on where this tech will go, it's going to be interesting times ahead - Paul
@@BMSWEB That was something I hadn't considered, that it can fly in different directions than just the one facing the camera. As soon as I saw it, I smiled and thought that it was clever and also a very logical step in being more efficient in some of the cases.
One of the things I was curious about was how long it takes for them to run the simulation to learn the track. It would be interesting to know the tradeoff of flying slower versus having a high enough resolution to fly the track. When you said something about 32 minutes since designing the track and about 40 minutes since the track was created, how long do they need to run the learning/simulating part of it for the drone to fly it?
wow, this blew my mind. I now need to at least try to use the TBS module with a pc.
The FPV feed looked BRUTAL, but is it really faster being that brutal ? normally smooth is fast. jerky just looks fast (at least in humans?)
edit: unless it's only fast for us? as in better for our brains to process faster? passing the gates at uncommon (for us) attitudes might be faster, but the jerkiness idk.
edit2: the throttle control it has is AMAZING
You have it basically right, I feel. Smooth is fast for humans in large part because they need to use the camera to see and control. The AI isn't even using the camera at all, so any sense of stabilization, for it is unnecessary.
Still quite impressive ofc, I just feel the FPV is a bit misleading because of that. It doesnt have any reason to point it in the right direction even.
Looking at the lap time, the ai did a lap in about 1/3rd the time of the human pilot.
So yes, faster.
Interesting to see how the AI uses the drones control in ways a human would not, Just based on beinng able to see where you are going as a human. It flips and turns Like a human flying it with their eyes closed Just based on muscle memory almost. Pretty trippy. Makes you realize these lil FPV drones are capable of a ton more then even the best pilots have achieved. Stuff a human may never be able to do.
Thomas awesome stuff, super cool to see the progress they make. Do you know what’s the next step for them? Is it always about fully controlled environment or they plan to develop something usable everywhere?
I assume they will finalise this experiment, do a scientific paper on in it and that will be it for now, until someone decides to develop it further. It just depends and where and how they see it used. - Paul Here
Awesome video.
Impressive feat and really interesting to see the fpv camera footage at the end -- didn't even appear to be racing tilt but was aimed very up, very inhuman flying.
I wonder if it can give you ideas for human racing, like deepblue, alphazero gave human chess players totally new ideas to approach the game
Cheers! We we're discussing the idea of using a variant of the software that flies, keeping the camera at a more conventional angle. The idea was you could throttle limit it in software to your pace and watch how it's flying and try to replicate it/learn from it. I think this is something that's totally possible!
Interestingly the code tends to favour holding full throttle and using a sharp turn in, to get some of the motors to wind down and thus reduce the momentum and allow it to turn. Basically it's a super sharp turn in and it slowly eases off the rotation, with the throttle command always pinned!
-Thomas
@@BMSWEB ah that’s interesting. Harder for a human with ~165 deg fov to replicate, but a good lesson.. probably also requires a bigger lipo on a big track!
I was also thinking it could be amazing for testing gear, once it got very consistent.
Eg, control test different props, motors, tunes, etc. and let lap times speak for themselves.
The visual AI version would be better for that as it would fly more human-like lines - not that the University would be interested in throwing in resources at this! But give it time
@@nickgreenemail We actually discussed that with them, since it's one of the things we have to deal with on a day to day basis. So when testing props or motors etc it requires a lot of flights that need to be consistent etc and then also a lot of SpreadSheet data to work out how the data is trending, while making notes on what data to ignore (ie clipped a gate or made a mistake etc) This would pretty much speed up that process in a big way. Yep many applications for this, but it could get pretty costly.
- Paul Here
@@BMSWEB I hadn't thought about using it for testing purposes. If the AI copied some human pilots flight styles the testing would be consistent and more true to real-life scenarios.
@@brian7android985 Yep for sure
I want to be able to map a city with the Mavic 3 or so. Or a 4x4 squared miles grid that I can then plug into a virtual flight and then fly the city in FPV simulator?
Can I recommend if you send viewers to watch another video before this one, to put the link in the description 😉
This was such a good watch! Some amazing results. Someone get these guys some ELRS gear 😅
Tracer is crazy fast though. They’re probably using it to minimize latency.
@@Flumphinator elrs can go 1000hz...tracer only 250.
There is a latency and consistency component to it, but on the refresh rate side their Tracer operates at 1000hz already :)
-Thomas
@@BMSWEB thanks for the clarification. Have you guys experimented with ELRS for your own racing? I have been pushing for a decent elrs + vtx combo board...
@@BengalCatChilli We did ages ago and it worked great, however stuck with TBS mainly due to approvals etc Now speaking of approved hardware, when Thomas competed in the Japanese JDL Series (Paul here btw) a couple of months ago, the only approved hardware/software for Japan is Futaba and BetaFPV Nano ELRS! Before heading over to Japan Thomas had heaps of help by the absolutely amazing ELRS guys and learned heaps, but when he got to Japan there seemed to be issues with the Hardware and hence reliability issues with it. Let me be absolutely clear here, the reliability issues are due to the Hardware and version of software that was approved for Japan. I think if you use the right hardware ELRS should work perfectly.
Does anyone know what langue they would use to write the program for the drone program?
How does the AI controller compare to a physics-based handcrafted method? With accurate, high-speed absolute position tracking you can control it basically with a physics model and PID tuning. I'd expect this to be much more reliable.
Hi! They've done this before! If I remember correctly they called it a model-based approach, or a roboticist's approach. Basically model the core scientific principles we understand of how the world works, and use that as the basis to guide the drone in 3D space. I believe the short comings vs AI approach was that even in a controlled environment, there are so many uncontrolled variables that the current model-based approach struggles to account for, so they have to limit its speed to account for error. I guess using a neural network, it can find patterns in these inconsistencies and push closer to the limit!
I believe they might have some coverage on the website and TH-cam channel for the Robotics and perception group about this!
-Thomas
Is there any way to get access to their PID tuning system for the quad? Imagine if that tech could be available for everyone to easy tune their quad based on flights.
Believe it or not that was running stock Betaflight PIDS! The AI just learns to input whatever RC Commands are necessary to make the drone do what it wants in the real world! After we got home, we sent them our PID tune so their motors would run cooler.
-Thomas
"oohh, that's nice, I like it a lot* -skynet
Was the drone flying 3d? I wanna see that next!!!!!!!!!
Also, external camera tracking is easy. Make it on board then it would be something
The first video covers this :D
-Thomas
When you guys were digitizing the track, you mentioned that you used the markers to define what angle the drone should fly through the gate at. When the AI trains, does it always respect this angle (and i guess inner gate position) or is it free to discover a "faster one" if it can? Do you guys make the gates a bit smaller in the simulation so it leaves some room for error in the real world?
It's free to discover it's own path and racing line. Only requirement is it passes through that section/waypoint in the correct direction - Paul
What happens if you move the launching spot few feet backwards? Does it affect everything?
since this one launches from a hover, it will just bring it self to the true starting point. I'm not sure about the race we did the day before, since those were from starting blocks!
-Thomas
@@BMSWEB very cool! I hope one day, I can be part of projects like that
The computer in the background at 3:00 looks so similar to mine that I actually looked over to make sure the computer I'm currently using was still there. 🤦♂
Not having to orient the fpv camera to actually see the course is another advantage the computer has here. The computer can fly in any orientation, since it's looking at the drone from the 3rd person perspective.
What sort of techniques does the AI team use here? Some sort of reinforcement learning?
Next gen military aircraft
it looks to me as if it were always snapping to either full throttle or no throttle. Also, doesn't the sending commands back and forth add significant delay? Humans adapt and train to such delays easily, but i imagine that for best performance, the main flight ai should reside within the drone with externally only some rough course correction data supplied. Additionally same ai should do the collision avoidance i think.
How long untill a drone can fly this well with just its onboard camera system and AI
In the previous video we do show their AI Vision Based drone and it does fly extremely well, however it's whole computing system is also on board which makes the drone much heavier. Also I doubt any onboard vision based drone will be able to the way this does because direction / orientation isn't relevant to this Vicon Drone
Amazing stuff! The AI feed is interesting. As drone pilots, we don't/can't fly that way because we need to have visual frames of reference. Is that on the roadmap?
Hi! The vision drone from the previous video purely flies from onboard vision! It's quite a bit heavier but it's an amazing piece of kit!
-Thomas
@@BMSWEB Oh wow. I had forgotten about the vision drone. Any chance you can get a hold of the video footage from the vision drone? I would love to see what the AI vision drone sees!
@@Waylandir It's in the previous video, we managed to get their footage from the vision drone. All the processing is done on board with that drone and hence the added weight and significantly less performance than this drone, but it is more realistic in terms of what is currently possible. I think the vision from it is in the part where Thomas explains what the Vision Drone is
- Paul
@@BMSWEB I see it now. It starts at 3:55. Really cool stuff. Interested to see the flight improvement. The small clip is impressive but is very reactive. The footage looks like a warm up lap of a human pilot just learning the track. But after learning the track, top fpv pilots learn where to shave time by anticipating and pre-turning. Very interested to see if the vision drone can "learn the track" and then apply that knowledge to anticipate and pre-turn. Thanks Paul! Hope you guys are doing great!
@@Waylandir Thanks buddy, the tech is really interesting, would honestly love to work more closely with them as I feel some of their logic is slightly off. And the flip side is we would also stand to gain from learning from them. The issue with the Vision Based system is it's currently very sensitive to light changes etc. So on some of the days it just didn't perform as expected. Having said that in time, I have no doubt they will resolve a lot of issues and have a more reliable system which is going to be insane.
We're doing great, lots of R&D happening now in house which is awesome! Exciting times ahead
I saw this video 2 month ago - how can this be? maybe it was on another channel
Hi, can we make AI fly Ukrainian FPV drones that target invadors?
Fascinating. How many simulation iterations are required for a 99%+ lap completion consistency? Also, what rate of iteration are the team using? Using phantom helper/guide gates was a slick move on the team's part!!! Good work Thomas, and Paul too!🥰
Cheers! You'll probably have to ask the university about that! They have an even newer version of the simulation software that I believe no longer requires helper gates! Pretty insane stuff!
-Thomas
@@BMSWEB thanks buddy! I'll email the team directly if I can. Exciting stuff!
wow alex vannover also involved
So was MarvFPV (Previous Video)
Main advantage is that the AI drone doesn't have to go "forward".
Das änglisch dönt chle schwizerdütsch irgendwie 3:25 hehehe
Well... Its interesting, but its pretty minimally useful. Its not flying FPV and none of the processing is onboard. It becomes really clear that its being treated as an abstract physic object when you see it going through the gates backwards and sideways in ways that a human wouldn't have enough information to make possible.
Its kind of like they took a stock game AI for flyers from something like Unreal 4 and glued it together with a massively expensive tracking system. Its just trying to fly a line through space. If you put a pole in its way it would smack right into it without even knowing it was there.
Hi! Refer to the previous video about the Universities work to understand why this specific part of the project exists, and yes your right, in its current form, this system doesn't have any object avoidance!
-Thomas
@@BMSWEB Yeah, I mean I know you understand exactly whats going on here, obviously, but my comment was more for other people in the comments section who seem to have literally no idea, despite the explanation, that this is NOT an autonomous vehicle using its onboard camera and some kind of onboard AI to fly it.
I'm not sure how they don't understand that none of those things are what is happening, but they don't.
my god
imagine chased by a killer drone like this
To check if movie is worth of watching check 6:50
Bagaimana dia melakukan itu
The quad got mad at a little mistake and smashed itself into the ground full speed😂
There's a lot of nerd testosterone in that room, I only wish I was worthy
순전히 재미로만 남고 나머진 ai로 대체될 듯
I really am concerned about the future...
4:47 - MineCraft ahahhah
The all mighty start of any science project XD
-Thomas
This is uncharted waters now, for millitary guys are salivating at the prospects of weaponizing these AI drones.
Leisure drone have been put to lethal use in Ukraine, now imagine what the addition of AI could do!
The future of AI drones is going to be scary af, I shudder to even think about it😯
eurver and eurver again
Now put face ID and strap some C4 onto it
NEVER piss off a drone pilot…🇦🇺
wake me up when the AI can control the drone from onboard drone camera alone
That already happened in the previous video lol