Magnus shocked by world’s most advanced climbing AI
ฝัง
- เผยแพร่เมื่อ 10 ก.พ. 2025
- Please like and subscribe if you want more videos like this!
Project's website: belay.ai
Regine's Instagram: regine_storaa
Shot by Markus Skaane
Edited by Sam Tuck / samtuclimbing
Magnus Midtbø tries Australia's national comp boulders (crazy hard)
I made an AI that emulates Magnus as your coach. But something is not working... basically says "More chalk, less shirt" all the time.
seems to work perfectly fine then
Magnus deep down would really prefer to climb ass naked and covered in chalk
smart AI
When you're on the wall it should say "come on" every 5 seconds
@@rlstrength good improvement! "Come on! Eaaaasy"
This guy should team up with Moonboard, with set routes and an option to film yourself on the route, it could end up being a personalised coach in each gym
Although this won't be setting or giving real advice on advanced climbs, it certainly could do very well on something static and predictable like a Moonboard, and with many hours of video to train the AI with. Other than that it will take years and tons of compute to even attempt to understand bespoke sets.
Ah that's a good idea. Exactly the type of application that this could work with. I wonder if you could even add pressure sensors to the holds to get any other useful data. Maybe, maybe not.
This is kinda crazy either way spray
@@Cyrribrae With how good the AI's ability to understand the visual data, combined with a basic understanding of gravity, angles, and folcrum, I'd say it could reasonably accurately predict how much pressure is being applied.
@@StewartCoates I'm actually not sure about that, I'm quite certain I could stand on the exact same position on the wall while putting more weight on different holds/feet. A super advanced AI with a 32k camera might still be detect it based on muscle contraction, but this is out of scope for the next 10 years at least.
@@CharveL88 maybe it could be helpful for beginners as well since it can have a lot more data for easier climbs.
This second channel is quickly filling up with the kind of quality content I've been missing. Good job!
I can't wait for an AI to tell me how I can improve my sit-up technique when getting out of bed for the morning, so that I maximize the energy I have for the day.
Call her Sendie
I threw my back out on that technique once
people sweating every aspect of life instead of live it
Sit up is the wrong beta bro, just roll
i sleep on the floor so that laying down is a negative resistance and getting up is a full body weight @@MeskDaKrull
Magnus, as a photographer/videomaker myself, I really apreciate the quality of your videos (filming, editing, sound... ). It would be cool to see a behind the scenes here in this channel!
Exactly, would love to see it too! Videomaker here too🙋♂
The dev missed a great opportunity to call it BelAI.
What about GlaDOS
@@El_Fling Probably copyrighted by the Valve Co.
beta sprAI
Yeah but it’s called BelayAI so it is similar ig 😭
@@neaituppi7306 Good point. GlaDnus with magnus voice
Insane quality for being a second channel lol. Doing gods work for climbing fans Magnus+Team!
love the simple climbing vids. looking forward to this channel more than the 1st
Too bad its missing data, but if it records many climbers on known route difficulties it could also potentionally set new routes and also predict the correct difficulty for a newly set route, you could also track progression in your climbing with this so measure how good you are becoming, also would be fun to compare performance, so if you have pro climbers then you can see how you stack up against pros and what you need to improve in your climbing. I think this guy is onto something, but there needs to be real app development and LOTS of data filmed from different gyms and routes etc for this to be interesting. Its very early stage, but the potential to build fun stuff ontop of it is endless, I really hope he has the drive or at least gets a team together that work on a climbing app with one or some of these features.
Agreed, I'd be really interested to see it set up in front of a moonboard or similar. Lots of movement options compared against known grades.
The website has career postings for multiple positions and a less than 2 min video that showcases many more functionalites compared to Magnus' video. Once the app is released it could possibly make grading a lot more streamlined with users flocking in and feeding data. Interesting how it'll play out.
@@hugoburgess3699 This would be perfect for the moonboard. It could get hundreds if not thousands of examples for each boulder
No, it can't set new routes because it is only dealing with the 2D physics of body position, not the angles of the wall; the size, shape and texture of the holds, or all of the other necessary factors for being able to set a route that matters.
This is a parlor trick good for generalized positional advice...IF it is trained on that specific set with lots of video of people doing it.
To have it set a new route that actually works it would have to understand the 3D shape of holds, wall, and body. Then it would have to understand friction factors of a particular hold, etc., etc. In other words, a long way to go without any data other than 2D body positions to work with.
@@FibberFinn Disagree completely. The only thing this AI will be good for is a cool app that helps beginners on known climbs, and generalized basic advice at best on climbs that aren't, so that leaves gym climbing out where sets are changed frequently.
Not saying there isn't a use-case for some app to use AI to help with basics, that will be pretty cool in itself, but don't expect an AI like this to be able to understand much more than that. Self-driving vehicles are only now just starting to get enough data to be useful and they have way less sensor input than the human body has and uses on a v7 climb, say.
The AI is only as good as the data it has to work with, and even though recent AI's can figure out basic physics on it's own, unprompted, there simply isn't any way of getting this to be more than generally accurate on a climb that doesn't have data of someone already climbing it.
I'm liking the side channel more and more!
Enjoyed the video and also really liked Regine's laugh when things went wrong. Fun collab
@Magnus, call her Beta 😊
Bertha
GlaDOS
Betty
Beta sprAI works well too!
BaiTA
Enjoyed the video a lot. Good to see someone trying the boulders with you too.
Having her there makes Magnus look even more superhuman. She's SO talented, but still can't send it like Magnus. Just puts into perspective how useless we'd all be in those same situations lmao
Magnus has always come off as so wholesome and genuine. I can sit back and watch these videos and feel like im sharing this time with a good friend.
The programmer did not do all the AI himself, a significant part of is from openpose, hence involving a team working of years.
I feel like that’s sorta just semantics. It’s like saying someone who made graphic art in Blender didn’t do it themselves because there’s a whole team who worked for years on Blender.
@@jamessmith4172 The right analogy would to take a finished blender picture and to add a few filters and claim "I made this picture". All the training and architecture is finished in open pose.
Hello, computer vision engineer here! Looks like this program combines an object detection model, pose detection model, and LLM together into a pseudo-coach. The model is likely trained in a fully supervised manner meaning the dev manually curated an annotated dataset of bounding boxes and poses from likely many TH-cam videos. As a coaching tool, unfortunately this will be hard to extend. The application does not appear to provide any added value than superimposing two videos and the LLM providing advice does not know about physics, hold quality, or body types which a real coach would factor into their feedback. Best to rely on human coaches for now.
Well, but this tool can give you extremely accurate movement distributions for every limb. Things like that is something no human can give with such precision. I think a tool like this can be expanded in many ways to give great insights into climbs. For one the ability to compare numerous results and automatically compile the best techniques for each move, like creating a compilation from the best moves out of many runs. This would get you closer to the perfect technique for your body specificaly with every attempt. I think the body type is factored in since the length of every limb is visible to the programm. I also think physics can be applied into such a tool, since it has movement data over time. There's tons of physical information in video, the software can in theory identify how fast things move and how limbs bend depending on speed on impact. There's a lot to this and the developers don't have to stick to the current models they are using.
Hello, also CV engineer here, think there is a lot of value to this system not directly as a coach but as a tool to objectively observe climbing and compare different approaches. I often have it while climbing that a friend struggles with one problem that is easy for me or the other way around and having an objective observer to tell you that you have a higher success chance if you move your hip differently or you need to shift your weight differently at this spot on the boulder would be great to understand what makes the difference between a successful and a failed attempt.
How often is the only advice you can give while climbing to "just do it" because it looks exactly the same between climbers, this system can see the difference and give feedback.
I won't have a coach while bouldering as it is a casual hobby, but just setting up my phone next the the boulder and getting some feedback would be great from time to time.
The LLM part of the demonstration is the flashy bit, but in my opinion the least useful, as you said, the context of the actual boulder are missing.
Fair point. But that's the way to use AI in most cases for now. I work in as an IT Engineer in the healthcare world. We have a lot of ongoing AI projects when it comes to research. It's very hard to have an AI that can come up with conclusion from all factors. AI for us is mostly researched as a tool for doctors to use and ease workload. I think there will be a time when AI can fully replace humans in some cases, but i believe it's gonna be a while until then. I believe this tool on combination with a coach could provide great success.
I think it can be an interesting tool to study the kinematics of the movement and output some statistics, nothing groundbreaking however. Also, there's plenty of literature on human detection and skeleton tracking, one can just use a pre-trained model or other approaches not based on supervised learning. The LLM part can be useful to generate some considerations starting from the data, but nothing compared to a real coach, that, as you said, can consider a lot of other factors such as the dynamics of the movement. These are just speculations however since we do not know in details the functionalities of the software.
@@generic13372 Like he said tho, you can do that by filming the two climbs and superimposing the videos. You'd be able to see the difference on your own that way.
i just got back from the climbing gym, so this was perfect timing to watch. great video and really interesting too
Awesome! As a person that is interested in motor learning researsch, I can say that the "angle and body position" data is not really valuable information to progress despite being interesting.
On the other hand, the conclusions/summary that the AI takes can be really really useful if appropriate language is used! (not focusing too much on body parts but rather on movement's result)
love this 😎
Feels like verbal explanations are hardly ever useful in climbing regardless of language. It works way better when someone demonstrates it.
@@777Mikos This might be true for the basic beta of a route/boulder (hand/feet placement, sequence) but once you get to more technical stuff, demonstration tends to be useless because it's about adjustment or body tension you just can't "see"
For example when technical moves require to shift your weight at precise locations/moments or when you need to engage your ankles to create body tension, verbal explanations is key
However I dont see how this can transfer to AI because I'm not sure it can infer this kind of technical advices from visual data only
Forgive my earlier dumb comment. I deleted it, honestly didn’t think you would notice, and I’m touched that you responded. The second channel is great so far and I, among many others I’m sure, have been enjoying all of it so far. Please keep uploading (I didn’t actually sell my kidney btw)
when magnus called the project an app i cried a little for the dev.
I had the privilege of not only visiting Norway last year, but also this gym. There are multiple floors you can Boulder on and a vast amount of top rope and lead climbs. The cafe had beautiful desserts and all the staff were super friendly. A wonderful gym that I know the community loves to have as a beacon for all climbers.
You just stated what was in the tour video...
@@Dr.Spatula Well maybe because I went to the exact place being toured… 🤣 I don’t need to lie in comments for clout homie.
Do more hard climbing videos. I love this and honestly learn so much. Keep these super awesome climbing focused videos coming. Love your channel. You are the reason I started climbing.
I love Regine's energy. Would love to see more videos with her.
she wants himmmm 15:37
a bit of a waste , putting her on strenght things she does not have. didnt get to start or finish anything . her gigles started to anoy me.. i actualy wonder why she was in the video .. she must be a girlfriend of some guy who was there..
For someone whose in the national team I was expecting more of her..
@@myjakuhintai8206That's what I was thinking.. it seems like marte is a stronger climber than all that. Maybe I'm wrong but. She seems stronger And more athletically gifted. Now if you want a super strong woman climber Magnus sister, you need to look no further. She's something else.
12:54 -- What a mind-bender of perspective. Looked at first like there's a hole in the wall.
I think the concept of filming many attempts, gathering data from them and then using AI to analyze that data to point out the differences and similarities is cool and probably quite useful, but I'd be really careful when it comes to letting the AI draw conclusions from that and tell you what to do differently. It can sound really convincing and smart when it tells you to "move your right shoulder a smidge to the left" or whatever, like a coach would, but you have to remember that literally everything it "knows" is just the data you've given it through the video footage, and then the generative AI just fills in everything else based on probabilities.
It will tell you what you are doing wrong if you give it enough examples of moves done right and wrong... but as a climber you'll know better at that point anyways.
One key of the AI part is in fact pose analysis/computer Vision. I guess the rest is comparing movements of each limbs.
AI can analyze you as a climber, know your limits, strenghts and weakneses.
Taken that in consideration witht the fact that it will self-develop more and more by watching more climbing videos,
He can really know what would be the best possible move for you personally.
Now even gym trainers will loose their jobs 😅
just like with anything else, it is more of a guidance rather than an absolute "truth" that must be followed. At all times, exercise good judgment and common sense.
@@Smixi35 Which should tell you why this isn't very useful for anything but Moonboard or beginner climbing. AI's are only as good as the data fed to it and this happens to be through 2D video right now which doesn't even account for the fingers, or friction, or a host of other necessary factors to be useful on anything bespoke and not set permanently, and trained watching climbers on that specific climb.
More Magnus channel is so chill that Magnus doesn’t even iron his t shirts 🤣
You iron your t shirts??? 😂
who does???
There should be an excited Adam Ondra voice option as well!
Really impressed by people who can fall on their backs from > 2 m and get up like nothing happened. Regine must have great neck muscles!
Well they aren't landing on concrete
Still, I've fallen from that height onto a similar mat onto my back, and it was scary. I thought I'd get whiplash but somehow I was fine.
This is such a cool project. I can’t wait to see this scale over time and get implemented more commercially!
I thought it was hysterical when the Ai told Magnus to slow his role.
You once said you are an introvert. Well as an introvert myself, I can appreciate this channel more than the other one. By that I'm not saying the other one isn't good of course. But this one is more relaxing to my brain.
Do more of this! So enjoyable
Sick video dude, would love to see a some outdoor climbing vids on this channel! A France trip where you try Rainbow Rocket would be cool, proper Magnus style boulder
I can't wait for AI route setting, that is going to be some sick stuff!
Great video, but too short. I miss old Magnus, when an hour of entertainment was not uncommon. Don’t listen to those marketing fools, Magnus. Follow your heart, make longer videos. Peace ✌️
The AI can't help Magnus that much when he simply flashes the boulder 😂
This could be big for speedclimbing I imagine!
That, and as a beginner climbing app advice is about all it's going to be able to do since the holds and setting is predictable.
It was mabye built/based on speedclimbing fotage, because its the same route and very easily comparable with other people
Regine's encouragement is great, I'd loved to have a climbing partner who is as encouraging as her
Loving the 'side' channel!
I think it would be super cool to see it as a analytic tool for championship
When lots of different people do speed climbing with this software, the data is going to be VERY interesting to see
Great idea to use a language model to make the results easily accessible (the speech synthesis is the cherry on the cake)
Wished you would have shown more of Regine's attempts
Maybe Magnus will wear a wig next time and 'pretend to be' ugh.
More Magnus is now by far the best climbing channel on YT.
Reminds me of a program we use in weightlifting where I train... just WAY more complex and advanced. Truly fascinating.
i've had a similar idea but i am not a climber so i never really pursued it. cool to see someone is!
my idea was to use 3d scans of the wall and a 3d actor to attempt moves and calculate the forces someone might endure to complete a move so root setters could have more insights into exactly what type of climber will struggle where
It was good with Regine there too. Maybe some videos of climbing with her and your sister, for the different styles and the interactions?
He should/could release the AI as an app.
Magnus, you should talk to the company that tested peoples deadlifts last year at the Shaw Classic strongman competition. They had me do a relatively light deadlift on camera and predicted my max, the hardest point in the lift, etc.
Rockelle/Rocquelle/Rocki absolutely should be this AI voice's name
This will really shine on system boards
Just look at Magnus' face and neck color from 9:00 to 9:45. It does the whole pale -> furious red -> back to normal gradient. Really interesting.
I have to see if I am a starfish climber, or frog climber.
Imagine having all magnus several hours training session filmed for data set. Would be insane to have it analyze all of it :)
at 12:55 there's a weird sort of optical illusion. When I first saw it I thought the camera was panning to the left making the left side of the wall come closer which made the undercling look inverted and my brain felt like it broke. It took me like five replays just to figure out what was happening.
12:55 that hold looked like an optical illusion! it looked like an actual hole in the wall! could be interesting to do create/set a route with more horozontal variation
The smart guy's(didn't catch his name) T-shirt is sick!!!
keep it up with these videos!!
her slab climbing was crazy!
This feels like the ice skating movie where she creates a program to analyse her moves with physics. AI wasn't a thing so no mention of it. This is wild
i think this might be almost more useful for other sports, as a huge part of climbing is in the fingers and hands while the ai seemed to focus much more on arm body and leg movements.
So thinking about olympic sports this might be incredible for certain disciplines
Would have liked to see climbing her more. Like a 50/50 split would have been nice - still a fun video though
I've been wanting to do something like this for a long time, the only thing is convincing a gym to invest in and buy cameras for speculative tech. Best of luck to the dev
This is so impressive! Exactly the type of implementation of AI that I think is beneficial for society in general.
At the end it would have been interesting to have your thoughts about the last input of the AI and also some final thoughts about the whole system.
Magnus: I took an arrow to the knee.
we absolutely need more videos like this, bouldering, maybe with some additions to them, but just your climbing sessions help to progress on my level by spectating)
As cool as the ai stuff was honestly would love to see the 2 of you just having a session trying hard without the distraction
This was fascinating, a really interesting use of AI. The potential efficiency gains alone would be valuable for competition climbers. That voice would annoy the hell out of me though, I think it should be John Cleese!
Can't tell you how thankful I am for the climbing content, Magnus ❤ feels like the old days that I'm super nostalgic for 😂
I wonder if adding more cameras for the software to work with, thus providing it with the sense of depth, will yield any interesting results
have you considered integrating lidar data so that you can scan outdoor climbing environments as well. the algorithm would have better access to the three dimensional space that climbers are working with, and really taking it to the next level, understanding the physics the climbers are overcoming and integrating to send their climbs :3
This is very exciting to see in action! As a data scientist and an avid climber, this is a great example of how AI can help! Will definitely check out the website to learn more on how it works etc. Can see future use cases to help train people, almost like a pocket coach! Which would be really cool! Nice work
Do you think that the given example of instruction from the AI in the video would help you on your problems?
Possibly. I think it would all come down to how it’s trained. Is it solely looking at my performance on one boulder, is it trained from other attempts on the same boulder from other climbers or does the algorithm find similar climbs and give tips based off those moves? I would certainly love to try it and see how it works in real time!
Would be cool to pair this with something like Kilter Board with repeatable boulders more people can access. Could even do a “climb like Magnus” type marketing.
Yeah it would work so much better with known repeatable problems to start filling in datasets.
Gotta be careful with the difference in climber sizes. Trying to climb like Magnus could be a good idea if you have a similar height and wingspan (granted, that's a lot of people), but even then, you won't have his strength. Maybe you need to do a figure 4 in a move where Magnus would "simply" do a one-arm pull-up.
I guess it would be interesting to find a strong climber who matches your own size and style.
@@andeolevain that's a good point. I'm just starting out, and I'm an older guy (46), so if I even tried to do something like him, I'd probably rip a tendon right off the bone. This video was interesting though because my career has been boring indoors programming and high performance compute cluster architecture. So blending in computing analytics with climbing is super interesting!
Especially so if we can learn more about safety and risk management.
Mi amigo es el más chingon de todos !!!
Estoy muy orgulloso hermano sigue rompiéndola 🔥🔥🔥🔥🔥🔥
Another vote for Magnus AI voice. :)
I love your new channel
You should call it Eve. Adam is the best climber in the world. Eve would be the best coach..
AI will solve the grading debate with individual grade calculators for every human that wants it.
Time to start building a dataset!
But every body is different in architecture (proportions, weight distribution, strengts and weaknesses) . You can't just compare limb positions and joint angles directly 💟🌌☮️
Indeed. It's a shame this wasn't addressed, because unless proportions and relative strengths are taken into account the numerical comparisons are quite limited in usefulness. That said, it's early days for people applying pose detection to climbing, and the relative numbers for one person over multiple attempts can still be valuable as a starting point (e.g. "this attempt your heel was lower and your left hand was higher on the piece, which worked better" kind of thing).
Over time this will hopefully allow scanning in physical geometries (or at least providing a set of limb lengths), and ideally will eventually make use of stereo / lidar-augmented footage to treat things as properly 3D (including the wall and hold geometries and poses), at which point it can get much more powerful, and provide more meaningful individual and comparative insights.
The dark side of the moon shirt ❤❤❤
All the falls looked like they hurt in this video
This is sick.... This would be super handy Asa newcomer to climbing.
I want this :D - with Magnus-Coach!
Loving the climbing content but could the editor put the boulder grades on the screen please? Like Magnus used to. I like to know what I'm looking at but I'm even worse at 'guess the grade' than Branden
Never mind. I rewatched it & noticed the gym has changed to colour grades. What a shame
Thank you for the new channel! Your climbing related videos are 🤌 excellent'e.
I'd watch all 3 hours
this is awesome, good climbers can help feed the AI with inputs and helping beginners that way all over the world ! great use of AI
I'm writing something similar for bowling, this is very well done.
Just reminded me of how beautiful some of your lines are Magnus. The mixture of balance, leverage, brute power and a thousand hidden 'strength' moments most of us probably don't ever grasp are happening. Properly delightful to watch.
haha, at 12:54 the black hold looked like a hole to me, I had to re-watch it 4 times before my brain understood what was happening xd
reminds me of the dinosaur head illusion
From the last gen that began in etriers and goldline hemp ropes? Gotta admit?
a little scary with the whole AI embrace
probably very helpful for speed climbing
I was thinking about something similar, where the AI would get a video of a route, so that it is able to really see all the boulders, how deep they are etc or like a 3D model of the boulders, to the find a way of betas. But that sounds really complicated :D
You could go the lazy way and just feed it a lot of videos of climbing videos, but i guess it wouldn't work to well. At least not when the boulders are filmed in a way that it can't know how steep angles are etc.
It’s a cool piece of software, but calling it “AI” is just laughable; it is a data analysis tool that pipes its results into an LLM (sounds like ChatGPT) and feeds that into a TTS program.
The bare minimum for something like this to be an “AI” would be having it predict the route or work off a single climb attempt.
Exactly, it is getting really tiresome how people are miscategorizing everything on a computer as AI nowaways...
It seems more like a trackman (in golf) than anything
Does computer vision pose estimation not count as AI anymore? Or text-to-speech?
Its literally just a frontend for pose estimation.
What is your definition of intelligence? It seems like even the leading thinkers can't agree on a definition of intelligence. Perhaps the best one I've heard is the ability to reach the same outcome through different means, and that still doesn't seem adequate.
Interesting AI project. Cool video, too!
Just like the vid of Magnus climbing, doesn't really need the hype AI element
A lot of people are getting on nerve by saying this, not ai. This guy didn't work hard enough.Bro cool down i get it you all know a lot but can you not be appreciative about the fact he thought of it and worked alone ?
Mate you did a great job.Maybe its not the best but keep working on the project