Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning
ฝัง
- เผยแพร่เมื่อ 21 มี.ค. 2022
- Nvidia presented parts of this work at GTC 2022, revealing our humanoid-quadruped transformer!
Title: Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning
Authors: Eric Vollenweider, Marko Bjelonic, Victor Klemm, Nikita Rudin, Joonho Lee and Marco Hutter
Paper submitted to IEEE/RSJ International Conference on Intelligent Robots and Systems in Kyoto.
Preprint: arxiv.org/abs/2203.14912
Abstract: In recent years, reinforcement learning (RL) has shown outstanding performance for locomotion control of highly articulated robotic systems. Such approaches typically involve tedious reward function tuning to achieve the desired motion style. Imitation learning approaches such as adversarial motion priors aim to reduce this problem by encouraging a pre-defined motion style. In this work, we present an approach to augment the concept of adversarial motion prior-based RL to allow for multiple, discretely switchable styles. We show that multiple styles and skills can be learned simultaneously without notable performance differences, even in combination with motion data-free skills. Our approach is validated in several real-world experiments with a wheeled-legged quadruped robot showing skills learned from existing RL controllers and trajectory optimization, such as ducking and walking, and novel skills such as switching between a quadrupedal and humanoid configuration. For the latter skill, the robot is required to stand up, navigate on two wheels, and sit down. Instead of tuning the sit-down motion, we verify that a reverse playback of the stand-up movement helps the robot discover feasible sit-down behaviors and avoids tedious reward function tuning.
Note: The following parts of the video are sped up:
- Door opening and closing when the robot stands inside the elevator between 00:20 and 00:21 (+200%)
- In-between the standing up and sitting down sequence between 00:49 and 01:10 (+200% only the navigation on two legs)
- Reaction with the crowd between 1:41 and 1:50 (+150%)
- Last drone footage after 2:01 (+200%)
Learn more at www.swiss-mile.com/
Disclaimer: Robot from ANYbotics; customized by ETH Zürich; strictly for research purposes. - วิทยาศาสตร์และเทคโนโลยี
![[TH] VCT Pacific Stage 2 - Week 1 Day 1 // DRX vs ZETA | BLD vs TS](http://i.ytimg.com/vi/AWaOqaxN6Y0/mqdefault.jpg)
It's amazing when it briefly goes one-legged at 1:31
Good catch :)
Technically, this robot is doing a wheelie!
True 🤔
Bro are you genius
@@leejunja 🤣
Wow. She is amazing!!!
She? He? It? They? Pronounce are hard to define for robots. :)
Wow
😋
This is amazing. I follow all things robots and you guys are on a different level...
What an honor! :)
This is Awesome!
Thank you :)
That's so cool!
Thank you :)
I loved the learning sessions. You were great with translating to the general viewer pretty tough topics. And I have a crush on your robot.
Thank you! 😃
how wonderful!
Nice I saw this from NVIDIA (from their presentation this year).
The robot was featured couple of times there :)
If you put the front wheels at the "elbow" instead of "wrist" level then the wrists are freed to mount useful grippers.
Good idea!
I LOVE this machine.
Thank you
Man that looks amazing! Macro size and slap a car chassis on this thing. You revolutionize the auto industry 😎👍
Working on that!
Wow, this is a really cool robot. Love to see more video about it. Thanks RSL!
More to come!
when do the wheels become ducted fans so it can fly?
Should be the next goal! The all in one robot
I was amazed that the robot could push the elevator button to take the elevator at 0:14 !!! Is this an intelligent autonomous action or is it caused by human control?
All you have to do now is to transform the wheels to a expandable/ collapsible fan wheels. 🙂
Great idea 💡
@@leggedrobotics It is Yes 🙂👍 If you need any help then let me know. 🖖
Yeah, like a boat or a jet-powered bicycle.
@@nathandavis1093 How do you mean?
It is so amazing to see anymal standing up! According to your answer few days ago, is it possible to send torque or current to the wheel directly? Maybe it will present better dynamic characteristics and help anymal balance itself in a more robust way . Have you tried it before?
We are controlling torque through velocity commands. Why more robust? The robot never falls, even when going down the stairs.
@@leggedrobotics Velocity commands to PD controllers even for the two-wheel balancing task? Or there exists some type of feedforward torque?
FANTASTIC!
Many thanks!
cool！！！
Thanks!
This looks so good I love it.💕 If you attach a robotic arm on its back, the Applications will most definitely increase, but I think it might affect the dynamics of the robot. But if you can manage it out somehow this will be great 👍
Why add another arm if you can transform two legs into arms? ;)
@@leggedrobotics Because the wheels are limiting what the arms can do. Are you going to tranform the wheels to fingers when it is standing on its other 2 wheels?
Is the robot able to climb all kinds of stairs? I saw you include an example of it climbing a small flight of stairs in quadrupedal mode. Or is it not going to be used in that kind of application?
Very impressive. Is this all done based on neural networks without any sort of PID control? Or are the transitions handled through the learned behaviour and balance still handled by PID?
All behaviors shown in this video are generated through a neural networks that send position and velocity targets to each motor.
@@leggedrobotics Those position and velocity targets are probably handled by PID control, no? So technically, no; but yeah any high level control relating to the bulk motion such as balance is handled by the neural network.
@@eelcohoogendoorn8044 That was my thought, a hybrid system driven by the neural network and maintained by PID. Wondering if there's also inverse kinematics going on to move to target positions or not. Exciting stuff if this could all be handled by a virtually trained NN rather than going through fiddly PID tuning.
@@andymitchell2146 The velocity/position targets for the motors are tracked by a high-frequency (400Hz) PID controller. The actual movement (8 times lower frequency) is a NN policy. In this work there is no inverse kinematics going on. Some parts (the style extractions for AMP motions did have forward kinematics though, since the discriminator observations contained the end-effector position in the body frame) The policy directly predicts joint angles/velocities.
You can find a link to the paper preprint in the description.
What are the acquisition requirements for these?
I really love your work in devloping learning-based controllers for legged robots. Moreover, i also wanted to know whether there any opportunities to work at your Lab. I am currently a final year undegrad. I aim to a do a masters degree in electrical engineering (specialized in robotics and control) and head for PhD position at a reputed lab like yours. Please shed some advice of how I could achieve my goals.
Check out the website of our lab for more information about opportunities and read our papers to see what kind of skill set you need.
@@leggedrobotics Thank you means a lot. I hope to gain expertise and research experience in this field and get a Ph.D. position at your lab. :)
Me too
This tech is terrifying
Mad cool
Thank you :)
Looks like the Wheelers from the Land of Oz
Looks like amazing work! Can you share your paper to Arxiv??
The paper will be soon online.
This is amazing. My favorite part was when it took the elevator. Is this going to make it out of the lab? When? If it does, I guess it will first be used for things like industrial plant inspection, like Spot. Y'all need to make dance videos like Boston Dynamics XD
We will work on it :)
I propose to implement a wheel lock to overcome obstacles on foot.
You should add a set of arms and hands that can replace to wheels when it is upright.
Good idea 👍
Like like like
Like!
GoBots live!
Finally!
im confused but pretty robot goes brrrrr
Why confused?
42 mins?? what hardware are you running it on?
We used a NVIDIA GeForce RTX 3080 Ti for training
And Nvidia Isaac gym for simulation:)
@@leggedrobotics Nice! What are you running the network on in real time?
Is this the next Mars Rover? given his agility and his abilities, he should be able to explore the red planet...
That should be our vision :)
The scary part of robots learning, is that one can learn for days and all the others can get that knowlegde instantly from copy paste like in Matrix 😮.
Exactly!
Me trying to figure out if it's an Autobot or a Decepticon 🤔
Autobot!
Next up: Transformers.
The first of its kind !
github plz
This is no doubt amazing & impressive. But I'm concerned about hygiene when I see the robot standing up and using its wheels to push the elevator buttons.
Not saying that elevator buttons are clean as of today, its comparable with someone using their shoes to push a button.
If that's your take away, I pity the drugs that keep your mental illnesses from allowing you to destroy yourself.
Make your wheels fold / unfold automatic by splitting up its legs and arms in to wheel spokes
in a way like this th-cam.com/video/mt2FpYsQ4IE/w-d-xo.html and this th-cam.com/video/f86N8Xp7DHI/w-d-xo.html
And then?
@@leggedrobotics
Then it will look like a humanoid robot standing up and walking.
Then it unfold wheels spokes on legs and drive away when in need of higher speed.
In hard terrain it just walk on all four legs.
It unfold all wheels spokes and go down on all four
when it need highest speed and stability .
Legs and arms unfold like this when it split up its arms and legs in wheel spokes , th-cam.com/video/f86N8Xp7DHI/w-d-xo.html
This is very cool but it freaks me out
Do not worry :)
mannnnnnn
Only competitor team can downvote this vid :D
😎
Da kann man bestimmt ein cal .50 Maschinengewehr drauf montieren.
Bitte nicht!