Amazing RL work! It's interesting that the energy-efficient gait learnt is diagonal so that it takes advantage of the both femur and coxa actuators for maximal velocity. Maybe adding compliance in the structure of the leg itself could result in further energy stored in the sagittal plane and thus a more symmetric gait?
'evolved'/learned movement for a designed/prescribed structure. Interestingly though perhaps not surprisingly we see an awkward result in the sideways gait. Lots of options from here. But if you're going to give the software freedom to evolve then (in sim) give the hardware some of the same freedom. If the sim doesn't allow that then re-balance the power/efficiency of the forward vs sideways motors and let the reinforcement learning bias forward movement. Or give the software what it seems to want which appears to be a round body/ spider-like. (apologies ...just thinking out loud).
I think the simulations have a dynamic model of each actuator -- hence what's happening there could be close to reality, barring manufacturing differences. But, you can work around that by tuning your motors and drive-train to ascertain the required gains for each leg subsystem. I do like your second point where you want to rework the body design to make the entire system more optimal or reorient the motors to achieve something similar.
@@harikrishnahariprasad2141 ' simulations have a dynamic model of each actuator' If i understand you correctly the sim informed the developers on the power for the actuators? ok, it seems to me that a sideways/odd gait is pretty much guaranteed in that case. I expect they'll want to keep structure of the hardware fixed as-is, simply because changing that will cost lots of time and money (but also for functional reasons ie navigating corridors etc). Re-tuning the motors seems to be an option as you say. Or (new idea) add a constraint to the reinforcement fuction that biases forward motion. It's reasonable to want that in order to navigate corridors etc and much easier than hardware changes like my spider-body thought.
Why do you need to force symmetry constraints? What do you mean by learning more normal gaits? The learned policy is minimizing the cost function based on the robot's dynamics for ambulating. It has found lower energy alternatives that might look different from a typical quadruped. That's a good thing and doesn't mandate symmetry constraints unless it impairs the task completion aspect of the robot, and from the video, it very clearly didn't even while navigating narrow spaces in the obstacle avoidance scenario. Refer to Yordan Tsvetkov's comment below.
Amazing RL work! It's interesting that the energy-efficient gait learnt is diagonal so that it takes advantage of the both femur and coxa actuators for maximal velocity. Maybe adding compliance in the structure of the leg itself could result in further energy stored in the sagittal plane and thus a more symmetric gait?
Great point, I would pay some serious money to see that!
Amazing RL work!
'evolved'/learned movement for a designed/prescribed structure. Interestingly though perhaps not surprisingly we see an awkward result in the sideways gait. Lots of options from here. But if you're going to give the software freedom to evolve then (in sim) give the hardware some of the same freedom. If the sim doesn't allow that then re-balance the power/efficiency of the forward vs sideways motors and let the reinforcement learning bias forward movement. Or give the software what it seems to want which appears to be a round body/ spider-like. (apologies ...just thinking out loud).
I think the simulations have a dynamic model of each actuator -- hence what's happening there could be close to reality, barring manufacturing differences. But, you can work around that by tuning your motors and drive-train to ascertain the required gains for each leg subsystem. I do like your second point where you want to rework the body design to make the entire system more optimal or reorient the motors to achieve something similar.
@@harikrishnahariprasad2141 ' simulations have a dynamic model of each actuator' If i understand you correctly the sim informed the developers on the power for the actuators? ok, it seems to me that a sideways/odd gait is pretty much guaranteed in that case.
I expect they'll want to keep structure of the hardware fixed as-is, simply because changing that will cost lots of time and money (but also for functional reasons ie navigating corridors etc). Re-tuning the motors seems to be an option as you say.
Or (new idea) add a constraint to the reinforcement fuction that biases forward motion. It's reasonable to want that in order to navigate corridors etc and much easier than hardware changes like my spider-body thought.
Being a simple human, I noted the "rocking" quasi guitar solo that starts at 1:17
Might be worth doing follow up research into finding a symmetry constraint which can help learn more normal gaits. Fantastic work!
Why do you need to force symmetry constraints? What do you mean by learning more normal gaits? The learned policy is minimizing the cost function based on the robot's dynamics for ambulating. It has found lower energy alternatives that might look different from a typical quadruped. That's a good thing and doesn't mandate symmetry constraints unless it impairs the task completion aspect of the robot, and from the video, it very clearly didn't even while navigating narrow spaces in the obstacle avoidance scenario. Refer to Yordan Tsvetkov's comment below.
Wow!
🤯