Interactive Hand Pose Estimation using a Stretch-Sensing Soft Glove (SIGGRAPH 2019)

SIGGRAPH 2019
igl.ethz.ch/projects/stretch-...
OLIVER GLAUSER and SHIHAO WU, ETH Zurich, DANIELE PANOZZO, New York University, OTMAR HILLIGES and OLGA SORKINE-HORNUNG, ETH Zurich
We propose a stretch-sensing soft glove to interactively capture hand poses with high accuracy and without requiring an external optical setup. We demonstrate how our device can be fabricated and calibrated at low cost, using simple tools available in most fabrication labs. To reconstruct the pose from the capacitive sensors embedded in the glove, we propose a deep network architecture that exploits the spatial layout of the sensor itself. The network is trained only once, using an inexpensive off-the-shelf hand pose reconstruction system to gather the training data. The per-user calibration is then performed on-the-fly using only the glove. The glove’s capabilities are demonstrated in a series of ablative experiments, exploring different models and calibration methods. Comparing against commercial data gloves, we achieve a 35% improvement in reconstruction accuracy.

[@mirak63]

You have already secretly done a fullbody suit, right ? xD

[@LhunVideo]

great, now write an OpenVR driver for it so it works exactly like a knuckles controller without developer support for games and apps, and you'll instantly have funding. That's the one failing of these other products, not how good they are. None of these gloves get off the ground because they don't have native, plug and play openvr driver support for steam's hand pose system.

[@orbelosulpolar830]

This looks great! I hope it will become a consumer product soon!

[@RobotMama]

Amazing work!

[@anonony9081]

Nice, looks great

[@lepermunna]

excellent work!

[@iryna_u4]

Where can I buy it?

[@distrologic2925]

I CAN'T WAIT!! THE POSSIBILITIES

[@CreationTribe]

Oh my god, this is brilliant! I may have to alter one of my projects to work with this instead of the other platform I chose.

[@machmar]

Impressive, kudos

[@jsnneill]

good work.

[@OutOfNameIdeas2]

This Is amazing and still terrifying at the same time. Can't wait for this to be a consumer product

[@benwilms3942]

This is amazing. It looks like it would be very useful for capturing particular sequences of articulation, that be computationally matched for likeness to a predetermined set of instruction giving movements, that then send driver information to soft robotic assisted extremities, for example a paraplegic with soft robotic exo suit leggings, that are actuated by simple instructions from a glove like this on the hand. Or even a sleeve like this, that does same for an opposed prostatic arm, for simple locomotive actions like swinging in time during walking for better balanced gate.

[@drager980]

Cool. If you could get 3D Space tracking resting on the wrist or back of the glove it would probably have lots of funding in VR. Or just as an attachable piece which is complementary to existing VR controllers.
I would love to see a VR solution where gestures are (sometimes) usable over controller solutions.

[@mylife462]

Ye kaha milega

[@shuyangnie2446]

Hi, which system did you use for groud truth acquisition?

[@matthewoduamafu8186]

Hello, Please is the product on the market?
If yes how can I get 1

[@apricotdog]

Very interested in this project

[@sipangkanthasema5846]

Can it measure pressure on it?

[@georgegreen3672]

what's name of the animation software?

[@Brechdog]

Something's happening!!!!!!!!!!

[@Obsi399]

Yo that's cool as fuck. I can't wait to play vr with it

[@Nerdule]

This seems like it would be *extremely* interesting in combination with DextrES, which was able to apply similarly compact and flexible force-feedback to a soft dataglove.

[@5600onG]

Please take my money

[@choiiohc1]

I used it in Siggraph 2019, but the accuracy of hand pose estimation was awful. This movie shows the just best case.