UW Graphics Lab
UW Graphics Lab
  • 53
  • 128 624
3D Reconstruction with a Tiny Distance Sensor
We reconstruct 3D geometry using distributed measurements from a low-cost time-of-flight based distance sensor. In CVPR 2024.
Project Page: cpsiff.github.io/towards_3d_vision/
Paper Title: Towards 3D Vision with Low-Cost Single-Photon Cameras
Authors: Fangzhou Mu, Carter Sifferman, Sacha Jungerman, Yiquan Li, Mark Han, Michael Gleicher, Mohit Gupta, Yin Li
Appearing in: CVPR 2024
00:00 Sensor Introduction
01:00 Our Method
02:04 Results
03:45 Applications
มุมมอง: 352

วีดีโอ

[supplementary video] IKLink: End-Effector Trajectory Tracking with Minimal Reconfigurations
มุมมอง 1103 หลายเดือนก่อน
Supplementary video for our ICRA'24 paper: IKLink: End-Effector Trajectory Tracking with Minimal Reconfigurations
Unlocking the Performance of Proximity Sensors by Utilizing Transient Histograms
มุมมอง 106K8 หลายเดือนก่อน
Directly utilizing low-level information generated by optical time-of-flight sensors allows recovery of planar geometry and albedo from a single sensor measurement. Code is now available: github.com/uwgraphics/ProximityPlanarRecovery Project website: cpsiff.github.io/unlocking_proximity_sensors/index.html Robotics and Automation Letters (RA-L) / To Appear: ICRA 2024 0:00 Intro to Transient Sens...
RangedIK: An Optimization-based Robot Motion Generation Method for Ranged-Goal Tasks
มุมมอง 208ปีที่แล้ว
Presentation video of our work: RangedIK: An Optimization-based Robot Motion Generation Method for Ranged-Goal Tasks This work is accepted at ICRA 2023. Link to pre-print: arxiv.org/abs/2302.13935
RangedIK: An Optimization-based Robot Motion Generation Method for Ranged-Goal Tasks
มุมมอง 528ปีที่แล้ว
Supplementary video of our work: RangedIK: An Optimization-based Robot Motion Generation Method for Ranged-Goal Tasks This work is accepted at ICRA 2023. Link to pre-print: arxiv.org/abs/2302.13935
Geometric Calibration of Single-Pixel Distance Sensors
มุมมอง 289ปีที่แล้ว
For more information, visit the project website: cpsiff.github.io/Geometric-Calibration/ Published in Robotics and Automation Letters, 2022 / In Proceedings International Conference on Intelligent Robots and Systems (IROS) 2022
Proxima: An Approach for Time or Accuracy Budgeted Collision Proximity Queries
มุมมอง 1.6Kปีที่แล้ว
In this video, we provide an overview , a new approach for computing a collision proximity function in robotics that formalizes the tradeoff between efficiency and accuracy and This video reflects the presentation at the Robotics: Science and Systems 2022 conference, where this work was published. A link to our paper can be found here: www.roboticsproceedings.org/rss18/p043.pdf Open-source code...
Robust Replay of Human Demonstrations Using Identified Constraints
มุมมอง 112ปีที่แล้ว
Demonstrations of methods that apply our constraint inferencing techniques to provide robust (compliant) execution of tasks based on human demonstrations. Both the nominal motion and the constraints are inferred from a demonstration.
Trinary Tools for Continuously Valued Classifiers (Paper Accompaniment Video)
มุมมอง 28ปีที่แล้ว
Video to accompany our paper "Trinary Tools for Continuously Valued Classifiers (Paper Accompaniment Video)" from the VisXAI 2022 workshop (and the Visual Informatics journal). This video accompanies our paper [Trinary tools for continuously valued binary classifiers](graphics.cs.wisc.edu/Papers/2022/GYC22/). The paper is about the CBoxer system. For more information see the [Boxer Project Page...
Trinary Tools for Continuously Valued Binary Classifiers (Pacific Vis 2022 VisXAI workshop Talk)
มุมมอง 44ปีที่แล้ว
Video presentation for the 2022 VisXAI workshop (part of Pacific Vis 2022) to accompany our paper "Trinary Tools for Continuously Valued Binary Classifiers"
Understanding Control Frames in Multi-Camera Robot Telemanipulation
มุมมอง 98ปีที่แล้ว
Presented at HRI 2022.
CollisionIK: A Per-Instant Method for Generating Robot Motions with Environment Collision Avoidance
มุมมอง 1.8K2 ปีที่แล้ว
CollisionIK: A Per-Instant Pose Optimization Method for Generating Robot Motions with Environment Collision Avoidance This paper was presented at ICRA 2021. Authors: Daniel Rakita, Haochen Shi, Bilge Mutlu, Michael Gleicher Abstract: In this work, we present a per-instant pose optimization method that can generate configurations that achieve specified pose or motion objectives as best as possib...
Strobe: An Acceleration Meta-algorithm for Optimizing Robot Paths using Concurrent Interleaved Pods
มุมมอง 2902 ปีที่แล้ว
This paper was presented at ICRA 2021. Authors: Daniel Rakita, Bilge Mutlu, Michael Gleicher Abstract: In this paper, we present a meta-algorithm intended to accelerate many existing path optimization algorithms. The central idea of our work is to strategically break up a waypoint path into consecutive groupings called "pods", then optimize over various pods concurrently using parallel processi...
Single-query Path Planning Using Sample-efficient Probability Informed Trees
มุมมอง 5522 ปีที่แล้ว
This video is our virtual presentation appearing at ICRA 2021 on our path planning algorithm SPRINT (Sample-efficient PRobability INformed Trees). The paper was published as an ICRA/ RA-L paper in 2021. Authors: Daniel Rakita, Bilge Mutlu, Michael Gleicher Abstract: In this work, we present a novel sampling-based path planning method, called SPRINT. The method finds solutions for high dimension...
Characterizing Input Methods for Human-to-robot Demonstrations
มุมมอง 973 ปีที่แล้ว
Human demonstrations are important in a range of robotics applications, and are created with a variety of input methods. However, the design space for these input methods has not been extensively studied. In this paper, focusing on demonstrations of hand-scale object manipulation tasks to robot arms with two-finger grippers, we identify distinct usage paradigms in robotics that utilize human-to...
User-Guided Offline Synthesis of Robot Arm Motion from 6-DoF Paths
มุมมอง 1783 ปีที่แล้ว
User-Guided Offline Synthesis of Robot Arm Motion from 6-DoF Paths
SubSpace Video Stabilization (ToG 2011)
มุมมอง 763 ปีที่แล้ว
SubSpace Video Stabilization (ToG 2011)
Content-Preserving Warps for 3D Video Stabilization (SIGGRAPH 2009)
มุมมอง 2513 ปีที่แล้ว
Content-Preserving Warps for 3D Video Stabilization (SIGGRAPH 2009)
Stylized and Performative Gaze for Character Animation
มุมมอง 1673 ปีที่แล้ว
Stylized and Performative Gaze for Character Animation
Re-Cinematography (2008)
มุมมอง 253 ปีที่แล้ว
Re-Cinematography (2008)
Authoring Directed Gaze for Full-body Motion Capture
มุมมอง 1253 ปีที่แล้ว
Authoring Directed Gaze for Full-body Motion Capture
HijackGL: Reconstructing from Streams for Stylized Rendering (2002)
มุมมอง 733 ปีที่แล้ว
HijackGL: Reconstructing from Streams for Stylized Rendering (2002)
Footskate cleanup for motion capture editing (2002)
มุมมอง 4643 ปีที่แล้ว
Footskate cleanup for motion capture editing (2002)
Simplicial Families of Drawings (2001)
มุมมอง 133 ปีที่แล้ว
Simplicial Families of Drawings (2001)
Motion Path Editing
มุมมอง 643 ปีที่แล้ว
Motion Path Editing
Projective Registration with Difference Decomposition (CVPR 1997)
มุมมอง 263 ปีที่แล้ว
Projective Registration with Difference Decomposition (CVPR 1997)
Spacetime Motion Editing (1997)
มุมมอง 993 ปีที่แล้ว
Spacetime Motion Editing (1997)
Image Snapping (SIGGRAPH '95 Paper Video)
มุมมอง 723 ปีที่แล้ว
Image Snapping (SIGGRAPH '95 Paper Video)
Boxer: Interactive Comparison of Classification Results
มุมมอง 523 ปีที่แล้ว
Boxer: Interactive Comparison of Classification Results
Briar: A Constraint-Based Drawing Program
มุมมอง 463 ปีที่แล้ว
Briar: A Constraint-Based Drawing Program

ความคิดเห็น

  • @HomoSapiensMember
    @HomoSapiensMember 21 วันที่ผ่านมา

    thank you for this amazing work!

  • @steinmar2
    @steinmar2 21 วันที่ผ่านมา

    Pretty nice (:

  • @icebluscorpion
    @icebluscorpion 4 หลายเดือนก่อน

    optimize it to use this for bed leveling in 3D printers other uses are pointless and stupid

  • @jumpstar9000
    @jumpstar9000 4 หลายเดือนก่อน

    Very cool. You could make a MIDI controller as a demo. Something like a theremin. It might also be nice to arrange a few of these in a hemispherical or triangulated configurations for a desktop device. Then you could use both hands as manipulators for a 3D input device. Also use with prosthetics. Really cool stuff guys 👍

  • @EPMTUNES
    @EPMTUNES 4 หลายเดือนก่อน

    Pretty cool stuff here!

  • @retinapoliyn7462
    @retinapoliyn7462 6 หลายเดือนก่อน

    آYou could easily do the same with the VL53L5X sensor

  • @user-ui8jt6cx2k
    @user-ui8jt6cx2k 6 หลายเดือนก่อน

    Great Work!

  • @luisca92
    @luisca92 6 หลายเดือนก่อน

    Just got a ur10e for R&D, can’t wait to try this

  • @1kreature
    @1kreature 6 หลายเดือนก่อน

    Would just like to point out: Arduino is NOT a microcontroller. It is a framework and development environment with certain compatible/compliant development boards containing a microcontroller from different companies such as Atmel and STM.

  • @LanceThumping
    @LanceThumping 6 หลายเดือนก่อน

    I'd be interested in seeing a version of this maximizing for linear accuracy. The setup I envision is the sensor is mounted to a linear rail with a board/reflector fixed at a right angle to one end. Then the sensor is set at a known distance to the board, the differentiation is run with known values for the distance/angle to calibrate for the surface reflectance. Finally the sensor runs on the linear rail, with the calibrated data and the fixed angle relative to the surface to provide only distance data. I'm curious how much all this boiling down of the data will increase the accuracy of the linear distance measurement. Reason being is that I've been interested in the idea of using these types of sensors for closed loop control of a 3d printer (I like the idea of knowing exactly where the print head is at a given time), but one issue is the accuracy. The sensor frequency would definitely be too slow but there are other comments addressing it that could be explored.

  • @mumblety
    @mumblety 6 หลายเดือนก่อน

    Cool!

  • @ConsultingjoeOnline
    @ConsultingjoeOnline 6 หลายเดือนก่อน

    YES!!! Amazing research guys! Keep it up!

  • @tombowen9861
    @tombowen9861 6 หลายเดือนก่อน

    I'm immediately thinking of all the super low cost ways you could now provide backup orientation data to an IMU. With the simplicity and price-point, you could use it to provide a crosscheck to combat sensor drift over long trips. If the sample rate is too low, use two! haha. So many lunar landers have gotten turned around over the years due to sensor confusion, and here's a few dollars that can provide surface geometry data that might be used to reorient. Really neat project!

  • @ryanreedgibson
    @ryanreedgibson 6 หลายเดือนก่อน

    This would be great to use with my thrust vectoring rocket.

  • @roninbadger7750
    @roninbadger7750 6 หลายเดือนก่อน

    When is this going to be in the next Samsung or Pixel, 6 months? Next Apple 5 years? next LG last year? "Ai assisted 3D modeling" next feature on a smart phone.

  • @friendlybear5924
    @friendlybear5924 6 หลายเดือนก่อน

    Very informative video. Thank you! Enjoyed it a lot.

  • @brotherdust
    @brotherdust 6 หลายเดือนก่อน

    Wow! This would be useful to implement as part of an automatic bed leveling routine on a 3D printer! Nice work! Subscribed!

  • @dmendesf
    @dmendesf 6 หลายเดือนก่อน

    What's the model of the robotic arm? It's really cool.

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      It's a Universal Robots UR5

  • @jakobjorgensen7773
    @jakobjorgensen7773 6 หลายเดือนก่อน

    I've been looking for a solution like this - excellent work! Thank you

  • @andybrice2711
    @andybrice2711 6 หลายเดือนก่อน

    Is this a similar principle to RADAR and ultrasound scanners?

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      Those often also operate on time-of-flight, so yes!

    • @andybrice2711
      @andybrice2711 6 หลายเดือนก่อน

      @@uwgraphics But also I think they have to try and parse out multiple reflections from complex surfaces and various layers of changing impedance.

  • @Lion_McLionhead
    @Lion_McLionhead 6 หลายเดือนก่อน

    Surprised these sensors evolved into low resolution 3x3 cameras. Definitely an advantage to apply offboard computing power to the histogram analysis. The integrated processor is a Cortex M0.

  • @uwgraphics
    @uwgraphics 6 หลายเดือนก่อน

    To clarify something we've seen many comments on: our results don't mean that the manufacturer's solution for distance estimation is "bad". The manufacturer's solution runs onboard the sensor, which has a minuscule amount of computing power, and was likely designed to support generic use cases for their sensor, not the very specific planar recovery that we perform.

  • @JaydenLawson
    @JaydenLawson 6 หลายเดือนก่อน

    3:55 wow your results are MUCH better than the proprietary distance estimates

  • @napent
    @napent 6 หลายเดือนก่อน

    Nice - would love to play with that data in Python

  • @VacuousCat
    @VacuousCat 6 หลายเดือนก่อน

    Works like echolacation. Animals can do echolocation to detect shape, distance, and texture.

  • @jhoughjr1
    @jhoughjr1 6 หลายเดือนก่อน

    this is awesome, earned a sub.

  • @MyrLin8
    @MyrLin8 6 หลายเดือนก่อน

    Excellent work. Impressive signal use.

  • @Joso997
    @Joso997 6 หลายเดือนก่อน

    what about the Sun?

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      Good question! SNR is lower in sunlight because ambient light crowds out many of the "good" photons. In our testing (separate of this paper) the sensor itself is still able to function up to about half a meter even in direct sunlight. We would expect the performance of our plane finding algorithm to perform more poorly in sunlight because of the low SNR. We haven't tested it thoroughly because it's hard to get a robot arm outside!

  • @pontosinterligados
    @pontosinterligados 6 หลายเดือนก่อน

    Wow! Kudos guys! 👏

  • @constantinosschinas4503
    @constantinosschinas4503 6 หลายเดือนก่อน

    Will adding more sensors, in possibly different angles, improve lower distance error across different materials?

  • @maxmyzer9172
    @maxmyzer9172 6 หลายเดือนก่อน

    cool!

  • @matthewsheeran
    @matthewsheeran 6 หลายเดือนก่อน

    I wonder if these are the new "induction" previously PIR sensor light sensors to detect a passing human and turn a light on?

  • @Mister_Fister.
    @Mister_Fister. 6 หลายเดือนก่อน

    aaaand ur a virgin

  • @DaveEtchells
    @DaveEtchells 6 หลายเดือนก่อน

    Brilliant work! I was surprised how well simple calibrated peak detection worked, compared to the much more complicated differentiable method.

  • @gluonsmx
    @gluonsmx 6 หลายเดือนก่อน

    Really smart! Thanks for sharing the rationale and code

  • @SirasPK
    @SirasPK 6 หลายเดือนก่อน

    Very good work guys. Outperforming a company this way makes me wonder if they let the interns write codes for these things.

    • @overflow7276
      @overflow7276 6 หลายเดือนก่อน

      They most likely do. I once bought a muscle sensor from Seed-Studio and the only code example they provided was using the raw unfiltered signal to light up an LED bar. This is so far off from the usual field of use for these sensors it really made me scratch my head why nobody over at seed studio has actually bothered trying to get a filtered and clean signal from their hardware, especially since the filter settings depend a lot on exactly that hardware.

    • @Fennecbutt
      @Fennecbutt 6 หลายเดือนก่อน

      It's cause all the C levels are just biz majors whose only purpose in life is to convince investors and customers that their company has some sort of "secret sauce". I fucking loathe companies that NDA their datasheets like Pixart Imaging (and plenty of others).

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      See the pinned comment - we actually don't think the manufacturer's solution is bad. It's just more constrained on compute and designed for more generic sensor use cases.

  • @acatfrompoland5230
    @acatfrompoland5230 6 หลายเดือนก่อน

    Very inspiring video :3

  • @bob2859
    @bob2859 6 หลายเดือนก่อน

    Very cool. Looking forward to seeing if you can get geometry reconstruction working!

  • @borisbadinov7757
    @borisbadinov7757 6 หลายเดือนก่อน

    Fantastic. Also the clarity of your technical descriptions overcame my youtube induced ADHD

  • @FullCircleTravis
    @FullCircleTravis 6 หลายเดือนก่อน

    Thanks for sharing this.

  • @ThisIsToolman
    @ThisIsToolman 6 หลายเดือนก่อน

    Very interesting. Looking forward to future videos involving this sensor.

  • @benargee
    @benargee 6 หลายเดือนก่อน

    One small step towards no longer having to pay tips to human waiters.

  • @u9vata
    @u9vata 6 หลายเดือนก่อน

    Two questions: - Is there source code available for just the plane reconstruction (excluding robot moving parts or such)? - Is there similar solution for higher range similar sensors? Would be pretty interested if there is and might have use case for AR. Maybe sennsor range is high already but not sure. I know some of these ToF sensors have longer range like around 10m at most (which would be enough for my case) but I guess maybe this exact sensor has smaller range.

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      Code has just been released! github.com/uwgraphics/ProximityPlanarRecovery The sensor used in this work typically has a max range of about 3 meters. We set it to "low range, high accuracy" mode to give it a max range of 1m for higher temporal resolution. Our solution could in principle work with higher range sensors. The highest I know of in this form factor have a quoted max of 5 meters (e.g. ST VL53L8)

    • @maciejurbanski6146
      @maciejurbanski6146 6 หลายเดือนก่อน

      @@uwgraphics brilliant work; thank you for publishing the code!

    • @paulwesley3862
      @paulwesley3862 6 หลายเดือนก่อน

      ​@@uwgraphicsthanks for publishing the code. you said you restricted it for<1m, what is the temporal resolution then?

    • @u9vata
      @u9vata 6 หลายเดือนก่อน

      @@uwgraphics Thanks! I just realized its 3FPS because of I2C though so likely need some sensor fusion, but this is a very cool project with low cost sensors becoming much more powerful than they by factory are! Awsome work!

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      @@paulwesley3862 There are 128 bins, but the first 15 or so are never used. The bin resolution is the equivalent of about 1cm. That doesn't mean the final output of the sensor is limited to 1cm resolution, though.

  • @bendingsands87
    @bendingsands87 6 หลายเดือนก่อน

    I wonder if this could be used to make a device for 3d printer bed leveling to either make a bed mesh or for real time manual bed leveling. This is very cool.

    • @robstamm60
      @robstamm60 6 หลายเดือนก่อน

      Unless you come up with some insane hack to increase the accuracy from a few millimeters to less than 0.1mm you are out of luck. These sensors measure the time of flight of light - to measure with 0.1mm accuracy you would need to measure with less than 1ps time resolution...

  • @therealnagia2461
    @therealnagia2461 6 หลายเดือนก่อน

    hooooley crap! who knew simple tof sensors had so much output you could gather such useful data from! thank you for this!

  • @TouYubeTom
    @TouYubeTom 6 หลายเดือนก่อน

    the source code is missing

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      It has just been released! github.com/uwgraphics/ProximityPlanarRecovery

    • @TouYubeTom
      @TouYubeTom 6 หลายเดือนก่อน

      @@uwgraphics cool thanks

  • @BitSmythe
    @BitSmythe 6 หลายเดือนก่อน

    1:55. The blue pane doesn’t match the table. It looks like the bottom left corner should be further away.

  • @chemicburn
    @chemicburn 6 หลายเดือนก่อน

    Incredible work thanks for sharing!

  • @AlaskanInsights
    @AlaskanInsights 6 หลายเดือนก่อน

    Nice, This is a cool work around fur sure.

  • @ethaneveraldo
    @ethaneveraldo 6 หลายเดือนก่อน

    I’ve been longing for an affordable, compact 3D scanner/solid state lidar for years. So many things you could do with something like this. I use a lot of single point ToF sensors in my projects, didn’t know there were 9 points measurement ones. Maybe by having a small array of them, each at a small angle, you could create a working 3D sensor? Video is incomplete. Need to see more

    • @uwgraphics
      @uwgraphics 6 หลายเดือนก่อน

      Combining data from multiple distributed sensors is an interesting direction for future work. There are higher resolution versions of these sensors available! Check out the ST VL53L8, which reports 8x8 zones. The VL53L8CH even reports histograms, but they're lower temporal resolution than the TMF8820 that we use.