The problem I have with Dr Mike's videos is that whenever I come here to learn about a specific thing, I end up binge watching many more of his vids, and then forget the reason I came here in the first place.
Very timely :) I have just finished stitching together several hundred lidar scans of a 2 km long road tunnel for a client. I relied pretty heavily on the auto-alignment software!
I was about to say "sheesh, couldn't you just pay somebody to do all that fiddly work" and then I realized that I'm a doofus and *you're* the one being paid to do fiddly work lol.
Usually these videos are shot in advance - not sure about Sean but I know Brady will make a trip to Nottingham and film enough for a handful of videos and then dole them out over time
Singular value decomposition is one of those things that translate extremely well to GPUs. In fact, Nvidia's CUDA library has purpose-built SVD support (the cusolverDnDgesvd function in its cuSolver package).
Sadly, SVD is by far not the bottleneck in this algorithm because it's only an SVD of a 3x3 matrix. Actually, the real thing that needs to be sped up on the GPU is nearest neighbors search, which is a bit tougher but doable
Enjoyed - nicely explained. Love the enthusiasm. Seems to me that it works well for the scenery but there is a whole set of other problems if there are moving components. First level difficulty a moving rigid object - say a car. Even more tricky a flexible item - say a person walking. For a real time application then more frequent scans means closest point is more likely to be the corresponding point but you have less time to process each scan - interesting trade off.
The goal of ICP is to reduce the amount of error, where "error" in this context would be defined (very roughly) as the sum of squared distances between the matched pairs of points. So movement in your scene would be handled (naively) the same way as the edges of the bunny scans in his example - the algorithm will minimize the error as much as possible but eventually get to a stopping point where most of the scene points overlap nicely but there are areas of large concentrated error, corresponding to the places where there was movement. (Hopefully that was clear enough.)
This is great, I've worked as a Lidar engineer for like a decade now, and ICP is one of those tools which will just tie everything together in a nice bow.
In astronomy we often need to map arbitrary images of the night's sky to Right Ascension and Declination coordinates (called plate solving). I wonder if this algorithm is involved, using stars as the points.
When your points have additional attributes, such as brightness, it's often helpful to treat them as an additional dimension. Besides that, what the matching program does is probably very similar. At least, as the final step. To improve convergence, it might, for example, start with just the brightest points, and add more as it gets closer to the solution.
I’m currently doing my dissertation on using lidar on iphones to create 3d meshes. I’ve been watching computerphile since its inception back when i was in high school, crazy to think how far i’ve come since then
I had this video in my watch later over 3 months now. My expectation was to learn something about photogrammetry and matching pairs to build a tree and then a point cloud. I had some ideas to how it might work with projection and interaction, but that seemed a little involved. Now with the topic of doing 2D or 3D tracking in video, I always assumed the 2D example would do an extrapolated spiral search pattern for points and just find the lowest difference in the search area. But it never occured to me to see matrix tricks like SVD show up. Such a crossover. I knew that a fourier transform on a matrix (like an image) would ignore any kind of translation or rotation, so you would use it for pattern matching for example. There was an actual optical application by Huygens Optics in a video a long while ago.
5:40 I understand the centre of mass of the points to shift the sample - but how do you decide the rotation? Do you just move it to centre of mass, then rotate and if it gets better, you rotate it some more , until it gets worse again? Or is there a better way to do it?
rotations are simple enough that they can be expressed as matrix multiplication. the resulting matrix will not be completely solvable because its an approximation, but SVD gets you the best solution you can get with linear algebra
@@alegian7934 yes he mentions SVD later but considering its 3 dimensions my mind still boggles how this can be done so quickly. Is there anything that helps a first guess?
I could be wrong in any of the following. But: if matrix A describes the 3d rotation, X is the matrix that contains as columns all points in one group and Y is the matrix that contains all points in the other group, we have AX=Y (approximately, remember, these cant truly be equal) so ideally if they were equal we would invert matrix X and obtain A=YX' (we found the rotation matrix) In reality X is not invertible at all but we can find its SVD that gets us as close to its inverse as possible. so we get something like A*UΣV=Y and we solve for matrix  (an approximation of the optimal rotation). And we apply  to X
@@alegian7934 I guess that's it! This is all new to me. We did cover 3d transformations back in my degree, but not inverse, only forward. At least that's what i remember! Very interesting, will have to do some more reading before I understand this enough.
@alegian Can't we just do the following: 1) build a vector A out of the selected points of the first shape in respect to their center of mass: A = Σ a_i - a_center 2) build a vector B out of the selected points of the second shape also in respect to their center of mass: B = Σ b_i - b_center 3) calculate angle between the vectors?
Our biggest scan project was around 20 Billion points. Took a while to register even on a Threadripper-based workstation. Our scanner measures 976,000 pts/sec.
Btw: 20billion points would take more than 230 days of non stop laser scanning (with your ~1M pts/sec laser scanner). Did you just scan the all earth or what ?!!! :D
@@videojeroki a zoo with a castle in the grounds. We used multiple scanners. Terrestrial lidar scanners , UAV lidar and a wearable mobile mapping unit. We also took over 40,000 61MP full frame photos which are used to texture the mesh.
Some programs might, but others use a method called RANSAC. For star fields, it’s not too much to ask of a computer to try to “brute force” match the brightest stars in 2 images.
Really interesting video! Are self driving cars doing this type of mesh stitching on the fly? And if so, would adding special reflective tape to the environment help with key point localization and reduce the amount of computation its doing? (Clearly I don't know anything about this stuff)
@@difflocktwo nope too stressful haha, i'm in grad school for biomedical engineering. but I have talked with surgeons who use registration technology with robotic assistance and it's super cool to see these concepts work in real situations
@@xsoggyeggroll4966 Liked the job there. I meant have you seen open surgery with these products been using or just simulations? I was talking about seeing a surgery 😄
I assume the reason ICP isn't N^2 is because computing the "centre of mass" is the same as computing an mean/average? i.e., you can just **add** to the current average/accumulator then perform a single division at the end?
KD trees are the underappreciated workhorse of machine vision. If done properly they can be blazingly fast compared to other spatial partitioning schemes. They don't provide some of the benefits of more complex structures but for nearest neighbor searches they're tough to beat. Interestingly enough the textbook implementation of KD trees (and the ones found in open source libraries like the PCL) are not optimal. They are "pure" but in practical terms there are tricks you can play to speed them up without much work.
The algorithm was originally going to be called Iterative Closest Union Point, but after an inexplicable outbreak of giggle fits, the name was changed.
What is a situation where you would already have the geometry from a point cloud and need to overlap it like he's demonstrating. From what I've seen protogrammatry and point cloud to mesh software already aligns the points before even doing a geometry pass.
Can you guys explain how those spinning lidar sensor are use to create a 2d map. (in term of how the raw data that lidar received get compute by the computers and then TADA we just have a map?)
Me: student at Stanford studying for my robot autonomy exam Mike: Have you heard of the stanford bunny Me: not really, I've just spent the past two weeks of my life working with the point cloud of this bunny. but that's about it Mike: Me: will i ever escape this bunny
that would be my first idea to solve this issue I guess. create a set of data describing distance and angles of a bunch of points, then find the same results in the new data. although saying that point cloud data is usually very different from scan to scan.
@@emmanueloluga9770 I discussed it this morning, it went very well. It is on an aerial point-cloud registration algorithm designed for urban scenarios. Basically it uses both image and point-clouds features to find an initial alignment as Mike does manually in the video and then applies ICP to refine it. I used an aerial Lidar scan of my uni city and ground clouds built with photogrammetry from a goPro(which stores a lot of position metadata) .
uh, I woulda just matched just compared the distances from the relative end points and worked my way inwards, if the distance stretches or contracts to much then it's not the same object, abort and try the next object, either way you need to identify the objects in the snapshot working with, might as well just compare the results to previous results to see what has the least difference in point distances **Edit:** the process could be further optimised by skipping objects that have to big a difference in the number of points also
Obligatory question that will probably be answered in the next minute: Are there any physics-based/momentum-like solvers? "We've been moving kind of left-ish at two meters per second. If we assume some points are close, and some are far, where would the last set of points fall?"
A couple of applications of this: 1. You have a lidar scan of terrain that includes woods, which has no color but has a few points on the forest floor, and a photogrammetric scan of the same terrain, which has color but no ground points. You align them and color the lidar points. (At least PerfectTIN, the program I wrote to make a TIN of point clouds, completely ignores color.) 2. You have a drone with a 32-head laser scanner. You fly it between two buildings over pavement and get 32 point clouds, one from each head. You align them and determine the calibration so that the point cloud you get from putting all 32 heads together is accurate.
Speaking as a grown up kid from the Commodore 64 days, may I just say, the kids are all right. You want self driving cars that save babies automatically? Let this up and coming generation design them.
"Maybe you have an artist sculpt something and then you scan it and get a nice mesh" well, a mesh anyway. point clouds don't yield clean topology, so one almost always has to retopo the mesh
But I think you are skipping over the main problem: how do the LIDAR scanners do the job of identifying common points that you did manually in an automated way?
i would guess that because of the high scanning frequency the points are already going to be pretty well aligned. each consecutive scan will be pretty similar to the last
Probably with the help of other sensor data. For example, wheeled robot can take the encoder reading at the wheel to estimate the change in direction and position of the lidar sensor itself between each lidar scan. This data can be used to first align the point clouds closely together enough before doing the ICP.
If you know the exact location of your camera and lidar, you can map lidar points to camera pixels, then use strategies like edge detection or color feature isolation to tag points.
His input was a rough guess, in any real-world application you can use ubiquitous accelerometer or GPS data to get that guess. It's just not relevant to the algorithm itself, which is the focus of the video.
yes to above answers. it's already close, but you can improve it with inertia measurements, feature detection in the point clouds, or feature detection from some color camera
Ultimately you end up having to solve the same problem. The example "cloud" shown in this video might give you the impression that you could just fit something like a spline or Bézier curve through the points and then "simply" align the two curves. First problem is that most real world data sets won't let you simply fit a curve..."Well, then maybe fit a surface" you might say... again, not always easy, or fast. But even if you do manage to fit nice representative surfaces to the clouds... how do you think you find the registration between two surfaces? You use ICP :)
When scanning a stationary object like this from all sides, why not record the camera position and direction for each scan? It seems like that data could be used in place of the manual rough alignment step, and then the whole thing would be automatic.
In practice, if you can solve a problem in software alone, it's usually cheaper than solving it using hardware that you'll have to buy, install and debug.
So I see no way to do this without scanning your two groups of items at least once. I mean, you have to know what you've got. So, as I did that, I'd accumulate data I needed to define the "principle axes" of the two groups, along with extent information along those axes. Then in the "general" (non-pathological) case, it should be as simple as matching those up. We could contrive situations where there were symmetries that would make that approach non-deterministic, but it sure seems like it would get you a long way.
The problem I have with Dr Mike's videos is that whenever I come here to learn about a specific thing, I end up binge watching many more of his vids, and then forget the reason I came here in the first place.
Happens to me all the time.
Very timely :) I have just finished stitching together several hundred lidar scans of a 2 km long road tunnel for a client. I relied pretty heavily on the auto-alignment software!
I was about to say "sheesh, couldn't you just pay somebody to do all that fiddly work" and then I realized that I'm a doofus and *you're* the one being paid to do fiddly work lol.
how did you do that
Mike's videos are always pretty engaging. This one is right there near the top.
Pretty cool stuff.
That video is rather unexpected - thought the next Mike Pound video would definitely be about Log4j.
He even said on Twitter he was working on it
That has to happen
Usually these videos are shot in advance - not sure about Sean but I know Brady will make a trip to Nottingham and film enough for a handful of videos and then dole them out over time
Singular value decomposition is one of those things that translate extremely well to GPUs. In fact, Nvidia's CUDA library has purpose-built SVD support (the cusolverDnDgesvd function in its cuSolver package).
I used that for live feeds of a self-driving simulation.
Sadly, SVD is by far not the bottleneck in this algorithm because it's only an SVD of a 3x3 matrix. Actually, the real thing that needs to be sped up on the GPU is nearest neighbors search, which is a bit tougher but doable
You'd have to use a parallelizable KD Tree
Enjoyed - nicely explained. Love the enthusiasm.
Seems to me that it works well for the scenery but there is a whole set of other problems if there are moving components. First level difficulty a moving rigid object - say a car. Even more tricky a flexible item - say a person walking.
For a real time application then more frequent scans means closest point is more likely to be the corresponding point but you have less time to process each scan - interesting trade off.
The goal of ICP is to reduce the amount of error, where "error" in this context would be defined (very roughly) as the sum of squared distances between the matched pairs of points. So movement in your scene would be handled (naively) the same way as the edges of the bunny scans in his example - the algorithm will minimize the error as much as possible but eventually get to a stopping point where most of the scene points overlap nicely but there are areas of large concentrated error, corresponding to the places where there was movement. (Hopefully that was clear enough.)
It's inspiring how you explain different computer science topics and still explain them all exceptionally well.
This is great, I've worked as a Lidar engineer for like a decade now, and ICP is one of those tools which will just tie everything together in a nice bow.
In astronomy we often need to map arbitrary images of the night's sky to Right Ascension and Declination coordinates (called plate solving). I wonder if this algorithm is involved, using stars as the points.
When your points have additional attributes, such as brightness, it's often helpful to treat them as an additional dimension. Besides that, what the matching program does is probably very similar. At least, as the final step. To improve convergence, it might, for example, start with just the brightest points, and add more as it gets closer to the solution.
Have a look at OpenCV + Homography.
I’m currently doing my dissertation on using lidar on iphones to create 3d meshes. I’ve been watching computerphile since its inception back when i was in high school, crazy to think how far i’ve come since then
I am loving these recent videos on Computer Vision! Thanks!
Finally a video on the technical aspects of Insane Clown Posse (ICP).
Magnets how do they work (more sixty symbols territory, perhaps a sociology channel as well ;) )
Fing LIDAR; how does that work?
whoop whoop
they got the dog beats
I've always had Mike Pound pegged as a Juggalo, so it's nice to have my suspicions finally confirmed.
Pleease more of this :3 great, interesting stuff perfectly presented as always
I had this video in my watch later over 3 months now. My expectation was to learn something about photogrammetry and matching pairs to build a tree and then a point cloud. I had some ideas to how it might work with projection and interaction, but that seemed a little involved. Now with the topic of doing 2D or 3D tracking in video, I always assumed the 2D example would do an extrapolated spiral search pattern for points and just find the lowest difference in the search area.
But it never occured to me to see matrix tricks like SVD show up. Such a crossover.
I knew that a fourier transform on a matrix (like an image) would ignore any kind of translation or rotation, so you would use it for pattern matching for example. There was an actual optical application by Huygens Optics in a video a long while ago.
I had no idea you could use meshlab like that. Thank you for making this.
the legend returns
Love Dr Pound
Very nice explanation! You could have also shown how bad it can get by doing the wrong initial point correspondence and hitting a local minimum.
Awesome explanation and demonstration!! Love it!
I'm a simple man. If I see Dr Mike Pound in a thumbnail, I click.
I think Mike is the best presenter on this channel.
Hope to see a video on log4shell, when you get around to it :)
Cheeky
This was the clearest explanation of any algorithm I've ever heard in my life.
Mike Pound's back!
The videos are great. Can you kindly provide with captions ?
13:15 The sound effect for ICP is brilliant.
5:40 I understand the centre of mass of the points to shift the sample - but how do you decide the rotation? Do you just move it to centre of mass, then rotate and if it gets better, you rotate it some more , until it gets worse again? Or is there a better way to do it?
rotations are simple enough that they can be expressed as matrix multiplication. the resulting matrix will not be completely solvable because its an approximation, but SVD gets you the best solution you can get with linear algebra
@@alegian7934 yes he mentions SVD later but considering its 3 dimensions my mind still boggles how this can be done so quickly. Is there anything that helps a first guess?
I could be wrong in any of the following. But: if matrix A describes the 3d rotation, X is the matrix that contains as columns all points in one group and Y is the matrix that contains all points in the other group, we have AX=Y (approximately, remember, these cant truly be equal) so ideally if they were equal we would invert matrix X and obtain A=YX' (we found the rotation matrix) In reality X is not invertible at all but we can find its SVD that gets us as close to its inverse as possible. so we get something like A*UΣV=Y and we solve for matrix  (an approximation of the optimal rotation). And we apply  to X
@@alegian7934 I guess that's it! This is all new to me. We did cover 3d transformations back in my degree, but not inverse, only forward. At least that's what i remember! Very interesting, will have to do some more reading before I understand this enough.
@alegian Can't we just do the following:
1) build a vector A out of the selected points of the first shape in respect to their center of mass: A = Σ a_i - a_center
2) build a vector B out of the selected points of the second shape also in respect to their center of mass: B = Σ b_i - b_center
3) calculate angle between the vectors?
now do one about SLAM (specifically with lidar maybe), would be a very cool topic
I thought the Utah teapot was the archetypal 3D model, never heard of the Stanford bunny!
I think Utah teapot is the classic hard surface modeling and Stanford bunny is more relevant to photogrammetry
Our biggest scan project was around 20 Billion points. Took a while to register even on a Threadripper-based workstation. Our scanner measures 976,000 pts/sec.
Maybe you where busy counting your points, but if you listen carefully around ~10:00 he is answering your comment
Btw: 20billion points would take more than 230 days of non stop laser scanning (with your ~1M pts/sec laser scanner). Did you just scan the all earth or what ?!!! :D
@@videojeroki, I know how it works, scanning is a large part of my job.
@@videojeroki a zoo with a castle in the grounds. We used multiple scanners. Terrestrial lidar scanners , UAV lidar and a wearable mobile mapping unit. We also took over 40,000 61MP full frame photos which are used to texture the mesh.
i assume this is what is done when they stack photo's say for astronomy. That would be an interesting video.
In astrophotography they can stack up to 100 hours of data to render a galaxy
@@giant3909 omg wow..thats a lot of data
Some programs might, but others use a method called RANSAC. For star fields, it’s not too much to ask of a computer to try to “brute force” match the brightest stars in 2 images.
Please add the subtitles! :) I love the other videos
Thank you for this little tutorial. I wouldn't have found meshlab otherwise.
MMFCL in here for this video, spreading the word for ICP! WOOP WOOP!
Really interesting video! Are self driving cars doing this type of mesh stitching on the fly? And if so, would adding special reflective tape to the environment help with key point localization and reduce the amount of computation its doing? (Clearly I don't know anything about this stuff)
im taking classses in computer assisted surgery and this couldn't have come at a better time lol
are you a surgeon?
@@difflocktwo nope too stressful haha, i'm in grad school for biomedical engineering. but I have talked with surgeons who use registration technology with robotic assistance and it's super cool to see these concepts work in real situations
@@xsoggyeggroll4966 Do you see those applications on practical?? That seems scary 😶
@@mertakyaz5359 icp is generally used to match medical images with the robots workspace and also to track tools in the workspace
@@xsoggyeggroll4966 Liked the job there. I meant have you seen open surgery with these products been using or just simulations? I was talking about seeing a surgery 😄
12:09 Where is FPFH (from PCL) or multiview registration when you need it?
Crazy, I am just doing a project in the next weeks on this for university
Thank you for this interesting video
Let's all take a moment to appreciate the existence of FOSS!
Very interesting. You can see the potential for some sort of memoization as well.
16:14 - "you can put the links in the description" -- is it possible to put the links in the description, please?
I assume the reason ICP isn't N^2 is because computing the "centre of mass" is the same as computing an mean/average? i.e., you can just **add** to the current average/accumulator then perform a single division at the end?
Yep
It's in NP
Very good mike
You mention translation and rotation, but in 3D I'm also thinking about relative scaling and skewing. I presume a generic affine transform is assumed?
Can we see a video on the TOR network compromise where a bad actor is running a bunch of relays?
The bad actor you speak if us the US government
Please do a video on Log4Shell!
Dr Pound said there would be links in the description...?
Oh yes, sorry, bear with me! -Sean
Kd Trees, would make an interesting computerphile video.
KD trees are the underappreciated workhorse of machine vision. If done properly they can be blazingly fast compared to other spatial partitioning schemes. They don't provide some of the benefits of more complex structures but for nearest neighbor searches they're tough to beat. Interestingly enough the textbook implementation of KD trees (and the ones found in open source libraries like the PCL) are not optimal. They are "pure" but in practical terms there are tricks you can play to speed them up without much work.
The algorithm was originally going to be called Iterative Closest Union Point, but after an inexplicable outbreak of giggle fits, the name was changed.
what's wrong with that name? I'm not a ntive english speaker so I think I'm missing the joke
@@julianatlas5172 Say the initials out loud.
What is a situation where you would already have the geometry from a point cloud and need to overlap it like he's demonstrating. From what I've seen protogrammatry and point cloud to mesh software already aligns the points before even doing a geometry pass.
man, this was pretty clear. nice what you can do with one sheet of paper XD
Can you guys explain how those spinning lidar sensor are use to create a 2d map. (in term of how the raw data that lidar received get compute by the computers and then TADA we just have a map?)
Feels pretty similar to k means clustering.
Basically both are gradient descent on a non convex function, involving distances in N dimensions.
Is this somewhat like 'persistence of vision'?
Check out Scale-Adaptive ICP too :)
What’s does he says at 9:46 ? Singularity composition ?
Singular value decomposition
A long shot but how important is it for it to be accurate?? Would the fast inverse square root solution work well for this?
how did u eliminate the unwanted points form depth camera
Are you going to make a video on the Log4j exploit?
Mike has a home? I thought he just lived in his office, seen in this video behind him
He appears at home only when he is on loan from the university.
@@drooplug I see
Understandable
Me: student at Stanford studying for my robot autonomy exam
Mike: Have you heard of the stanford bunny
Me: not really, I've just spent the past two weeks of my life working with the point cloud of this bunny. but that's about it
Mike:
Me: will i ever escape this bunny
is this also used for 3D dental scan technology?
I'm pretty sure yes
Please do a video on the recent log4j exploit!!
I guess you can also do some calculations in other domain, for example distance(angle) instead of cartesian x(y). It can help.
that would be my first idea to solve this issue I guess. create a set of data describing distance and angles of a bunch of points, then find the same results in the new data. although saying that point cloud data is usually very different from scan to scan.
OMG YES! !! THANK YOU THANK YOU (I guess my pfp gives away my enthusiasm)
I discuss my masters thesis in two days and this is something i used in it. nice timing 🤣
Good luck on that!
Awesome. What's your master thesis on?
@@emmanueloluga9770 I discussed it this morning, it went very well. It is on an aerial point-cloud registration algorithm designed for urban scenarios. Basically it uses both image and point-clouds features to find an initial alignment as Mike does manually in the video and then applies ICP to refine it. I used an aerial Lidar scan of my uni city and ground clouds built with photogrammetry from a goPro(which stores a lot of position metadata) .
@@guoah9 oh wow thats very interesting and exciting work. Congratulations on that accomplishment. Is there a paper out yet
uh, I woulda just matched just compared the distances from the relative end points and worked my way inwards, if the distance stretches or contracts to much then it's not the same object, abort and try the next object, either way you need to identify the objects in the snapshot working with, might as well just compare the results to previous results to see what has the least difference in point distances
**Edit:** the process could be further optimised by skipping objects that have to big a difference in the number of points also
Obligatory question that will probably be answered in the next minute: Are there any physics-based/momentum-like solvers? "We've been moving kind of left-ish at two meters per second. If we assume some points are close, and some are far, where would the last set of points fall?"
Instinctive physics-undergrad answer: represent each point in 6D, being 3 spatial coordinates and 3 momentum coordinates. Then do ICP in that space?
This was an Advent of Code puzzle this year
very interesting
Sir you played an amazing role in Spiderman no way home !
Huh? Explanation?
A couple of applications of this:
1. You have a lidar scan of terrain that includes woods, which has no color but has a few points on the forest floor, and a photogrammetric scan of the same terrain, which has color but no ground points. You align them and color the lidar points. (At least PerfectTIN, the program I wrote to make a TIN of point clouds, completely ignores color.)
2. You have a drone with a 32-head laser scanner. You fly it between two buildings over pavement and get 32 point clouds, one from each head. You align them and determine the calibration so that the point cloud you get from putting all 32 heads together is accurate.
No 3d scans of Lenna?
Could we get some captions please? I'm finding it very hard to make out what Dr Mike is saying...
Speaking as a grown up kid from the Commodore 64 days, may I just say, the kids are all right. You want self driving cars that save babies automatically? Let this up and coming generation design them.
How does this relate to SLAM?
Video on Log4?
The missile knows where it is, because it knows where it isn’t
You know the guy knows what he's doing when you see the MX master
gradient descent?
"Maybe you have an artist sculpt something and then you scan it and get a nice mesh" well, a mesh anyway. point clouds don't yield clean topology, so one almost always has to retopo the mesh
But I think you are skipping over the main problem: how do the LIDAR scanners do the job of identifying common points that you did manually in an automated way?
i would guess that because of the high scanning frequency the points are already going to be pretty well aligned. each consecutive scan will be pretty similar to the last
Probably with the help of other sensor data. For example, wheeled robot can take the encoder reading at the wheel to estimate the change in direction and position of the lidar sensor itself between each lidar scan. This data can be used to first align the point clouds closely together enough before doing the ICP.
If you know the exact location of your camera and lidar, you can map lidar points to camera pixels, then use strategies like edge detection or color feature isolation to tag points.
His input was a rough guess, in any real-world application you can use ubiquitous accelerometer or GPS data to get that guess. It's just not relevant to the algorithm itself, which is the focus of the video.
yes to above answers. it's already close, but you can improve it with inertia measurements, feature detection in the point clouds, or feature detection from some color camera
Facial recognition algorithms must be using this as well
Looks like this would be used in fingerprint Scan or qr code scan maybe.
n squared? Isn't exhaustive search going to be n factorial?
Use thios quite alot for game art. Photogrammetry though, so dont know if it work the same.
why not fit a curve to each set and rotate until those curves match. translation is automatically accounted for
Ultimately you end up having to solve the same problem. The example "cloud" shown in this video might give you the impression that you could just fit something like a spline or Bézier curve through the points and then "simply" align the two curves. First problem is that most real world data sets won't let you simply fit a curve..."Well, then maybe fit a surface" you might say... again, not always easy, or fast. But even if you do manage to fit nice representative surfaces to the clouds... how do you think you find the registration between two surfaces? You use ICP :)
When scanning a stationary object like this from all sides, why not record the camera position and direction for each scan? It seems like that data could be used in place of the manual rough alignment step, and then the whole thing would be automatic.
In practice, if you can solve a problem in software alone, it's usually cheaper than solving it using hardware that you'll have to buy, install and debug.
You do that with inclinometers and GPS, but the precision you get is not enough. It is enough for the initial estimation, after that you use ICP
Is it just the lighting or does Dr. Pound have blue and green eyes?
Please add subtitle. Love from oversea.
the structure of the algorithm feels kind of similar to the face id algorithm that was shown in computerphile many years ago.
Is that a vivarium ?
I started working on a project involving ICP and point clouds this month LOL.
Yes
05:08 -> Now it's clear to me that drawing lines doesn't make any sense
07:30 -> Can I just use my lines? How clever... :P
Please request him to talk about log4j
Whoop whoop!
So I see no way to do this without scanning your two groups of items at least once. I mean, you have to know what you've got. So, as I did that, I'd accumulate data I needed to define the "principle axes" of the two groups, along with extent information along those axes. Then in the "general" (non-pathological) case, it should be as simple as matching those up. We could contrive situations where there were symmetries that would make that approach non-deterministic, but it sure seems like it would get you a long way.
Mike is back? instant klick
sorry but he i just the best host on here, second would be miles.