0:42 the man moves his arm and it looks like the arm's shadow is cast on his chest. Is it really a shadow or just points that disappear? If it's really a shadow, does it mean your point cloud is affected by lighting like a classic mesh object would? Or is it something else ?
It's a shadow, but in the context of the video capture. It's not a virtual shadow cast by a virtual light. This is because the 3d points are calculated from the estimation of the distances of the points in the image to the sensor (I see that they are using their own DepthAI platform to estimate depth directly from video. But it can be done with dedicated hardware like a Kinect, a Lidar, etc). So, we have what we call a depth video, in which the values of the pixels represent a distance (depth). Then we convert this pixels to 3d points in space departing from their x,y in the image and their distance value. So, if the arm stands in between the chest and the sensor, the depth captured will be the closest one, and so we'll have a shadow of the arm projected in the chest - no 3d points in the chest can be calculated at this very moment. If you are not familiar, searching for kinect depth map and kinect point cloud may clarify🖖
Would be great to get a tutorial on setting something like this up.
Please follow/star the repository: github.com/luxonis/depthai-unity will push soon demo scenes for basic streams and point cloud vfx
0:42 the man moves his arm and it looks like the arm's shadow is cast on his chest.
Is it really a shadow or just points that disappear?
If it's really a shadow, does it mean your point cloud is affected by lighting like a classic mesh object would? Or is it something else ?
It's a shadow, but in the context of the video capture. It's not a virtual shadow cast by a virtual light. This is because the 3d points are calculated from the estimation of the distances of the points in the image to the sensor (I see that they are using their own DepthAI platform to estimate depth directly from video. But it can be done with dedicated hardware like a Kinect, a Lidar, etc). So, we have what we call a depth video, in which the values of the pixels represent a distance (depth). Then we convert this pixels to 3d points in space departing from their x,y in the image and their distance value. So, if the arm stands in between the chest and the sensor, the depth captured will be the closest one, and so we'll have a shadow of the arm projected in the chest - no 3d points in the chest can be calculated at this very moment. If you are not familiar, searching for kinect depth map and kinect point cloud may clarify🖖