I’ve used the Kinect, Kinect Azure, and Realsense sensors with success. I have found the Kinect Azure to work really well for WFOV depth sensing. Since it’s been discontinued, I was hoping the Orbbec Femto Bolt would be a drop in replacement. I updated to 2023.11340 and set up an Orbbec TOP. The only settings I changed were the depth resolution to 1024x1024 and the Image to Point Cloud. Then I’m instancing a bunch of boxes to visualize the point cloud.
Here is a capture of the Kinect Azure point cloud:
Here is a capture of the Orbbec Femto Bolt point cloud:
I put a ladder in the middle of the capture area as a demonstration, but the entire point cloud seems warped. Is there a setting that I can adjust on my end? I’d love to set it so that this is a drop in replacement of the Kinect Azure’s “Wide FOV - Unbinned (1024x1024)” point cloud.
Thanks for the comments. I can confirm with my Femto Mega that there is definitely more distortion with the Orbbec compared to the Kinect.
You can see in the top images, that the depth camera is largely identical (kinect on the left, orbbec on the right), which is what I would expect since they are supposed to be using the same camera hardware internally. However, as you noted the projection of the points is far less distorted on the kinect than the orbbec.
Based on this, it appears the intrinics on the Orbbec aren’t being applied properly to undistort it. That is currently being done by the Orbbec SDK, but I will take a look and see if there’s anything we can do to improve it.
I’m running in the same issue, Orbbec implementation in TD seems to be skipping some undistort command. Found a mention of distortion it in the sdk, could it be a solution?
ob::CoordinateTransformHelper::calibration2dTo3dUndistortion(param, sourcePixel, depthValue, OB_SENSOR_DEPTH, OB_SENSOR_DEPTH, &targetPixel);
Here is a comparison between the Femto Mega and the Kinect Azure in TD. The Femto pointcloud is very distorted at the edge.
@k.anti Thanks for the tip. I will take a look into that function.
@tylersmithfkb I did another test just now using the Kinect Azure TOP in the Orbbec mode, and that seems to fix the distortion. The only catch is that I encountered a crash if you try to use the Wide FOV with the default 30 FPS - this normally just causes an error with the Kinect, but seems to crash the Orbbec for some reason I’m looking into. However, if you set the FPS down to 15 before setting the Depth mode it seems to be ok.
I’ll let you guys know when there is an update available.
Quick update: It looks like that undistort function was added in the 1.9 version of the SDK and we’re currently using the 1.8 version.
We will look at updating the SDK in TouchDesigner for our next update, but in the meantime hopefully you can use the Kinect Azure TOP as mentioned above.
@robmc I’m glad it wasn’t just me. Thanks for the quick fix and for looking into a longer term solution! I always appreciate how quickly TD adds compatibility with new tools/sensors.
@k.anti Nice contribution!