ZED 2i camera transform drift issue

Hardware
,
1

Hello dear TD community,

I’m encountering an issue with the automated positioning of the ZED 2i camera, particularly in low-light environments. The problem is that the camera’s position drifts in 3D space (XYZ) when there is movement in front of it, as if it believes it’s moving—even though it’s fixed in place.

I tried disabling the Camera Transform parameter in the ZED TOP, but it seems this only hides the transform values rather than disabling the behavior. I’ve tested this with three different ZED 2i cameras, and the issue persists.

Case 1 – Large-Scale Setup:
I’m using two fixed ZED 2i cameras to cover a 12-meter-wide LED screen. When a group of people moves in front of one of the cameras, the ZED seems to interpret the crowd’s motion as its own, and the position drifts.

Case 2 – Small Game Setup:
In a smaller setup with a 2x2 meter LED screen, a single person stands in front of the camera and is meant to be centered on the screen. When they move quickly or come close to the camera, the Camera Transform changes as if the camera itself were shifting.

TouchDesigner version: 2023.212370

Attempts at Solutions:

  1. Periodically Resetting the Camera Transform:
  • Issue: Causes the body tracking to freeze for a few frames every time it’s reset.
  1. Manually Offsetting the Body Tracking Data by Subtracting the Camera Transform:
  • Issue: Results in the skeleton floating or jittering left and right during movement.
  1. Automatically Resetting the Camera Transform When the XYZ Sum Exceeds a Threshold (e.g. 1):
  • Issue: When the person moves fast, the system resets too often, causing instability.

I found this thread on the Stereolabs forum, which discusses modifying ZED SDK parameters to disable automatic positioning. However, the solution is Unity-specific, and since TD uses a custom SDK integration, I don’t seem to have access to those internal parameters.

Attached is a .toe file demonstrating my attempts to deal with this issue:
ZED_skeleton issue.toe (14.8 KB)

It would be incredibly helpful to have an option to lock or disable the XYZ position tracking while still keeping the rotation values (which work perfectly). Any way to completely turn off automatic position updates for fixed-camera setups would be greatly appreciated.

Thanks so much for your time and support!

4

Hello,

We can expose the same parameter to disable the camera self calibration at camera opening. This should be available in the next Experimental release. However It doesn’t look like this will solve the issue you are having with the moving camera. I will take a closer look and let you know what I found.

Best,

Huck

5

Thank you Huck, I really apprieciate, can’t wait to test it!
For the small scale (one-person experience), fixed, centered position of hips is doing the trick but I am still testing what would be a good way to deal with big-scale installation. For now I am keeping the camera reset but in bigger intervals.

7

In the ZED TOP, there is a list parameter for a point of reference, which is World or Camera. Having access to this in the ZED CHOP would solve this issue.
Best,
B