Hello, I’m using a new one Orbbec Femto Bolt (for the 1st time) in my project, for body tracking up to 2 persons, so I’m using the Kinect Azure TOP and the Kinect Azure CHOP.
After a lot of test I’ve discovered that:
1 - in the CHOP operator the number of persons detected move randomly from 0 to 1 or 2. Also if there’s just one person standing in front of the camera, a lot of times the “id” channel go to 0 (zero) for a while, or go to 2 (maybe a ghost?)
2 - this issue is continuous (more than every half second) using every depth mode in the TOP operator. The only depth mode that is a bit more useable is “wide binned 512x512”
3 - with every depth mode (except wide binned) some times TD close silently (I suppose because of this instability in the body detection)
4 - So, by force, I’m using the wide binned depth mode, but the hands tracking (I’m just tracking the hands) is not satisfying at all. I have a Kinect2 that work better…
So, what’s the problem? I don’t think this camera should work like that.
Is it an issue of TD drivers (the Kinect Azure CHOP or TOP) ?
Many thanks for your help!
For reference, the camera is only responsible for sending the depth and color images to the computer … all of the body tracking logic is done locally on your system by the same Microsoft libraries for either the Kinect or Orbbec cameras.
By default the body tracking is run on your GPU, so it might be worth making sure your video drivers are up to date. As a test, you could also try choosing the CPU option on the TOP. This generally isn’t fast enough for practical use, but it might identify the problem.
Is there any chance your camera orientation is wrong, like if it is mounted from a ceiling? It can cause subtle problems for body tracking if the image is not in the expected direction and there is an option on the Kinect Azure TOP to change this.
Have you looked at the player index image for your scene? This might show if it is accidentally detecting something else as a person or if it is not detecting the whole body.
Regarding the crashes, to debug crashes that do not leave a dump file you could try using the WinDbg tool. It can attach to a running copy of TouchDesigner and should be able to catch additional crashes: Using WinDbg to Debug Crashes - Derivative
Also, you might want to check your USB cable and port on your machine. The Kinect / Orbbec cameras can be sensitive to the quality of the USB port and if the camera was potentially dropping frames it might cause problems with the body tracking. Azure Kinect USB Host Controller requirements - Microsoft Q&A
Thanks for your kind reply Rob!
I’ve already tried in CPU mode, it’s obviously slower, but nothing change, same problems.
I’m also using a 3m usb3 cable.
The Orbbec is actually located at 3m in height, rotated down 40 degrees.
BUT, I’ve experienced the same issues with the Orbbec at 1.6m in height, with 0 degrees (just looking on the front), with the provided usb3 cable.
Maybe the Orbbec is broken ?
If you install the OrbbecViewer application, you might be able to better diagnose what the raw data feeds look like. These are the same as what you can get out of the Orbbec TouchDesigner operator, but I prefer this application for diagnosing issues with the cameras themselves by removing TD from the equation.
Thank-you Whorl, but my primary issue is related to body detection with Orbbec camera, and the Orbbec viewer does not give any information about it (as far as I understood).
To clarify, I don’t have problems with the Kinect Azure TOP in Touchdesigner, I get all the images (depth, color, ir) without any problem, in any mode.
BUT I must use a Kinect Azure CHOP too, so to retrieve the players id number, and their hands positions. This is when the issue occur. Indeed, as I wrote, the body (player) detection is unstable: I can stand in front of the camera, without moving, for minutes, and you will see that my player id number goes from 0 (no player) to 1 (player found) a lot of time during all the period, randomly.
Unfortunately I’m on mobile now, and I don’t know how to upload a picture that I’ve took from the screen, but I’ve put a Trail CHOP for the player1 id, and you will see this trail (that should be always = 1) to jump from 1 to 0 a lot of times, randomly.
Moreover, this is the behaviour when using the “Wide Binned” mode in the Kinect Azure TOP. If I use any other mode (Near Binned, etc) you will see this trail to jump from 1 to 0 like a crazy, not “just” a lot of times, and after some time TD crash silently.
I hope I have better explained my (big) problem with this camera in TD
Many thanks you all for your help
The reason I was suggesting debugging the actual frames returned from the camera is because flakiness in depth cameras is often related to the environment (if you’re ruled out the other things robmc suggested). Needing to use a binned configuration suggests you might be running up against the environment. At the very least, I’d rule things out by moving the camera to a very controlled environment to test. We run our Femto Megas at the limit of the NFOV binned range and, while we are not using the Kinect SDK for this project, it requires significant post processing to deal with the noise that appears when reaching the limit of the cameras.
I think that maybe I’m starting to understand, when you talk about image post processing.
But I’m using the camera in a very controlled environment (it’s a small room 2x2 meters, with curtains on the entrance, totally black - floor, walls, and a 3x2 meters screen on the front of the player, and the Orbbec is mounted on the top of the screen, pointing down with 40 degrees).
Looking at the depth image from the Kinect Azure TOP it seem to me quite stable, all the time. No more than one person enter in the room right now, but the issue is still there. I really don’t know how I could make this one more useable for the Kinect Azure CHOP…
