Accurately Record Azure Kinect TOP Video + CHOP Body/Joint Data

What’s the best way to save the Azure Kinect’s 4K color video, while simultaneously saving the Kinect CHOP joint/body positions into a .chan file? I’m working on a data science project and the goal is to record at the highest FPS for the Kinect with no dropped frames in the video and no duplicate joint data in the .chan. We’re going for recorded accuracy and are looking to have a row of joint data in the .chan associated with each frame of the video.

I currently have the ‘Sync Image to Body Tracking’ turned ON, but I’m running into the problem that the Kinect joint data seems to hang up and write duplicate / many repeated rows of data into the .chan file. Also in the movie file TOP out, I have the fps set to ‘me.time.rate’ to match the current fps of TD. (note when Realtime is on I’m getting ~58-61 fps)

One thing to note is that the Azure Kinect hardware maxes out at 30fps, but I get the smoothest recorded video either running with real-time off, or at 60fps with real-time on.

My intuition would be to keep real-time off, but we get tons of repeated data in the .chan file.

Any pointers would be greatly appreciated!

Hi @tblankensmith,

maybe the attached component will help. It records the skeleton data via a Record CHOP and at the end of the recording saves out the channels as a bchan file. The Record CHOP is set to Record the current frame to avoid discrepancies with the movie recording.
I also added a local time component which sets the rate of the component to whatever is select for the kinect framerate.

When watching the frame channel from the Kinect CHOP, it seems that the skeleton tracking is actually just operating at 15fps - hence the duplicated values in the recording?

cheers
Markus
kinectRecord.tox (52.8 KB)

Finally figured it out!

Turns out the duplicate data we were seeing from the Kinect wasn’t the problem. We needed to run TouchDesigner at ~120fps in order to ensure that we don’t miss any frames from the Kinect. We were able to verify this by looking at the Azure Kinect CHOP’s Frame channel (like you recommended) and making sure the frame is either the same or that it increments by values of 1.

Also, found that the Kinect video recorded best setting the Movie File Out FPS to 60.

Thanks for your help Markus!