How to generate point cloud from Azure Kinect pre-recorded data (mkv file)

Hey guys!

I am pretty new to TouchDesigner and I have the following problem. I use the Azure Kinect. I watched the tutorial according Azure Kinect and how to use the live data stream to generate all sorts of intersting stuff. My question is how I can do the same process but with prerecorded data.

The recorded file from the Azure Kinect is stored as mkv file. I already tried to extract the data from the file to an ply file but it did not really work. Do I have to pre-process the mkv file or can I just open it in TouchDesigner and und select the kind of data I want to use?

TouchDesigner doesn’t currently support multiple channel MKV files, but I think there are external tools that can be used to split the depth and color into separate files which you can then load with the Movie File In TOP.

You can use the depthProjection tool in the palette under point clouds to then convert the depth image into 3D points.

Unfortunately it gets more complicated if you need to align the depth and color streams since that is normally done inside the Kinect Azure node.

Thanks for the fast response!

I understand. I tried (still trying) to do the job with open3d or mkvtoolnix. It should be possible with Azure Kinect Sensor SDK as well but I dont get it so far. Do you have any experiences or hints how to do best?

Unfortunately, I haven’t actually worked with the MKV files myself so I can’t really say specifically what you need to do there.

I’ve done a little work with playback of EXR sequences captured from the Movie File Out TOP. The EXRs can take up more space, but you can record the point cloud directly so you don’t need to do any projections.