Hello guys,
Im trying to record a kinect stream inside touch designer. Do you know if its possible to record as a mov a point cloud color? Or i need to do it outside touch with another program like brekel or kulfex.
Thanks
1 Like
Yes, you can do it with the Movie File Out TOP. But caution, it depends which data you are recording from the Kinect as Depth and Point Clouds come in different pixel formats… For example a Point Cloud data is 32-bit float pixel format, so the only way to record 32-bit float movie is as an .EXR image sequence since most codecs don’t support that high bit depths. .EXR image sequences can be written out and read back into TD, but they are not super fast, your system specs will need to be good.
If you are recording other streams its pretty easy.
I was also reading here that the Kinect Recorder that comes with Kinect Studio does a good job and can feed playback directly into TouchDesigner. (not sure if its just Kinectv2 or Azure as well)
Hello, I am using Kinect Azure and I have recorded with Kinect Studio the mvk files and put them in the Kinect Viewer. It all works good, but TouchDesigner is still streaming from the main sensor and not the recordings. Looks like is not going through the Kinect Studio. Am I missing something? Thank you!
Microsoft change things quite a bit with the Kinect Azure and I don’t believe it supports streaming from the kinect studio (or viewer) apps anymore. There is a library for reading those files directly, but that feature isn’t currently implemented in TouchDesigner.
There is some additional information on extracting the data from the kinect mkv files here: Use Azure Kinect Sensor SDK to record file format | Microsoft Learn For example, you could use the ffmpeg command line tool to extract the depth information as a series of PNG files that could be loaded into TouchDesigner
Additionally, as Ben mentioned above, we do support reading/writing sequences of EXR files as a method of storing uncompressed 32-bit floating point depth data.
Hey @robmc - do you know if the changes broke the ability to play back Kinect v2 .xef data from Kinect Studio into TD? I’m having trouble getting that to work right this moment.
As far as I know, streaming from Kinect Studio 2 into TouchDesigner should still be working. The azure stuff is all separate with its own viewer app and nodes, so it didn’t really affect the original Kinect systems.
Unfortunately, I don’t have the kinect 2 hardware available at the moment to double check it.
hi @robmc
i’m trying to record point cloud from Azure kinect with movie file out TOP, i choose ‘image sequence’, ‘openEXR’ and record. but when i add the result file to movie file in TOP it doesn’t play as a video or image sequence; it just shows a static frame. please help me, how to get back video?
To play it back as a movie, change the filename in the Movie File In TOP to just be the folder where there EXR files are located. It will automatically play back all of the files in that folder in sequence.
Are you still including the me.fileSuffix reference in your Movie File Out filename? You need to use that python expression for it to update the file name for each frame, otherwise it will just keep using the same name for each frame and you’ll only get one file.
@robmc i tried but didn’t understand. if i want to save all files to folder X with names x1, x2, x3, etc (or what suffix will be) i type X/x.me.fileSuffix.exr in filename or what? i tried several options but nothing works
Make sure you have the parameter in Expression mode for the TOP to properly evaluate the me.fileSuffix code. You can tell it’s in expression mode if the text is light blue rather than grey and when you expand the parameter the second box (the blue one) is filled in. I’ve attached a screenshot of what it should look like:
On a separate node, I recommend increasing the ‘Max Thread’ parameter to something greater than 1 since saving EXRs can be fairly slow and the node won’t likely be able to keep up with only 1 thread.
Thank you so much!!!
1 Like
No problem. Let me know if you have any other issues.
Actually I have a question. Is it possible that Azure kinect being opened in Touchdesigner has less point cloud quality than opened in azure kinect viewer (oficial app from microsoft for azure) or other app? It seems like so when i look at point cloud but i’m not sure
There shouldn’t be any differences in the source data between Touch and the Kinect Viewer; however, they do render them in different ways (both in 2D and 3D), so they might not look exactly the same.
When you say the quality is different, are you talking about the number of points, the frame rate, or something else? The number of points is controlled by the image resolution which is tied to both the Depth Mode parameter and also the ‘Align Image’ toggle.
Hi Ben,
Can you explain a bit how to record with the kinect studio and replay on TD ??
My specs are not good enough to record and replay the .exr files so I want to try the method with the kinect studio.
Hi @hardworkparty, bit late but for whoever may find this useful, I found out that having a Kinect TOP active receiving unexistent data from the.xef file makes the other Kinect TOPs don’t work when Kinect Studio is streaming from a file playback. In my case the file I was using had no color data and having a Kinect TOP receiving color data in TD made the others stop working, turned off the color Kinect TOP, and they were working fine. Spent couple hours fighting with it, so I’m commenting here for future me to remember.
1 Like
Hey Rob. Been following your guidance, but I’m not being able to properly record from Kinect with Movie File Out. For some reason is only storing just a few frames from the whole sequence (around 2 frames per second, sometimes one, sometimes any.)
You know what could be happening?