I am trying to organize a project where my client (located in a different country) would send me Kinect data/footage so that I could create some sort of 3D point cloud effect.
Is this possible through a combination of multiple Movie File In TOPs or is having the Kinect directly plugged into TD via the Kinect TOP the only way to extract depth info etc?
To clarify; my client does not have TD but has access to a Kinect. I’m simply wondering if there’s any way he can record video footage with depth information that I can somehow use in TD on my system?
I am hoping to achieve a 3D/spatial effect similar to this Holly Herndon video:
Any pointers or tips on how to achieve this would be greatly appreciated!
In the new experimental release we’ve added support to the Movie File Out TOP to save multi-channel EXR images that can store both the color and depth images from the Kinect (Azure or earlier models) in the same file. You can then use the Point File In and Point File Select TOPs to read the file sequence back into Touch.
If you prefer to use the Official build, you can achieve similar results using two Movie File Outs to save the depth and color images to separate folders and then loading them in using two Movie File Ins.
If I understand what you’re saying correctly, my client would need to use TD with the Kinect and then use movie file out, correct?
This wouldn’t really solve the problem as my client does not have Touchdesigner - the main thing I’m trying to figure out is whether there is a way for my client to record info from the Kinect (saved as various mp4 files or whatever else it can be saved as) which he would then send me so that I could load it into TD.
Essentially, my client would be asking a few of his contacts to record video data from their homes with a Kinect, so the data capture process needs to be as easy as possible (it would be challenging to get everyone to download touch just to send me these files). The final project would feature Kinect info from each of these people’s homes which I would then combine into a “master” project on my own.
This video explores a .toe Malcolm posted, is there any way to replace this project file’s 2 Kinect TOPs with RGB and XYZ data from 2 Movie file In TOPs from video data captured without touchdesigner?
Sorry, I missed that you were looking for a non-TouchDesigner solution.
If you’re using a Kinect 2 then you can use the Kinect Studio application in the SDK on the end-user’s machine to capture a playback stream that includes depth and color information. When you play it back on your machine, TouchDesigner will capture the data from the studio app just like it would if the device was directly connected to your machine (when you hit play in the studio it overrides the live feed).
I just tried it out here and it seemed to work pretty well. The new Kinect Azure also has recording/playback options but I haven’t tried that one yet.
So in order for TD to process the info gathered by the users from the SDK, I would need to have the SDK installed as well? Could you expand upon this process a little more please?
It’s probably worth mentioning that this project isn’t going to be real-time.
Yes, both you and the end users will need the Kinect for Windows 2.0 SDK installed to access the Kinect Studio application and to access the device (I’m still assuming you’re using a Kinect 2 for this). You can get the SDK here if you don’t already have it: https://www.microsoft.com/en-us/download/details.aspx?id=44561
For your end users, they will use the Record tab in Kinect Studio to connect to the camera and to record an ‘xef’ file. Make sure they check the Depth and Uncompressed color boxes on the left of the screen to include both in the output file.
On your end, you should be able to open the toe file from the example that has two Kinect TOPs in it (one with the color and one with the point cloud). You can then select the ‘xef’ from the File menu in Kinect Studio and use the Play tab to playback the stream. As soon as you hit the play button, TouchDesigner should automatically start capturing the streams from the file as if the device was directly connected to your machine.
You can then hook up a Movie File Out to the Render TOP to save off the final output for later editing or playback.
I haven’t tried it myself, so I’m not sure if it works the same way for recording and playback as the Kinect 2.0 I’ll see if I can find any answers on that or maybe someone else on the forum has tried it.
I can say that you will get significantly better resolution with the Kinect 2.0 if you decide go with that option.
First get it working on screen in Kinect Studio by by browsing your file, selecting play tab and hitting little black play arrow. It should play back your selected streams. Assuming that is ok, and you see the motion recording say with both 2D and 3D playback windows in Kinect Studio, all you have to do in TD is drop in a Kinect TOP, select type of image in parameters (default Depth) or Infrared, or Player Index, or Color Point cloud - assuming they recorded the correct streams in the first place, and you will see the playback immediately in the Kinect TOP. Make sure timeline is running of course.
