Kinect Feature Request -User Silhouette Extraction

Hardware
1

One of the most useful utilities of a Kinect is getting the silhouette of a tracked user to use as a mask or generative starting point. I believe that the player id is embedded into the depth data coming from the Kinect. Is this data still available in the depth map output by the current version of the Kinect Top (SDK 1.7)?

If not, I’d love to see an implementation of this. I believe there’s an example included with the Developer’s Toolkit on how this information is extracted, also explained in this StackOverflow post.

Thanks for the consideration!

2

Try using a Kinect Top connected to a Cache top. Cache the incomming kinect background.

Then using a composite subtract top you can subtract you cached backround kinect image with your live kinect feed which should leave you with a silhouette. some hsv adjust or other tops can clean the noise.

Let me know how this works for you!

3

Hey emintzer, thanks for the reply but it’s not what I’m looking for. The idea here is that the Kinect can do this actively - using a user’s tracked skeleton and known depth, it can detect whether a given pixel belongs to a person or not. In this way, you can use it with any kind of busy background, and even things moving in front of the user while still detecting a pretty clean outline.

4

+1 on this request.

Examples of how this could be used are odopod.com/blog/working-kine … and-flash/

5

I need this too. +1

6

me too!

7

How would you want this data to be made available to you? As a different texture or embedded in the current depth map texture as another channel?

8

Another texture seems the simplest to deal with. Are there any advantages to having it in a separate channel?

9

I vote for a separate texture, as well. Otherwise we’d just have to extract it from the depth texture in order to use it, right?

10

Doesn’t make much difference to me either way, it is easy enough to pull the channel we need if there’s a benefit on your side to keeping it all in one input.

11

Bump! Anything we can do to help make this happen?

12

This is done, but didn’t make it into the 12000 series experimental. It’ll show up when we post a 14000 series experimental, which will occur sometime after the 12000 series has gone Official.

13

Oh, awesome! I’ve got a project I’m planning out, perfect for Touch with this feature implemented.

14

I see this feature has been implemented as Player Index in the experimental release. When I try to use it, TouchDesigner freezes up on my machine.

Windows 7 64-Bit, using TD Build 16360 (32 and 64-bit versions both freeze). NVidia GTX 660 graphics card on driver 335.23, Kinect For Windows Hardware with SDK 1.8 installed.

15

Thanks mmm, able to reproduce it here…

16

huuu, great its impelemted!
testing it right away, hopefully the crashing will be solved soon…
thank you for your effort!
luca

17

Just tried it, it’s neat!

How would you go using this as a mask for the rgb stream?
I’ve played with the tox at viewtopic.php?f=22&t=5909&p=23417&hilit=kinect#p23417 to match rgb and depth streams (there are some offset/scale on the color stream basically) but is not perfect.

In the depth with color sample in the sdk they use this function :
NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution
Is there a proper way to emulate it in touch?

Thanks!
Vincent

18

Vincent - barakooda was asking about this in another thread. I’ve been trying to figure this out this evening, haven’t got it working, but posted some info here :

[url]Kinect 2 questions - General TouchDesigner Discussion - TouchDesigner forum