How to project image to person using kinect?

Hello, I’m new to TD.

Now I’m working on a project that ‘projecting image to person’ with using kinect/projector calibration.

The calibration was successful by referring to this document.
(Kinect2 - Projector Calibration)

And the image that I want to project to person is this.
(https://youtu.be/PNr_owSi50o)

I successfully did all two things, but can’t combine them!!

Please help me… Is there any way to combine these two?

Like projecting the silhouette image exactly to person using kinect calibration.

image

.
.

++ I want to make like these videos. Image follows the moving person.

Hey @Ahyeon ,

Which Kinect are you using?
I never tried this again but was thinking that you should be able to do this with the Player Index projected onto a rectangle placed at the players position away from the kinect. The rectangle should have as texture coordinates the type Perspective From Camera and reference a camera that uses the kinect’s Depth Cam intrinsics (you can get to via the python method .depthCameraIntrinsics of the Kinect Azure TOP Class). Then rendering this from the Projectors view should give you the right result.
I don’t have a kinect here so it’s all a bit theoretical. Will try this out next week at the office.

Cheers
Markus

Hello, @snaut !
Thank you for kindest reply and helps!
I’m using Kinect 2.
I’m very new in TD so it looks little bit difficult for me but I’ll try my best following your advise!

I’ll wait for your next week’s try.
Thank you so, so much again!

Hello, @snaut !

I made a new .toe file like this but the image doesn’t follows me(moving person) correctly…
I tried to follow your advise but I couldn’t find Perspective From Camera type so did like this… sorry.

Can you check my .toe file’s problem or share yours?
I’m really struggling with this…:sob:

I’ll wait for your reply.
Thank you!

Hey @Ahyeon,

sorry this took so long. Included is a file with 2 approaches:

  • one renders the pointcloud and just uses the playerindex as color
  • the other tries to place a rectangle at the players distance from the camera and projects it onto it.

The first approach seems to be the most successful only falling short at points where you see individual particles. The second approach has a lot of shortcomings as it can’t properl stretch the perspective.

Have a look and hopefully this works.
Cheers
Markus
PlayerIndexMapping.toe (30.0 KB)

1 Like

Hey @Ahyeon,

i just realized that you said you had a Kinect2 - I build this with a Kinect Azure.
Please try the attached file which implements the poincloud render with the player index as the color.

Cheers
Markus
PlayerIndexMapping.Kinect2.toe (26.2 KB)

1 Like

I am trying this exact same thing, except I want a completely black silhouette over the player (as in theoretically zero lit pixels hit their body, kind of idea)

When i hit CALIBRATE inside of the KinectCalibration tool, the lower right pointcloud preview goes black and stays black. Closing and reopening the calibration window doesn’t bring it back. It seems like CALIBRATE doesn’t work?

Also, my render window is always empty (just transparent checkerboard) even though I am showing up on other cameras in the network

@snaut I’m also getting no output (transparent background only) for the render operator when using the kinectcalibration operator. I’m getting successful calibration, but it doesn’t seem like I can easily swap out the normal cam comp for the usual non-calibrated cameras that automatically seem to produce an output when connected to the render top. Could you help with this? Thank you!

Hi @LoveAndMagickMaker,

are you performing the steps approximately as outlined in this video?

cheers
Markus

To combine Kinect calibration with image projection, ensure the Kinect captures the person’s silhouette. Use depth data to map the silhouette and project the image onto it dynamically.

Can you elaborate more on this? I’m trying to use the calibration TOP but connecting anything other than an uncalibrated camera COMP yields no output. I made sure that the calibration was successful and that the Kinect picks up my body but it doesn’t seem to work well when connected to the render TOP

Hello,
I use another technik to project on body from kinect:

  • point cloud where I limit what I want in glsl
  • instance to create a volume with rectangle facing camera
  • camera at the place of the beamer
    you can use the volume as cache or project the texture to the volume.

Here is an example.