Kinect for virtual set - Help to remove weird artifacts and noise from TOP

Hello all,
I’m using a Kinect2 and I’ve created a coloured point cloud of the Player, using:
1 - the Player Index image to detach the player from the background
2 - the Point Cloud image to create the 3d points
(and the Color image for the texture).

My issue is that both images are very noisy, so the result is very noisy as well, and in my opinion my point cloud of the Player is actually just good for a rough demo, so I should refine them to try to avoid artifacts as much as possible.

Due to the low resolution and quality of the produced images from this device, them are quite unstable (floating false small area here and there) and very poor on the edges (no AA).

I’ve worked with a Orbbec Femto Bolt too, but despite its price it didn’t give me more quality than my old Kinect2

I’ve created a .TOX with some operators (erode + dilate + blur, etc) but it doesn’t seem to make its job so well.
Anyway you can find it here
player_index_refiner.tox (4.2 KB)

So I’m wondering if there are some tips to make these images from the Kinect to work better than what they do out of the box, or maybe if someone can suggest me a better (cheap) way, or better devices (not motion control camera) to create a virtual set with a real person (without green screen).

I know that’s hard to do with no money :grimacing:
Many thanks for your help!

1 Like

Hi @Issimissimo,

i don’t have a solution for you but what I think your are looking for is real-time semantic segmentation of a pointcloud.
Maybe if you research that topic online some more ideas will pop up.

cheers
Markus

1 Like