We are trying to map several virtual objects in a 3d scene to their physical counterparts in real space. The physical tokens are supposed to alter the projection (which is trying to emulate water with FLEX particles) by acting like obstacles. We have a kinect v2 tracking the scene.
So far we have managed to track reflective surfaces by their IR value as well as objects by their color. The solution of tracking by IR value does not scale very well to more objects since it seems hard to make good thresholding work well - we will be projection onto a physical space with different heights, making it harder to discriminate between what and what not to track. Tracking by color gets tricky since a projected color image will always be present in the physical scene, affecting the color value of the tokens.
Any idea how to solve this? We have started looking towards fiducial markers/qr codes. Is that something that works well in TD, considering we have to track several objects at once. How would one go about doing that?
Greatful for your input!