Hi everyone,
I’m working on an interactive installation and I need some advice. I’m building a 2x2 LED wall that should behave like a big touchscreen. The content is an interactive puzzle made in TouchDesigner, and the idea is that users physically touch the LED surface to interact with it (ideally multi-user).
In my equipment I have two Kinect v2 (Xbox) and one OAK-D Lite. I’m trying to understand what the most solid and stable workflow would be to detect touch on the LED wall surface.
What I mainly need to solve is surface calibration and touch detection. Has anyone built (or shared) a .tox or example project for 4-corner calibration of a surface using depth data? I’m looking for a clean way to map depth coordinates to a defined quad, something like a homography setup inside TD, with proper perspective correction and alignment.
Regarding touch detection: is depth thresholding (detecting when fingers get close to a plane) stable enough with Kinect v2 in your experience? Would you recommend using a single Kinect centered in front of the wall, or two Kinects left/right for better coverage? Or does it make sense to build a hybrid workflow, for example Kinect for body presence and general interaction and OAK-D Lite for more precise hand/finger proximity?
I’m particularly interested in stable threshold logic for detecting a “touch plane”, good strategies for noise reduction, and how to deal with possible IR interference from LED panels. Latency is also a concern.
If anyone has experience using OAK-D Lite for surface interaction mapping, I’d love to hear pros and cons compared to Kinect v2, especially for short range depth precision (around 1–2 meters) and calibration workflow in TouchDesigner.
My current idea is to define the LED wall as a calibrated quad in TD, convert depth to grayscale, apply a threshold close to the LED plane, blur/dilate to stabilize, extract blobs, and then map blob positions to the puzzle logic. Does this sound like a robust approach, or is there a better method you would recommend?
If you have a reusable .tox, GitHub repo, example file, or just experience doing touch detection on LED walls with depth cameras, I’d really appreciate your input.
Thanks a lot.