need help for an art project with motion and sound

Beginners
, ,
1

Hello everyone,

I am a French art student currently working on an interactive installation project.

I am using a camera as my main input and I am now looking to transform body movement data into sound and interactive visuals using TouchDesigner. My interest in TouchDesigner comes from seeing it as one of the most effective tools to translate physical movement into real-time audiovisual interaction.

The core idea of my project is to create a system where the presence, movement, direction, and intensity of the body in space directly influence:

  • sound generation and sound textures,

  • visual elements such as shapes, colors, or motion-based graphics.

At this stage, I am able to capture video input from the camera, but I am still exploring the best ways to:

  • extract meaningful movement data from the image,

  • convert this data into stable and expressive control signals,

  • map these signals to audio parameters (volume, texture, spatialization),

  • and connect the same data to visual behaviors in a coherent way.

As an art student, this project is both technical and artistic, and I am still learning TouchDesigner. I would greatly appreciate practical advice, recommended workflows, or patch-building strategies for transforming camera-based body movement into sound and interactive visuals.

Any help, examples, or references would be extremely valuable.

Thank you very much for your time and support.

Best regards,
A French art student exploring body movement, sound, and interactive media

2

Hello,

I am french and I was teacher in art school (I continue to give workshop in some places) :slight_smile:

Concerning your project, there is some tutos explaining how to do that, mainly on http://alltd.org

I have not the time now to give you more but here are some paths to explore:

– kinect chop is the easier way for movement analysis (need kinect and Windows)

– Optical flow top give info on movement and speed

– Body track Chop (nVidia only) give valuable data

Searching it you can fing many useful examples.

Enjoy TD, Jacques

PS Vous pouvez demander à votre école d’organiser un workshop TD autour de ma venue :slight_smile:

3

Hey, check out MediaPipe and YOLO Tracking from @blankensmithing !