AI-Controlled Parties with TouchDesigner + TDMCP — concept, feedback welcome

General TouchDesigner Discussion
, , , ,
1

Hi everyone,

I’ve been exploring a concept I’m calling AI-Controlled Party: a way to use AI as a kind of “show director” for live events, while keeping TouchDesigner as the deterministic real-time engine.

The idea is not to let an LLM directly control everything on stage. Instead, AI would operate at a higher level: suggesting or selecting pre-approved cues, changing the visual mood, reacting to show context, drafting announcements, or helping the operator navigate a live visual system.

TouchDesigner would still handle the actual real-time execution: visuals, audio-reactive systems, cue playback, dashboards, mappings, and operator controls.

I’m building this around my tool, TDMCP, which connects AI assistants to TouchDesigner workflows:

https://pantani.github.io/tdmcp/guide/ai-controlled-party

Right now, this is still a concept / dry-run pattern, not a fully validated live production system. The current direction is based on a safer architecture:

AI input → show intent → policy decision → approval queue or dry-run plan → TouchDesigner / TDMCP execution only through operator-safe mappings.

For example, AI could request a pre-approved cue like band_intro, suggest a mood change, or queue a fog request for operator approval. More hazardous actions such as lasers, moving heads, blackout, PA control, or audio routing should remain blocked or operator-only by default.

I’m especially interested in feedback from the TouchDesigner community:

How would you approach this kind of AI-assisted show control?

Would you use it only for visuals, or also for cueing, announcements, dashboards, and rehearsal workflows?

What safety boundaries would you consider essential?

Do you see this as a useful direction for live events, clubs, installations, performances, or VJ workflows?

Would love to hear what you think.

Thanks!
Pantani

2

Why would we want an AI to do all the fun stuff?

What would we do in the meantime? Nanny the Ai?

I totally fail to see the benefit, but I am happy to learn another view :slight_smile:

all the best

stefan

3

Hi Stefan,

Fair point — I also don’t think AI should “do all the fun stuff”.

My view is that the fun is still in designing the TouchDesigner system, the visual language, the rules, and the boundaries. The AI would just become another real-time control layer, not the artist itself.

Where I see value is in events that do not have a huge pre-produced show. A small club, bar, or local party usually does not have custom projections planned for every artist. With TouchDesigner + TDMCP, the operator could control or adapt the whole visual system in real time, based on music, mood, microphone input, crowd energy, schedule, or performer cues.

So it is less:

“AI replaces the VJ”

and more:

“AI helps the VJ/operator control a much larger live system with less preparation.”

I agree that nobody wants to nanny an AI during a show. That’s why I think the system should be based on safe mappings, pre-approved actions, dry-run mode, approval queues, and human override.

For me, AI is not the replacement for the fun part. It is another instrument to play with.