Derivative TDMCP: an MCP server for TouchDesigner (beta)
Excited to receive feedback on the MCP. Let us know your first impressions.
Oh sweet! Looking forward to play
Awesome, thanks for sharing this gem @Jarrett!
Sounds very interesting Jarrett. For beginner to intermediate users, would this potentially be useful for optimizing our project to run faster when we have exhausted our attempts with probe, etc? Or is that beyond capabilities. I have not used Claude yet but had lots of issues with ChatGPT untruths previously…
Hey Gord,
Good question. The agent has tools to look both at stats / infos of nodes so it can analyze cook times for example. An agent can be very helpful to read an existing network, but I do find that even the top frontier models are likely to struggle to figure out how to actually build a fully fledged complex network in TouchDesigner.
Installing Skills will generally improve the agent behavior. What is most interesting is if you let a top of the line frontier model work with a small TouchDesigner problem and work with them you will probably get somewhere and learn something too. As long as you know they can’t be completely trusted then you will probably have some success.
A top of the line model can clearly explain most any TouchDesigner network and relevant code with good fidelity. It will miss nuance and relationships when the references are buried in parameters and expressions. You have to image how the agent is looking at the file - it’s like they are in the textport learning about your file by running commands. It’s not necessarily going to give you good information on how to make the network better, but you should try to understand what the agent is suggesting. You have to verify everything yourself. Once you understand this, then the agent can serve as an ongoing help system connected to your file and available whenever you need it.
An important skill you can run is /td-colab. This will tell the agent to work with the human and not try to run ahead and solve things (which they tend to do if left to their own devices). Just typing “/tdcolab” will cause the agent to chill out and communicate with you the human - the td-colab skill tells the agent that TouchDesigner environment is if for the human and not for the agent to run around messing things up. It will generally defer back the human before doing anything. If you ask for help from the agent and tell it not to change anything but help you understand what the network is doing and how you might make it faster, you will probably find success together through a collaborative discourse.
Ask the agent simple verifyable questions. Don’t ask high level questions like “can you make my network faster”. Ask it to use a performance DAT to help you analyze the network and step through the file one thing at a time. The agent can visually navigate around the network with you showing you places where there might be problems etc.
It’s best to have an agent work on tasks where you can measure its success - for example when an agent writes a shader, if the shader compiles and looks good, then you can measure its success because it works. When an agent generates a large network of OP spaghetti, I would doubt the network is fully functional.
The latest frontier models are very smart so they can be extremely helpful with assisting you in understanding a network you didn’t build. I have seen models make quite sophisticated solutions with TouchDesigner networks, but those networks are usually somewhat simple topology of connections and then heavily driven by code on disk where the agent does what it is designed to do - write code. Once you figure out this pattern you will find you can create simpler networks for yourself, while using the agent to help you support a device or build a shader you might not have had time to figure out on your own.
No AI model has ever been directly trained on TouchDesigner and this is why the models are not TouchDesigner experts. They are general experts at a ton of things in the realm of code and general systems so this is why they can move around inside TouchDesigner with impressive dexterity.
The latest Opus 5.5 model from Anthropic is generally excellent so I do recommend you trying it out with a $25 USD account. An entry level account that includes Claude Code or Open AI Codex can serve you as a general TDMCP collaborator, docs guide, and Python expert all day every day. Make sure when you install TouchDesigner that you install the on disk documentation which is an optional setting in the Windows installer. Docs are always installed on MacOS.
With heavy coding and MCP use, your account usage can get paused or down-graded to a cheaper model. Having this happen is good to experience so you get to know how far you can push a top of the line model and also how useful older, faster and cheaper models can be. I have been using a $100 / month account for over a year which feels very fair - I tend to reach my limits with Fable but Opus 5.5 is just as good as Fable and it feels inexhaustible with TDMCP running all work day long 5 days a week doing a ton of coding. I rarely use TDMCP for heavy building of TouchDesigner networks - maybe that would use up tokens more quickly so beware of TDMCP for token burning. Use the Claude Code /status and /context commands to analyze your usage. It will show you how much you use on MCP vs coding etc.
Thanks for the question Gord. Please report back if you give things a try.
Jarrett
Thanks very much Jarrett. That is very helpful info. I will start learning how to do this. Much obliged.