@hardworkparty and @monty_python - sorry for the delay, we’ve been doing a little internal re-org for our practice and I wanted to share something that was closer to a better reflection of our current practice.
Disclaimers
- This doesn’t address stubs from TD, and instead is a focus on how to better auto complete the code you author as extensions.
- This doesn’t work for all projects, and doesn’t work for all developers.
- This does require a little bit of Python and TD wrangling to get this all behaving, but has been worth it to us.
- This isn’t as portable as would be ideal… in a perfect world it would be easier to pluck one of these subcomponents up and drop them into another project. In our case after a lot of internal discussion we settled on the reality of working with a base project structure that needed better auto-complete support as our primary focus.
A sample project file can be found here:
Extension Auto-complete
Detailed write-up
Coming soon
Overview
Our primary objective here was to help solve our big challenge of Python interfaces authored by two different devs that needed better auto-complete support. This is pretty straightforward in a pure python context but is more difficult a Python class belongs to a TouchDesigner op.
Our biggest changeover was to find a happy solution between the behavior of local/modules and how an editor like vsCode understands Python libraries. Our team largely uses vsCode but in theory this same idea should work in an editor that has similar mechanics.
Python Type Hinting
The real magic for us really came from Python’s type hinting. If you’re unfamiliar, the structure of this looks something like:
def foo(val1: int, val2: int) -> int:
return val1 + val2
A type follows a colon after the argument. This works for any default type, but is also valid for any of your own objects. For example:
class TDFoo:
def __init__(self):
self.name = "Derivative"
pass
def TDTest(my_object:TDFoo) -> str:
return my_object.name
Here we can create a custom class, and use that class as a type hint for a function. That’s all well and good, but why that’s really helpful is because your code editor can use that hint to autocomplete for you:
Gotchas
The tricky part of this is that both your code editor and TouchDesigner need a way to resolve your Python use. In vsCode that’s not too wild, but in TouchDesigner we need to be a little more creative. For the sake of simplicity, we’ve moved all of our class objects to local/modules we’ve done this in part since it’s easy to access Python modules in TouchDesigner if they live in this path with the pattern mod.YourDATName.YourFuncOrClass. This separates your extensions from the Ops that use them… which is not ideal, but for a sufficiently complex Python implementation it’s been worth it.
vsCode Set-up
In a code editor this doesn’t look too unfamiliar. We tend to treat operators that do big work in our projects as singletons - we might have an op called DATA that is the Data store for our project, and we only have one of them. In these cases we tend to treat the Python for that op as it’s own library. In that case we create a directory for that library and include an __init__.py file to create a single instance of the class we’ll be using as a singleton:
Where the magic happens is in the use of another Python library we typically call lookup:
lookup allows us to create a variable that points to an op with a global op shortcut, but is hinted as the class object used for its extension.
Rather than using an op.someGlobalOpName pattern to access these ops we instead locate that operator through our lookup module. Which means we now get auto-complete for any of the methods that belong to that object:
TouchDesigner Implementation
To make this magic work in TouchDesigner, we have to do a little op organization in local/modules. Here we can add each of the python files that contain our class objects (skipping the __init__.py file which is really our helper for our code editor). The only exception is that we need to pull in the __init__.py file for lookup, but put it in a DAT we name lookup. TouchDesigner will allow us to import a DAT by name - so while our code editor will resolve the directory called lookup → lookup/__init__.py, we have to help TouchDesigner complete that same idea.
If we look at these side-by-side you can see how this might look in vsCode and in TouchDesigner:
Takeaways
We tried several different permutations of this before landed on this particular implementation… and I can’t say that we wont change in the next year if we find something easier. That said, it’s been a huge benefit to have code completion between extensions for us. So far, despite being a little cumbersome to set-up, it’s been worth a little bit of juggling.
Also a huge shout out to @ishelanskey as he’s driven a lot of our exploration and kept the fires burning for better code completion in our projects.
@Ivan - I’m also going to tag you in here. I know you’re always interested in how the community is making python work in TouchDesigner.
)
