@ben thanks for info. Unfortunately project.paths doesn’t help much in this area… I mean it looks like a nice concept, but I am primarily looking for a way to automatically configure component paths from environment variables. Configuring them manually with such prefix (and then providing a json for every single file) is an option to a certain extent, but it doesn’t allow for the level of flexibility TD needs in larger pipelines.
So far I have been trying to get this kind of workflow in TD:
Lets say I am developing various td modules (lets call them packages) in their respective git repositories. These are then being deployed (either using CI or manually) to one network location available for all users / servers / render nodes. Its structure could be as simple as this (names are made up just to illustrate this workflow):
/packages
├── td_utils
├── 1.4.0
├── 1.4.1
├── td_rocks
├── 1.0.0
├── 1.1.0
├── td_trees
├── 3.0.2
├── 3.1.0
├── td_forest
├── 2.1.0 (depends on td_rocks-1.0.0 & td_trees-3.0.2)
├── 2.2.0 (depends on td_rocks-1.1.0 & td_trees-3.1.0)
Each version contains stuff necessary for given package to work - such as tox files, scrips, shaders, assets, unit tests, etc. One could then handle whole pipeline just by using environment variable TD_PATH
, that would contain paths meant to be searched for toxes. This setup makes it easy for a project to define package dependencies (with specific versions being targeted).
For example project foo would define its dependencies:
td_utils-1.4+
td_forest-2.1.0
To start this project, only TD_PATH
environment variable has to be configured. Depending on complexity, this could be either handled manually in some bat script, or with something like Rez in more complex environments.
TD_PATH=/packages/td_utils/1.4.1;/packages/td_forest/2.1.0;/packages/td_rocks/1.0.0;/packages/td_trees/3.0.2
Once TD starts, it looks for toxes within these paths in TD_PATH
, and if it finds some, it updates it local library of toxes. Its library could then look like this:
/packages/td_utils/1.4.1/bottom_bar.tox
/packages/td_utils/1.4.1/buttons.tox
/packages/td_utils/1.4.1/ui_panels.tox
/packages/td_forest/2.1.0/forest.tox
/packages/td_rocks/1.0.0/rock.tox
/packages/td_trees/3.0.2/tree.tox
Then it goes through all the components in the scene and loads components with matching toxes (available within this library) from disk. This way correct versions of utils and forest tox is loaded (also correct versions of rock and tree toxes are loaded {recursively} within the forest component).
In case we decide to change version of some package, it is as easy as changing dependency specification. TD_PATH
environment variable is generated automatically using Rez - thanks to its dependency resolution. It is worth mentioning that Rez can also handle python package dependencies. For example if td_utils would require tensorflow-2.10.0
, Rez would take care of getting it from PyPI and configuring proper environment for TD.
For simpler use-cases (where Rez isn’t necessary) we can just manually change bat script used for setting up our environment.
Long story short - as long as we have TD_PATH
configured, we can start any scene or any tox without having to worry about toxes not being correctly loaded.
I guess there are many ways to approach actual implementation on TD side, but I feel like there is a need to support this kind of workflow as it allows to integrate TD in larger pipelines. Right now I am doing this with some custom component that takes care of building the library of toxes based on TD_PATH
and then it goes through components in the scene, loading correct versions of toxes. However many times I stumble upon some issues that are quite hard to overcome. It would be great if this sort of workflow would be enabled by default in TD. It could also later serve in many other ways - for example such library could make it possible to extend OP create dialog by custom components. That way could user add some path containing custom components to TD_PATH
and they will be right away available in OP create dialog. 
Please do you think some Pro support hours could help in this case? I would be glad to use mine for it, but I guess it might take much more time that I have
Anyway thanks for reading through this.