Some TDPyEnvManager thoughts & RFE's

General TouchDesigner Discussion
1

I’m testing to integrate TDPyEnvmanager into our projects.
(Caveat: I think I read all documentation, but it could very well be I missed something)

First: I fully endorse & applaud this project, as we’ve been managing various self-built solutions over many years, and a single streamlined approach for all would be very welcome.

Some thoughts:

  • default venv name
    I would recommend to make the default venv name something short and the same for each project, because the venv often needs to be started from console (usually on projects we have standalone python apps running alongside TD). And when I’m working in project “MyProjectName” there is no need to repeat that projectname for any folders inside that project.
    So I have an RFE to make the default venv name .venv instead of MyProjectName_vEnv
    (also .venv is the same default name as uv uses, which would help while moving between different projects during the day)

  • pyproject.toml
    since PEP 518(2016), 517(2017) and specially PEP 621 (2020) the standard on which we are converging is a singlepyproject.tomlfile which replaces many different files (like setup.py, requirements.txt, pytest.ini, .pytest-version)
    So why not generate a (or add to an existing) pyproject.toml file by default, instead of adding another new specs file (TDPyEnvManagerContext.yaml) to our projects.
    I would strongly prefer a single file as the single source of truth for all my TD/python project specs.
    Also currently when I have all the TDPyEnvmanager settings already in my pyproject.toml under [tool.touchdesigner.TDPyEnvManagerContext] , TD will still always auto-create a new TDPyEnvManagerContext.yaml file on TD startup, which seems unnecessary and is confusing - because which file is the source of truth now?

  • uv
    We, and a very large part of the Python community (projects like FastAPI, Pydantic, Hugging Face, Jupyter, Rye, Polars) have moved to uv since a long time, as the speed difference of the uv package solver / installer / venv manager compared to pip and others is just not funny anymore. And even though uv was started by commercial company Astral, it’s a fully open source project(currently more than 500 contributors) with Apache2 and MIT licenses, so uv will not go away, no matter what its founding company does. It would be very much appreciated if uv becomes a firstclass bundled member of TDPyEnvmanager/TouchDesigner.

  • auto creation of venv on start
    (this is linked to the pyproject.toml / uv RFE’s above).
    The auto creation & initialization of a preconfigured venv on TD project start is an awesome idea.
    Implementation would be really streamlined if it could use our project requirements in pyproject.toml (at least I could not manage making this work - it seems this only works when adding a separate requirements.txt file). If this worked we would only have to maintain one file (single source of truth for da win!). And then if TD could use uv to auto create & populate the venv it would be a lot faster to startup as well.

last nitpick: As we are already in TD Python land, would it not be easier to talk about this software if we called it simply VenvManager instead of TDPyEnvmanager :wink:

Thanks! Idzard

3 Likes
2

Hi @nettoyeur

Thanks for the thoughtful feedback and for taking the time to test the tool. Really appreciate it.

Which version of TouchDesigner are you using?

.venv is actually the default name starting with 2025.32280, so that should already align with the convention used by tools like uv.

The .toml / .yaml duplication you mention was actually a bug, which has been fixed in the upcoming 2025.30k+ release (coming very soon):

Regarding pyproject.toml vs requirements.txt, they unfortunately serve somewhat different purposes.

requirements.txt is often used as a resolved environment snapshot (commonly generated via pip freeze), whereas pyproject.toml is a much more flexible specification. It can include dependency groups, optional dependencies, dev dependencies, build-system requirements, etc.

Because of that flexibility, it becomes tricky to determine which subset of dependencies should be installed automatically in all situations. For example, a project may include a pyproject.toml for development tooling, but a runtime environment may only want the pinned dependencies from requirements.txt.

We try to avoid making assumptions on behalf of users here, since automatically installing everything declared in a pyproject.toml could unintentionally bloat the .venv. That said, this is an area we’re still exploring and refining.

To add to the mix, with something like the following:

[project]
dependencies = ["numpy"]

[project.optional-dependencies]
dev = ["pytest", "black"]

[tool.poetry.group.docs]
dependencies = ["mkdocs"]

Each of these tools may resolve and install a different environment:

  • poetry install
  • uv sync
  • pdm install

Our own interpretation in the TouchDesigner context might again be different.

TDPyEnvManager originally grew out of some existing internal tooling and workflows (including the conda article that circulated a while back). Basic venv support was added early because it is part of the standard Python library, which guarantees it is available in every TouchDesigner build.

Supporting uv would require a fair amount of additional work and infrastructure, so it hasn’t been implemented yet. It’s something that may be considered in the future.

Historically we’ve seen similar requests around tools like Poetry, whose key ideas were later incorporated into the broader Python packaging standards. In general we try to avoid tightly coupling TD to specific ecosystem tools like these, which while open source, are also developed and maintained in a different language and tech stack.

That said, the architecture is currently being reworked to be backend-agnostic, which should make it easier to support alternative environment backends in the future.

I think those are related to above points about .toml use.

Noted.

Thanks again for the feedback. Glad you find those tools useful.

2 Likes
3

Regarding the requirements.txt, it might be worth to lok at PEP 751 which got accepted and describes a pylock.toml which allows for a much clearer description of the environment. It seems that it is already possible to create a pylock.toml using pip lock but no clear indication of being able to pass the lockfile to pip install
A big upside for deyploments using uv is that we can run uv sync to ensure that all installed dependencies are not just fulfilling the requirements described in the pyproject.toml but are pinned to the exact version/hashed and sources.

So I suppose this would go much more in the direction of shifting away from requirements.txt and use pylock.toml instead as the definition of the env.
(uv simply takes care of this already without having to think much about it. Running uv add makes sure that the pyproject.toml and uv.lock are set up and up to date, reducing friction by a lot.)

2 Likes
4

uv sync would not guarantee pinning if it’s not strictly using the lockfile, would it?

This seems to be discussed here: Add --frozen flag to "sync-and-running" documentation · Issue #10793 · astral-sh/uv · GitHub
and here: Have `uv sync` default to `--locked` · Issue #12372 · astral-sh/uv · GitHub

We’re focusing on the development experience here which is that uv sync and uv run here should “just do what they’re supposed to” not throw an error if you have modified your pyproject.toml. This is the same thing that Cargo does. In contrast, the deployment experience is more of a “write it once” thing and it’s reasonable to opt-in to additional safeguards.

I think those two workflows serve slightly different purposes.

We will certainly support as much of pyproject.toml as possible moving forward. But when it comes to matching something like requirements.txt together with the idea of a lockfile, the ecosystem is still evolving. Efforts like PEP 751 are moving in that direction, but for now we try to stay close to what the standard Python tooling supports out of the box.

It’s also about finding the right balance between enabling useful features and avoiding unnecessary complexity or technical debt. The list of flags and workflow variations around uv sync alone is already non-trivial to cover nicely within the TouchDesigner context.

1 Like