Since I discovered it a year ago, I only use Poetry as package manager for my python projects. It’s amazingly simple, beautiful, and easy and slick as you wish pip would be. It’s so well designed it brings a tear to my eye
The cool thing is for each project you create with Poetry it creates a local environment which you can easily commit to git update/change/send to a client. By default Poetry uses a venv it fires up for each project, but that will not work with TD and we also don’t need the venv feature, we just need Poetry to manage packages and that TD knows about them being installed locally.
I currently already use poetry with TD projects, but it requires some hacks for each DAT to be able to import these locally installed packages which makes it all a bit cumbersome to use.
My dream is that TD would ship with poetry included, and that I for instance would only have to type poetry add pycuda in the textport to include the pycuda python package with my current .toe project.
That would mean from then on I can type
include pycuda
in all my TouchDesigner DATs and it would just work. (just like the Requests library that ships with TD)
Also that would mean a new directory Lib/site-packages would be created next to my .toe file where in this example the pycuda package (and its dependencies) will be installed to (so they are not system-wide installed but per project instead).
poetry does all sorts of amazing things in the background, like create project.toml and .lock files which you can commit to Git so it’s guaranteed any client / coworker opening a .toe project 6 months from now would get those exact same package versions I specified when delivering the project.
thanks for reading all this and try to take look at poetry if you have some spare time !
cheers idz
I have a working prototype of a poetry COMP. It creates a local lib/site-packages and installs all your (resolved) python and tox dependencies . And you can create python packages holding tox files, publish those to pypi … works nicely, but I definitely prefer a native solution and therefore fully support that RFE.
My PIP-Implementation (windows only) also creates a library-folder in the project directory and adds the refferences on startup. So, in theory you can also just put the libs in the Lib folder of the project.
Also, the Import_Library method will come in handy as it will download the module from PIP and install if it does not exist and returns the module afterwards (or directly if it already exists)
Nothing against the RFE, def nice to have tight integration.
imho this should not just use poetry for managing python packages, but to also handle tox files as python packages. This way you automatically get proper version and dependency management for tox files. And you can use pypi or any custom hosted python repository as your tox repo
Another interesting opportunity that would arise if using poetry as a built in package manager is to use the “necessary” pyproject.toml to configure the toe file. Stuff like project.paths or any custom variables could all be group in that one
Another feature: any project-specific packages which one defines using Poetry should always take preference over any system-wide installed package, or a package shipped with TD. This means it would be trivial for a user to override built-in NumPy with a newer/older version using Poetry. This is just a matter of ordering the include paths I think.
Alas, quite a few things got put ahead of this. I did see a lot of talk today about an issue you’re having with packages. Didn’t get all the details but the team is working on a solution for that!
Any push to reopen this line of thinking? Package management seems to be the biggest obstacle for effectively sharing and maintaining complex systems, and some sort of dependency management system is becoming essential. It sounds like there would be a lot of buyin for a standard like Poetry, is there anyone I can bribe to bump the priority?
It seems like Poetry is too limited but we are doing some work to figure out a solution to this. The coming experimental build will include some externalization changes which will help in this department.
