Package managment helper Python

Wishlist and RFEs
1

I’ve done a few interesting python gymnastics to make it easier to work with external libraries, especially for non python experts. The particular idea I’m currently chasing is using the python version distributed with TD to determine which version of a library to install in a directory that’s at a known location from the project file.

The idea here is that you then essentially end up with set of dependencies that can travel with your project rather than making changes to the TD install, or maintaining your own python installation that matches TD’s.

What I’ve been playing with is using a suprocess call to specify which python to use, and where to install the requested package.

Big gotchas so far are that TD is missing some essential pieces to make this easier:

  • pip
  • setuptools
  • pkg-resources

There are a few pieces above that I think I can work out, but it will for sure break when I install a new version of TD.

To that end, something that would be beyond helpful here would be a helper in the project class. That might look like:

def importExternalLibrary(library_string_name, destination_folder):
    # does a pip install of the requested library
    # in the specified directory
    # adds specified directory to project Paths

def importExternalReqs(path_to_requirements_txt, destination_folder):
    # does a requirements install with the path to the requirements txt
    # in the specified directory
    # adds specified directory to project Paths

I know this one isn’t as exciting as some of the other REF’s, but boy golly would it be helpful :slight_smile:

13 Likes
2

+1 20char20char20char20char20char

1 Like
3

Totaly in, but for now, did you take a look into my PIP Component? For one it enables PIP-Control (install, uninstall) but second it also holds the PIP library in the component and installs it into the local python library. I think it should be rather easy to extend to instead export in a project folder and add the path to the project folder to the search path.

td_pip.tox (1.4 MB)

4

Thanks @alphamoonbase - I’ll give this a look. My first attempt gives me a warning that I need to restart TD and run as admin, which is another barrier that I’ve been trying to avoid.

I’m curious if Derivative has some plans here - these kinds of handles would really be a great help IMO.

5

+1

pip install twentycharacters
import twentycharacters as thread_bump

1 Like
6

+1
Bad at python but would like to be able to use more python

1 Like
7

+1 from me as well. This would really help

8

Thats in fact based on the new installer. It will write protect the install folder for some reasson. The old installer does not give that error. But I will see if I can implement it in a way that it is project dependant. Should not be to hard (last famous words)

9

+1
also curious if there’s a public list / RFE list for libraries that TouchDesigner is shipping along with Python installed by default. I think it was great when requests started shipping by default in TouchDesigner (I’m pretty sure it doesn’t ship with default Python? I could be wrong though, I don’t follow the Python news as much these days). Numpy was also a good add. I could imagine a few other libraries that might be worth considering default addition like pillow or similar.

1 Like
10

+1, would love to have something like this built in. Super useful!

11

+1 :slight_smile:

I’ve done some hackity hacks to get pip packages into Touch but I don’t feel great about commiting the source for a given version of package inside my project repo, or using a combination .gitignore and shell scripts to install the packages as needed.

12

I took your work into consideration and reworked everything a little bit.
The component now creates a LIB Folder in the Project folder and installs PIP and everything coming from PIP directly into that folder (and adds it to the import path for python.) So no more problems with Admin rights.

2 Likes
13

Very nice @alphamoonbase! This works great - I still want this in TD itself, but this is a nice work around.

14

+1 Having an integrated way of installing and managing python libraries would make it much easier to build tools and systems that can be shared with users and interoperate with other software.

15

I get the following error when trying Install a package: :frowning_face:

python >>> 
Traceback (most recent call last):
  File "/td_pip/parexec1", line 19, in onPulse
  File "/td_pip/extPIP", line 24, in InstallPackage
  File "C:\Program Files\Derivative\TouchDesigner\bin\lib\subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:/Program Files/Derivative/TouchDesigner/bin/python.exe', '-m', 'pip', 'install', 'qrcode', '--target', 'D:/TI_Sandbox/TouchDesigner/Lib']' returned non-zero exit status 1.
python >>>
16

Hmmm, hard to tell.
I switched the link to the file to a link to the olib with the latest version. If you did not already, maybe try this one instead:

Also, which OS and TD Version are you using?

17

Thanks for the response!

I’m running TD 2021.16410 on Win 10.

The updated version didn’t return the same error message, but I get this when I try and install a package:

TD_PIP
TD_PIP1176×576 27.7 KB

18

You can try two things:

  1. Delete the Lib folder and restart the touch project. Newer versions of tdpip install setuptools directly with the init. But I assume you still had the lib folder in your project, so it assumed pip with initialised correctly.
  2. Install the setuptools package, but I suggest going with the first way!