I’m having an issue with the Engine COMP where it reports that it cannot find an external Python Lib. My setup:
Setup a venv of Python 3.9.5 to match the version in Touch 2022.35280 Commercial
pip install -t ./libs [library name] - To store the library locally to the .toe file
TouchDesigner 64bit python path set to ./libs, relative to the .toe path.
Container COMP - Text DAT - from google.oauth2 import service_account to import the lib – Confirm this works in the Container COMP. I also set up an In CHOP and a ChopExec DAT to handle an external button press to init the Text DAT. Add an Error DAT and Out to the Container
Export the Container COMP to a .tox file.
Load the .tox file into an Engine COMP, rig up the button to init. The tox is stored in the same location as the .toe file, so the ./libs pathing should be the same.
Disable the source Container COMP
Press the Init, receive an error through the Out DAT
Traceback (most recent call last): File "/container1/chopexec1", line 2, in onOffToOn td.tdError: File "/container1/loadLibs", line 1 r = previousimport(*args, **kw) ModuleNotFoundError: No module named 'google' Results of run operation resulted in exception.
Is there a trick to referencing external Python libs from an Engine COMP?
Can you set the path as absolute rather than relative?
I just gave the setup a test in both python 3.9.5 (latest stable) and Python 3.11 (latest experimental) and I could get it to work with a simple test case where I override the Numpy version.
In some cases with complex libraries, what I noticed is that adding site-packages to the path can possibly be not enough. If this is still failing, you might want to add to your engine path using an Execute DAT onCreate. Take inspiration from the Anaconda article for the script: Anaconda - Managing Python Environments and 3rd-Party Libraries in TouchDesigner
I traced the os path from both the container and engine and both report the same directory. Also tried changing to the Experimental version to see if that was the issue. (no change) C:\Program Files\Derivative\TouchDesigner.2023.10130\bin\Lib\ (Both the .toe and Engine tox hit the same directory)
import os
import inspect
op('libPathOutput').text = inspect.getfile(os)
JetXS, I followed your advice and went through the Conda approach. I have a td environment set up and have installed the google-cloud-aiplatform. After calling the Execute CHOP based on the Conda instructions, in the Touch Textport I can import aiplatform. (from google.cloud import aiplatform)
For the Tox file, I copied the Execute CHOP code and placed it into an Operator inside the tox file. The hope here is that when the Engine Tox initiates it sets up the Conda environment. I still receive the same error. No module named 'google'.
I’ll have to give a try by upgrading numpy just as a sanity check.
Really thinking about copying the libs into the Touch python directory at this point. Still not working but thanks for your help!
Yes, google-cloud-aiplatform is the main package I’m using. I had the LLM working in the .toe file, but it needed to be threaded to make the system run smoothly. Looking to use the engine to better manage the params I’m feeding into the system.
The only issue I’m seeing here is the user and condaEnv variables. If I were to share these in a repo, I’d have to change these based on the development system. Will have to see if ~ will work to find the local directory.
Hi @bmarshall and @JetXS ,
Sorry to bother you but I’ve been having the same issue, with the module scikit-learn, and the solution from @JetXS doesn’t seem to work.
I want to run a K-means color segmentation script inside of an engine comp, using scikit module.
The script works fine in a container, but not in an engine comp.
I’m using the conda method, there are execute DATs both in the main project and inside the .tox file.
The former adds the conda paths onStart() and the latter onCreate(), but I still get the error
There is a bug currently with some errors sticking to COMPs when they are not in an erroring state anymore, so this might be misleading.
The issue however in what you shared is just that the Create toggle of the Execute DAT is not turned on.
Additionally. remove whatever you have in the callbacks of the Engine COMP.
Those callbacks aren’t affecting the Engine COMP itself, they are being executed in the context of the host / main process, when the Engine Init / Start… etc.
With those changes, it should all run as expected. I have things running properly on my end.
Oh thanks @JetXS !
I had enabled the create toggle but somehow the engine wasn’t using the updated .tox after hitting “reload”. I had to delete and recreate (with ctrl-Z) the engine for the newest .tox to be used.
Strange but now it works, thanks!
Still not working but thanks for your help!
