[2023.11340] No "GPU" parameter is on Nvidia Upscaler TOP?

Hi,

I am having trouble with Nvidia Upscaler TOP.
My Touchdesigner is always stuck when I connect to Nvidia Upscaler TOP (I have to force shut down TD.)

I believe that my environment causes this trouble: 2 GPUs (1 is on eGPU), and the trouble happens in the very particular case below.

My environment:
Win11
CPU: Intel Core i7
GPU: GeForce 3080 Laptop, GeForce 3090Ti (on eGPU)
TD: build 202311340
Drivers: The latest version, 546.33

Cases:

  1. Disconnect eGPU (only 3080 Laptop): Work.
  2. Connect eGPU (TD’s graphic: Windows Decide): Work.
  3. Connect eGPU (TD’s graphic: 3080 Laptop): STUCK!
  4. Connect eGPU (TD’s graphic: 3090Ti): Work.

I run TD on my 3080 Laptop and use 3090Ti for Stable Diffusion simultaneously. (As my observation, it is the fastest… )

I guess this trouble can be solved if I can select the GPU (3090Ti) that the Nvidia Upscaler TOP works in. Is there any plan to add the GPU parameter on Nvidia Upscaler TOP?
(I found the parameter GPU on the documentation page, but TD 202311340 does not have it. Why…)

Hi @Peachman

I have logged an issue regarding the GPU selection parameter being missing, for a developer to look into it and get back to you.

Best,
Michel

1 Like

The GPU index feature was actually removed from the final 2023 release, but was accidentally left in the documentation from an earlier test case. I’ve removed the reference now.

The reason it was removed was because we reworked all of the Nvidia Maxine-based nodes to use direct GPU memory transfer rather than go through the CPU as it did before. This is much faster, but it means that the GPU used for Upscaling can only be the one used for the rest of the network. It’s not possible (or efficient), to transfer the image from the primary GPU to a secondary one for the Upscaling, and then back to the primary GPU.

With that said, we’d still like to look into why it is getting stuck in that one configuration. You describe it as the eGPU is connected, but TD is running on the laptop’s 3080. Are you using the GPU affinity options to target the 3080, or something else?

If you use the task manager, you can create a dump file for a stuck process. If you zip that and send it to support@derivative.ca we might be able to see where it is stuck. I suspect, it’s a Vulkan dependency loop, but it would be good to confirm that.

2 Likes

Hi @JetXS @robmc,

Thank you very much for your prompt response.

This one,

Are you using the GPU affinity options to target the 3080, or something else?

I assigned my 3080 laptop to my TD this way.
System > Display > Graphics > Touchdesigner > Options > 3080 Laptop

I emailed support@derivative.ca, but the dump file is huge, so I uploaded the dump file to my Google Drive and attached the drive link…

The email subject is
“[Dump File Shared][2023.11340] No “GPU” parameter is on Nvidia Upscaler TOP?”

Best,

I’m not actually sure if that setting is used by us (Vulkan) for GPU selection. Have you tried starting with -gpuformonitor as a command line option to specifically bind the process to a GPU?

1 Like

@malcolm @JetXS @robmc

Thank you very much.
I didn’t know TD has the function GPU Affinity and Monitor DAT. That’s interesting.
Anyway, this is information on my Monitor DAT.

Does this mean this?
-gpuformonitor 0 - 3090Ti (on eGPU)
-gpuformonitor 1 - 3080Laptop

And, I don’t know why, my way, the Affinity certainly works.

I assigned my 3080 laptop to my TD this way.
System > Display > Graphics > Touchdesigner > Options > 3080 Laptop

However, using the bat file,

set APPPATH=C:\Program Files\Derivative\TouchDesigner.2023.11340\bin
 start "%APPPATH%" "%APPPATH%\TouchDesigner" -gpuformonitor 0 TDAffinityCheck_3090ti.toe
 start "%APPPATH%" "%APPPATH%\TouchDesigner" -gpuformonitor 1 TDAffinityCheck_3080laptop.toe

2 toe files are opened, but the Affinity both does not change… (indeed, the TD.exe file is located in TouchDesigner.2023.11340. And I turned off my way above, back to “Windows Decides.”)

I just noticed that not only does the Affinity not change, but the gpu_name is both 3090Ti. Why…

My setting is;

LaptopPC (Monitor1) [3080Laptop] =(ThunderBolt)=> eGPU [3090Ti] =(HDMI)=> Monitor2

TD seems only to use 3090Ti on my Task Manager.
I don’t know, but this might be normal behavior if connecting to eGPU…?

image

Hmm ya, the driver may be lying to us based on the settings you are choosing to try and trick us into picking the GPU you are selecting using the driver settings.
We don’t have an eGPU here to test this unfortunately.

@malcolm

Yup, eGPU makes a mystery sometimes…
Thanks, anyway, the GPU affinity seems to work (the way is not TD genuine, though…)

  1. TD [GPU => 3080Laptop]
  2. Stable Diffusion [GPU => 3090Ti]
    But Nvidia Upscaler TOP causes the error on TD.

Maybe I should try to upscale on the Stable Diffusion side for now…