Render top excessive GPU cook time [ win 11 / 2021.39010 ]

if I make a simple default render network in the latest experimental, the gpu cooktime reads at:
3-4 ms, very high. I know it’s early, not sure if this is a known issue yet or not, so figure I’d post:

EXPERIMENTAL:

as you can see, it is drastically lower in the stable at ~0.05

STABLE:

default pbr scene.1.toe (5.4 KB)

Seems like a measurement issue, if you drop down more the GPU times for them go down.

sorry, what do you mean by drop down?

I mean just create more copies of the Render TOP. The time for each isn’t 5ms, so it’s more that the timing isn’t finding it’s end-point accurately since the GPU is so under-used.

I’ve noticed that GPU compute times are waaaay down according to Windows Task Manager for 30k experiemental series. It’s amazing comparing identical networks between stable (30%) and experimental (3%) @malcolm. Obviously results will vary but I’m seeing profound differences on most of my projects and components.

2 Likes

Awesome! Yeah I was specifically looking to see how much cpu overhead has dropped on the render top. Great to hear it’s just a measurement bug. Seems like for that default network I uploaded roughly 3x improvement in experimental from the stable.

@dylanroscover neat about compute speeds, was this expected though @malcolm ? I thought vulkan was more about optimizing cpu side and gpu shader speeds would be more or less comparable?

Somewhat yeah. It’s possible the GPU % measurement is counting stalls, which may be reduced with Vulkan

Hey there, just wanted to check up on this issue as I am experiencing this in the latest release. It sounds like the large cook times are inaccurate? I am seeing high gpu cook times (5-6ms) with super simple render TOPs. I confirmed that if I clone the render TOP the cook times in the cloned one are way lower (<1ms). Even when I make the original render TOP be 64x64px the cook time in the middle mouse click overlay are still 5-6ms.

Is this being considered as a bug? It’s making tracking down long render causes a bit impossible. Would love some insight or work arounds. Thanks!!!

Is your project dropping frames at this point? In general to get good measurements you need to be running at full load on the GPU, so you arn’t counting bubbles and stalls in the pipeline.

I am getting sporadic hiccups mostly with moviefilein TOPs, so unrelated to the related to the render TOPs I’m seeing these cooktimes with.

I have a pretty large project that has a lot of compartmentalized parts (for VJing), and it’s typically putting CPU and GPU at high load. I have things in place to monitor how much load all of my VJ scenes have. With the new version when I go to view gpu cook times render TOPs are showing as the culprits but upon further inspection their cook times appear to be inaccurate as I mentioned in my comment from yesterday.

I could try and make an isolated example if that would be helpful. Right now it’s a bit hard to discern since my project is quite big. Thanks!!

Hey there, I wanted to attach some example files that hopefully show the issue.

I made a simple network that has a moviefilein TOP. I then have a base comp that’s trailing the cpu and gpu cook times. I also through in a large noise TOP to try and simulate GPU load.

16960 build:
moviefilein-16960-load.toe (4.7 KB)

26590 build:
moviefilein-26590-load.toe (4.9 KB)

In the 16960 build I’m seeing cpu cook times of around 0.4ms and gpu of 0.3ms.
In the 26590 build I’m seeing cpu cook times of around 0.1ms and gpu of 1.5ms.

Even though 1.5ms isn’t that long I am seeing much longer times in the wild closer to 5ms. Let me know what else I may be able to do to debug this. In my larger projects which are loading in multiple videos these cook times are bringing things to a halt. Thank you!!