FIXED: [2022.31030 win] Unusual high gpu cooktimes on inTOP

Hi there,

I seem to get very high gpu cooktimes on the first inTOP I use after a renderTOP. First I thought it was just a printing error, but after some testing it really seems to take that amount of gpu time. I’ve also tried it using a selectTOP, but seems to give the same results.

I’ve attached a proof of concept of the problem. Please check the gpu time of the inTOP inside the base1 comp.

Cheers,
tim

ps. system used: windows 11, nvidia 2080ti


gpucooktimes.1.toe (8.6 KB)

Hey @timgerritsen

Thanks for the report.

Unfortunately I am not seeing the same readings here. 0.002. Granted I’m on a 3090 but it seems fine.

Can you test again in build 2022.32050 ? I’m guessing your drivers are up to date ?

Best,
Michel

Thanks michel for checking. Forgot to check my driver version, so gonna check it when i get home. Also will try the newest TD version.
Checked today on a 4090 and had the same issues but also didnt check the drivers.
Will get back to you asap :slight_smile:

I just installed the latest nvidea driver (528.49) and latest touch (2022.32050), still having the same issue. It’s a bit weird since the first moment it looks like its working normal, but as soon as I move to the parent the gpu time shoots to 2ms. Seems like an old version 2021.16410 doesn’t show this problem. Also perform mode seems to be working better (about 0.2ms for the selectTOP, which I think is still too much than it should).
I’ve made a screen capture to show the issue in more detail. (sorry for the low resolution :D)

(was too big to upload here)

Cheers,
tim

gpucooktime_issue.toe (4.1 KB)

1 Like

This is just a bug with the sampling somehow, since these nodes don’t do any GPU operation unless you are causing them to do a resize (which the Select can, but the In can not)

Hi malcolm,

Yes indeed, I doubt the operator itself is doing anything to produce such load, however I had the feeling it did had impact on my framerate. Where I counted the gpu times of different operator and them all together was less than 16ms, while the patch ran only 50fps. But might be that also the times of those other operators are not fully trustable :D.

What would you suggest to take as a good way to test gpu performance? Perhaps external software outside of touch ?

Cheers,
tim

If you’re feeling like doing a DEEP dive at everything TD does in the Vulkan space, give RenderDoc a shot, there’s an option to show command durations in the Event Browser:

1 Like

It’s also possible your bottleneck isn’t the GPU, and is instead the CPU. In general the best way to see if you are GPU bottlenecked is to try increasing the GPU load (high resolution TOPs), and see if that changes the performance. If it does then you are hitting a GPU bottleneck. If not then your bottleneck is somewhere else

1 Like

So I’ve confirmed this is just a bug with how downstream nodes interact with the Render TOP that cooks upstream from it. It makes their timing incorrect. I know how to fix it, but I think the fix will be too large to merge into the 2022.20000 series, so it may need to wait until the 2022.40000 series. Thanks for the report!

2 Likes

Ah, thanks Malcolm! Great to know it’s just the timing!

Cheers,
tim

Confirmed it’ll be fixed in 2022.40000. Thanks!

2 Likes