Optimization Questions

General TouchDesigner Discussion
1

My apologies for the long post. Hopefully this will help some others having similar issues.
I’m working on optimizing a theatre performance network that has lots of individual scenes/cues that are selected and sent to a mapping/blending network then out to 7 projectors. I’m using only live video feeds and movie in objects, no 3D rendering. It seems like the more I work on the network, the lower my frame rate goes and I’m desperately trying to get some of those frames back.

I’m also trying to get a better understanding of how best to optimize TD in general.

I am running windows 7 64bit, 16GB RAM, SSD, TD v26000. , Nvidia Quadro K5000, driver v341.21. black magic capture card.

These are some of the things I’ve discovered from searching the forums.

selective cooking
derivative.ca/Forum/viewtopic.php?f=4&t=599

multi composite/mover (seems like a good replacement for multiple OVER for transposing many items on a canvas)
derivative.ca/Forum/viewtopi … =22&t=5537

Cook time with scripts (how to view cook time of your nodes)
[OPs vs. Script Cook time test)

Base vs Containers: (bases are more efficient than containers)
viewtopic.php?f=4&t=1388&p=4357&hilit=optimize#p4357

Turning on and off the cooking flag: (not the best solution for optimization because it may cause pause when reenabled)
viewtopic.php?f=27&t=1840&p=5933&hilit=optimize#p5933

Saving GPU Ram: (can be achieved by reducing resolution of TOPS to 1 x 1 and then cooking)
viewtopic.php?f=20&t=1601&p=25641&hilit=1x1&sid=571c6845e57b9c326749fd76e3bccdc6#p25641

Panel drawing and performance: (not super important because perform mode takes care of it)
viewtopic.php?f=4&t=5383&p=19810&hilit=optimize#p19810

I’ve also checked out all the related wiki documents on optimization and cooking.

When I put a HOG CHOP in I only see a 1-3 frame drop so I assume I’m having more of GPU bottleneck than a CPU issue.

Some questions/concepts I still don’t understand:

-When I cause a cook to occur, it adds to the total GPU memory used (middle click → ‘GPU memory all tops’ and’ total GPU memory’), and stays there even if it is no longer cooking/being displayed. Does this mean that those GPU resources are still ‘in use’ and not available for other ‘active’ nodes to use?

-What happens when this total GPU memory reaches/exceeds the GPU memory of my graphics card? Is it released in a ‘first in first out’ kind of way? (I’m thinking not because the ‘total GPU mem’ seems to be able to exceed the limit i.e. 1500 MB of 1000 MB)

  • As a general practice, should I be actively reducing the resolution to 1 x 1 of all my TOPs after I cook them and don’t need to display them anymore to ensure that they are not using up GPU resources? (I tried doing this with some of my TOPs, expecting to see some increase in ‘total GPU memory’ or frame rate, but it didn’t seem to have any effect)

  • Maybe I’m putting too much thought into the GPU memory used. Is ‘total GPU memory’ even a good way of determining how hard the GPU is working?

  • Bypassing a container does not seem to stop the objects inside of it from cooking but disabling cooking on them does. One of the posts above says this isn’t a good way to optimize. Is this still true?

-According to the hierarchy list within my performance monitor I can see that certainCHOPS are causing others to cook. Specifically I see that exporting a null channel to a play parameter (on/off) causes the null (and everything upstream of it) to cook constantly. Why?

  • If my computer system CPU monitor says everything is fine while running TD (i.e my ram and processor are well below 50% used) would higher cook time chop objects (as seen in the performance monitor) be having any effect on my frame rate?

-Regarding TD Frame rate: All my cameras are capturing at 30fps and I have no video content that is higher than 30fps and the refresh rate of my projectors is 60hz. Does running TD at 60fps affect playback performance? Should I be running TD at 30fps instaed of the default 60fps?

-Maybe this is more of a Nvidia question, but if the specs on my graphics card say the DVI output can handle ‘Up to 330M Pixels/sec: (ex 2560x1600@60Hz)’ and I am running (3 x 1024x768 = 3072 x 768) am I exceeding the maximum horizontal resolution of the card’s output and thus causing a frame rate drop in TD?

Sorry again for all the questions. I’ve been trying like crazy to figure this stuff out for the last week and have run into a wall.

Thanks for any advice you may have.

3 Likes
2

I think these are some great questions, I am also dealing with many similar frame rate losses and would love to see some of this clarified.

3

This is great.

I am in the same boat.

Replies would be great

4

Sounds like you’re running into a pixel shader bottleneck probably from doing a lot of things like this. If you can, definitely use the multi composite, GLSL is very fast, and many Over TOPs can definitely take a toll.

Are you doing a lot of calculations in your scripts? If not I wouldn’t worry too much.

A really handy tool to use is Inspektor, it’s in the tox file area of the forum, follow the instructions with it and you can get a very visual overview of the cook time across your network to find bottlenecks pretty quickly.

Containers render a UI whereas Base COMPs don’t, so if you need a UI use a Container, otherwise I use Base COMPs more often.

In your situation I’m guessing your slow downs are in the Pixel shading stage of the graphics card, so you might be better off doing the below to dump things.

This is how we generally clean out comps when not in use, but before you get here, you might be able to just optimize your network to work how you want it. So do this last, not first.

Doesn’t sound like this is your problem.

I’ll preface this next section by saying GPU memory is a complex thing, so the next bits are just my understanding of it.

Yes, that’s why a lot of people drop the resolution of the whole TOP pipeline to something small like 2x2, so that the old memory gets released.

Lately, reports of crashing. Nothing good in general though. I’m not 100% sure of the FIFO, but I would guess the GPU mem would get fragmented soon enough and that’s a real slow down for it to clean up.

It is pretty general practice on heavy projects, but you should try to do as much as you can before that. You really only need to change the size of the generators at the beginning of the chain, everything else will drop down in resolution once they cook, and yes you should see your GPU mem clear up.

Not how hard it’s working, but how much memory is being used. You could have a single giant texture that takes up all your GPU memory, but if its not cooking, its not “working”, its just hogging all the memory.

Bypass just stops whatever youre bypassing, ie a single op. Cooking wil turn off the nodes inside. Not the best way, probably better to do the TOP method of dropping resolution in this case, but still do everything else first.

Is your Null CHOP always cooking?

Remember that a single TouchDesigner process is on a single CPU core except for specific things that have been programmed to be multi-threaded like movie loading.

Depends what you want. It doesnt effect performance, it dictates it, in that you have 16ms to do everything in your project, and if you go over, you drop a frame. If you dont need 60fps for your actual processing or animations or anything, you can switch to 30fps, which gives you double the amount of time per frame (33ms), then set your window COMP “FPS is Half Monitor Rate” switch on, and it’ll double the frames on output.

What’s your actual output setup look like? You going into a Triple head or similar?

No prob

1 Like
5

Thanks so much for the reply Elburz (and the countless other informative posts on the forum) ! All of this is very useful.

This is just what I need. I’ll check it out.

Totally makes sense. Thanks for clearing that up.

Yes I think I should do this. Thanks.

Yes I have two tripleheads attached to two DVI ports driving 6 x 1024x768 projectors. Each matrox creates a 3072 x 768 rectangle that I arrange horizontally next to each other in the windows display prefs. I then use a I then drive my monitor and seventh projector from the remaining two display ports. My perform window is 7168x768.

6

So are your outputs working correctly? Shouldn’t really be an issue except maybe you might get some tearing because you have a mixture of triple heads and other outputs coming off the same card. You can read the wiki page on tearing and stuttering to deal with those issues, but they wouldn’t effect framerate, so maybe one step at a time :smiley:

7

If you get really in a jam, post your patch and we’ll take a look.

8

Hi there,
I’ve been playing around quite a bit with optimizing my system and I think I’ve made some good progress.

I created this optimize button that searches though the network it’s inside of and reduces all the resolutions of all the TOPS to 1x1 and then force cooks them, freeing up much needed GPU memory. It also creates a table that stores the resolutions of all the TOPS so that they can be restored when the button is toggled off. I’ve attached it below. *Feel free to laugh at my sloppy python, I’ve only been programming for about 6 months :wink:.

The problem now is that, depending on how many TOPS I am optimizing/deoptimizng, my system occasionally slows down for second while it is cooking each of the TOPS. This is particular noticeable when I am playing a movie while doing the optimization.

As you can see there, I tried something where instead of cooking all the TOPS at once, I actually iterate through, cooking each one individually, with a small frame delay in between.

This still doesn’t help. My FPS is fine otherwise so I’m curious why a few relatively small cooks are causing my system to hiccup?
optimizeButton1.tox (5.01 KB)

9

:sweat_smile: 8 years later…

Just found this. Quite an interesting read.

Is this 1x1 trick still the best practice, or are there now better ways to optimize?

@neonlimbo did you ever figure out how to workaround that last issue with the force cooks?

10

Hey @hekuli,

i would caution on the 1x1 resolution setting - while it frees up memory, it will fragment your GPU memory more and more as well as causing hiccups when all the required memory has to be allocated again.

optimization is a case by case challenge and often has to do with project architecture in general. Certainly a bigger effort.
Disabling cooking and freeing texture memory this way is a last resort solution for me.

cheers
Markus

1 Like