2020.25380 - Engine COMP frame-rate and performances issues

Bugs
1

Hello folks,

I am not sure it is a bug exactly or if I just don’t understand how things works correctly so feel free to move that post wherever !

I am finally digging a bit more in the Engine COMP.

The first little exercise that I am trying to achieve is to run a MovieFileOut in its own Engine so that recording of short clips doesn’t affect the main process.

On paper it sounds pretty straightforward and easy to achieve.

My first question comes to the type of exchanges between the main process and an engine.

  • I thought that behind the scene, for CHOPs, it was all Shared Memory buffers going from one thread to the other with close to none impact on performances while exchanging data.
  • But, if I look at the performances, there is an average of 1 to 2ms being used by the engine COMP, while the content of the exact same TOX outside of the engine and straight within the main process is using only 0.02ms…
  • If the origin of that difference is not the exchange between processes, I could guess that it is something related, maybe, to having all operators of an engine comp in Force Cook all the time ? I’m just supposing things here. Nothing scientifically accurate.

The second question is related to a lack of resources and documentation around the engine COMP.

Clock:

  • How does being Independent or in Sync can affect performances ? If I use info CHOPs, it seems that when the Engine COMP is set to Independent, it struggles to reach the 60fps, and average at 50fps on my current seetup. With performance monitoring tools, I can tell that my CPU and GPU have no issues and there is plenty of room left for everything to run at 60fps.
  • When setting the Clock to Sync, the Engine COMP is dropping to 30fps while the main process is running at 60fps. I could understand why here, but it would mean that if I ask the Movie File Out within the component to record at 60fps it’s actually not 60fps, even tho the video outputted state that it is a 60.

Wait for render:

  • Wait for Render wait - Wait during cooking rather than dropping late frames from TouchEngine.
  • This is what we can read in the Wiki. It is not quite clear, or pardon my English :wink: Who is actually waiting ? How does it affect potentially negatively the TouchEngine or the Main process ?

So far, what I thought would be a very simple exercise is actually tedious. I think that recording whatever comes from the main process without dropping frames in both the main process and the Engine is achievable, but I didn’t find the correct recipe yet (and I tried a whole lot of different settings and setup) which I think is just coming to a mix of Engine Comp settings, Movie File Out settings and potentially a better understanding of how things work behind the scene (if there is no bug ofc !).

I hope it’s all quite clear and thanks in advance for your time !
Michel

1 Like
2

Thanks for this report, we’ll have to look into a lot of it before replying with answers. Then we’ll distill down the bugs vs the documentation vs the RFEs.

3

Yes, I’m very excited to see development of the engine comp but have been getting quite a lot of bugs and performance issues in my testing so far, but I know it’s early days. :slight_smile:

I hope we’ll eventually be able to go “inside” the engine comp one day to edit the component too rather than re-save the original component etc.

I’ve been seeing slow-downs and freezes whilst running several engines and needing to force quit them in task manager to get performance back. But it could well be that it’s best suited for more simple process at the moment and my components were not ideal for them.

Most of my performance bottlenecks are more in the GPU rather than the CPU. Is there a possibility of an engine type of process for the GPU one day?
So then you could assign certain processes for a secondary GPU for example, this would be great :slight_smile:

4

Thanks for the feedback @Carnarts, please report any specifics here in the bugs forum, anything reproducible we would love to look at.

Have you tried the recent build 2020.26630 with your engine Comp setup? There were a few dozen improvements to it in this build.

5

Hi Ben, ok when I test some more I’ll try and get something reproducible together with a small amount of components. Yes I have been trying it with the most recent build. :slight_smile:

6

The Engine COMP (in the main TouchDesigner process) will wait if the frame it wants to output is not ready. If you don’t rely on the output of the Engine COMP in the main process (which you won’t for a Movie File Out) then you can leave this turned off.

If you do want to wait for the output frame because you’re using output in the main TouchDesigner process, then the Out buffer on the Tune page can help by rendering a number of frames ahead. With a buffer, the Engine COMP effectively has some padding if the frame-rate is variable, so it will have to wait less often (or not at all).

I’ve tried adding a few words to the description on the wiki to make it clear where the waiting is happening.

Regarding your performance issues, it would be super helpful to have a .toe and .tox so we can fully understand the problems you’re seeing. I’d like to iron out the performance issues - and annoyances - people are running in to.