FreeD data update timing relative to cooking and rendering

General TouchDesigner Discussion
,
1

I would like to understand the timing of FreeD/network data reception relative to operator cooking and rendering in TouchDesigner.

Specifically, I would like to know the order and timing of the following processes within a frame:

  1. When does TouchDesigner begin cooking operators for each frame?
  2. At what point is incoming FreeD/network data received and made available to operators?
  3. If new FreeD data arrives before an operator is cooked, is that latest data used during the same frame, or is it reflected in the following frame?
  4. Is there a specific stage in TouchDesigner’s frame loop when network input is processed relative to operator cooking and rendering?

For example, is the processing flow approximately:

FreeD/network data received → OPs cook → Render

Or can FreeD data reception and updating occur asynchronously at a different point during the frame?

I am particularly interested in the latency and synchronization behavior between incoming FreeD camera tracking data and the rendered output.

For example, if FreeD packets are arriving at approximately 60 Hz while the TouchDesigner project is also running at 60 FPS:

  • How does TouchDesigner determine which FreeD packet is used for each rendered frame?
  • If a FreeD packet arrives while a frame is already being cooked or rendered, can it affect that same frame, or will it only be reflected in the next frame?
  • Is network reception handled asynchronously from the main TouchDesigner cook/render loop?

The reason I am investigating this is to better understand whether timing differences between FreeD packet arrival and TouchDesigner’s frame loop can introduce one-frame latency, uneven tracking updates, or visible camera/CG jitter.

Thank you.

2

Some great questions and I’ll do my best to answer, but it does touch on quite a few different systems so I’m not 100% certain on everything.

Generally speaking, the networking connection is running on a separate thread that is always listening for incoming UDP messages from the FreeD server. When the FreeD In CHOP cooks, it looks to see if there are any new messages and updates its data to the latest values.

If multiple packets came in since the last cook, the older packets will get dropped. If no packets came in, then the chop will hold onto the previous values. If you connect an Info CHOP, you can see the number of valid packets processed. This will include packets that were skipped because more than one arrived on the same frame.

As far as when the FreeD In CHOP cooks within a TouchDesigner frame, that is a lot more complicated. TouchDesigner cooks nodes on demand, so the order depends on what other nodes are relying on the data in that FreeD In CHOP. So a Render TOP may rely on a Camera COMP which relies on the FreeD In CHOP, in which case the order would be FreeD → Camera → Render TOP.

Each node will only cook once though, so if the FreeD In CHOP cooked earlier in the frame because something else called it, then it will not recook again for the Render TOP. The values in the FreeD CHOP will not change again in the middle of a frame.

So overall, you can’t really guarantee when a node will cook within a TouchDesigner frame, but it should always be before whatever is using that data. And nodes will only cook once, so whatever value is latest when the node cooks will remain for the rest of that frame.

Hope that helps. Happy to dig into anything further if you have more questions.