Summary
I am trying to create a TV production pipeline using the NDI In/Out TOP.
What I am trying to do is:
- Input live video in Log format to TouchDesigner using a Blackmagic capture card.
- Send it to Unreal Engine using NDI.
- Convert the received NDI signal from Unreal Engine from Log to RGB.
- Perform real-time compositing in Unreal Engine.
- Output the result from a Blackmagic capture card.
That is the flow of the system.
Issue
While building the system, I noticed that the colors were changing between the input video from the camera and the output video from Unreal Engine.
At first, I thought the colors were changing every time they passed through the Blackmagic device, but as I narrowed down the scope, I found that the color change at the NDI In/Out TOP stage was the biggest.
Specifically, there may be a mismatch in color space conversion between RGB and YUV between the NDI Out TOP and NDI In TOP.
In the system I am currently developing, I am trying to address this issue by creating a custom NDI Output TOP using a CPlusPlus TOP. However, the performance is not very good, so I hope that the issues with TouchDesigner’s NDI In/Out TOP will be fixed in the future.
I have summarized the verification details in case it can be helpful for problem-solving. The following is the information:
Hey, thanks for the detailed report. I spent a lot of time today looking at this. I did find a few issues with our Rec2020 color matrix, but the rest of the matrices we use are correct. We currently use the 8-bit offsets for YUV (0.0627, 0.501, 0.501), however using the 16-bit offsets didn’t change the issue.
I then tried doing a YUV->RGB conversation in the NDI Out TOP with the same buffer that I give to the NDI SDK, and that looks correct. So I can confirm our shaders do RGB->YUV->RGB correctly without loss of data.
I also don’t notice the issue you are reporting when using SDI inputs/outputs, which use the same YUV<->RGB conversion shaders.
So I think the issue we are facing is possibly related to the NDI compression. Do you have an NDI Out TOP custom OP that you are using that does not show the issue?
Thank you for your response. I have also been investigating this issue for over a week, and I have come to the conclusion that the colors are indeed changing due to NDI compression. I agree with that point.
However, I believe that the YUV offset of 0.501 should be as close to 0.5 as possible. (Is the calculation correct, 128/255 = 0.50196078431?) This is because the UV offset of the YUV component is equivalent to white balance. Both sRGB and Rec.709, Rec.2020 use the same D65 white point, so when converting a grayscale image to RGB->YUV, ideally the UV component should be 0.5.
In the shader I created, 0.50049 yielded the best result.
When performing NDI transmission and reception between TouchDesigner, there is no issue because they can use the same YUV profile. However, when communicating with other applications or professional equipment, there is a possibility that the colors will change if the profiles are not the same. In the case of NDI, I think the NDI Tools Test Pattern should be the reference, so it is best to align with that specification as much as possible.
I have created a custom NDI In/Out TOP and an operator compatible with the NDI Tools Test Pattern, so I will share it. out/test.toe is the project file.
Outputs
A: Custom NDI Output
B: NDI Tools Test Pattern
C: TouchDesigner NDI Output (16-bit)
D: TouchDesigner NDI Output (8-bit)
Inputs
1: Custom NDI Input
2: TouchDesigner NDI Input
By using combinations A1 (Custom NDI Output->Custom NDI Input) and B1 (NDI Tools Test Pattern->Custom NDI Input), it was possible to align the white balance.
Thanks for the further sample. Indeed the offsets are the issue for this example. I’ve fixed them so 8/10/16 offsets are used appropriately. Thanks for the amazing examples!
1 Like
