How to get DirectX shared texture handle that is used in TouchEngine TOP Input?

Hello,
I would like to ask how could one go about getting handle of shared DirectX texture that is used for passing texture to TouchEngine?

I am assuming DirectX texture sharing is used in TouchEngine In/Out TOP based on this post:

I would like to utilize it in a scenario for passing texture from UE4 to TD (editor) while using TouchEngine-UE4 Plugin as a middleman. I am assuming it would be possible to directly consume UE4 texture inside of TD (editor) by using DirectX In TOP with proper handle.

This is related to following topic. It doesn’t solve genlocking, but consuming texture from UE4 directly inside of TD (editor) would reduce latency and overhead.

If you are only using TouchEngine for this purpose, because you want a shared DirectX texture, this won’t work using the UE4 plugin as-is. One could in theory modify it but you would have to create and fill the DirectX shared textures yourself, at which point, why involve TouchEngine at all? It sounds like you need a custom plugin at least on the UE4 end, and possibly on the TouchDesigner end, to manage a shared surface yourself. You might get something close to sync if your shared texture is backed by a DXGI keyed mutex and you are careful in your acquire/release calls.

1 Like

Aha, I see, thank you very much for explanation. I just thought I could somehow use TouchEngine-UE4 Plugin - so that I won’t have to write custom plugin. I guess that was a bit naive thinking :slight_smile:

@bangnoise please may I ask whether you know about any plans on supporting Texture Share available in Unreal Engine? It seems to support synchronization mechanisms and I thought this might be a great solution for TD <-> UE texture sharing without frame drops.

My skills in this area are limited, so before diving into this rabbit hole I thought I might ask if Derivative isn’t already working on some “UE Texture Share TOP”? Thanks.

Due to licensing constraints of the TextureShare API we aren’t able to implement it within TouchDesigner directly unfortunately.

Oh, thanks for info. Does that mean I would be also hitting these licensing issues in case I would somehow manage to implement it myself in TD?

Also, please are there some other texture sharing options (that would support synchronization) you are currently considering (in terms of TD <-> UE)?

I can’t speak to the licensing for what you want to do, you’ll have to check with Epic. The only thing we have right now is the TouchEngine-UE plugin, which is still in development.

Thanks for info :slight_smile:

I have just noticed LightAct has support for UE TextureShare API and it made me wonder if there isn’t some way around the licensing that would make this possible… I mean it would be great stuff.

I am not sure what is Smode using for texture sharing, but the video on this page is showing really nice workflow between the two applications.

I believe this kind of workflow would be a game changer for TD :slightly_smiling_face:

May I ask how did you guys at Derivative managed to get more info regarding Texture Share licensing restrictions? I have tried contacting them several times but without success (I tried writing on their forum {but received no answer there}, then writing directly to Epic {but was told to contact people from Unreal}, and finally Unreal’s licensing support {but all they told me was that their standard license is what I am looking for and that it should cover my needs}). I have no idea who else to ask there so I thought I might ask you about it. Thanks

I used their UDN to ask the question. It confirmed that their standard licensing model is what the TextureShare is covered under. That model does not work for us so we can’t implement it

Aha, I was wrongly thinking there are some Texture Share specific license restrictions. This makes sense, thank you very much for info.

Please may I ask one thing that related to this topic? Let’s say I would implement Texture Share for TD using C++ TOP. Please is there a way to explicitly set this TOP (or thather the branch to which it belongs) to be cooked last? While thinking about synchronization (when sharing texture from UE → TD) I thought it might cause some issues if this C++ TOP would cook too early - resulting in long stall before getting new frame from UE and therefore unnecessary delaying cook of other operators.

If you are pulling a texture into TD, then when that node cooks with be based on when it’s being used. There is no way to control that directly since that’ll be entirely based on how it’s being used, not the C++ TOP itself

I see, thanks for info. Please is there possibly something else that could be done in order to prevent this kind of thing from happening (in case one wouldn’t want to add latency by queuing incoming frames)?

Have a background thread acquiring frames from UE if that requires a wait, then your TOP cook uses the latest completed frame.

Yes, adding a 1 frame queue will remove the stall, but add a frame of latency. It all depends what your requirements are.

If you need perfect sync, you will need to stall. However if UE is generating frames at a consistent rate and TD is cooking consistently, and you have vsync off in your perform window, then after the first stall TD will have shifted it’s cooking window so that the TOP will end up cooking a little later (since everything else will have started later due to the previous stall), so the next stall will be much shorter.

Thank you very much, I haven’t realized this. That makes total sense.

However this won’t work in case one would like to use new ‘Sync To Input Frame’ mode (with AJA input device) as cooking window won’t shift, right? I guess 1 frame queue would then be the only option.

Or… Maybe one could experiment with offsetting UE cooking window to eliminate stalls on TD side (not sure if somehow possible, but one brute force approach could be to genlock UE to blackmagic / aja device that would have its reference signal shifted when compared to reference signal plugged into aja device used in TD)… :face_with_monocle: A bit crazy, but I guess it might work. :smile:

Yes, you can only sync to one source, if you try to sync to both the AJA sync pulse and the UE frame coming in, you may end up with long stalls. If you can get the offsets right things can line up, but this will be hard to get right for sure

1 Like

While reading through release notes of 2022.28040 I got quite excited to see RenderStream support. I have thought it would be great to use RenderStream purely for pushing textures from UE to TD. However when looking closer at licensing I am getting kind of confused - thinking it might not actually work just like that :sweat_smile:

Please do you think it is possible to use RenderStream without actually having Disguise hardware or software license? I guess the base case scenario would be if one needs to have only RenderStream license. Since UE plugins on github state following (using or), I am guessing it might be the case?

Anyway I thought I might ask here if you know more about whether it could work the way I imagined, or there are more pricey requirements. Thanks for help :slight_smile: