TouchEngine Plugin for Unreal Engine: UE5 Beta available - 2022-07-14 02:41

Great, good to know. Thanks.

So you mean that in your testing, the data generated by a TouchEngine instance would be passed to the “master”, and then from the Unreal master to other nodes of the cluster, those nodes were all on sync ?

Best,
Michel

On my nDisplay master computer i have touchdesigner and unreal running.
I have 4 kinects connected to that computer.
I merge the kinects in touchdesigner in one big image do some processing to it then send it to the ndisplay master that is running on the same computer and to two ndisplay slaves that are running on two other computer with touchOut->touchIn.
In unreal i use a tox loaded with the touchengine plugin that get the image via touchIn and pass it to Niagara system to drive some particle.

I’m working on a new version of that performance.


I will present it during the Ars Electronica Festival in the deep space projection room.
They use two computer for the rendering one for the wall and one for the floor.
If anybody is interested on how they use unreal and nDisplay they provide a template that is available to everybody on github.

Cheers,
Colas

3 Likes

Hi @JetXS,

It’s working when i do it in touchdesigner or if i load my tox with the touchengine in touchdesigner but not when i’m loading my tox with the unreal plugin.

That tox is working in unreal


noUnpacking.tox (1006 Bytes)

That one doesn’t
It seem that unreal can’t get an image out of it but i don’t get any error.

unpacking.tox (1.1 KB)

Cheers,
Colas

Hey @Tyrell

I think we just got the first issue fixed on the dev branch.

For the .tox issues, I spent some time testing.

In the samples project, you see on the left a case where the samples for Engine perform TOP is unmodified, but in the second case on the right, it is updated with the texture being sent out being 16bits RGB rather than RGBA.

This seem to be the issue.

I will log this for a developer to look into it.

Best,
Michel

Hi @JetXS,
Thank you for your quick answer.
Converting the image to rgba in touch before sending it to unreal is fixing my problem.


On that image you can see my setup running locally on my machine.
I have 3 ndisplay instance running they receive a image from touchdesigner via the UE plugin and use it to drive a particle system.
I tried to run that setup on multiple machine (one per ndisplay instance) and it’s also working
Thanks a lot for your help.
Cheers,
Colas

Hey @Tyrell

Great stuff.

I’m surprised that a particle system is fine over 3 machines. Particle systems are notoriously bad at being spread across clusters. Or you need at the very least to compute all positions / physics in one machine.

I guess that is what you do ? Then you render on the others ?
It’s a decent setup, It’s just one machine is under heavy stress, likely, compared to the others…

Anyways, good luck with the project and feel free to shoot an email at support@derivative.ca when you are done with the project… We’d be curious to hear more about it and get into some of the technical details + get feedback.

Best,
Michel

Hey @Tyrell,
This is really cool! I’m also working on some integration with Unreal and Touch over NDisplay.
Maybe you’ve already outlined this, but for your setup, did you also need a Touch player license on each of the slave computers? Currently working on an LED wall with 4 slave computer and was wondering how this would work?

Edit: I see above where you asked the same question, I think I understand how this would work now!

Amazing to hear you are bringing TouchDesigner back to the Deep Space 8K room! Shuhei Matsuyama and our friends at Think and Sense worked on a 8K VR project with NHK back in 2019 in the same venue. Crushing amounts of pixels were served! :smile:

2 Likes

Hey @JetXS ,

I had to install 3 workloads in Visual Studio installer and then everything worked fine with the samples project!

Now I have something else:
I created a new project, following the instructions from Github:

Everything works as described but I have an issue of how to get the information out of the Get TouchEngine Output node. For example, I am outputting from the tox file just one channel of 1 sample (an LFO CHOP), and I want to get this float number in my blueprint. How can I do that? What is the format of the Value output?

Screenshot 2022-08-31 at 12.16.57 AM

I also noticed somewhere in the samples project there is a Get Channel node that may be needed for the task but I cannot find it with search on my project’s blueprint.

Thank you!

Hey @morphe

What you want to use is Get Channel and feed it the CHOP Object that you got out of “Value”, and an index for your channel.

This will return an array of floats. You can then work with those like any other array of floats in Unreal blueprints.

If it doesn’t show in the list, untick the “Context sensitive” checkbox and they should show.

image

Hi @JetXS,
I have strange problem with the touch engine in the deep space setup of the ars electronica museum.
They are using two quadro RTX A6000 linked with NVlink.
Each graphic card is connected to two projectors, the 4 projectors are blended together.
we are running one instance of unreal that is using the two graphic cards at the same time.
One graphic card is getting the image coming out from the touch engine but the other one doesn’t.
Do you have any idea how we could solve that problem ?
Cheers,
Colas

EDIT : ok we found the solution that has nothing to do with the touchdesigner plugin we have the same problem with the spout plugin.
It’s a bug of unreal with directx 11 it seems that some niagara buffers are not correctly shared between the two graphic card switching to directx 12 fix the problem.

1 Like

Hey @JetXS,

Thank you for all the support, plugin works great!
I have one question, can we have the Component to run in Editor Mode? I managed to get a tick in the Event graph without having to press Play and I would like for development purposes to have the plugin run in Editor Mode. Is that possible? I have tweaked some options in the Component’s details but still I have not found the way.

Thank you!

1 Like

In the screenshot above I also had a switch to start first the TouchEngine and then starting to listen for any channel change (Get TouchEngine Output)

Hey @morphe

When you mean in “editor mode” you mean, not at runtime (without pressing play) but just while you are editing your BPs?

This is not possible at the moment, and I doubt this could ever be possible but I will discuss it with one of the developers.

Best,
Michel

Hey @Tyrell

Not sure I follow - The plugin is not supposed to be compatible with DX12 at the moment unless I miss something.

Care to share more details about the setup? Was the process using the TouchEngine switched to DX12 ?

Best,
Michel

Hey @JetXS,

Yes, while editing BPs. I managed to run OSCServer in editor (there is a function called SetTickinEditor:
Set Tick in Editor | Unreal Engine Documentation), and I was wondering if TouchEngine nodes could have this option so we don’t have to do the back-and-forth Play-edit-play-edit while developing.

Best,
Vassilis

1 Like

Hi @JetXS,
The problem we had with unreal was solve by switching to DX12 but then the plugin was not working anymore.
Therefore i finally didn’t use the touch engine plugin.
I run touchdesigner separatly and used spout to send the data to unreal.
I will do a blog post with more details about the projects and the technical setup.
Cheers,
Colas

Apologies you encountered so many blockers @Tyrell

We are working on getting DX12 support at the moment.

I’ll keep everyone updated as soon as we have something to share.

A blog post about this project would be amazing, feel free to share it and let us know!

Best,
Michel

1 Like

Hey there @JetXS

Loving this plugin! I see that in September you mentioned that you were looking to support DirectX12.

Can you provide any more info on that? Estimation of when that will be rolled out?

Hey @MobileMotionMedia

It is available in the UE5.0-Dev and UE5.1-Dev branches.

So far it is tested by a small team, so you will want to test extensively if it’s for a production environment.

You’ll want to compile from source and you will need a new TouchDesigner build that should come in the next 10 days. Or reach out by email to support@ for a dev build.

A precompiled version of the plugin should be available by the end of the year.

Best,
Michel