Good day team, I have a very important question for us, I hope you can throw some light into our doubts.
We are building a complex particle system based project in touchdesigner, using instanced particles combined with GLSL. And that’s all good.
The issue comes because with that we will produce a video, which is easy from TouchDesigner. But we also need to take that particle system and export it somehow to other platforms to produce an AR and VR version of the work.
The questions my team have are:
a) Is there any way to export that particle system to file formats like OBJ, FBX, GLB, etc?
b) If not, is there any way to export touchdesigner projects to platformas like Aero, Unreal, etc
c) We found this TouchDesigner-Unreal plugin: User Guide | Derivative , is this working well? what does it allow you to do? if I exported this particle system + glsl touchdesigner project with that plugin, what would I be able to do then with it in Unreal? would I be able to interact with it?
d) any other suggestions to interface between a touchdesigner project of this kind and other AR/VR development platforms?
e) finally, in the worst case that nothing of this was possible, what would be the possibilities that touchdesigner itself offers to wrap an AR and/or VR experience around the particle system?
It depends on what your approach is. If your particle system is CPU-based then yes you could probably try to get that exported using an alembic sequence.
You could possibly export as a point cloud image sequence (textures, or TOPs in TouchDesigner) where each pixel stores the (or some) data of each particle, you could achieve that using a combination of the particlesGpu in the palette, the PointClouds tools in the palette, the Point Transform TOP, the MovieFileOut TOP set to EXR
Note that “exporting” your particle system is like pre-rendering it, so it might complicate any further interactivity you might want to add later down the chain.
There is multiple ways to send either a render of the particles to another software, or the data itself. There is plenty of resources available at alltd.org using Unreal or UE in the search bar.
The plugin is still in Beta. There are examples of what it allows you to do in the Samples repository.
Too many to list them all, eheh. Many users are relying on OSC I believe, mainly for its ease of use and setup. I can think of MIDI, Web protocols… pretty much all the ins / outs of TouchDesigner.
@JetXS thank you Michael, really appreciate your feedback, regarding the first point, I doubt that this would work for us, as I am creating these particle systems using instanced particles + GLSL code that manipulates them so obviously they are on the GPU and we would only work with GPU based stuff to have the best performance. Also we need to preserve the capability to change the behaviour of the particles, in the same way that the Unreal Plugin allows us to do that by sending messages to and from the .tox project so that the GLSL TD code can change the behaviour of the particles depending on parameters that we send. I understand that this would probably not be possible in other different platforms outside of Unreal nowadays?
"It depends on what your approach is. If your particle system is CPU-based then yes you could probably try to get that exported using an alembic sequence.
You could possibly export as a point cloud image sequence (textures, or TOPs in TouchDesigner) where each pixel stores the (or some) data of each particle, you could achieve that using a combination of the particlesGpu in the palette, the PointClouds tools in the palette, the Point Transform TOP, the MovieFileOut TOP set to EXR
Note that “exporting” your particle system is like pre-rendering it, so it might complicate any further interactivity you might want to add later down the chain."
Are you using a GLSL Mat, or a GLSL Top? Are you rendering anything in TouchDesigner, or just doing the maths / physics / position with TouchDesigner ?
IMHO, the best approach would be to do all your maths in TouchDesigner, send the data either as CHOP or TOPs to Unreal and render in Unreal. There are examples of this in the Sample project of the Unreal Plugin. Or other approaches without the plugin in alltd.org.
I’m not sure I’m understanding your question here, can you please rephrase it?
@JetXS Michael, thank you again for your great feedback, here I go:
" IMHO, the best approach would be to do all your maths in TouchDesigner, send the data either as CHOP or TOPs to Unreal and render in Unreal. There are examples of this in the Sample project of the Unreal Plugin. Or other approaches without the plugin in alltd.org."
Forgive me if this is an absurd point, but if I render all in unreal, and only do math in TD, what’s the point? I could then do the whole thing in unreal, no? The whole point of me wanting to use this plugin is that I work and do all with TD and we just want to put it into unreal and then add other things around it and interact with it; If I was to render with unreal I guess we could do the whole thing with unreal, maybe this is an absurd point, I dont know. But ok maybe I get you, maybe you mean, send the data as a TOP to unreal and then the rendered particles can be better integrated with other Unreal stuff? or is it because the unreal renderer is better than the TD renderer? but no, because if we send the TOP what would be the difference? we want to preserve the look in the TD Project, so really what would be the advantage of rendering it with Unreal?
what I mean in the second question is if the kind of thing that we can do with your great Unreal plugin can be done at the moment to interface with other platforms different from Unreal; Are there nowadays other similar bridges between TD And other platforms, or currently there is also the one with Unreal? (in the sense of allowing you to embed a whole TD project in that other platform while maintaining the capability to interact with it and drive it from that other platform)
@JetXS
we have another question, key one:
they tell me that we have to produce our output for the AR experience for mobile phones and tablets, android and iPhone, so the question is, if the output of Unreal will be for mobile phones, will the TD plugin work ok for that as well? any advices for that scenario?
thank you
It won’t work on any mobile phone . And even if it did I think you would need a TD license for each device you want to run your app on.
TDs internal VR tools are very low level compared to what you get in unreal or unity, so together with the ”1 license per install” requirement I would not recommend using TDs VR tools for anything else than previz
That’s entirely up to you and what you feel the most comfortable with.
Yes, mostly had in mind blending your contents together.
If you send the TOP coming from a Render TOP in TD, you have a “flat” image that then will get applied to a plane or geometry as texture in Unreal. It might feel extremely out of place unless you spend a bit of time tweaking effects and most probably use post-processing in Unreal quite heavily. You could try and pass the depth buffer of your TD Render to unreal and make use of this as well, but again, I believe it would be quite some work to have a smooth integration.
On the other hand, if you just pass your particle system data from TD to Unreal, it could allow you to render it in Unreal and have it feel and look more integrated in your space.
I’m not the biggest Unreal developer. But that’s my take on it and maybe you know of techniques in Unreal that will make my points invalid.
See my points above.
That’s the one that came to mind at the moment, but I believe some other third parties/customers are starting (or looking into) integrating the TouchEngine API into their proprietary software.
Note that if you just want to pass your render from TD to Unreal, you might not want to use or need to use our TouchEngine-UE4 plugin. You can share your texture from Touch to Unreal using Spout and have a setup relying on OSC for interactivity. Many developers are still approaching similar challenges to yours this way.
@JetXS Michel, thank you for your great feedback. Yes, at the end, because we need to produce the output for mobile, we see as the only option to basically embed a rendered video of the particle system within an unreal app for mobile, basically that´s it, so in that case it will be just a rendered video of the particles so it doesn´t matter anymore the plugin stuff i guess,
thank you very much for your help
