My name is Feng, and I’m working on a real-time interactive waterfall in TouchDesigner that responds to my dance movements. However, the interaction is too slow and unresponsive, and I’m struggling to optimize performance and improve reaction speed.
My case- I want the waterfall to splash and bounce dynamically when my body moves, but it’s not working as expected.
What I’ve Built So Far:
Pose Detection (Using Mediapipe OpenPose)
Applied Blur + Threshold to create a solid body shape from the detected pose.
Extracted nose position from Pose_Tracking CHOP for an exact body reference point.
Used Transform TOP to align the solid body shape and the nose position.
Added OpticalFlow to this Transform so that I can connect it to the Waterfall.
Waterfall Setup (Using ParticleGPU)
Created a waterfall effect with ParticleGPU.
Added Turbulence + Wind to create natural movement.
Interaction Setup
Connected OpticalFlow output to ParticleGPU to drive interaction.
Set Hit Behavior to “Bounce on Contact” in ParticleGPU.
Added Feedback for motion persistence.
Problems I’m Facing:
Pose interaction is extremely slow – It lags behind my movements and cannot keep up with fast dance gestures.
No visible splash or bounce effects – The waterfall does not react strongly when my body moves.
Performance concerns – My MacBook Pro (2019, 2.6 GHz 6-Core Intel Core i7) may be struggling with real-time processing.
I’m still a TouchDesigner beginner, and I’ve been struggling with these issues for a while.
Do you have any tips on improving responsiveness or making the splashes/bounces more dynamic? Any advice would be greatly appreciated! It’s great to connect with other interaction designers and artists.
regarding the delay between movement and projection: try to narrow down where the latency is introduced. When you watch the camera stream in the Video Device In TOP, is this already showing the extreme delay?
regarding splash and bouncing: the optical flow input to particlesGpu acts as a additional force on the particles and does not take the Hit Behavior into consideration. So for example when waving your hands, a sideways force can be added to the particles via the Optical Flow input. You can experiment with the effect by increasing the Optical Flow Magnitude parameter dramatically.
The Hit Behavior only pertains to the limit box that surrounds the simulation. Once a particle reaches the extend of that box as setup via the Pos Limit Plane and Neg Limit Plane parameters, it will act as described in the Hit Behavior parameter.
Hi @snaut
I appreciate your time and help. The detected pose in each node looks fine. I tried running only the waterfall, and it runs fast. When I run only Mediapipe, the pose detection is also fast. However, when I run both together, the pose slows down slightly, but the waterfall becomes extremely slow. Could this be because both Mediapipe and the ParticleGPU waterfall are running on the GPU, causing a performance bottleneck? How can I improve efficiency?
Regarding the splash and bouncing effects, I tried adjusting the parameters in OpticalFlow and added a Math CHOP to multiply the values dramatically, but I still don’t see any noticeable effects.
performance wise this is difficult to optimize given your computer specs. As Mediapipe will take whatever resources it needs, your only optimization might be reducing the number of particles in particlesGPU.
For the rest, it’d be easiest if you could post your file to check on your optical flow setup directly.
I tried reducing the number of particles in ParticleGPU to 50 and 100, and it still didn’t help. Below is my folder. MediaPipe TouchDesigner.1.toe (693.9 KB)
If you already have Mediapipe please ignore the following info(Thanks a lot for your help and efforts!!!).
If you don’t have, there is the link to download: Release Release v0.5.0 · torinmb/mediapipe-touchdesigner · GitHub Just scroll down and click release.zip to download.
regarding performance, I would suggest deleting all elements of your network that are not required. Currently the only extra Mediapipe components in use are the OpenPose and pose_tracking components, so others, like hand_tracking, object_tracking and more can be deleted.
Your network then should look like this and hopefully performance would improve:
For the interaction, try to adjust the parameters of the Optical Flow a bit. For example setting the Force parameter on the Optical Flow to “2” and the Optical Flow Magnitude parameter on the particlesGPU to “1” will already give you a more visible effect when moving.
There is a nice video by @elburz going over Optical Flow interaction with particlesGPU that might also be helpful
@snaut OMG! You totally solved my problem! The effects look so much better now, and the reaction speed has improved significantly. I can’t thank you enough for your time—I really appreciate your expert advice and generous sharing.
