Performance Issues with Multiple Video Playback + MediaPipe Pose Detection

[colorful_invisible]

I’m hitting some performance issues with my current project and would love some advice.

Here’s what I’m trying to do:

I’m using MediaPipe Pose Detection to control multiple videos and one audio track based on hand position. When my hand moves left/right, it scrubs through the videos backward/forward. The videos switch automatically when reaching their end frames. All videos will be designed to have matching end/start frames and that will make the magic of this experience.

The problem is I’m getting really bad frame rates. Right now I’m just testing with 3 videos (will need 10+ for the final), and it’s super laggy even though I have a decent machine.

I think the issue might be that I’m trying to control all videos simultaneously to keep them in sync, but maybe there’s a smarter way to handle this?

Any tips on how to improve performance while keeping the videos synchronized?

Dropbox Capture

Thanks a lot

[Alaghast]

Hello @colorful_invisible,
not sure which video codecs you are using for these videos, but for scrubbing things you have to convert them in HAP codecs to manage it in GPU.
You can easily convert using some software like Shutter Encoder if you use MacOS, or HandBrake.

[colorful_invisible]

Works like magic and happy 60 FPS

Thanks @Alaghast

Do you know if there is any way I can scrub just the videos that are on the row, the actual and the next on the line? At the moment I am scrubbing all the videos on my list at once, eating performance for no use.