I’m trying to use best practices for seamless video switching as explained here.
It works well for video only applications and most videos with audio as well.
However on some videos with audio I get an excruciatingly long frame, effectively freezing TD for up to half a second, where (I assume) the Audio Movie CHOP is “Waiting for Audio Read”:
It is not happening on the same video if I disable this CHOP. My setup is waiting for the video to be preloaded before switching to it as described in aforementioned guide, so none of it is affecting my TD playback, except when I use its audio, leading me to believe that preload() does not take audio tracks into consideration.
I’d assume that “Open Timeout” parameter would allow to avoid this by setting it to a very low value, but it does not help my case at all or seem to have any effect whatsoever.
I also apply same principles that I use on the Movie File In tops for the Audio Movie CHOPs as well, disabling their preview, toggling “Play” off while their videos are being preloaded, etc…
I’d appreciate any help with this, suggestions for workarounds, or just knowing that this is in fact a shortcoming of the Audio Movie CHOP
It seems to happen more frequently on video files that have their audio encoded with Opus (in webm container), but not only.
I will DM you a link to one of the files.
To clarify, I am not so much seeking help in making audio tracks load faster, but more in allowing it to load asynchronously for as long as it needs without interrupting the main TD thread, either by playing the video mute until it catches up or preloading the audio together with video.
On a “lazy” HDD it can even last seconds:
I won’t be able to control all video files that come into my setup in the end and I am okay with some of them taking unusually long to properly load as long as I can handle that gracefully in TD.
Did you get a chance to look at it so far? I would like to get a hint if it’s likely something to be worked out or if it’s on hold indefinitely.
If there are no news, I need to look into workarounds because this completely halts my installation at times and breaks the flow. The only thing I can imagine is having the playback handled inside an Engine COMP.
Hi. The values were set to a low number like 0.1, but even at 0 it still occurs.
Were you preloading to arbitrary moments in a video? Because that’s what happens in my setup. I don’t pass any number into preload() but their play mode is set as Timecode which I assume makes it preload into whatever is dictated by the Timecode object at that time.
From the docs:
If specified the movie will be opened at the specified frame index. If not, then the movie will be opened at the index specified by its parameters.
If you still can’t reproduce it I will try to disassemble that part of the project into a most stripped down version possible that still reproduces it together with several video files and share that with you.
Thanks for the hint. I was specifying the frame index in the preload() parameters. However I’m still unable to reproduce the issue. Are you able to reproduce in a simple .toe file you can post?
It features a video/audio output and a button that triggers a randomized video switch from a given folder of videos. The switch unloads the current video, toggles it away until a new video is loaded, and toggles it back on once isFullyPreRead returns True, which to my knowledge is a correct way to handle video preloading without hanging the main thread.
This is a very simplified mechanism of what I’m doing in my actual project.
Some more observations and comments:
The long audio read seems to happen only if preloading to non 0 index of video files. So in my demo it randomly picks one to provoke the behaviour.
While initially I thought this was related to certain audio codecs and video sources, that seems not to be the case. I can’t confidentely pin point specific characteristics other than that most videos are in common web friendly formats and bitrates (audio being in AAC or Opus for instance). The videos that I’ve tested have durations ranging from several minutes to an hour. I maybe suspect that longer videos trigger longer hangups.
I hope this helps to reproduce it. And thank you for being patient.
The performance monitor still reports the same “waiting for audio read” and I see dropped frames. The drops however are a lot less noticeable in my isolated simplified setup despite using the same drive and same video files. (for a moment I thought it’s gone until I checked the actual data)
And it’s already set to 0 in my main project after you’ve suggested it so I ruled it out. The stalls are just the same with ir without this tweak.
Ok thanks for the extra information. I think what is happening here is that the Audio Movie CHOP isn’t cooking until you start playing the audio, so it isn’t opening it’s track yet until right when you want the audio data.
What I’m going to do is add a preload() method to the Audio Movie CHOP. This will cause both the Audio Movie and the Movie File In TOP it’s pointing to start a preload process at the same time.
For this issue I’ve added a new preload() method for the Audio Movie CHOP, that will cause both the audio and the video to be preloaded at the specified frameIndex. You only need to call it on the Audio Movie CHOP, it’ll flow-through to the Movie File In TOP.
Other notes about this example. The way you are choosing your preload index, but then using a % value for the Cue seems problematic, since those will almost never match, unless know the length of them movie you are opening before you open it.
This is great news malcolm! Thank you for coming up with a solution.
Shall we expect this feature in the next release then?
And the expression on the Cue parameter was a leftover from my testing, before I changed it to cue in via preload index parameter. Afaik it didn’t affect anything since “cue” was not being pulsed.
In my actual project I also only use preload with index specified and cue and cuepoint are untouched. I retrieve exact indices I want by first building a database of all video files (including their durations) using a script with pymediainfo.
