OpticalFlow IMAG experiments

I had made a note to do some opticalflow IMAG particle experiments. I thought I’d seen someone post something about IMAG POP tests here, but I can’t find the thread now.

Regardless, I figured I’d do a “classic” Notch-style IMAG effect with particles emitting from regions with lots of deltas.

It wound up turning into a shootout between the two optical flow models in TD. TLDR: The nvidia one works great for footage from a webcam of a person in their office waving their hands around, but blows up from minor blacklevel variations when you try to feed actual IMAG footage into it. From my admittedly limited testing it seems basically worthless for any live cam footage (front lit subject on a dark background) I tried to throw at it. I think their ML models aren’t well adapted to to that use case.

Here’s a video featuring me (with a hilarious case of bedhead) as well as some test footage featuring ‘Mike’, a roadie being mercilessly mocked by the rest of the crew after agreeing to stand for some test footage for us to use during development of a tour several years ago:

edit: link to video the embed isn’t working for some reason

In the video you can see that I can get great results when I stand there and wave my hands. As soon as I switch over to Mike, the whole frame goes haywire. Swapping over to the palette version tones everything back down and seems to properly respond to the subject (allowing for Mike being a bit low energy).

If someone wants to straighten out any misconceptions I might have about the nVidia optical flow, that would be great. :slight_smile:

Project is attached.

optclflow.toe (9.5 KB)