Optical Flow

Shared .tox Components
21

hi all, I was really loving this little network, but it looks like somewhere along the lines towards 088 it stopped working. At least it doesn’t work here with 088 10500…

any tips, tricks, ect? it would be great to see this thing work again :slight_smile:

a side by side output from 077 (left) and 088 (right) of the the HSFlowGLSL looks like this:

hsflow_issue.PNG.jpg
hsflow_issue.PNG.jpg2524×709 460 KB

I guess something changed in the way 088 handles things. Sadly Im not the guy to understand what. I searched through the changelogs, but alas, nothing jumped out at me…

it looks as if the previous frames are not being subtracted in 088.

22

Hi Delray.
We’ve confirmed a difference in the glsl output between 077 and 088 in this case and will let you know what we discover.
Thanks for posting.
Cheers,
Rob.

23

great thanks…

24

Attached is an update version of the .tox file with the shaders fixed up to work with 088. The issue was there were some variables being used (gl_MultiTexCoord1-7 in particular) that have undefined values. They randomly worked in 077 but no longer work in 088.
Uses the latest version of 088.
opticalflow088.tox (32.5 KB)

25

thanks malcolm - I also got a note from itaru and posted a fix too in the other thread. I should have put that up here too. (viewtopic.php?f=36&t=5058)

26

Hello,

Trying out this, it looks like it broke again at some point, glsl1 is looking for a camera_out TOP that does not exist.

But else if I want to do motion blur on 3d objects, I’ll guess I’ll get better results if I try to output a velocity pass from the render as well?

I guess there have been some refinements since the classic stupid opengl tricks pdf (2003!), but it’s probably a good starting point.
nvidia.com/docs/IO/8230/GDC2 … Tricks.pdf

27

The camera_out has always been missing from the .tox, even in the original. The fact it’s not using a Feedback TOP and has a loop in it’s network is an issue also possibly.

28

I’ll dig the optical flow thing more later.

Fore now I was able to get the one from the pdf working (which is also in latest nvidia sdk docs.nvidia.com/gameworks/index. … sample.htm)

I get nice results but since it’s using normals to stretch meshes it doesn’t work with flat or hard edge geometry, so I’m trying to get the advanced sample to work now.
docs.nvidia.com/gameworks/index. … sample.htm

29

Are there any updates on this working in 088?
thanks

30

I wrote a research paper on hardware accelerated motion blur in 2003 for 3D using the vertex data. There is a pdf and video on my website.

clementshimizu.com/hardware- … eneration/

This full image space optical flow is very exciting.

Clement

31

Hey,

just spend a little time with this and fixed it up to work (or so I believe) in the experimental branch of 088.

All Parameters are now on the component as custom parameters.
I think this is what it’s suppose to do - feedback welcome.

cheers
Markus
opticalFlow.jpg
opticalFlowFx.tox (4.01 KB)

32

Hi Markus,

Thanks, just having a very quick look, I don’t think it’s supposed to be blue, is it?

edit : I guess I was still half asleep it’s actually working as it should, just the parameters saved with the tox make it blue.

33

Hey guys,

Sorry to revive an old thread.

With this example, is it possible to pull out the vector information into chop channels.

I saw this implementation here: vimeo.com/114542945 and would love to be able to use the vector data for driving other animations / visualizations.

Cheers,

O.

34

Hey, yes you can always extract the vectors information using a TOPtochop, make sure to reduce image resolution and use the next frame mode, otherwise it can be pretty slow.

35

Vinz, I would love to see an example of how you might do that. I’ve been scratching my head all afternoon trying to work it out.

36

Hey Jason,

Sure I’ll post the setup in the next couple days.

37

Yes! indeed vinz , seeing an example of how to execute that would be great!

38

10 years later, here you go!

Cheers,
Vincent
optical_vectors_display.toe (6.96 KB)

39

Thank you, thank you, thank you! Will test it out when I’m back in the studio :smiley:

40

Hi Vinz

Is this meant to be the vectors of pixel movement like optical flow or something else? Maybe I just don’t get it, but if I connect a more simple source to this setup to see it more clearly(a circle top for instance, moving left to right) it’s showing vectors going in almost all directions, when I would expecct it to be only the axis of movement. Am I missing something or is the shader not actually designed to show this?