Capture Mouse Pointer With Screen Grab

I’m trying to make a video switcher for creating tutorials and I don’t see my mouse pointer when I am using the screengrab TOP.

Is there a trick to getting it to record the cursor?

I’m on the latest 2021.11180 on windows.

I typically use OBS these days when recording tutorials. I know that it adds another tool to your process, but IMO it’s been easier to use an existing app rather than build all of those options from the ground up.

I know several users who have built their own screen recording tools, and they typically use a mouse in with a PNG to composite in their own pointer.

1 Like

Well that’s not an option I’m trying to straight up replace OBS. :slight_smile:

Do you happen to know the technical reason it doesn’t pick up the mouse?

I don’t know for sure, but I’d guess that the TOP just isn’t currently written in a way that exposes the parameter to capture the cursor. I don’t think that’s a technical limitation, just a limitation as the TOP is currently written.

I’m sure @malcolm can better speak to the mechanics here - this might just be something that can be turned into a RFE. :slight_smile:

The algorithm we’re using right now returns the pointer information separately (position and shape buffer), which it looks like you can use to composite the cursor back into the capture.

I’ve added this to our RFE list, but in the meantime you might be able to get a similar result just with the Mouse In CHOP

Hmmm, Interesting. The mouse in chop doesn’t seem to want to work for me, in that it seems to be limited to my primary monitor, and I’d like to stream my two lower monitors.

The screen grab chop is a little buggy for me as well, which is possibly to be expected due to my duct tape and bailing wire video setup. :slight_smile:

Two of my monitors are on display cables, one on HDMI, and one on DVI, all to a NVIDIA GeForce GTX 1060 6gb. (That probably won’t change until the crypto market collapses again.) The two DVI monitors work to display their individual monitors, but to actually capture my other two displays I have to do crops from the screen grab catching the full desktop, which seems to work fine for the moment.

yeah, the mouse chop may only work over a touchdesigner window, so that might not be a solution if you’re capturing a different app.

Do the other monitors appear in the source list e.g. Display 3, Display 4, etc but just don’t work, or are they missing entirely? Are you using anything special to manage your desktop like Mosaic, or is it just the basic windows display settings?

Vanilla windows install. All the monitors appear, just the two non display cable monitors will only show the checkerboard transparent background unless I do a full display capture and crop the monitors I want individually.

If I could somehow spout the spout the displays in to TD that might be an option…but I’m by no means a world class programmer or an expert on video streams.

Here’s my monitor info. I notice offhand that there’s some slightly different frame rates happening…

Which are the two displays that aren’t capturing properly in your list there? I’m wondering if there’s a bug with the capture coordinates, because selecting a display from the source menu should behave the same as manually cropping it from the full desktop.

I’m able to capture the two lower resolution monitors individually through the screen grab chop. As I said they are both on display cables, if that matters. Then one hdmi one dvi for the other one.

To capture all four I have to use the full desktop option of the screen grab top and do crops of each individual monitor with 4 screen individual screen grab tops. It works.

I’ve also noticed that the lower res monitors have some empty space, which I assume is down to the way windows treats multiple monitors of varying resolutions. Windows seems to lie to itself about the full available resolution to be able to treat the whole thing as one giant square field. At least when they are arraigned two up two over. I’ve not tried it as 4 monitors arraigned in the control panel as a single row…I suppose I could, but using monitors that are laid out different virtually than they are in real life is quite a pain.

Screen grabbing individual applications just results in a blank checkerboard for me as well, despite what monitor the application might be on.

I tried yesterday setting the 1080s resolution down to match the smaller monitors, and that didn’t seem to change things either.

Back on the cursor grab end of things, having a capture cursor flag available on the screen grab top would be great as I’d like to do some tutorials on programs that use a lot of different cursors in different scenarios, and it’s not really practical to try and paint in the right cursor when you want to display something like the different ways ableton can extend a clip. :slight_smile:

I appreciate all your hard work and you taking the time to follow up on this.

Update:

So, If I put all my displays virtual line in Windows’ display settings, then I can grab the displays individually by selecting the display in the screen grab chop, and I can now successfully grab individual applications as well (I could not do that at all before.). I suspect this is possibly related to the difference in the way Windows and TD consider a monitors position/label?

If I recall, TD starts with the monitor physically located in windows settings as being at the far left atand counts up, where as Windows…I don’t really know how Windows decides to number displays, or how TD deals with identifying vertically stacked monitors.

Curiously there are still some transparent pixels chilling at the bottom of my 1920x1080 screen grabs in the working config. Not a big deal.

At any rate, it’s not too counter intuitive at the moment to just move the mouse farther right to access my top row of monitors

If there’s any other testing I can do to provide useful info please let me know.

Semi-unrelated, is there a way to get a table of the applications the ScreenGrab Top offers application capture for? I can see some useful stuff happening with a replicator and auto generating sources for applications.

Thanks for all of the information. It seems to confirm my suspicion that TD was having problems with the specific layout of your system … I’ve only got two monitors here at the moment, but I’ll see if I can reproduce the problem.

The only way to access the application list at the moment is through the Par class i.e. n.par.source.menuLabels Not sure if that gets you what you need.

A quick update: there was a bug with cropping displays or applications that were below the primary window. I’ve fixed this now and will have an update soon that you can test out if you want.

I also looked into the mouse cursor issue a little further and I think we’ll be able to do it, but it’s going to take some time and there are a lot of other bugs/requests on our list so I’m not sure when that might be available.

Back to grabbing mouse position outside of the TouchDesigner window. This little COMP from Idzard is absolutely the best!

3 Likes

Thank, I’d be happy to test the new version.

Thanks! I’ll check it out.

There is a windows hack to make pointers appear in every screengrap software. In your windows mouse settings go to “Additional mouse options”, go to Tab “Pointer Options”, and check “Display pointer trails”. If you set the trail to short you will have just one pointer in the screen grab.

6 Likes

Thanks Manuel, that’s an awesome tip.