Wildly fluctuating frame rates

Doing some performance testing on a Azure Kinect application (test early, test often!) and experiencing wildly fluctuating frame rates.

Simple network, some Math CHOP range calculations and compositing a few circles on the RGB feed from the Kinect.
40-50 fps for several minutes in Perform mode then gradually decrease to under 10 fps over a period of around 30 seconds (with corresponding decrease in both CPU and GPU usage) then slowly increase again.
This cycle seems to repeat, fast, slow, fast slow etc.
Anyone care to guess what might be causing this?
And is there any way of ‘forcing’ a constant frame rate?
Windows 10 TD 2020.22080