I am trying to use my OAK-1 camera for hand tracking. I am working on Windows 11 and attempted to use the OAK examples provided in TD, located at: C:\Program Files\Derivative\TouchDesigner\Samples\OAK\OAKExamples.toe.
After disabling the “Use3D” button in the “handTracking” container and initializing the camera, everything appears to work fine. However, as soon as I place my hand in front of the camera, TouchDesigner freezes and crashes. I noticed that the “oakselect_joints” CHOP is receiving data right before TouchDesigner freezes.
The same crash also occurs when using the OAK-D S2 camera.
I am currently using TouchDesigner version 2023.12120.
could you roughly estimate how long this might take? We are currently working on a project that needs to be completed by the end of February, and we have already purchased the OAK hardware.
Apologies for the curious questions.
Thank you,
Bene
we are actively investigating but haven’t found a solution yet.
Can you see if you can generate a OAK crash report with this script:
import cv2
import depthai as dai
from json import dump
from os.path import exists
# Connect to device and start pipeline
with dai.Device() as device:
if device.hasCrashDump():
crashDump = device.getCrashDump()
commitHash = crashDump.depthaiCommitHash
deviceId = crashDump.deviceId
json = crashDump.serializeToJson()
i = -1
while True:
i += 1
destPath = "crashDump_" + str(i) + "_" + deviceId + "_" + commitHash + ".json"
if exists(destPath):
continue
with open(destPath, 'w', encoding='utf-8') as f:
dump(json, f, ensure_ascii=False, indent=4)
print("Crash dump found on your device!")
print(f"Saved to {destPath}")
print("Please report to developers!")
break
else:
print("There was no crash dump found on your device!")
You can run it in a seperate TouchDesigner process right after the camera / TouchDesigner crashes.
I just tried to run the script in a separate TD file right after TD crashed. But got an error message that i could not connect to the Device. When running the script as soon as TD closes it prints “There was no crash dump found on your device!”
It looks like this is an issue with the depthai SDK, Luxonis has said they will come with a with a fix for their depthai SDK near end of Febuary that should address this problem. If you have a pro license with pro support hours, we can work on getting a separate build with a downgraded stable working SDK. If not you will probably need to wait until Luxonis releases their fix.
Hey, thanks for the message.
Unfortunately we only have a Touchdesigner Commercial license, so we have to wait until the end of February. We also ran the original deapth_ai_handtracking model and it works on Mac, but unfortunately not on Windows.
Cheers,
Bene
Thanks for your feedback! I just wanted to check in again—do you know when this update will happen? I really need a concrete answer now, as we have to deliver the exhibit by March 19th. If there isn’t a link by next week, I’ll have to explore other options. Appreciate your help!
I have now tried to replace the depthai.cp311-win_amd64.pyd file and other folders in "C:\Program Files\Derivative\TouchDesigner\bin\Lib\site-packages" with the DepthAI version 2.29.0.0.dev0+818211d7d970a931cc128b0b98f495af631092f4.
Within TouchDesigner, I can also verify the updated version using the following commands in a text DAT:
import depthai as dai
print('depthai version:', dai.__version__)
print('depthai location:', dai.__file__)
Unfortunately, when initializing the OAK Device CHOP, I get the same error as described in the existing forum post:
Is there perhaps a simple workaround to fix this error without having to wait for a newly updated build?
Sorry for the delay, while updating to that specified build I was still having issues with the HandTracking in both the original Github project and in TD. I contacted Luxonis to see if they are able to reproduce. It seems that they are planning to release a new version of DepthAI on Friday with fixes. In the meantime I am currently building a version of TD with the version of DepthAI specified in that command and will post a link by today. Please let me know if you are still having problems in the new build.
Hi huck,
thanks for the quick response. I’ve had the version running all day, and the camera only froze once, but TouchDesigner didn’t crash because of it.
What other issues did you encounter when testing this version?
Thanks again for your help!
Hey huck,
Thanks for your help. Our installation is now up and running, but recently the camera has been freezing more frequently, even though it used to run stably for several days or even weeks. Do you have any idea what might be causing this freeze?
Also, I noticed that a new version of TouchDesigner was released — does it already include a newer DepthAI version, the one with the updated hand tracking?
Hm, you can try setting the TOUCH_TEXT_CONSOLE environment variable to 1 to get some extra debugging information. When a freeze or crash occurs with the DepthAI library it should output additional information to the console.
The new release of TD still has the old version of DepthAI, I do however have an updated build that also has the most recent release of DepthAI (v2.30.0) that you can try out. In the end I think this is an issue on the DepthAI SDK side, so the amount help I can do is limited. Does the freeze happen consistently by some action (i.e. I was seeing freezing when moving my hand very quickly towards and away from the camera) or does it happen when no tracking is done?
