Custom cpp top operator for Zed 2 body tracking

General TouchDesigner Discussion
#1

Hello everybody. I am writing a top operator for the ZED 2 camera that can perform body tracking.
Basically, I am trying to do in TD what is available in the examples here:

I have compiled that examples locally, I have followed this tutorial https://www.youtube.com/watch?v=9wBwRDMS2I4 and I was finally able today to initialize the zed camera in my TOP operator.

The problem that I am facing now, is that if I add this line in the constructor

ObjectDetectionParameters obj_det_params;

I receive an obscure error:

“Exception thrown at 0x00007FF912C7A458 (ntdll.dll) in TouchDesigner.exe: 0xC0000139: Entry Point Not Found.”

Any idea about what it could be?

Are there any other people working on the Zed 2 integration? or is it a process that happens just inside Derivative? It would be helpful to share some insights.

1 Like
#2

This error likely means it’s failing to find a required .dll when it first tries to use that feature. Do you have all the required .dlls next to your plugin .dll?

#3

Thanks @malcolm for your reply. In visual studio, I have compiled the examples from the zed repository. Then, I have copied all the settings(from the Linker section to the Additional libraries in the c++ section) from the zed official examples to the visual studio solution containing my custom operator.

Is there a way to check which dll is missing? I am new to windows programming

#4

You can use this Dependencies tool for that

#5

Yep that’s the tool I would suggest too.
Visual studio doesn’t do any of this automatically for you. It’s literally about copying the .dlls to the directory where your plugin .dll resides. You can make post-build rules for that. But nothing about the include/linking will do that for you.

#6

Thanks @malcolm and @nettoyeur for your reply. I Have downloaded the tool and I have started to play around with it. I have first run the tool with the option “Child Only”, and everything seems ok

Normal
Normal1453×571 65.5 KB

With options recursive I have a bunch of missing libraries, with 2 types of error. The first one, is “module could not be found on disk”. Actually it is a long list and I hope I do not have to install them one by one.

Missing
Missing1369×324 25.3 KB

The second error is happening six times and it says “module x has missing imports”

Boh
Boh1361×211 27.1 KB

I am quite new to windows at these levels, do you have any hints about how could I fix this .dll?

#7

In the meantime, I have uploaded the code here https://github.com/edap/zed-examples/blob/master/zedTop/cpp/src/TOP_main.cpp with the instructions about how to build it.

The camera is active, the operator loads in TD, but if I uncomment the line “//ObjectDetectionParameters obj_det_params;” it does not load anymore in TD. Although it compiles.

#8

Your screenshots include sl_zed64.dll Is that DLL next to your compiled C++ TOP?

#9

Thanks for your reply @DavidBraun. If for “next” you mean in the same folder, no, it isn’t. Do you think it should be? All the .dlls or just this one?

#10

Yes sl_zed64.dll should be next to your custom DLL. Everything else from the dependencies.exe screenshot looks normal.

#11

I have first tried to copy sl_zed64.dll from the ZED sdk to the folder where my custom TOP .dll is, then I have tried to copy all the .dll that are in the ZED sdk folder to same location. Tthe error still persists, I can compile the lib but as soon as I uncomment the line “//ObjectDetectionParameters obj_det_params;” the TOP operator can not be loaded in TD.

#12

When using the Pytorch libtorch SDK, for some reason I had to place the DLLs inside the TouchDesigner executable’s bin folder, wherever it may be, like in C:/Program Files/Derivative/TouchDesigner/bin. Your location might be different. This is a strange thing to try because it’s not in the wiki, but maybe you can try it. Be prepared to delete the zed DLLs from this location if it doesn’t fix it.

#13

Well @DavidBraun, it worked, many thanks :wink:
there was already a file called sl_zed64.dll in that folder, but it was probably related to an old version of the SDK. With the 3.3.3 version it works. I have uploaded the code that I have until now and an updated readme file here https://github.com/edap/zed-examples

#14

I think the issue here is the .dll needed for object detection is being what’s call ‘delay-loaded’. That means that it doesn’t try to load the .dll until that line of code you execute it tries to use it.
The issue is that the search path for extra .dlls only includes your plugin directory during initial load of your .dll. After that it falls back to the default search, which won’t include your plugin dir.
I think the current solution for this is to use a delay load hook:

We’ll look at ways to make this cleaner though, or at least give an example of using this.

#15

Looking at this more, the above workflow only really works if your .dll is the one that specifies the /DELAYLOAD. I’m not sure if this is the case for your particular project configuration.
Another solution would be to manually load the .dlls you need using LoadLibrary() in your constructor. That way they are already loaded before they are referenced later on, so the .dll location is already resolved.
You can use the OP_NodeInfo->pluginPath to determine where our plugin is located on disk, and look for other .dlls setting next to that manually.
All add this info to the wiki.

#16

Hello everybody. I have got the Zed2 camera with body detection working in TD, the code, unpolished and unfinished is available here https://github.com/edap/zed-examples/blob/master/zedTop/cpp/src/TOP_main.cpp. Any suggestion is welcome. I am basically porting this code https://github.com/edap/zed-examples/tree/master/body%20tracking/cpp/src into a TD TOP operator.

@malcolm I am not a Windows expert, if replacing the sl_zed64.dll shipped with TD with the new version of the Zed SDK (3.3.3) works, I think I will stick to this solution as it is the most reliable at the moment.

#17

HI
Please tell me How to use your zed in TD
I wonder you solution to connect Zed sdk(C++) and TD
THX

#18

Short update here. The TOP operator has been updated to version the Zed SDK version 3.3.5. I have added some instructions to the readme in case someone want to try out skeleton tracking on TD

#19

Hello!
I just finished implementing your code and it works great! However since I copied all the dll, TouchDesigner crashes everytime I try to use a ZED nodes (TOP, CHOP, SOP). Can you tell what caused this? Could you specify the TD version you are using?
I’m using:

-ZED 2i
-SDK v. 3.6.5 (Unable to find 3.6.1)
-CUDA 11.1
-CMake 3.23.1
-TouchDesigner 2021.14360

Thanks!

#20

Thanks for your ZedTOP. It’s work fine.
I’m trying to make CHOP version of object tracking, and i’ve got an error when initializing “ObjectDetectionRuntimeParameters detection_parameters_rt”. When I comment this line, everything work fine.

Unhandled exception at 0x00007FFD1E8DD288 (ucrtbase.dll) in TouchDesigner.exe: An invalid parameter was passed to a function that considers invalid parameters fatal.

It’s my first TD cpp project, so I don’t know what to do next and how to resolve this error. I’ve copied all dll’s to project folder and TD build folder.

Thanks.