Camera Oak-D how Import new python examples?

Hi, i just got a OAK-D LITE, it’s working well in the experimental version, at the moment only i got the RGB camera with the python code that comes with the oakdevice CHOP.
someone tried with other examples from DepthAI? I’m a bit lost… how to correctly enter the other python examples in the DAT callbacks.

Thanks so much.

I am exploring this as well. I want to play with the depth info from a Oak-D Pro POE. In the code example on luxonis’s site it i writing out to cv2.imshow(). What is the method for writing to the pipeline instead?

This is the example I want to try: Depth Preview — DepthAI documentation | Luxonis

1 Like

Hey guys, the DepthAI API is documented extensively at
https://docs.luxonis.com/projects/api/en/latest/

Some DepthAI terminology:

A Pipeline is a complete workflow on the OAK, consisting of nodes and the connections between them.
A Node is a single functionality (a single operator, in TD terms) of the DepthAI. Nodes have inputs and outputs, and have configurable properties. Just like the nodes in TouchDesigner.

So a good starting point is to start reading about the available DepthAI Nodes, such as ColorCamera and XLInkOut (this last one is important, as it is the node which sends data to TouchDesigner):
https://docs.luxonis.com/projects/api/en/latest/components/nodes/

And about their inputs and outputs, such as the inputs and outputs of the Colorcamera node:
https://docs.luxonis.com/projects/api/en/latest/components/nodes/color_camera/#inputs-and-outputs

The essentials how to connect nodes in the pipeline are laid out in the example in the oakdevice1_callbacks of the OAK Device CHOP, in the method createPipeline():

first we create the pipeline:

pipeline = dai.Pipeline()

Now we define two nodes, a ColorCamera and a XLInkOut:

# Define nodes
camRgb = pipeline.create(dai.node.ColorCamera)
xoutRgb = pipeline.create(dai.node.XLinkOut)

Below we give the XLinkOut stream(s) to TouchDesigner a unique name.
To receive this stream in TD use the same name “rgb” in your OAK Select TOP’s stream parameter:

# Stream names
xoutRgb.setStreamName('rgb')

Then we set some properties (parameters in TD terms) of the ColorCamera node:

# Properties
camRgb.setPreviewSize(300, 300)

An now we will link two nodes together - we will send one of the outputs of the ColorCamera node, in this case the “preview” output to the input of the XLinkOut node, which will send it to
TouchDesigner, where you can select it with a OAK Select TOP

# Linking
camRgb.preview.link(xoutRgb.input)

Now that all nodes are set up and connected our pipeline is complete, so we return our pipeline definition to the OAK device so it can be processed there:

return pipeline

@amazingrobot in that code example you linked to, everything after line 46 is happening on the host computer, so not on the OAK device. So any actions done there, you would do inside TouchDesigner.
In your example cv2.imshow is just used to open a window and show a frame from the stream named “disparity” in there. Not something you would want when using with TouchDesigner. If you wanted to show the exact same thing inside TD, you would place a OAK Select TOP (set it’s stream parameter to “disparity” (defined on line 23) and then you could use cv2 in a Script TOP for image manipulation as done here on line 54 and 59 .

So, basically you can fully run that example if you only used lines 4 to 42, and then end with “return pipeline” as in the TD example.

Here’s another example, all node linking & pipeline setup on the OAK are done in line 3 to 37.
Everything after that line is happening on the host computer (not on the OAK). In this case it’s just writing a video file from an output (“enc”) we already created earlier (line 35-37). So when using TouchDesigner your would replace everything after line 37 with:

return pipeline

And then use a OAK Select TOP to receive the “enc” stream and wire it to a Movie File Out.

4 Likes

Thanks for the explanation. I have a big project coming soon and due to local unavailability of kinects we’re checking out oak-d cameras.
In Touch documentation for Oak Cameras ( Experimental:OAK-D - Derivative ) there is a reference to a sample DepthaiTestSuite.toe file, but the link doesn’t work. Do you know if it is available anywhere?

I, too, cannot find this DepthaiTestSuite.toe file anywhere. Perhaps someone from Derivative can post it here? :smirk:

2 Likes

At first I also had errors using another python examples, but this trick did it for me:
the issue is trying to pass the encoded image to TouchDesigner. This seems to be causing the hard crash there.

Instead just pass the disperity directly to the output:

        dispOut = pipeline.create(dai.node.XLinkOut)
 dispOut.setStreamName('disp')
     depth.disparity.link(dispOut.input)

Thanks to Snaut=Markus Heckmann
OAK D depth.toe (4.7 KB)

1 Like

would be also interested in DepthaiTestSuite.toe

The file is in \Samples\OAK of TouchDesigner’s installation folder. It’s now named OAKExamples.toe

1 Like

Where to find that folder on a Mac?