Orbbec Gemini 2 Setup On Mac

I just took delivery of my new Orbecc Gemini 2 camera. Having previously used a Kinect with TouchDesigner, and having done some research to find that my new Mac M1 no longer worked with the Kinect in TouchDesigner, I thought this camera would play nice with TD.

While I can see the RGB camera output in TD with a Video Device In TOP, I am at a loss on how to access depth information or skeleton tracking so that I can capture OSC data to drive interaction. I see reference to an Orbbec TOP here - https://docs.derivative.ca/Orbbec_TOP - I cannot find where to access or download the TOP.

I’m afraid as a non-developer, I am not going to be able to use this camera for an upcoming collaboration with a dance troupe. Maybe it is beyond my skill level?

Any thoughts or direction on where to start will be much appreciated.

Thanks!

The Video Device In TOP doesn’t understand how to handle depth camera images, so you will need the Orbbec TOP to access the depth image from the gemini camera.

The Orbbec TOP was released for MacOS in the 2023.11510 version of TouchDesigner, so you will need at least that version to use it. You should find it in the TOP page of the OP Create menu. Note the information on the docs page about launching TouchDesigner with the sudo command which is currently required on MacOS, otherwise it will not be able to connect to the camera.

Unfortunately, the gemini camera does not support skeleton tracking … only colour, depth and point cloud images. The Orbbec Femto cameras are compatible with the Kinect Azure skeleton tracking system, but those are Windows only.

If you are interested in skeleton tracking on MacOS, you may want to look at the OAK-D cameras: OAK-D - Derivative They are a little more complicated than the Kinect or Orbbec systems, but they are fully programmable and support a wide range of AI features including skeleton tracking.

Hey thanks. I am using the Orbbec TOP now. Should have bought the OAK-D.

So I found a solution after all. This Mediapipe .toe is working nicely on my Mac M1 for face, hand, object, and pose tracking.
https://www.youtube.com/watch?v=Cx4Ellaj6kk

1 Like