I’m reporting a bug where I’m unable to retrieve the color image from an Orbbec Femto Mega connected via Ethernet. This issue occurs with both the Kinect Azure TOP and the Orbbec TOP. Other streams, such as Depth and IR, are working correctly.
Environment
TouchDesigner Version: 2023.12370
OS: Windows
Device: Orbbec Femto Mega
Firmware Version: v1.3.1
Connection: Ethernet
Error Log
I checked the OrbbecSDK.log and found the following key error messages related to the MJPEG stream:
[08/04 15:59:01.196074][warning][27056][ObException.hpp:40] Failed to initiate the "VIDEO/OB_FMT_MJPEG" subsession: RTP payload format unknown or not supported
[08/04 15:59:01.196102][warning][27056][RTSPStreamPort.cpp:265] Execute failure! A libobsensor_exception has occurred!
- where:265#libobsensor::pal::RTSPStreamPort::startStream
- msg:Failed to initiate the "VIDEO/OB_FMT_MJPEG" subsession: RTP payload format unknown or not supported
[08/04 15:59:01.204863][warning][27056][VideoSensor.cpp:657] Execute failure! A libobsensor_exception has occurred!
- where:657#libobsensor::VideoSensor::start
- msg:Failed to initiate the "VIDEO/OB_FMT_MJPEG" subsession: RTP payload format unknown or not supported`
The core of the issue seems to be: RTP payload format unknown or not supported.
Investigation and Suspected Cause
After some research, I think the root cause is a version mismatch between the device firmware and the SDK bundled with TouchDesigner.
The Orbbec SDK version included in TouchDesigner 2023.12370 appears to be v1.10.8.
According to the official Orbbec release notes, the latest Femto Mega firmware (v1.3.1) requires Orbbec SDK v1.10.16 or newer.
To verify this, I tested with the standalone Orbbec Viewer using SDK v1.10.8 and confirmed that it also fails to display the color image, reproducing the same issue.
This strongly suggests that the SDK in the current TouchDesigner build is not compatible with the latest Femto Mega firmware for color streaming.
Question
While the ultimate solution would be to update the bundled Orbbec SDK in a future TouchDesigner release, is there any known workaround for this problem?
Thanks for all of the information and sorry you’re having problems with this. Unfortunately, the color video stream is not supported over ethernet because it requires a decompression library that we don’t currently have support for. There is a brief mention of this on the Orbbec TOP page, but I’ll try to make that more prominent.
On a separate note, we are currently looking at upgrading our version of the Orbbec SDK since we realize that some newer camera models are no longer supported. Unfortunately Orbbec has not released version 2 for MacOS which makes it a little more challenging to update.
Hi Rob, hope you’re doing great! I’m wondering if there has been an developments on adding color streaming to the orbbec top via ethernet.
I’m on a job that requires us to stream depth and color data via ethernet. The only work around I can think of is trying to stream the data via the video stream top because the same as kiyu i’m seeing the color stream via the orbbec viewer.
Our focus at the moment is getting the initial 2025 official release out as soon as possible, but we’ve discussed this issue internally and are hoping to allocate some time to it after launch.
Sorry I can’t be more precise than that, but we appreciate everyone’s interest in this.
In our latest 2025.31310 experimental release, we updated the Orbbec SDK to 1.10.16. Our Femto Mega is on firmware 1.2.7, so we are wondering if you could test the color stream with ethernet for us using your camera. Our Femto Mega is able to see an RGB output from the color stream via ethernet on the Orbbec TOP and Kinect Azure TOP. Could you also tell us which firmware version you are on? Thanks a lot!
I can confirm that 2025.31310 experimental release and Femto Firmware 1.3.1 works with Color Channel receive over ethernet and also in combination with the other channels (just try out the different fps and resolution options to see what fits because there are some strange compatibility rules (for example certain resolutions such as 1440p work best with 25 fps and depth on 576p).
Hope this change also comes to the stable version of Touchdesigner. Thank you for your great work
The only thing I would be really interested (but probably impossible) is if there was a camera control option to handle the focus / iso / temperature etc. because the color quality of this camera seems to be quite bad to be used for some artwork. It would be really great if that was controllable through ethernet also.
Thanks for testing it for us. Glad to hear that it’s working.
We have moved from experimental releases to official, so you should be able to get an RGB output from the color stream via Ethernet on the latest official builds.
Regarding your point about setting camera options, it appears that there has been a regression in setting camera properties when we updated to Orbbec 1.10.16 SDK. We’ll work on a fix for that. Thank you for reporting it.
